大型网所使用的工具.ppt

Connecting to HBase Java client get(byte [] row, byte [] column, long timestamp, int versions); Non-Java clients Thrift server hosting HBase client instance Sample ruby, c++, java (via thrift) clients REST server hosts HBase client TableInput/OutputFormat for MapReduce HBase as MR source or sink HBase Shell JRuby IRB with “DSL” to add get, scan, and admin ./bin/hbase shell YOUR_SCRIPT Thrift a software framework for scalable cross-language services development. By facebook seamlessly between C++, Java, Python, PHP, and Ruby.? This will start the server instance, by default on port 9090 The other similar project “rest” $ hbase-daemon.sh start thrift $ hbase-daemon.sh stop thrift References 趨勢科技HBase 介紹 http://www.wretch.cc/blog/trendnop09 Hadoop: The Definitive Guide Book, by Tom White HBase Architecture 101 /2009/10/hbase-architecture-101-storage.html * 大型網站所使用的工具 Perlbal - /perlbal/ 多個網頁伺服器的負載平衡 MogileFS - /mogilefs/ 分散式檔案系統 有公司認為 MogileFS 比起 Hadoop 適合拿來處理小檔案 memcached - / 共享記憶體?? 把資料庫或其他需要經常讀取的部分，用記憶體快取(Cache)方式存放 Moxi - /p/moxi/ Memcache 的 PROXY More Resource: /p/memcached/wiki/HowToLearnMoreScalability /techdude/scalable-web-architectures-common-patterns-and-approaches How to scale up web service in the past ? Source: /mallipeddi/scalable-lamp-development-for-growing-web-apps Source: /2008/07/memcached-for-mysql-advanced-use-cases_09.html Source: /northscale/moxi-memcached-proxy Source: /northscale/moxi-memcached-proxy * HBase Intro 王耀聰 陳威宇 jazz@.tw waue@.tw 教育訓練課程 HBase is a distributed column-oriented database built on top of HDFS. HBase is .. A distributed data store that can scale horizontally to 1,000s of commodity servers and petabytes of indexed storage. Designed to operate on top of the Hadoop distributed file system (HDFS) or Kosmos File System (KFS, aka Cloudstore) for scalability, fault tolerance, and high availability. Integrated into the Hadoop map-reduce platform and paradigm. Benefits Distributed storage Table-like in data struc