2011年9月30日 星期五

[記事] Cassandra installation (cluster mode)

屬於阿宅世界的技術文章,想看的再點開,切莫自誤 !

配置

Server 1 : 192.168.1.1
Server 2 : 192.168.1.2

安裝

shell> cd /home
shell> wget http://ftp.twaren.net/Unix/Web/apache//cassandra/0.8.5/apache-cassandra-0.8.5-bin.tar.gz

shell> mv apache-cassandra-0.8.5 cassandra

shell> mkdir -p /home/cassandra/data
shell> mkdir -p /home/cassandra/commitlog
shell> vi /home/cassandra/conf/cassandra.yaml
# directories where Cassandra should store data on disk.
data_file_directories:
- /home/cassandra/data

# commit log
commitlog_directory: /home/cassandra/commitlog

# saved caches
saved_caches_directory: /home/cassandra/saved_caches

# 設成空白表示讓cassandra 自己決定
listen_address:

# 設成空白表示讓cassandra 自己決定
rpc_address:

# Seed provider
seed_provider:
- class_name: org.apache.cassandra.locator.SimpleSeedProvider
parameters:
- seeds: "192.168.1.1, 192.168.1.2"



shell> mkdir /home/logs/cassandra

shell> vi /home/cassandra/conf/log4j-server.properties
log4j.appender.R.File=/home/logs/cassandra/system.log


shell> vi /home/cassandra/conf/cassandra-env.sh
#unmark and set ip if you're behind a firewall /nat
JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=192.168.1.1"


# 兩台 server 都執行下列指令啟動
shell> /home/cassandra/bin/cassandra -f -p /home/logs/cassandra/cassandra.pid &

shell> netstat -anltp | grep java
tcp 0 0 0.0.0.0:35271 0.0.0.0:* LISTEN 13773/java
tcp 0 0 192.168.1.1:9160 0.0.0.0:* LISTEN 13773/java
tcp 0 0 192.168.1.2:7000 0.0.0.0:* LISTEN 13773/java
tcp 0 0 0.0.0.0:50782 0.0.0.0:* LISTEN 13773/java
tcp 0 0 0.0.0.0:7199 0.0.0.0:* LISTEN 13773/java


透過 nodetool 這個指令可以看到這個 cluster 下所有node 的狀況
如果有溝通不良的情事,請檢查上面列出來的 port 是不是被 firewall 檔了
shell> /home/cassandra/bin/nodetool -host localhost ring
Address DC Rack Status State Load Owns Token
120681864659184910919105277533403393170
192.168.1.1    datacenter1 rack1 Up Normal 6.55 KB 44.26% 25841189369168188429084524995506191071
192.168.1.2    datacenter1 rack1 Up Normal 6.55 KB 55.74% 120681864659184910919105277533403393170


建立成功後應該就可以透過 cassandra-cli 下達指令了
shell> /home/cassandra/bin/cassandra-cli -h 192.168.1.1
Connected to: "Test Cluster" on 192.168.1.1/9160
Welcome to the Cassandra CLI.

Type 'help;' or '?' for help.
Type 'quit;' or 'exit;' to quit.

[default@unknow] help;
Getting around:
? Display this help.
help; Display this help.
help ; Display command-specific help.
exit; Exit this utility.
quit; Exit this utility.

Commands:
assume Apply client side validation.
connect Connect to a Cassandra node.
consistencylevel Sets consisteny level for the client to use.
count Count columns or super columns.
create column family Add a column family to an existing keyspace.
create keyspace Add a keyspace to the cluster.
del Delete a column, super column or row.
decr Decrements a counter column.
describe cluster Describe the cluster configuration.
describe keyspace Describe a keyspace and it's column families.
drop column family Remove a column family and it's data.
drop keyspace Remove a keyspace and it's data.
drop index Remove an existing index from specific column.
get Get rows and columns.
incr Increments a counter column.
list List rows in a column family.
set Set columns.
show api version Show the server API version.
show cluster name Show the cluster name.
show keyspaces Show all keyspaces and their column families.
show schema Show a cli script to create keyspaces and column families.
truncate Drop the data in a column family.
update column family Update the settings for a column family.
update keyspace Update the settings for a keyspace.
use Switch to a keyspace.


cassandra-cli 的指令和我們一般習慣的SQL指令有很大的差別,反正都是不會
再安裝個 CQL (Cassandra Query Language)出來,看會不會好一點
目前的CQL client 有三種選擇 (http://www.datastax.com/docs/0.8/dml/about_clients)
我們選python 的版本來 玩玩看。
不過它有點挑食,需要 Python 2.6 以上、 3.0 以下的版本 = = ! ,如果不幸都沒有的,只好重頭裝起

shell> mkdir -p /home/python/src/
shell> cd /home/python/src/
shell> wget http://www.python.org/ftp/python/2.7.2/Python-2.7.2.tgz
shell> tar zxvf Python-2.7.2.tgz
shell> cd Python-2.7.2
shell> ./configure --prefix=/home/python
shell> make; make install

將 python 2.7 的執行目錄加進目前的path 裡
shell> vi /etc/profile

PATH=$PATH:/home/python/bin
export $PATH


先幫 python 裝  easy_install 這個模組,方便後續其他模組的安裝

shell> cd /home/python/src
shell> wget http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg#md5=fe1f997bc722265116870bc7919059ea
shell> sh setuptools-0.6c11-py2.7.egg --prefix=/home/python

再安裝必要的兩個模組

shell> easy_install readline
shell> easy_install thrift

再接著安裝 CQLSH
shell> cd /home/python/src
shell> wget http://www.apache.org/dist/cassandra/drivers/py/cql-1.0.3.tar.gz
shell> tar zxvf cql-1.0.3.tar.gz
shell> cd cql-1.0.3
shell> python2.7 setup.py install

如果以上執行沒有任何錯誤訊息產生,就可以試著進入CQL shell
shell> cqlsh 192.168.1.1 9160
cqlsh>

只是啊,這個CQL雖然說可以類似SQL的方式進行資料操作,但功能實在是陽春到不行,沒有 cassandra-cli 提供的一些諸如 show keyspaces之類的基本功能,唉,就不能和在一起弄個撒尿牛丸嗎 :~

最後....說好的HIVE呢 @@a

Ref:

0 意見:

張貼留言