zt-ubuntu 12.04 安装 Cloudera Manager 4.8.2(Mysql)

http://www.idefs.com/recordsubuntu-12-04-cloudera-installation-manager-4-8-2-mysql.html

Cloudera可以简化Hadoop的安装配置过程,自动在集群节点上安装hadoop相关组件,创建用户。Cloudera是Hadoop生态系统中,规模最大、知名度最高的公司

1.准备

系统版本

1 iflab@cdh1:~$ uname -a
2 Linux cdh1 3.11.0-15-generic #25~precise1-Ubuntu SMP Thu Jan 30 17:39:31 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
3 iflab@cdh1:~$

角色分配(需要修稿hostname、hosts)

1 IP               主机名
2 222.249.250.211  cdh1    manager master 
3 222.249.250.212  cdh2    slave1
4 222.249.250.213  cdh3    slave2
5 222.249.250.214  cdh4    slave3

确保所有机器root密码一致或者sudo免输入密码用户的用户名、密码一致!

2.添加Cloudera源

1 sudo sudo nano /etc/apt/sources.list.d/cloudera.list

写入

获取apt key

刷新源列表

1 sudo apt-get update

3.安装java环境

1 sudo apt-get -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold -y install oracle-j2sdk1.6

4.安装mysql以及JDBC驱动

1 sudo apt-get install mysql-server libmysql-java

5.配置mysql

1 sudo nano /etc/mysql/conf.d/mysql_cloudera_manager.cnf

写入

1 [mysqld]
2 transaction-isolation=READ-COMMITTED
3 # Disabling symbolic-links is recommended to prevent assorted security risks;
4 # to do so, uncomment this line:
5 # symbolic-links=0
6
7 key_buffer              = 16M
8 key_buffer_size         = 32M
9 max_allowed_packet      = 16M
10 thread_stack            = 256K
11 thread_cache_size       = 64
12 query_cache_limit       = 8M
13 query_cache_size        = 64M
14 query_cache_type        = 1
15 # Important: see Configuring the Databases and Setting max_connections
16 max_connections         = 550
17
18 # log-bin should be on a disk with enough free space
19 log-bin=/var/log/mysql/mysql_binary_log
20
21 # For MySQL version 5.1.8 or later. Comment out binlog_format for older versions.
22 binlog_format           = mixed
23
24 read_buffer_size = 2M
25 read_rnd_buffer_size = 16M
26 sort_buffer_size = 8M
27 join_buffer_size = 8M
28
29 # InnoDB settings
30 innodb_file_per_table = 1
31 innodb_flush_log_at_trx_commit  = 2
32 innodb_log_buffer_size          = 64M
33 innodb_buffer_pool_size         = 4G
34 innodb_thread_concurrency       = 8
35 innodb_flush_method             = O_DIRECT
36 innodb_log_file_size = 512M
37
38 [mysqld_safe]
39 log-error=/var/log/mysqld.log
40 pid-file=/var/run/mysqld/mysqld.pid

编辑my.cnf
sudo nano /etc/mysql/my.cnf
把下面这一行注释掉

1 #bind-address           = 127.0.0.1

6.配置innodb

1 sudo su
2 mv /var/lib/mysql/ib_logfile* /var/tmp/
3 logout

7.初始化数据库

1 sudo service mysql restart
2 mysql -uroot -p
3
4 create database amon DEFAULT CHARACTER SET utf8;
5 grant all on amon.* TO 'amon'@'%' IDENTIFIED BY 'amon_password';
6 grant all on amon.* TO 'amon'@'cdh1' IDENTIFIED BY 'amon_password';
7 create database smon DEFAULT CHARACTER SET utf8;
8 grant all on smon.* TO 'smon'@'%' IDENTIFIED BY 'smon_password';
9 grant all on smon.* TO 'smon'@'cdh1' IDENTIFIED BY 'smon_password';
10 create database rman DEFAULT CHARACTER SET utf8;
11 grant all on rman.* TO 'rman'@'%' IDENTIFIED BY 'rman_password';
12 grant all on rman.* TO 'rman'@'cdh1' IDENTIFIED BY 'rman_password';
13 create database hmon DEFAULT CHARACTER SET utf8;
14 grant all on hmon.* TO 'hmon'@'%' IDENTIFIED BY 'hmon_password';
15 grant all on hmon.* TO 'hmon'@'cdh1' IDENTIFIED BY 'hmon_password';
16 create database hive DEFAULT CHARACTER SET utf8;
17 grant all on hive.* TO 'hive'@'%' IDENTIFIED BY 'hive_password';
18 grant all on hive.* TO 'hive'@'cdh1' IDENTIFIED BY 'hive_password';

8.安装 cloudera-manager以及agent(因为master也是一个节点)

1 sudo apt-get install cloudera-manager-daemons cloudera-manager-server  cloudera-manager-agent

9.配置cloudera-manager-server数据库

1 sudo /usr/share/cmf/schema/scm_prepare_database.sh mysql  -uroot -p --scm-host localhost scm scm scm_password

10.配置其余节点(cdh2、cdh3、cdh4)

配置其余节点的源

1 sudo scp iflab@222.249.250.211:/etc/apt/sources.list.d/cloudera.list /etc/apt/sources.list.d/cloudera.list

获取apt key

刷新源列表

1 sudo apt-get update

在其余节点安装jdk

1 sudo apt-get -o Dpkg::Options::=--force-confdef -o Dpkg::Options::=--force-confold -y install oracle-j2sdk1.6

在其余节点安装cloudera-manager-agent 和cloudera-manager-daemons

1 sudo apt-get install cloudera-manager-agent cloudera-manager-daemons

修改agent的配置文件
sudo nano /etc/cloudera-scm-agent/config.ini
修改server_host=cdh1

重启每个节点的agent

1 sudo service cloudera-scm-agent  restart

11.重启控制节点cloudera-manager以及agent

1 sudo service cloudera-scm-server restart
2 sudo service cloudera-scm-agent  restart

12.访问web控制台

http://222.249.250.211:7180/ 用户名admin密码admin

QQ20140329-16@2x

选择免费版。。

QQ20140329-1@2x

继续

QQ20140329-2@2x

选择当前管理的主机,并勾选所有节点

QQ20140329-3@2x

此处我们使用自己的parcel,您可以默认,然后继续。

QQ20140329-4@2x

等待下载完,然后继续

QQ20140329-5@2x

检查主机正确性

QQ20140329-6@2x

选择需要安装的服务,然后点击 检查角色分配

QQ20140329-7@2x

检查并确认角色分配,然后点击 继续
QQ20140329-8@2x

输入之前创建的数据库用户名及密码

QQ20140329-9@2x

确认无误后点击 检查连接正确性,检测通过后点击继续

QQ20140329-10@2x

审核配置,点击 继续

QQ20140329-12@2x

启动集群结束后点击 继续

QQ20140329-13@2x

然后ok~

QQ20140329-14@2x

恭喜,enjoy!

QQ20140329-15@2x

 

Posted in Uncategorized.

edustack

edustack webmaster

发表评论

电子邮件地址不会被公开。 必填项已用*标注

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.