kafka-manager 설치

kafka-manager를 설치해 보자.

할 건 많고, 시간과 체력, 나의 뇌 성능으론 버겁다.

우리나라에선 철수한 Yahoo!!에서 만들었다. 이것도 철수네.철수가 참..

https://github.com/yahoo/kafka-manager/releases

sbt build다. 2년전에 써보고 오랜만이다. 시간이 꽤 걸린다.

[root@big-master ~]# adduser kafka; echo 'kafka' | passwd --stdin kafka; usermod -G datagroup kafka
Changing password for user kafka.
passwd: all authentication tokens updated successfully.

[root@big-master ~]# cd /usr/local/src/
[root@big-master src]#
[root@big-master src]# wget https://github.com/yahoo/kafka-manager/archive/1.3.3.17.tar.gz
--2018-05-07 18:48:51--  https://github.com/yahoo/kafka-manager/archive/1.3.3.17.tar.gz
Resolving github.com (github.com)... 192.30.255.113, 192.30.255.112
Connecting to github.com (github.com)|192.30.255.113|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/yahoo/kafka-manager/tar.gz/1.3.3.17 [following]
--2018-05-07 18:48:52--  https://codeload.github.com/yahoo/kafka-manager/tar.gz/1.3.3.17
Resolving codeload.github.com (codeload.github.com)... 192.30.255.120, 192.30.255.121
Connecting to codeload.github.com (codeload.github.com)|192.30.255.120|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: ‘1.3.3.17.tar.gz’

100%[========================================================>]  954,002      302KB/s   in 3.1s

2018-05-07 18:48:56 (302 KB/s) - ‘1.3.3.17.tar.gz’ saved [954002]
[root@big-master src]# tar zxvpf 1.3.3.17.tar.gz
[root@big-master src]# cd kafka-manager-1.3.3.17/
[root@big-master kafka-manager-1.3.3.17]# ./sbt clean dist
Downloading sbt launcher for 0.13.9:
  From  http://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/0.13.9/sbt-launch.jar
    To  /root/.sbt/launchers/0.13.9/sbt-launch.jar
Getting org.scala-sbt sbt 0.13.9 ...
downloading https://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/relation/0.13.9/jars/relation.jar ...
        [SUCCESSFUL ] org.scala-sbt#relation;0.13.9!relation.jar (5782ms)
#많이 시간이 흐른 후~~~
[info]   Compilation completed in 17.279 s
model contains 684 documentable templates
[info] Main Scala API documentation successful.
[info] Packaging /usr/local/src/kafka-manager-1.3.3.17/target/scala-2.11/kafka-manager_2.11-1.3.3.17-javadoc.jar ...
[info] Done packaging.
[info] Packaging /usr/local/src/kafka-manager-1.3.3.17/target/scala-2.11/kafka-manager_2.11-1.3.3.17.jar ...
[info] Done packaging.
[info] Packaging /usr/local/src/kafka-manager-1.3.3.17/target/scala-2.11/kafka-manager_2.11-1.3.3.17-sans-externalized.jar ...
[info] Done packaging.
[info]
[info] Your package is ready in /usr/local/src/kafka-manager-1.3.3.17/target/universal/kafka-manager-1.3.3.17.zip
[info]
[success] Total time: 108 s, completed May 7, 2018 7:46:57 PM
[root@big-master kafka-manager-1.3.3.17]#
[root@big-master kafka-manager-1.3.3.17]# unzip -d /bigdata/ target/universal/kafka-manager-1.3.3.17.zip
[root@big-master kafka-manager-1.3.3.17]# vi /bigdata/kafka-manager-1.3.3.17/conf/application.conf
kafka-manager.zkhosts="big-master:2181,big-slave1:2181,big-slave2:2181"
:wq!
[root@big-master kafka-manager-1.3.3.17]# chown -R kafka.kafka /bigdata/kafka-manager-1.3.3.17/
[root@big-master kafka-manager-1.3.3.17]# ln -s /bigdata/kafka-manager-1.3.3.17 /kafka-manager
[root@big-master kafka-manager-1.3.3.17]# vi /etc/systemd/system/kafka-manager.service
[Unit]
Description = kafka-manager ( ver. 1.3.3.17 )
After = network.target

[Service]
Type=simple
User=kafka
Group=kafka
WorkingDirectory=/kafka-manager
Environment=JAVA_HOME=/jdk
ExecStart = /kafka-manager/bin/kafka-manager -Dconfig.file=/kafka-manager/conf/application.conf -Dhttp.port=9000 -Dapplication.home=/kafka-manager
SuccessExitStatus=143

[Install]
WantedBy=multi-user.target
:wq!

[root@big-master kafka-manager-1.3.3.17]# systemctl daemon-reload
[root@big-master kafka-manager-1.3.3.17]# systemctl status kafka-manager.service
● kafka-manager.service - kafka-manager ( ver. 1.3.3.17 )
   Loaded: loaded (/etc/systemd/system/kafka-manager.service; disabled; vendor preset: disabled)
   Active: inactive (dead)
[root@big-master kafka-manager-1.3.3.17]# systemctl enable kafka-manager.service
Created symlink from /etc/systemd/system/multi-user.target.wants/kafka-manager.service to /etc/systemd/system/kafka-manager.service.
[root@big-master kafka-manager-1.3.3.17]# systemctl status kafka-manager.service
● kafka-manager.service - kafka-manager ( ver. 1.3.3.17 )
   Loaded: loaded (/etc/systemd/system/kafka-manager.service; enabled; vendor preset: disabled)
   Active: inactive (dead)
[root@big-master kafka-manager-1.3.3.17]# systemctl start kafka-manager.service
[root@big-master kafka-manager-1.3.3.17]# systemctl status kafka-manager.service
● kafka-manager.service - kafka-manager ( ver. 1.3.3.17 )
   Loaded: loaded (/etc/systemd/system/kafka-manager.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2018-05-08 20:45:16 KST; 12s ago
 Main PID: 5691 (java)
   CGroup: /system.slice/kafka-manager.service
           └─5691 /jdk/bin/java -Duser.dir=/bigdata/kafka-manager-1.3.3.17 -Dconfig.file=/kafka-manager/conf/application.conf -Dhttp.port=9000 -Dappli...

May 08 20:45:22 big-master kafka-manager[5691]: [info] k.m.a.c.KafkaStateActor - Starting kafka managed Topic Offset Getter ...
May 08 20:45:22 big-master kafka-manager[5691]: [info] k.m.a.c.KafkaManagedOffsetCache - key.deserializer=org.apache.kafka.common.serializatio...rializer
May 08 20:45:22 big-master kafka-manager[5691]: [info] k.m.a.c.KafkaManagedOffsetCache - auto.offset.reset=latest
May 08 20:45:22 big-master kafka-manager[5691]: [info] k.m.a.c.KafkaManagedOffsetCache - bootstrap.servers=big-slave2:9092,big-slave3:9092
May 08 20:45:22 big-master kafka-manager[5691]: [info] k.m.a.c.KafkaManagedOffsetCache - enable.auto.commit=false
May 08 20:45:22 big-master kafka-manager[5691]: [info] k.m.a.c.KafkaManagedOffsetCache - group.id=KMOffsetCache-big-master
May 08 20:45:22 big-master kafka-manager[5691]: [info] k.m.a.c.KafkaManagedOffsetCache - value.deserializer=org.apache.kafka.common.serializat...rializer
May 08 20:45:22 big-master kafka-manager[5691]: [info] k.m.a.c.KafkaManagedOffsetCache - exclude.internal.topics=false
May 08 20:45:22 big-master kafka-manager[5691]: [info] k.m.a.c.KafkaManagedOffsetCache - security.protocol=PLAINTEXT
May 08 20:45:22 big-master kafka-manager[5691]: [info] k.m.a.c.KafkaManagedOffsetCache - Consumer created for kafka offset topic consumption f...-cluster
#방화벽 활성화 상태시 port 열기
[root@big-devel kafka-manager-1.3.3.17]# firewall-cmd --permanent --zone=public --add-port=9000/tcp