在Debian上搭建Kafka集群需要以下步骤:
wget https://download.java.net/java/GA/jdk11/9/GPL/openjdk-11.0.2_linux-x64_bin.tar.gz
tar -xf openjdk-11.0.2_linux-x64_bin.tar.gz
sudo mkdir /usr/lib/jvm
sudo mv jdk-11.0.2 /usr/lib/jvm/
sudo vim /etc/profile
在底部追加如下内容:
export JAVA_HOME=/usr/lib/jvm/jdk-11.0.2
export PATH=$PATH:$JAVA_HOME/bin
source /etc/profile
sudo mkdir -p /data/kafka
sudo chown -R $(whoami):$(whoami) /data/kafka
cd /tools
wget https://downloads.apache.org/kafka/3.5.2/kafka_2.12-3.5.2.tgz
tar -xzf kafka_2.12-3.5.2.tgz
sudo mv kafka_2.12-3.5.2 /usr/local/kafka
server.properties
文件,配置如下参数:# data-vm1节点
node.id=1
process.roles=broker,controller
listeners=plaintext://data-vm1:9092,controller://data-vm1:9093
advertised.listeners=plaintext://data-vm1:9092
controller.quorum.voters=data-vm1:9093,data-vm2:9093,data-vm3:9093
log.dirs=/data/kafka
num.partitions=3
offsets.topic.replication.factor=3
transaction.state.log.replication.factor=3
transaction.state.log.min.isr=data-vm1,data-vm2,data-vm3
/etc/hosts
文件,添加各个节点的IP地址和主机名:192.168.112.10 data-vm1
192.168.112.130 data-vm2
192.168.112.129 data-vm3
# 启动Zookeeper
/usr/local/kafka/bin/zookeeper-server-start.sh config/zookeeper.properties
/usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties
/usr/local/kafka/bin/kafka-topics.sh --create --topic test-topic --bootstrap-server data-vm1:9092 --replication-factor 3 --partitions 3
/usr/local/kafka/bin/kafka-console-producer.sh --broker-list data-vm1:9092 --topic test-topic
/usr/local/kafka/bin/kafka-console-consumer.sh --bootstrap-server data-vm1:9092 --topic test-topic --from-beginning
以上步骤应该可以帮助你在Debian上成功搭建一个Kafka集群。如果在搭建过程中遇到问题,请检查配置文件和日志文件以获取更多信息。