1 运行Kafka单实例
1.1 Windws
1.1.1 安装包下载
官网下载地址:https://kafka.apache.org/downloads,截至此文,最新版本是3.5.1。
压缩包解压至本地目录,例如:D:0_programmingkafka_2.12-3.5.1。
1.1.2 修改环境变量
KAFKA_HOME=D:0_programmingkafka_2.12-3.5.1
修改Path变量:%KAFKA_HOME%bin
1.1.3 修改配置文件
新建目录:D:0_programmingkafka_2.12-3.5.1kafka-logs
修改Kafka配置文件:D:0_programmingkafka_2.12-3.5.1configserver.properties
修改Zookeeper配置文件:D:0_programmingkafka_2.12-3.5.1configzookeeper.properties
1.1.4 启动Kafka单机版
依次运行如下命令,每次新打开一个命令行窗口:
启动Zookeeper:
zookeeper–server–start.bat %KAFKA_HOME%configzookeeper.properties
启动Kafka:
kafka-topics.bat –create —bootstrap–server localhost:9092 –topic test —partitions 1 —replication-factor 1
启动生产者:
kafka-console-producer.bat –broker-list localhost:9092 –topic test
启动消费者:
kafka-console–consumer.bat —bootstrap–server localhost:9092 –topic test
在消费者控制台收到消息:
1.2 Linux
环境:Windows 11 WSL2
Linux发行版本:Ubuntu 22.04.2 LTS
1.2.1 安装包下载
下载安装包
wget https://downloads.apache.org/kafka/3.6.0/kafka_2.13-3.6.0.tgz
解压
tar xvf kafka_2.13-3.6.0.tgz -C /usr/local/bin
1.2.2 创建目录
sudo mkdir $KAFKA_HOME/kafka-logs
sudo chmod -R o+w $KAFKA_HOME/kafka-logs
sudo mkdir $KAFKA_HOME/logs
sudo chmod -R o+w $KAFKA_HOME/logs
1.2.3 添加环境变量
vim ~/.bashrc
export KAFKA_HOME=/usr/local/bin/kafka_2.13-3.6.0
export PAHT=$PAHT:${KAFKA_HOME}/bin
加载环境变量:
source ~/.bashrc
1.2.4 修改配置文件
ip addr
vim $KAFKA_HOME/config/server.properties
listeners=PLAINTEXT://172.26.143.96:9092
advertised.listeners=PLAINTEXT://172.26.143.96:9092
log.dirs=/usr/local/bin/kafka_2.13-3.6.0/kafka-logs
1.2.5 运行Kafka
zookeeper-server-start.sh $KAFKA_HOME/config/zookeeper.properties
kafka-server-start.sh $KAFKA_HOME/config/server.properties
1.2.6 停止Kafka
kafka-server-stop.sh
zookeeper-server-stop.sh
2 搭建Kafka集群
2.1 搭建Zookeeper集群
过程参考:
在WSL单机搭建Zookeeper伪集群
2.2 搭建Kafka集群
2.2.1 新建集群目录
创建集群目录:
cd /usr/local/bin
sudo mkdir kafka-cluster
2.2.2 配置环境变量
vim ~/.bashrc
添加:
export KAFKA_CLUSTER_HOME=/usr/local/bin/kafka-cluster
source ~/bashrc
2.2.3 创建节点目录
将Kafka安装目录复制3份:
sudo cp -rf kafka_2.13-3.6.0 kafka-cluster/kafka-1
sudo cp -rf kafka_2.13-3.6.0 kafka-cluster/kafka-2
sudo cp -rf kafka_2.13-3.6.0 kafka-cluster/kafka-3
sudo rm -rf kafka-cluster/kafka-1/kafka-logs
sudo rm -rf kafka-cluster/kafka-2/kafka-logs
sudo rm -rf kafka-cluster/kafka-3/kafka-logs
sudo mkdir kafka-cluster/kafka-1/kafka-logs
sudo mkdir kafka-cluster/kafka-2/kafka-logs
sudo mkdir kafka-cluster/kafka-3/kafka-logs
sudo chmod -R 777 kafka-cluster/kafka-1/kafka-logs
sudo chmod -R 777 kafka-cluster/kafka-2/kafka-logs
sudo chmod -R 777 kafka-cluster/kafka-3/kafka-logs
sudo rm -rf kafka-cluster/kafka-1/logs
sudo rm -rf kafka-cluster/kafka-2/logs
sudo rm -rf kafka-cluster/kafka-3/logs
sudo mkdir kafka-cluster/kafka-1/logs
sudo mkdir kafka-cluster/kafka-2/logs
sudo mkdir kafka-cluster/kafka-3/logs
sudo chmod -R 777 kafka-cluster/kafka-1/logs
sudo chmod -R 777 kafka-cluster/kafka-2/logs
sudo chmod -R 777 kafka-cluster/kafka-3/logs
2.2.4 修改配置
sudo vim $KAFKA_CLUSTER_HOME/kafka-1/config/server.properties
broker.id=0
listeners=PLAINTEXT://172.26.143.96:9092
advertised.listeners=PLAINTEXT://172.26.143.96:9092
log.dirs=$KAFKA_CLUSTER_HOME/kafka-1/kafka-logs
zookeeper.connect=172.26.143.96:2181,172.26.143.96:2182,172.26.143.96:2183
sudo vim $KAFKA_CLUSTER_HOME/kafka-2/config/server.properties
broker.id=1
listeners=PLAINTEXT://172.26.143.96:9093
advertised.listeners=PLAINTEXT://172.26.143.96:9093
log.dirs=$KAFKA_CLUSTER_HOME/kafka-2/kafka-logs
zookeeper.connect=172.26.143.96:2181,172.26.143.96:2182,172.26.143.96:2183
sudo vim $KAFKA_CLUSTER_HOME/kafka-3/config/server.properties
broker.id=1
listeners=PLAINTEXT://172.26.143.96:9094
advertised.listeners=PLAINTEXT://172.26.143.96:9094
log.dirs=$KAFKA_CLUSTER_HOME/kafka-3/kafka-logs
zookeeper.connect=172.26.143.96:2181,172.26.143.96:2182,172.26.143.96:2183
2.2.5 启动集群
$KAFKA_CLUSTER_HOME/kafka-1/bin/kafka-server-start.sh $KAFKA_CLUSTER_HOME/kafka-1/config/server.properties
$KAFKA_CLUSTER_HOME/kafka-2/bin/kafka-server-start.sh $KAFKA_CLUSTER_HOME/kafka-2/config/server.properties
$KAFKA_CLUSTER_HOME/kafka-3/bin/kafka-server-start.sh $KAFKA_CLUSTER_HOME/kafka-3/config/server.properties
创建Topic并查看:
$KAFKA_CLUSTER_HOME/kafka-1/bin/kafka-topics.sh --create --bootstrap-server 172.26.143.96:9092 --topic test --partitions 10 --replication-factor 3
$KAFKA_CLUSTER_HOME/kafka-1/bin/kafka-topics.sh --bootstrap-server 172.26.143.96:9092 --describe --topic test
Topic: test TopicId: yhX-1uT2SAaYj-HrreKEPw PartitionCount: 10 ReplicationFactor: 3 Configs:
Topic: test Partition: 0 Leader: 0 Replicas: 0,2,1 Isr: 0,2,1
Topic: test Partition: 1 Leader: 2 Replicas: 2,1,0 Isr: 2,1,0
Topic: test Partition: 2 Leader: 1 Replicas: 1,0,2 Isr: 1,0,2
Topic: test Partition: 3 Leader: 0 Replicas: 0,1,2 Isr: 0,1,2
Topic: test Partition: 4 Leader: 2 Replicas: 2,0,1 Isr: 2,0,1
Topic: test Partition: 5 Leader: 1 Replicas: 1,2,0 Isr: 1,2,0
Topic: test Partition: 6 Leader: 0 Replicas: 0,2,1 Isr: 0,2,1
Topic: test Partition: 7 Leader: 2 Replicas: 2,1,0 Isr: 2,1,0
Topic: test Partition: 8 Leader: 1 Replicas: 1,0,2 Isr: 1,0,2
Topic: test Partition: 9 Leader: 0 Replicas: 0,1,2 Isr: 0,1,2
2.2.6 停止集群
$KAFKA_CLUSTER_HOME/kafka-1/bin/kafka-server-stop.sh
$KAFKA_CLUSTER_HOME/kafka-2/bin/kafka-server-stop.sh
$KAFKA_CLUSTER_HOME/kafka-3/bin/kafka-server-stop.sh
原文地址:https://blog.csdn.net/yunyun1886358/article/details/133829672
本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。
如若转载,请注明出处:http://www.7code.cn/show_2737.html
如若内容造成侵权/违法违规/事实不符,请联系代码007邮箱:suwngjj01@126.com进行投诉反馈,一经查实,立即删除!