kafka 删除 topic 及数据

删除topic后，在重新创建相同名字的topic，报错了。下面总结一套，可行的方案。

一，创建报错如下

[root@namenode1 kafka]# bin/kafka-topics.sh --create --zookeeper namenode1:2181,namenode2:2181,datanode1:2181 --partitions 3 --topic track_pc

WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.

Error while executing topic command : Topic 'track_pc' already exists.

二，解决方案

1，停止所有的producer和consumer，不要停止kafka进程，停止kafka后，是创建不了topic，会报以下错误

[root@namenode1 kafka]# bin/kafka-topics.sh --create --zookeeper namenode1:2181,namenode2:2181,datanode1:2181 --replication-factor 2 --partitions 3 --topic track_pc

WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.

Error while executing topic command : Replication factor: 2 larger than available brokers: 0.

[2019-04-30 17:27:52,807] ERROR org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 2 larger than available brokers: 0.

(kafka.admin.TopicCommand$)

2，修改server.properties，添加以下内容

auto.create.topics.enable=false
delete.topic.enable=true

auto.create.topics.enable默认是true的，如果在没有创建topic的前提下，启动producer，会自动创建topic

delete.topic.enable默认是false的，如果不设置成true，只能进行逻辑删除，并不能真正的删除

[root@namenode1 kafka]# bin/kafka-run-class.sh kafka.admin.TopicCommand --delete --topic track_pc --zookeeper namenode1:2181,namenode2:2181,datanode1:2181

Topic track_pc is already marked for deletion.

3，删除topic及数据

[root@namenode1 kafka]# bin/kafka-run-class.sh --delete --topic track_pc --zookeeper namenode1:2181,namenode2:2181,datanode1:2181

删除kafka存储目录（server.properties文件log.dirs配置）相关topic的数据目录。如果有多个分区，要到kafka群里的每台机器上，删除相关topic的数据目录。

到这儿一个topic就完全删除了，通过以下命令，查看一下topic列表

# bin/kafka-topics.sh --list --zookeeper namenode1:2181,namenode2:2181,datanode1:2181

如果通过以上操作，还是不能完全删除，可以进行以下操作

4，登录zookeeper，删除相应的topic

# sh zkCli.sh -server namenode1:2181,namenode2:2181,datanode1:2181

[zk: namenode1:2181,namenode2:2181,datanode1:2181(CONNECTED) 5] ls /brokers/topics
[track_app, track_toutiao_click, track_toutiao_show, __consumer_offsets, track_pc, track_wap]

[zk: namenode1:2181,namenode2:2181,datanode1:2181(CONNECTED) 6] rmr /brokers/topics/track_pc

[zk: namenode1:2181,namenode2:2181,datanode1:2181(CONNECTED) 7] ls /brokers/topics
[track_app, track_toutiao_click, track_toutiao_show, __consumer_offsets, track_wap]

[zk: namenode1:2181,namenode2:2181,datanode1:2181(CONNECTED) 4] ls /admin/delete_topics
[]

[zk: namenode1:2181,namenode2:2181,datanode1:2181(CONNECTED) 8] ls /config/topics
[track_app, track_toutiao_click, track_toutiao_show, __consumer_offsets, track_pc, track_wap]

删除这/admin/delete_topics，/config/topics，/brokers/topics三个目录下的相关的topic