developer tip

"삭제 표시"항목은 언제 / 어떻게 최종적으로 제거됩니까?

copycodes 2020. 11. 15. 11:25
반응형

"삭제 표시"항목은 언제 / 어떻게 최종적으로 제거됩니까?


주제 삭제 명령을 내 렸습니다.

./bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic  vip_ips_alerts

행복한 응답을주는 것 같았습니다.

[2014-05-31 20:58:10,112] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
Topic "vip_ips_alerts" queued for deletion.

그러나 이제 10 분 후에도 항목이 --list명령에 계속 나타납니다 .

./bin/kafka-topics.sh --zookeeper localhost:2181 --list
vip_ips_alerts - marked for deletion

그게 무슨 뜻일까요? 주제는 언제 정말로 삭제 되나요? 이 프로세스를 신속하게 처리하려면 어떻게해야합니까?


tl; dr 은 Kafka 중개인을 설정 delete.topic.enable = true하고 config/server.properties인내심을 갖습니다.

Kafka 0.8.3-SNAPSHOT 의 최신 개발 버전에서 발생합니다 .

➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗ ./bin/kafka-topics.sh --zookeeper localhost:2181 --create --topic my-topic --partitions 2 --replication-factor 1
Created topic "my-topic".

➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗ ./bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic my-topic
Topic:my-topic  PartitionCount:2    ReplicationFactor:1 Configs:
    Topic: my-topic Partition: 0    Leader: 0   Replicas: 0 Isr: 0
    Topic: my-topic Partition: 1    Leader: 0   Replicas: 0 Isr: 0

➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗ ./bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic my-topic
Topic my-topic is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.

➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗ ./bin/kafka-topics.sh --zookeeper localhost:2181 --list
➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗

포인트는 것입니다 delete.topic.enable=trueconfig/server.properties당신은 카프카의 브로커를 시작하는 데 사용하는.

➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗ grep delete.topic.enable config/server.properties
delete.topic.enable=true

브로커의 로그에서 설정이 true인지 확인할 수도 있습니다.

➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗ ./bin/kafka-server-start.sh config/server.properties
[2015-07-24 22:33:26,184] INFO KafkaConfig values:
        ...
        delete.topic.enable = true

Kafka 8.2.2를 사용하는 경우 다음 항목에서 수동으로 항목을 삭제해야했습니다.

  1. Kafka 브로커 머신에서 주제 폴더를 삭제합니다.
  2. 사육사에 로그인하고-

    hbase zkcli
    rmr /brokers/topics/{topic_name}
    rmr /admin/delete_topics/{topic_name}
    

넌 할 수있어.

sudo ./zookeeper-shell.sh localhost : 2181 rmr / brokers / topics / your_topic


나는 같은 문제에 직면했고 문제를 확인하기 위해 며칠을 보냈습니다. 주제를 삭제하라는 명령을 트리거했지만 주제가 삭제 표시되었지만 삭제되지 않았습니다.

  1. 먼저 올바르게 설정된 구성을 확인했습니다. 아래 : server.properties

    모든 브로커에 대해 delete.topic.enable = true

  2. 토픽이 삭제되었는지 확인하기 위해 브로커를 다시 시작했습니다 (No !!).
  3. / kafka / data 폴더에서 데이터를 확인하지 않습니다.
  4. 보존 시간이 초과 될 때까지 기다리는 옵션도 고려했습니다.

아무도 도움이되지 않았습니다. 마침내 zooker에 로그인해야했습니다

./zkCli.sh # and delete the topics using 
rmr /brokers/topics/<<topic>> and rmr /admin/delete_topics/<<topic>>

이 후에 kafka를 다시 시작하는 것을 잊지 마십시오. 이것이 문제를 해결하기를 바랍니다.


정답은 실제로 다음과 같습니다. Kafka 사용자의 그룹 이메일 배포 목록에서 HOT :

François Langelier *@gmail.com über kafka.apache.org 05:57 (vor 1 Stunde)

an users The delete topic isn't working ATM

I think it will be available in the next release https://issues.apache.org/jira/browse/KAFKA-1397


For kafka version 0.10.0.0 it is enough to enable the topic deletion by setting:

delete.topic.enable

The topic is deleted within few moments

kafka-topics --delete --zookeeper your-zk:2181 --topic yourTopicName

You can confirm that it is gone with the following command:

kafka-topics --describe --zookeeper your-zk:2181 --topic yourTopicName

My issue was something similar. I deleted a topic and it was giving me the same message when I was listing all the topics.

I brought down the zookeeper and the broker. set the delete.topic.enable=true in my broker config file started zookeeper and the boker

The topic was gone...thanks to Jacek Laskowski


Check status of topic at zookeeper

bin\windows>kafka-topics.bat --list --zookeeper localhost:2181
Output: topic shows marked for deletion

Setting delete.topic.enable=true in server.properties was not working as well.

Solution: Check the zookeeper data directory location in zookeeper.properties file. it was dataDir=/tmp/zookeeper.
Issue got resolved after updating "dataDir" to a new location.

dataDir=zk-temp
# the port at which the clients will connect
clientPort=2181
# disable the per-ip limit on the number of connections since this is a non-production config
maxClientCnxns=0

We had this issue when deleting topics. The topics had been created when delete.topic.enable=true had not been set. We set that in config, restarted kafka to apply the new config. Deleted the topics and saw the "marked for deletion". We then restarted kafka again. After 40 minutes though all topics had been deleted (9 topics with total partitions in the thousands). The topics with higher numbers of partitions seemed to take longer, which initially made it look like nothing was happening.


The deletion happened for me almost immediately (v 0.9). I believe that it should be the same for you.

Once marked, the deletion will be triggered on the kafka node which is the topic partition leader. The thing to remember is that the the topic partition leader must be configured correctly (in terms of listeners), otherwise it will not receive the instruction to delete the logs.

I had my "marked for deletion" topics stuck at that state until I corrected my server properties and restarted the respective node.


For kafka version 0.10.0.0 it is enough to enable the topic deletion by setting:

delete.topic.enable

The topic is deleted within few moments. No restart needed.

kafka-topics --delete --zookeeper your-zk:2181 --topic yourTopicName

You can confirm that it is gone with the command:

kafka-topics --describe --zookeeper your-zk:2181 --topic yourTopicName

참고URL : https://stackoverflow.com/questions/23976670/when-how-does-a-topic-marked-for-deletion-get-finally-removed

반응형