-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Getting: In the middle of a leadership election, there is currently no leader for this partition and hence it is unavailable for writes. #272
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Can you try: |
Hi, @wvanbergen
code:
code execution:
etc. Thanks :) |
Please check that the broker is actually reachable from where you are running your go program (since it doesn't look like you're running it on the same machine) - could there be a firewall or routing problem in the way? I have a suspicion we are simply reporting the wrong error, and the right error is a lower-level networking problem. If that doesn't help, please set |
it's not a simple network issue: 192.168.1.76 == localhost, it's all on my laptop. Fedora 21 more or less stock. output of sarama debug:
And now it look like the client is doing some sort of host lookup to complete the connection: gclaptop is my machine name. Going to add that to hosts... And that solved the problem, thanks @eapache and @wvanbergen |
Kafka advertises itself by hostname through zookeeper, which can lead to some funny behaviours like this. Sarama can do a better job here by returning the right error in the first place though, I will try and fix that. |
Previously, if the cluster metadata was giving us back a broker which we suspected was unavailable (since it was already in our 'dead' set) then we would wait for the connection, and mark it as unavailable if the connection failed (otherwise, we simply do what the cluster tells us and let the producers/consumers deal with the connection errors). This was handy since it let us back off nicely if a broker crashed and came back, retrying metadata until the cluster had caught up and moved the leader to a broker that was up. I'm now of the opinion this was more trouble than it's worth, so scrap it. Among other things: - it does network IO while holding an important mutex, which is a bad pattern to begin with (#263) - it can mask real network errors behind "LeaderNotAvailable" (#272) The unfortunate side-effect of scrapping it is that in the producer and consumer we are more likely to fail if we don't wait long enough for the cluster to fail over leadership. The real solution if that occurs is to wait longer in the correct spot (`RetryBackoff` in the producer, currently hard-coded to 10 seconds in the consumer) instead of this hack.
Oh interesting, so kafka registers itself via hostname, makes some sort of sense. I wonder how the console commands work? In any case, thanks allot. |
The issue is that macOS laptops use a hostname that doesn't resolve by default, and the error message is very misleading (it doesn't contain the real error, which is |
So I am running into the same error.. Although it doesn't like my macbook trying to resolve it's own hostname is the culprit.. hrmm Sarama logs :
|
The logs seem to indicate that one (or more) of the partitions is leader-less. Sarama can only consume from and produce to leaders of partitions. Normally this situation is temporary because it's waiting for the cluster to elect a new leader, but that appears to be not happening. Sarama is handling this as designed: it will try a bunch of times but eventually give up. What does Kafka's |
I was able to resolve the problem by ensuring the kafka topic after bringing up a new cluster.. It seems that the following did the trick, obviously plug in your own values..
The script can be found from the Kafka downloads page This issue is now resolved for me, albeit looking forward to some better vernacular in the future regarding the other case. |
Has this problem some result? I encounter the same problem. |
1 similar comment
Has this problem some result? I encounter the same problem. |
Adds |
I think this may be a bug of Kafka. I got the same situation with @kris-nova. Kafka just fails to elect a new leader. So the connection of Sarama is disconnected. And this problems just happened suddenly. |
use 127.0.0.1 instead of localhost ,it work for me. |
Sometime may the kafka is elect leader, you should add retry times . }` |
Sha: 888a760
Kafka Version: kafka_2.9.2-0.8.2.0
I've created a kafka server with 3 nodes, completely based on the quickstart in the kafka documentation.
Everything works fine with the quickstart tutorial, I can create topics, list them, publish and consume. Including the topics listed in the below code. IE the CLI works for 'NewTopic' but the code errors out.
However when I attempt to run the producer / simple producer code locally I get this failure:
Code:
The text was updated successfully, but these errors were encountered: