Redpanda compare to Kafka

[evanzhang87]

Which version of Kafka can I use to compare with Redpanda ?
I use a low version (0.9.0.1) kafka, use kafka-producer-perf-test.sh to do some performance tests , but it didn't work.

[BenPope]

Try with tools from Kafka 2.4 and let us know how it goes.

Also, make sure you try it with the latest Redpanda.

[evanzhang87]

Wow, it works well! I'm going to do some tests these days and paste the results here

[evanzhang87]

Redpanda machine : 3 brokers

Kafka machine : 3 brokers

	CPU&RAM	SYSTEM	Filesystem
Redpanda	4C8G	CentOS7.6（VM）	xfs
Kafka	4C8G	CentOS7.6（VM）	xfs
Shell test	4C8G	CentOS7.6（VM）	xfs

disks:
  - mountpoint: /var/lib/redpanda/data
    read_iops: 2633
    read_bandwidth: 70204744
    write_iops: 5356
    write_bandwidth: 972321792

Redpanda : 10000000/40000 = 250sec ~= 4min

10000000 records sent, 39997.600144 records/sec (38.14 MB/sec), 13.49 ms avg latency, 1521.00 ms max latency, 2 ms 50th, 4 ms 95th, 597 ms 99th, 1119 ms 99.9th.

Kafka (ACK=1): 10000000/40000 = 250sec ~= 4min

10000000 records sent, 39993.601024 records/sec (38.14 MB/sec), 43.77 ms avg latency, 1862.00 ms max latency, 20 ms 50th, 97 ms 95th, 549 ms 99th, 1757 ms 99.9th.

Redpanda io monitor

Kafka io monitor

[emaxerrno]

What filesystem. What acks on the client settings. What’s the iops on the disk. Hard to tell from these numbers about the setup. How long did you run the bench. Are they VMs. Etc. Need more details

[emaxerrno]

this is what we used: https://github.com/vectorizedio/openmessaging-benchmark which is a fork of confluent's bench.

but the bash script you mentioned worked. there are a LOT of settings (sadly) to the kafka client.

so the benchmark is a stable code for you to test if you want apples to apples

[emaxerrno]

you got it! either this repository or the blog post are good docs :) feel free to join the slack https://vectorized.io/slack

[evanzhang87]

Wow,when I tried openmessaging-benchmark ,some problems appeared.

And the test results are unsatisfactory.

Is there anything wrong?

It's my test case

name: Redpanda
driverClass: io.openmessaging.benchmark.driver.redpanda.RedpandaBenchmarkDriver

# Kafka client-specific configuration
replicationFactor: 3
reset: true

topicConfig: |

commonConfig: |
  bootstrap.servers=10.2.27.178:9092,10.2.27.178:9093,10.2.27.178:9094

producerConfig: |
  acks=1
  linger.ms=1
  batch.size=131072

consumerConfig: |
  auto.offset.reset=earliest
  enable.auto.commit=false

name: Max rate 1 producer on 1 topic / 1 partition

topics: 1
partitionsPerTopic: 16
messageSize: 1024
payloadFile: "payload/payload-1Kb.data"
subscriptionsPerTopic: 1
consumerPerSubscription: 1
producersPerTopic: 1

# Discover max-sustainable rate
producerRate: 100000

consumerBacklogSizeGB: 0
testDurationMinutes: 5

[emaxerrno]

what's the hardware, how are you running it, etc. need more details, what environment, what memory, how many cores, ... which benchmark did you run.

Also, if you are using cloud instances, make sure you run them for 1.5hrs. the first 30mins are not predictive to performance due to cloud quotas, so you need to go beyond the cloud quotas to understand steady state.

[evanzhang87]

Redpanda is running on a centos7.6 machine with 8G and 4 cores, I started it with docker ,and --memory 1G ,I ran 1-topic-16-partions-1kb benchmark on a ubuntu18.04 machine, it's my result of ping redpanda machine. Well I think I find the reason, is it caused by my network?

[emaxerrno]

so --memory 1G and 1 core doesn't make sense for benchmarking

use this: https://github.com/vectorizedio/deployment-automation and then benchmark.