[otel-arrow-rust] Benchmarking test infrastructure

[daviddahl]

• Benchmarking test suite will help guage performance between PRs and commits in comparison to the Golang collector's performance

[lquerel]

@clhain is working on this issue

[cijothomas]

I’m planning to work on adding end-to-end (E2E) pipeline performance benchmarks to the repo. From what I understand, this issue is focused on microbenchmarks, while my work will target E2E pipeline-level benchmarking. I believe there shouldn’t be any conflicts or duplication, but I’ll start joining the SIG meetings to share my plans in more detail and gather feedback.

(I'll open a separate issue with some more details to avoid noise in this)

Edit: Link to new issue created : #377

[clhain]

Hi Cijo! I was working on porting approximately what's in https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/testbed to rust, with a few tweaks along the lines of what you laid out in #377 (mostly as a learning exercise, I'm completely new to rust - so totally fine if this is all throw away). I'll try to get a draft of what I have pushed up somewhere later today in case it's useful, but so far I only had basic functionality for:

• Run a monitored child process (e.g. collector) with static config, and track cpu / memory
• Generate traces and send to a otlp grpc endpoint (e.g. child process collector), using the rust sdk, at a specified rate

Love the vision you laid out in 377, very happy to contribute / collaborate on that!

[cijothomas]

@clhain Nice to meet you. Let's have a chat offline (could you ping me on slack. I couldn't find you there..), or I can join the next SIG call to discuss more!
I did start with the collector testbed, and felt it is better to have a more neutral perf test, so it is easy to use the same to test Arrow pipeline, OTLP pipeline, various Otel client SDKs etc.

I also figured it is much easier to use Python for scripting this up, instead of Rust (disclaimer - I am one of the OTel Rust maintainers, and have no Python experience before.). Eventually I hope that, we can make a pure config based customization of perf tests, but that's a long way out - so initially most users would need to do code based customizations. (and Rust is not so beginner friendly, as you may have noticed already.)

Love the vision you laid out in 377, very happy to contribute / collaborate on that!

Likewise! I’m really looking forward to collaborating on this.