Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,31 @@
# Changelog
All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).


## 0.2.0 -- Essential Bug Fixes, Compression

### Breaking Changes
- `Record::{key,value}` are now optional, following the underlying Kafka protocol (#93)
- compression support, `PartitionClient::produce` requires `compression` parameter (#82, #91, #92, #94)
- `PartitionClient::get_high_watermark` was replaced by `PartitionClient::get_offset` (#100)
- `StreamConsumer::new` `start_offset` parameter changed from `i64` to `StartOffset` type (#104)
- rework features (#107)

### Features
- record deletion (#97)

### Bug Fixes
- ignore `InvalidReplicationFactor` (#106)
- fix rare panic in `BatchProducer` (#105)
- filter out records that were not requested (#99)
- terminate consumer stream on `OffsetOutOfRange` (#96)

### Performance
- faster CRC calculation (#85)


## 0.1.0 -- Initial Release
This is the first release featuring:
Expand Down
20 changes: 18 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -53,16 +53,32 @@ tracing-subscriber = { version = "0.3", features = ["env-filter"] }
uuid = { version = "0.8", features = ["v4"] }

[features]
default = ["transport-tls"]
default = [
"compression-gzip",
"compression-lz4",
"compression-snappy",
"compression-zstd",
]

full = [
"compression-gzip",
"compression-lz4",
"compression-snappy",
"compression-zstd",
"transport-socks5",
"transport-tls",
]

compression-gzip = ["flate2"]
compression-lz4 = ["lz4"]
compression-snappy = ["snap"]
compression-zstd = ["zstd"]
fuzzing = []

transport-socks5 = ["async-socks5"]
transport-tls = ["rustls", "tokio-rustls"]

unstable-fuzzing = []

[[bench]]
name = "write_throughput"
harness = false
Expand Down
16 changes: 9 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,14 +90,16 @@ For more advanced production and consumption, see [`crate::client::producer`] an

## Features

- **`compression-gzip`:** Support compression and decompression of messages using [gzip].
- **`compression-lz4`:** Support compression and decompression of messages using [LZ4].
- **`compression-snappy`:** Support compression and decompression of messages using [Snappy].
- **`compression-zstd`:** Support compression and decompression of messages using [zstd].
- **`fuzzing`:** Exposes some internal data structures so that they can be used by our fuzzers. This is NOT a stable
feature / API!
- **`compression-gzip` (default):** Support compression and decompression of messages using [gzip].
- **`compression-lz4` (default):** Support compression and decompression of messages using [LZ4].
- **`compression-snappy` (default):** Support compression and decompression of messages using [Snappy].
- **`compression-zstd` (default):** Support compression and decompression of messages using [zstd].
- **`full`:** Includes all stable features (`compression-gzip`, `compression-lz4`, `compression-snappy`,
`compression-zstd`, `transport-socks5`, `transport-tls`).
- **`transport-socks5`:** Allow transport via SOCKS5 proxy.
- **`transport-tls` (default):** Allows TLS transport via [rustls].
- **`transport-tls`:** Allows TLS transport via [rustls].
- **`unstable-fuzzing`:** Exposes some internal data structures so that they can be used by our fuzzers. This is NOT a stable
feature / API!

## Testing

Expand Down
2 changes: 1 addition & 1 deletion fuzz/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ cargo-fuzz = true
[dependencies]
libfuzzer-sys = "0.4.0"
pin-project-lite = "0.2"
rskafka = { path = "..", features = ["fuzzing"] }
rskafka = { path = "..", features = ["unstable-fuzzing"] }
tokio = { version = "1.14", default-features = false, features = ["io-util", "rt"] }

[[bin]]
Expand Down
8 changes: 4 additions & 4 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@ mod backoff;
pub mod client;

mod connection;
#[cfg(feature = "fuzzing")]
#[cfg(feature = "unstable-fuzzing")]
pub mod messenger;
#[cfg(not(feature = "fuzzing"))]
#[cfg(not(feature = "unstable-fuzzing"))]
mod messenger;

#[cfg(feature = "fuzzing")]
#[cfg(feature = "unstable-fuzzing")]
pub mod protocol;
#[cfg(not(feature = "fuzzing"))]
#[cfg(not(feature = "unstable-fuzzing"))]
mod protocol;

pub mod record;
Expand Down
2 changes: 1 addition & 1 deletion src/messenger.rs
Original file line number Diff line number Diff line change
Expand Up @@ -258,7 +258,7 @@ where
}
}

#[cfg(feature = "fuzzing")]
#[cfg(feature = "unstable-fuzzing")]
pub fn override_version_ranges(&self, ranges: HashMap<ApiKey, ApiVersionRange>) {
self.set_version_ranges(ranges);
}
Expand Down