[stefexporter] Improve STEF exporter and receiver docs (#40408)

Added more details to exporter documentation, included benchmarks, a
disclaimer of instability and explained interoperability expectations.

Author: tigrannajaryan

exporter/stefexporter/README.md

@@ -13,16 +13,67 @@
 [contrib]: https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-contrib
 <!-- end autogenerated section -->
 
-Export data via gRPC using
-[Otel/STEF format](https://github.com/splunk/stef/tree/main/go/otel) format.
+## Overview
 
-## Getting Started
+Exports data via gRPC using
+[Otel/STEF format](https://github.com/splunk/stef/tree/main/go/otel).
+Otel/STEF is a compact and fast telemetry format. It is currently the fastest
+benchmarked metric format in the Collector. Here are recent 
+[benchmarking results](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/15381307862/job/43272589155),
+comparing CPU usage when using a few different formats (lower is better):
+
+| Test                    | CPU Avg% | CPU Max% |
+|-------------------------|---------:|---------:|
+| Metric10kDPS/Carbon     |    174.9 |    179.3 |
+| Metric10kDPS/OpenCensus |     59.2 |     61.3 |
+| Metric10kDPS/OTLP       |     44.8 |     45.7 |
+| Metric10kDPS/OTLP-HTTP  |     33.5 |     34.0 |
+| Metric10kDPS/SignalFx   |     78.9 |     88.0 |
+| Metric10kDPS/STEF       |     20.7 |     21.7 |
+
+STEF in this benchmark outperforms all other formats, including OTLP, which was previously
+known as the fastest format in the Collector.
+
+STEF is also very compact on the wire. In uncompressed mode, it typically yields payloads 
+that  are more than 10x smaller than OTLP payloads. In compressed mode, STEF is
+typically 5-7 times more compact than OTLP.
+
+Here are some 
+[STEF benchmark results](https://splunk.github.io/stef/benchmarks/results/benchmarks.html),
+comparing STEF and OTLP payload sizes for a few sample payloads produced by Collector:
+
+<img src="images/astronomymetricssize.png">
+
+<img src="images/astronomymetricssize.png">
+
+(This exporter implementation uses unsorted STEF format, labeled "STEF Unsorted" in 
+charts above).
+
+There are currently no known formats that match STEF's compactness and performance.
+
+STEF exporter can be used to send metrics from the Collector to any STEF-compatible 
+backend. STEF can also be used to send metric data between Collector instances, in which
+case on the receiving side a [STEF receiver](../../receiver/stefreceiver/README.md)
+should be used.
+
+STEF format, STEF exporter and receiver implementations are currently in alpha 
+stage of development, during which **the format may undergo breaking changes.** 
+To ensure interoperability between sending and receiving Collectors make sure 
+you are using versions of exporter and receiver that are compiled with the same 
+version of STEF library.
+
+Feedback about STEF format and implementation is welcome in the form of issues
+in this repository or in [STEF repository](https://github.com/splunk/stef/issues).
+
+## Configuration
 
 The following settings are required:
 
 - `endpoint` (no default): host:port to which the exporter is going to send STEF metric data,
   using the STEF/gRPC protocol. The valid syntax is described
-  [here](https://github.com/grpc/grpc/blob/master/doc/naming.md).
+  [here](https://github.com/grpc/grpc/blob/master/doc/naming.md). When sending to 
+  another Collector you will typically use `<hostname>:4320` as the endpoint, since 
+  port 4320 is the default port used by STEF receiver.
   If a scheme of `https` is used then client transport security is enabled and overrides the `insecure` setting.
 - `tls`: see [TLS Configuration Settings](https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/configtls/README.md)
   for the full set of available options.
@@ -43,7 +94,7 @@ exporters:
 ```
 
 By default, no compression is enabled. The only supported compression method is zstd.
-To enable, configure as follows:
+To enable compression, configure as follows:
 
 ```yaml
 exporters:
@@ -52,15 +103,15 @@ exporters:
     compression: zstd
 ```
 
-## Advanced Configuration
+## Advanced Settings
 
-Several helper files are leveraged to provide additional capabilities automatically:
+STEF exporter supports the following advanced settings:
 
 - [gRPC settings](https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/configgrpc/README.md)
 - [TLS and mTLS settings](https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/configtls/README.md)
 - Queuing, timeout and retry settings, particularly:
   - The `timeout` setting controls how long the exporter waits for ACK of a data sent
-    over STEF/gRPC stream.
+    over STEF/gRPC stream. Increase this value if you see timeouts in the logs.
   - The `num_consumers` setting defines how many unacknowledged batches can be in-flight.
     If the destination is slow to acknowledge then increasing this
     number can help increase the throughput.

exporter/stefexporter/images/astronomymetricssize.png

@@ -13,14 +13,16 @@
 [contrib]: https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-contrib
 <!-- end autogenerated section -->
 
-The STEF receiver runs a gRPC server accepting data using the Sequential Tabular Encoding Format (STEF).
+Receives data via gRPC in Otel/STEF format. Otel/STEF is a compact and
+fast telemetry format. For more about Otel/STEF, see
+[STEF exporter docs](../../exporter/stefexporter/README.md).
 
 ## Configuration
 
 - `endpoint` (default = localhost:4320 for grpc protocol):
   host:port on which the receiver is going to receive data. Refer to
   [naming documentation](https://github.com/grpc/grpc/blob/master/doc/naming.md)
-  for valid syntax.
+  for valid syntax for host part.
 - `ack_interval` (default: 10ms). The periodical interval of time when sending acknowledgements back to the client.
 
 Example:
@@ -30,7 +32,7 @@ stef:
   endpoint: 0.0.0.0:4320
 ```
 
-Several common configuration structures provide additional capabilities automatically:
+STEF receiver supports the following advanced settings:
 
 - [gRPC settings](https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/configgrpc/README.md)
 - [TLS and mTLS settings](https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/configtls/README.md)