Skip to content

Commit 2911b2c

Browse files
constanca-mconstanca-m
authored
[extension/awslogsencodingextension] Fix semantic conventions and update README (#39207)
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue. Ex. Adding a feature - Explain what this achieves.--> #### Description Fix semantic conventions and update README. <!-- Issue number (e.g. #1234) or full URL to issue, if applicable. --> #### Link to tracking issue Fixes #39051. <!--Describe the documentation added.--> #### Documentation README updated.
1 parent a160355 commit 2911b2c

File tree

4 files changed

+92
-2
lines changed

4 files changed

+92
-2
lines changed

‎.chloggen/vpc-fix-conventions.yaml

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Use this changelog template to create an entry for release notes.
2+
3+
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
4+
change_type: enhancement
5+
6+
# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
7+
component: awslogsencodingextension
8+
9+
# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
10+
note: Rename aws.eni.id attribute to network.interface.name and update README
11+
12+
# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
13+
issues: [39051]
14+
15+
# (Optional) One or more lines of additional information to render under the primary note.
16+
# These lines will be padded with 2 spaces and then inserted directly into the document.
17+
# Use pipe (|) for multiline entries.
18+
subtext:
19+
20+
# If your change doesn't affect end users or the exported elements of any package,
21+
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
22+
# Optional: The change log or logs in which this entry should be included.
23+
# e.g. '[user]' or '[user, api]'
24+
# Include 'user' if the change is relevant to end users.
25+
# Include 'api' if there is a change to a library API.
26+
# Default: '[user]'
27+
change_logs: []

‎extension/encoding/awslogsencodingextension/README.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@
1313

1414
This extension unmarshals logs encoded in formats produced by AWS services, including:
1515
- [Amazon CloudWatch Logs Subscription Filters](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html).
16+
- [VPC flow log records](https://docs.aws.amazon.com/vpc/latest/userguide/flow-log-records.html) sent to S3 in plain text.
17+
- Parquet support still to be added.
1618
- (More to be added later.)
1719

1820
Example for Amazon CloudWatch Logs Subscription Filters:
@@ -26,3 +28,63 @@ receivers:
2628
endpoint: :1234
2729
encoding: awslogs_encoding/cloudwatch
2830
```
31+
32+
Example for VPC flow logs:
33+
```yaml
34+
extensions:
35+
awslogs_encoding/cloudwatch:
36+
format: vpc_flow_log
37+
vpc_flow_log:
38+
# options [parquet, plain-text].
39+
# parquet option still needs to be implemented.
40+
file_format: plain-text
41+
```
42+
43+
#### VPC flow log record fields
44+
45+
[VPC flow log record fields](https://docs.aws.amazon.com/vpc/latest/userguide/flow-log-records.html#flow-logs-fields) are mapped this way in the resulting OpenTelemetry log:
46+
47+
| Flow log field | Attribute in OpenTelemetry log | Available? |
48+
|------------------------------|-------------------------------------------------------------------------------------------------------|:------------:|
49+
| `version` | `aws.vpc.flow.log.version` | 🟢 |
50+
| `account-id` | `cloud.account.id` | 🟢 |
51+
| `interface-id` | `network.interface.name` | 🟢 |
52+
| `srcaddr` | `source.address`: if `pkt-srcaddr` not filled or the same <br> `network.peer.address`: otherwise | 🟢 |
53+
| `pkt-srcaddr` | `source.address` if filled | 🟢 |
54+
| `dstaddr` | `destination.address`: if `pkt-dstaddr` not filled or the same <br> `network.peer.address`: otherwise | 🟢 |
55+
| `pkt-dstaddr` | `destination.address` if filled | 🟢 |
56+
| `srcport` | `source.port` | 🟢 |
57+
| `dstport` | `destination.port` | 🟢 |
58+
| `protocol` | `network.protocol.name` | 🟢 |
59+
| `packets` | `aws.vpc.flow.packets` | 🟢 |
60+
| `bytes` | `aws.vpc.flow.bytes` | 🟢 |
61+
| `start` | `aws.vpc.flow.start` | 🟢 |
62+
| `end` | Log timestamp | 🟢 |
63+
| `action` | `aws.vpc.flow.action` | 🟢 |
64+
| `log-status` | `aws.vpc.flow.status` | 🟢 |
65+
| `vpc-id` | `aws.vpc.id` | 🟢 |
66+
| `subnet-id` | `aws.vpc.subnet.id` | 🟢 |
67+
| `instance-id` | `host.id` | 🟢 |
68+
| `tcp-flags` | `network.tcp.flags` | 🟢 |
69+
| `type` | `network.type` | 🟢 |
70+
| `region` | `cloud.region` | 🟢 |
71+
| `az-id` | `aws.az.id` | 🟢 |
72+
| `sublocation-type` | `aws.sublocation.type` | 🟢 |
73+
| `sublocation-id` | `aws.sublocation.id` | 🟢 |
74+
| `pkt-src-aws-service` | `aws.vpc.flow.source.service` | 🟢 |
75+
| `pkt-dst-aws-service` | `aws.vpc.flow.destination.service` | 🟢 |
76+
| `flow-direction` | `network.io.direction` | 🟢 |
77+
| `traffic-path` | `aws.vpc.flow.traffic_path` | 🟢 |
78+
| `ecs-cluster-arn` | `aws.ecs.cluster.arn` | 🔴 |
79+
| `ecs-cluster-name` | `aws.ecs.cluster.name` | 🔴 |
80+
| `ecs-container-instance-arn` | `aws.ecs.container.instance.arn` | 🔴 |
81+
| `ecs-container-instance-id` | `aws.ecs.container.instance.id` | 🔴 |
82+
| `ecs-container-id` | `aws.ecs.container.id` | 🔴 |
83+
| `ecs-second-container-id` | `aws.ecs.second.container.arn` | 🔴 |
84+
| `ecs-service-name` | `aws.ecs.service.name` | 🔴 |
85+
| `ecs-task-definition-arn` | `aws.ecs.task.definition.arn` | 🔴 |
86+
| `ecs-task-arn` | `aws.ecs.task.arn` | 🔴 |
87+
| `ecs-task-id` | `aws.ecs.task.id` | 🔴 |
88+
| `reject-reason` | `aws.vpc.flow.reject_reason` | 🟢 |
89+
90+

‎extension/encoding/awslogsencodingextension/internal/unmarshaler/vpc-flow-log/testdata/valid_vpc_flow_log_expected.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ resourceLogs:
1313
- key: aws.vpc.flow.log.version
1414
value:
1515
intValue: "2"
16-
- key: aws.eni.id
16+
- key: network.interface.name
1717
value:
1818
stringValue: eni-0eb1e4178af74336c
1919
- key: aws.vpc.flow.start

‎extension/encoding/awslogsencodingextension/internal/unmarshaler/vpc-flow-log/vpc_flow_log_unmarshaler.go

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -318,7 +318,8 @@ func handleField(
318318
case "az-id":
319319
record.Attributes().PutStr("aws.az.id", value)
320320
case "interface-id":
321-
record.Attributes().PutStr("aws.eni.id", value)
321+
// TODO Replace with conventions variable once it becomes available
322+
record.Attributes().PutStr("network.interface.name", value)
322323
case "srcport":
323324
if err := addNumber(field, value, conventions.AttributeSourcePort); err != nil {
324325
return false, err

0 commit comments

Comments
 (0)