Skip to content

feat: upgrade logger #1265

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: feat/upgrade-logger
Choose a base branch
from

Conversation

tckeong
Copy link

@tckeong tckeong commented Jun 20, 2025

This PR upgrade the original logger, add a new logger backend (logger driver). The new logger backend, CloudWatch Logger, enabling buffered delivery of your application logs to AWS CloudWatch using ExAws.


How It Works

  1. Initialization
    On startup, this backend reads configuration options (:log_group, :log_stream, :level, :format, :metadata) to set up its internal state.

  2. Buffering Logs
    When the application emits a log, the handle_event/2 callback formats it, attaches metadata, timestamps it, and buffers it.

  3. Flush Trigger
    Logs are flushed in two cases:

    • The buffer reaches the configured @max_buffer_size (immediately triggers async flush).
    • A periodic timer (@flush_interval) sends a :flush_buffer message, flushing any pending logs.
  4. AWS Auth Check
    Before sending, the logger verifies that AWS credentials and region are configured via environment variables. If missing, the flush is skipped (avoiding endless retry loops).

  5. Retry Logic
    Logs are sent using ExAws.request/1, in %ExAws.Operation.JSON{} format. On failure, it retries up to @max_retries, with a delay of @retry_delay between attempts. An error is logged on each failure.


Configuration Options

Add to your config/*.exs:

config :logger,
  backends: [:console, {Cadet.Logger.CloudWatchLogger, :cloudwatch_logger}]

config :logger, :cloudwatch_logger,
  level: :info,
  format: "$time $metadata[$level] $message\n",
  metadata: [:request_id],
  log_group: "cadet-logs",
  log_stream: "#{node()}-#{:os.system_time(:second)}",
  • :log_group / :log_stream - CloudWatch Logs group and stream names.
  • :level, :format, :metadata – Same options as Logger.Formatter.

This PR integrates with Elixir’s Logger (:gen_event), offering buffered, retry-enabled delivery to AWS CloudWatch.

@tckeong tckeong self-assigned this Jun 20, 2025
@tckeong tckeong requested review from Copilot and RichDom2185 and removed request for Copilot June 20, 2025 10:05
Copilot

This comment was marked as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant