Serilog for Dotnet Aspire

[BartNetJS]

I'm exploring the new Dotnet 8 Aspire dashboard.
Out of the box it is providing structured logs (from opentelemetry) and tracing as shown in the print screen:

As soon i add Serilog Use like this

builder.Host.UseSerilog();

I loose the structured logs.
For example here i added the UseSerilog in the web frontend:

The serilog instrumentation is like this:

Log.Logger = new LoggerConfiguration() .WriteTo.Console() .Enrich.FromLogContext() .Enrich.With(new TraceIdEnricher()) .CreateLogger();

How can i use Serilog and Aspire together?

[davidfowl]

Did you configure Serilog to write to dashboard? (the configured OTLP server?). See this doc. It explains how the system currently works. To make serilog work with aspire, you need to tell to send serilog logs to the otlp endpoint.

dotnet/aspire#1872

[BartNetJS]

indeed, i forgot to configure the OTLP exporter.
I ended with this code and it started to work fine:

var useOtlpExporter = !string.IsNullOrWhiteSpace(builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]);
var logBuilder = new LoggerConfiguration()
 .Enrich.FromLogContext()
 .WriteTo.Console();

if (useOtlpExporter)
{
    logBuilder
       .WriteTo.OpenTelemetry(options =>
     {
         options.Endpoint = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"];
         options.ResourceAttributes.Add("service.name", "apiservice");
     });
}

Log.Logger = logBuilder.CreateBootstrapLogger();

builder.Logging.AddSerilog();

[davidfowl]

I filed dotnet/aspire-samples#106

[Sen-Gupta]

Hello @BartNetJS

We attempted adding Serilog as well.
I Need a confirmation!
Are you seeing two entries? As shown below,

@davidfowl is this expected?

Created another static method in ServiceDefaults

public static void AddCustomSerilog(this IHostApplicationBuilder builder)
{
var useOtlpExporter = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"];
var appName = builder.Configuration["AppMeta:AppName"];

if (string.IsNullOrEmpty(appName))
{
    throw new ArgumentException("Must set AppMeta.AppName for serilog to work correctly in the configuration");
}

var seqServerUrl = builder.Configuration["SeqServerUrl"]!;

var logBuilder = new LoggerConfiguration()
    .MinimumLevel.Debug()
    .ReadFrom.Configuration(builder.Configuration)
    .Enrich.FromLogContext()
    .WriteTo.Console()
    .WriteTo.Seq(seqServerUrl)
    .Enrich.FromLogContext()
    .Enrich.WithMachineName()
    .Enrich.WithExceptionDetails();

if(!string.IsNullOrEmpty(useOtlpExporter))
{
    logBuilder
       .WriteTo.OpenTelemetry(options =>
       {
           options.Endpoint = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]!;
           options.ResourceAttributes.Add("service.name", appName);
       });
}
Log.Logger = logBuilder.CreateBootstrapLogger();
builder.Logging.AddSerilog();

}

Note: Ignore the Seq bits!

The App Name is configured in settings as shown below
"AppMeta": {
"AppMode": "Server",
"Env": "qa",
"StateStore": "aarya-statestore",
"AppName": "agents",
"ProductName": "Insights"
}

and finally the projects are added in the AppHost as shown below:

var builder = DistributedApplication.CreateBuilder(args);

builder.AddProject<Projects.Insights_Grpcs_Agents>("agentsgrpc")
.WithDaprSidecar("agents");

builder.AddProject<Projects.Insights_Grpcs_Analytics>("analyticsgrpc")
.WithDaprSidecar("analytics");

builder.AddProject<Projects.Insights_Worker_Collector>("collectorgrpc")
.WithDaprSidecar("collector");

builder.Build().Run();

as we see AppName:"agents" and sidecare is registered with "agents".

But seems, telemetry is identifying them as two different sources and appending random strings to ensure listing of the sources.

Any clues?

[davidfowl]

Did you remove the other logger provider that sends telemetry directly to the dashboard or are you logging with both serilog and the default setup?

[Sen-Gupta]

@davidfowl @TGrannen,

The only extra thing needed was to clear the providers, it was already there in our production code but I missed during customiztation of the code snippet.

public static void AddCustomSerilog(this IHostApplicationBuilder builder)
{
var useOtlpExporter = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"];
var appName = builder.Configuration["AppMeta:AppName"];

if (string.IsNullOrEmpty(appName))
{
throw new ArgumentException("Must set AppMeta.AppName for serilog to work correctly in the configuration");
}

var seqServerUrl = builder.Configuration["SeqServerUrl"]!;

var logBuilder = new LoggerConfiguration()
.MinimumLevel.Debug()
.ReadFrom.Configuration(builder.Configuration)
.Enrich.FromLogContext()
.WriteTo.Console()
.WriteTo.Seq(seqServerUrl)
.Enrich.FromLogContext()
.Enrich.WithMachineName()
.Enrich.WithExceptionDetails();

if(!string.IsNullOrEmpty(useOtlpExporter))
{
logBuilder
.WriteTo.OpenTelemetry(options =>
{
options.Endpoint = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]!;
options.ResourceAttributes.Add("service.name", appName);
});
}
Log.Logger = logBuilder.CreateBootstrapLogger();
builder.Logging.ClearProviders().AddSerilog();
}

Seems like @TGrannen has beaten me for the sample. ;-)

[alex-todorov-j2]

This used to work - until I updated to latest VS Preview. Now only the Console and File sinks create Serilog logs but I get nothing in Structured in Dashboard. I confirmed that OTEL_EXPORTER_OTLP_ENDPOINT is correct. And if I call ClearProviders before AddSerilog, I get no structured logs in any sink. Any ideas?

[davidfowl]

https://learn.microsoft.com/en-us/dotnet/aspire/whats-new/preview-5?tabs=dotnet-cli#security-updates

[alex-todorov-j2]

I added in appsettings.json:

"Dashboard": {
  "Otlp": {
    "AuthMode": "ApiKey",
    "ApiKey": "a8e18bea-609d-4761-aa6b-40858d7c057b"
  }    
}

And now the code sends the configured api key to the OTLP collector endpoint in the extension method:

public static IHostApplicationBuilder AddOtelSerilog(this IHostApplicationBuilder builder, IConfiguration config)
{
    ...
    logBuilder.WriteTo.OpenTelemetry(options =>
    {
        options.Endpoint = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]; 
        options.Headers.Add("x-otlp-api-key", otlpApiKey);
        options.ResourceAttributes.Add("service.name", "apiservice");                
    });
    ....
}

But still I don't see the logs in Structured in Dashboard.

What else has to change?

[alex-todorov-j2]

Any update? Should I open an issue for this?

[davidfowl]

Easiest way to make this would is to read the official environment variables and passing them to serilog (https://opentelemetry.io/docs/languages/sdk-configuration/otlp-exporter/#otel_exporter_otlp_headers)

OTEL_EXPORTER_OTLP_ENDPOINT
OTEL_EXPORTER_OTLP_HEADERS
OTEL_EXPORTER_OTLP_PROTOCOL

logBuilder.WriteTo.OpenTelemetry(options =>
{
    options.Endpoint = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]; 
    var headers = builder.Configuration["OTEL_EXPORTER_OTLP_HEADERS"]?.Split(',') ?? [];
    foreach (var header in headers)
    {
        var (key, value) = header.Split('=') switch
        {
            [string k, string v] => (k, v),
            var v => throw new Exception($"Invalid header format {v}")
        };
    
        options.Headers.Add(key, value);
    }
    options.ResourceAttributes.Add("service.name", "apiservice");                
});

PS: It would be good to file an issue on serilg to read the default OTLP environment variables.