Get Startededit

Step 1: Configure application loggingedit

The following logging frameworks are supported:

Serilog
NLog

Add the dependencyedit

The following target frameworks are supported

Name	Target Framework Moniker (TFM)
.NET Framework 4.6.1	`net461`
.NET Standard 2.0	`netstandard2.0`
.NET Standard 2.1	`netstandard2.1`

The minimum required Serilog version is 2.9.0.

Add a dependency to your application project file:

<Project Sdk="Microsoft.NET.Sdk">
    <!-- other details excluded for brevity... -->
    <ItemGroup>
        <PackageReference Include="Elastic.CommonSchema.Serilog" Version="${ecs-logging-dotnet.version}" />
    </ItemGroup>
</Project>

or install with the .NET CLI

dotnet add package Elastic.CommonSchema.Serilog --version ${ecs-logging-dotnet.version}

or nuget CLI

Install-Package Elastic.CommonSchema.Serilog -Version ${ecs-logging-dotnet.version}

The following target frameworks are supported

Name	Target Framework Moniker (TFM)
.NET Framework 4.6.1	`net461`
.NET Standard 2.0	`netstandard2.0`
.NET Standard 2.1	`netstandard2.1`

The minimum required NLog version is 4.5.4.

Add a dependency to your application project file:

<Project Sdk="Microsoft.NET.Sdk">
    <!-- other details excluded for brevity... -->
    <ItemGroup>
        <PackageReference Include="Elastic.CommonSchema.NLog" Version="${ecs-logging-dotnet.version}" />
    </ItemGroup>
</Project>

or install with the .NET CLI

dotnet add package Elastic.CommonSchema.NLog --version ${ecs-logging-dotnet.version}

or nuget CLI

Install-Package Elastic.CommonSchema.NLog -Version ${ecs-logging-dotnet.version}

Use the ECS integrationedit

Serilog Text Formatter

EcsTextFormatter is an ITextFormatter implementation in Elastic.CommonSchema.Serilog that formats Serilog events into a JSON representation that adheres to the Elastic Common Schema specification.

It can be configured in conjunction with a Serilog Sink. To configure with the file sink for example, first install Serilog.Sinks.File nuget package

Install-Package Serilog.Sinks.File

Then configure the file sink to use EcsTextFormatter:

var logger = new LoggerConfiguration()
    .WriteTo.File(new EcsTextFormatter(), "/path/to/log.txt")
    .CreateLogger();

Now, when logging events

logger.Information("Checkout contains {ItemCount} items", 12);

log a message with the information level

Each Serilog log event will be formatted as single line JSON in the file /path/to/log.txt.

NLog Layout

EcsLayout is a Layout implementation in Elastic.CommonSchema.NLog that formats an NLog event into a JSON representation that adheres to the Elastic Common Schema specification.

It can be configured as the layout of an NLog Target. To configure with the file target for example, using code configuration:

var config = new LoggingConfiguration();
var fileTarget = new FileTarget()
{
    Layout = new EcsLayout(), 
    FileName = "/path/to/log.txt"
};
config.AddRule(LogLevel.Info, LogLevel.Fatal, fileTarget);
LogManager.Configuration = config;
var logger = LogManager.GetCurrentClassLogger();

Create a new instance of ECS layout

In addition to code configuration, EcsLayout can be configured using XML configuration in NLog.config:

<nlog>
  <extensions>
    <add assembly="Elastic.CommonSchema.NLog"/>
  </extensions>
  <targets>
    <target name="file" type="File">
      <layout xsi:type="EcsLayout">
      </layout>
    </target>
  </targets>
  <rules>
    <logger name="*" minLevel="Info" writeTo="File" />
  </rules>
</nlog>

Now, when logging events:

logger.Info("Checkout contains {ItemCount} items", 12);

log a message with the information level

Each NLog log event will be formatted as single line JSON in the file /path/to/log.txt.

EcsLayout Parameter Options

The following properties determine which properties should be included or excluded as metadata of an event

Parameter name	Type	Default	Description
`IncludeAllProperties`	bool	`true`	Include LogEvent properties as metadata
`IncludeMdlc`	bool	`false`	Include NLog Scope Context Properties as metadata
`ExcludeProperties`	string		Comma separated string of properties to exclude from metadata

The following are NLog Layout properties used to capture information for an event

Parameter name	Type	Default	Description
`EventAction`	string		The action captured by the event
`EventCategory`	string		The category of the event.
`EventId`	string		The unique identifier for the event
`EventKind`	string		The kind of event. High level information about what kind of information the event contains
`EventSeverity`	long	NLog level * Trace, Debug:: 7 * Info:: 6 * Warn:: 4 * Error:: 3 * Fatal:: 2	The severity of the event
`AgentId`	string		The id of the agent collecting events
`AgentName`	string		The name of the agent collecting events
`AgentType`	string		The type of the agent collecting events
`AgentVersion`	string		The version of the agent collecting events
`ProcessExecutable`	string	`${processname:FullName=true}`	Absolute path to the process executable
`ProcessId`	long	`${processid}`	The process id (pid)
`ProcessName`	string	`${processname:FullName=false}`	The name of the process
`ProcessThreadId`	long	`${threadid}`	The id of the process thread
`ProcessTitle`	string	`${processinfo:MainWindowTitle}`	The title of the process
`ServerAddress`	string		The address of the server
`ServerIp`	string		The IP address of the server
`ServerUser`	string	`${environment-user}`	Information about the user that is relevant to the event
`HostId`	string		Unique host id. Hostnames are not always unique, so use an id that is meaningful in your environment
`HostIp`		`${local-ip:cachedSeconds=60}` (requires NLog 4.6.8+)	The IP address of the host
`HostName`	string	`${machinename}`	The name of the host
`LogOriginCallSiteMethod`	string	`${exception:format=method}`	The name of the function or method which originated the event
`LogOriginCallSiteFile`	string	`${exception:format=source}`	The file containing the source code which originated the event
`LogOriginCallSiteLine`	string		The line number of the file containing the source code which originated the event

Step 2: Enable APM log correlation (optional)edit

If you are using the Elastic APM .NET agent, log correlation can be configured to inject trace id fields into log events.

Step 3: Configure Filebeatedit

Follow the Filebeat quick start
Add the following configuration to your filebeat.yaml file.

filebeat.yaml.

filebeat.inputs:
- type: log
  paths: /path/to/logs.json
  json.keys_under_root: true
  json.overwrite_keys: true
  json.add_error_key: true
  json.expand_keys: true

Make sure your application logs to stdout/stderr.
Follow the Run Filebeat on Kubernetes guide.
Enable hints-based autodiscover (uncomment the corresponding section in filebeat-kubernetes.yaml).
Add these annotations to your pods that log using ECS loggers. This will make sure the logs are parsed appropriately.

annotations:
  co.elastic.logs/json.keys_under_root: true
  co.elastic.logs/json.overwrite_keys: true
  co.elastic.logs/json.add_error_key: true
  co.elastic.logs/json.expand_keys: true

Make sure your application logs to stdout/stderr.
Follow the Run Filebeat on Docker guide.
Enable hints-based autodiscover.
Add these labels to your containers that log using ECS loggers. This will make sure the logs are parsed appropriately.

docker-compose.yml.

labels:
  co.elastic.logs/json.keys_under_root: true
  co.elastic.logs/json.overwrite_keys: true
  co.elastic.logs/json.add_error_key: true
  co.elastic.logs/json.expand_keys: true

For more information, see the Filebeat reference.

« Introduction