Three elephants that support logging in Windows

Let's start with the fact that the agent itself mostly does not care by what technology data collection is carried out. However, when implementing agents into your own infrastructure as part of a security policy, there are always some restrictions on the use of certain tools, ports, and other things. In this regard, we will consider possible sources of data transmission for agents.

If we want to collect data from endpoints on Windows OS, there are three main tools. The first is the Windows audit subsystem, which stores events in the EventLog format. This is a Windows event logging tool that records operating system messages. Events are divided into several groups – logs:

Moreover, some applications write their own events to a separate folder, for example:

This folder may contain logs of both the processor and software (sound drivers or the same Kaspersky).

There are also separate logs for:

• DNS Manager

• Failover Cluster Manager

• IIS Access

• Task Scheduler History

• Windows Component Service

System log events in EventLog format already have some initial normalization and contain the following fields:

• Date in the format: month, day, time, and year

• Event category as an integer

• Event level

• Windows security ID

• Windows source

• Windows event log keywords

• Windows event ID

• Message text

For event logs, it is possible to change read, write, and clear permissions. In this sense, the security log (Security) stands apart - the right to write to it is reserved exclusively for the local security authority of Windows (LSASS). It is also possible to add event types to the Security log and assign a source from registered applications.

For primary analytics, and even more so for incident investigations, information from EventLog alone is not always sufficient. In this regard, another utility with extended event audit functions - Sysmon, included in the Sysinternals package, is actively used. Essentially, it is a Windows system service that provides detailed data on incoming events, network connections and drivers, file changes, and much more. I found this map of interesting data that can be collected using Sysmon (for details, I recommend looking here.

To ensure an additional level of security, Sysmon is usually installed together with other endpoint protection solutions, such as traditional antivirus, event management system (SIEM), or endpoint detection and response (EDR). The driver of this service allows for collection immediately upon device power-on, i.e., before the main components of the computer start.
Additionally, this tool is quite well described and popular, which means that there are many ready-made configurations on the internet for various event audit requirements.
The only significant drawback of Sysmon is its weak compatibility with the Linux OS class. For such OS, only some of the events registered in Windows systems are recorded (Event ID 1, 3, 4, 5, 9, 11, 16, 23).
To search for logs in the "Event Viewer" interface, look in the Applications and Services Logs\Microsoft\Windows\Sysmon log. Here is an example of an event:

And finally, the third primary event collection technology is Event Tracing for Windows (ETW). The technology is provided in the OS by default and operates at the kernel level. ETW records event logs created by kernel drivers and user applications. Sysmon itself generates some of its events based on it. To understand the essence of ETW, we will need to use several concepts:

• Providers (providers, event providers) are applications (e.g., DLLs) that have event registration functions. It is the provider that monitors the state of applications (or the system) and sends events to the session.

• Sessions (sessions, trace session, logger) are entities that collect events from providers and pass them to consumers. One session can consume events from one or more providers. Sessions also sort events among themselves to pass them to the appropriate consumers.

• Controllers (controllers) are applications that manage sessions. They can create or delete sessions, as well as change their parameters.

• Event consumers (consumers) are applications that can receive and process events from one or more sessions, as well as from files.

Providers generate and transmit events about states occurring within processes or the kernel, and consumers use these events for their own purposes (e.g., performance analysis). The vendor presents the data transfer process as follows:

ETW in its basic configuration is a simple inter-process communication mechanism and operates only locally, and in the vast majority of studies is used as a "defensive" tool.

Events from this source can be found in the Applications and Services Logs and Windows Logs\Security tree.

If you want to delve into the logic of ETW operation and basic administration, I recommend studying the following article, which describes the internal processes of the tracing service in an understandable language.

So, we figured out where and with what we can collect data, but what should the logs themselves be like? After all, as the commentator under the previous article correctly said it is known, for the correct operation of the infrastructure and subsequent event analysis, it is not enough to simply collect "everything I see." There is a high risk of cluttering the processing of devices, and at the same time making it impossible to respond to events promptly. So I propose some criteria for evaluating the quality of the transmitted data:

• Events must be authentic, meaning that we need to be sure that the recorded events actually happened and that the event attributes are correct;

• The event log must be complete, meaning that it provides a full picture of the actions;

• Any recorded events must have a clearly defined semantics, meaning that different types of events must be distinguished.

This article is for informational purposes only and in no way suggests which specific event collection tools you should use. The article will be more useful for those who are just diving into the audit of Windows endpoint events - friends, now you know where to dig further. In the next article, we will talk about journaling in Linux-class OS.