By 
Organizations devote plenty of attention to security, yet it can remain patchy. There are always high chances of cracks and loopholes. Adding to the difficulty is the lack of sufficient context on the security data log. Consider an IP address found in a network log. By itself, it reveals little. If you, however, enrich it with additional information, such as geographical location, whether it is known to be part of a botnet, or the number of login attempts, then you have a clearer picture of whether the IP address poses a threat.
Role of data enrichment in ensuring organizational security
Data elements, such as user ID and IP addresses, individually cannot provide sufficient insights. For example, IP addresses may often change and may be shared by multiple people. They are thus not a good indicator of malicious activity. They may, however, be exploited by bad actors by switching IP addresses and sharing information. Enrichment provides additional information that helps separate the malign activities from the benign ones.
For a useful analysis of security log data, individual data identifiers have to be enriched with contextual information obtained from additional sources. If a username is found in an application log, for instance, it can be cross-referenced against the identity and access management system to obtain the user’s privileges, roles, department, and so on. The log data can similarly be enriched with IP addresses by referencing known threats (if any) associated with a specific address or appending geolocation. This makes the raw log data more meaningful, and the additional context offers extra analysis and detection options.
Data enrichment can be done either at the time of collection or analysis. Enriching data at the time of collection is more tedious but it gives a more accurate picture of context. It prevents misrepresentation of log data that may happen as a result of changes in the network environment, as in the case of IP addresses provided via Dynamic Host Configuration Protocol. Enrichment of data at collection also necessitates replicating the original log files to reconstruct the events and facilitate investigations.
Enrichment at the time of analysis removes the need to retain copies of log files and puts less strain on the analysis platform but at the cost of accuracy. Reconstructing the chain of events and identifying the root cause is difficult, if not impossible. Information that was not captured initially may be lost irretrievably and subtle patterns could be missed.
Contextual information sources for security data enrichment
Information for the enrichment of security log data can be acquired from different sources and appended to the raw data points. A few examples of information sources include the following.
- Directory services: Information includes user identity details, asset identity information, and roles and access privileges. These are useful for threat analysis and detection.
- Identity and access management systems: This provides information such as user identity, usernames, and aliases. Enriching security events with these provides clear audit trails and enables correlation of users and activities.
- Vulnerability scanner: Contextual information from this source includes applications in use, patch levels, vulnerabilities identified, and other known exploits. These allow security events to be weighed based on the vulnerability of the target.
- Penetration tester: This provides information such as the failure or success of exploitation and the methods used, evasion techniques, etc. These details add crucial context to an attack vector.
- Threat database: This source provides valuable information on threat intelligence, and includes details, provenance, and remediation of exploits, malware, etc.
How data enrichment helps improve security
Accurate threat detection
Enriching the raw log data with crucial details makes it easier—or at least less tedious—to distinguish real threats from noise. It provides context to suspicious events and clarifies whether they are malicious or benign activities. An alert indicating an unusual file download does not say much. But when enriched with contextual information showing the file’s hash value to match known malware, it becomes clear that the unusual activity is a potential security threat.
Reduced false positives
The sheer volume and varied nature of security data can lead to misleading alerts. Security logs and events stream in from numerous sources, often rapidly and in vast quantities. This can cause benign activities to trigger alerts based on intrusion detection system (IDS) signatures, resulting in false positives.
False positives increase the signal-to-noise ratio, inflating the number of security events. This makes it difficult not only to detect real threats but also eats up precious security analysts’ time. Enrichment adds context to security events, enabling the noise to be separated from the signals. It thus helps reduce the number of irrelevant alerts, saving time as well as resources.
Improved threat investigation
Data enrichment not only helps threat identification but also facilitates deeper investigation and effective response. Enriched logs with detailed contextual information, like timestamps, user roles, and affected systems, enable more in-depth forensic analysis. This helps in understanding the nature, scope, and impact of the breach.
Enhanced event correlation
Enrichment improves event correlation by adding relevant context to security events. This makes it easier to spot patterns indicative of attacks that may not otherwise be evident from isolated data logs. For example, correlating login attempts with geolocation data can help identify suspicious activities that might be missed by looking at login attempts alone.
The events are collected from different sources such as firewalls, switches, and authentication services. They are then normalized before they can be correlated.
Effective incident response
Should there be an attack or a potential attack, enrichment helps in fast triage. By providing details about the attack vector, techniques, and procedures used by malicious actors, it helps in understanding the incident. Enrichment also helps security teams prioritize security breaches based on the severity and relevance of the incident. For example, enriching an alert about a potential malware infection with threat intelligence data enables the security team to take remediation measures accordingly.
End note
Data enrichment plays an important role in enhancing security and managing threats. The additional context that data enrichment provides to raw log data leads to faster and more accurate threat detection and more effective response. This has benefits besides greater security. The reduction in false alarms leads to less distraction and also minimizes wastage of time and resources.
But as important as data enrichment is to security, it is a tedious task and requires expertise. If resources are scarce and expertise limited, reach out to data enrichment service providers to fill the gaps. They can help patch the loopholes.
In short, irrespective of the approach you choose, it is crucial to remember that enriching data is significant toward organizational security and hence should never be overlooked.
Comments