From Default to Detection: My First Dive Into SIEM Tuning and Threat Detection

Bridging Theory and Practice

Security tools like Wazuh and Sysmon are designed to catch common malware out of the box, but they’ll quickly flood log storage with useless alert noise. You don’t get real visibility that way. Security teams have to actively modify policies and write custom rules just to catch threats lurking in the shadows. To make matters worse, indicators like credential dumping, log clearing, and persistence mechanisms hide in plain sight as low-severity events, if they trigger an alert at all. For this project, I modified a pre-built lab environment to see what it takes to cut through the log clutter and validate how well a SIEM can catch active adversary techniques and turn them into concrete, actionable alerts. By the time I finished, I’d learned some surprising lessons about where default rules fail and why tuning is everything.

Before we look more closely at this lab, it’s probably worth sharing my angle on all this. My main motivation for getting into cybersecurity was to be part of the team building the shield to stop data leaks and the chaos adversaries leave behind. Somewhere along the way, I realized my long-term goal is the analytical forensics side of DFIR. The idea of being the one to figure out the “what, where, and why” of an attack is what pulled me into this field. But to solve those high-stakes puzzles, you have to get down in the weeds where the data lives and make sure that you’re seeing the logs and alerts that matter. Ultimately, that’s why I decided to focus this project on modifying Active Directory to separate the signal from the noise, ensuring that key events hit the centralized system to catch potential adversarial activity.

This project is my first contribution to the security community, marking the leap from textbook theory to hands-on detection. I hope someone finds something useful in my experience! Let’s dive in.

The Telemetry Pipeline

While the lab environment included several other endpoints, I opted to streamline the scope. To keep the telemetry clean and manageable, I focused my project on a three-node setup: an Active Directory endpoint (ad01) to host target services and execute attacks, a command-line SIEM backend (Wazuh_SIEM) for log processing and custom detection, and an analyst system (Blue-Team Workstation) to visualize the results.

Juggling these virtual (albeit a little outdated) machines to keep the pipeline alive was a constant headache.

Early on, I ran into a really confusing roadblock where my Wazuh dashboard was missing large chunks of time from the logs. The annoying part was that the dashboard didn’t even flag the agents as offline. Everything looked fine on paper, but no new logs came through when I wasn’t staring at it. I eventually discovered that virtual machine idle suspensions were messing with background states on limited resources, causing the connection between the host and Wazuh to silently lock up. To fix it, I had to troubleshoot the pipeline by verifying active TCP socket connections for ad01 on port 1514, clearing the local agent cache, and manually restarting WazuhSvc to force a fresh connection. Once I figured out that rhythm, the plumbing stayed stable.

The 4 Experiments

#1 Credential Dumping

Before running experiments, I had to isolate ad01 from the other active agents shipping logs to the manager, which is a standard reality of any multi-endpoint environment. Dashboard filters were essential just to separate the simulated Atomic attacks from background noise.

The first goal I had in mind was to isolate credential dumping telemetry and crank up its severity. With the default settings, if an adversary breached this lab environment and used rundll32.exe to dump OS credentials via KRShowKeyMgr, Sysmon would label this as process creation, and Wazuh saw it as a low-level (3) alert. While it’s nice that Sysmon was already forwarding the raw security events, if I want actionable alerts from Wazuh, I need to write custom detection rules.

The rule I initially made worked, but with a lot of caveats:

<!-- Atomic Red Team T1003.005 Credential Dumping Detection -->
<rule id="100007" level="10">
  <if_sid>61603</if_sid>
  <field name="win.eventdata.commandLine">KRShowKeyMgr</field>
  <description>Atomic Red Team: Potential OS Credential Dumping via keymgr.dll (T1003.005)</description>
  <mitre>
    <id>T1003.005</id>
  </mitre>
</rule>

Enter fullscreen mode Exit fullscreen mode

While the Atomic tests for this experiment successfully sent logs that Wazuh read as an elevated severity threat, it seemed to only catch the NTDS shadow copy extractions and not the exact extraction utility as cleanly as desired.

I was excited that it worked on the first try, but looking back, I rushed into the deployment before finishing my MITRE ATT&CK research. If I were to write this rule again for a production-ready environment, there are two major flaws I would need to fix to change it from a tactical detection to a more strategic one:

• Case-Sensitivity Bypasses: Wazuh’s default string matching is case-sensitive, meaning an attacker could completely bypass this rule just by typing krshowkeymgr or KrShowKeyMgr.
• Wrong MITRE Tag: While T1003 focuses on cached domain credentials, the keymgr.dll utility actually interacts with application password stores, which technically maps to T1555.

To fix these gaps, here is what the hardened version of that rule looks like:

<!-- Consolidated Credential Dumping Detection -->
<rule id="100007" level="10">
  <if_sid>61603</if_sid>
  <field name="win.eventdata.commandLine" type="pcre2">(?i)KRShowKeyMgr|rundll32.*keymgr\.dll</field>
  <description>Suspicious Credential Dumping: Abuse of keymgr.dll / KRShowKeyMgr</description>
  <mitre>
    <id>T1555</id>
  </mitre>
</rule>

Enter fullscreen mode Exit fullscreen mode

Additionally, other techniques exist, like T1552 (unsecured credentials), which involves hunting for plaintext passwords that sit in config files, scripts, and registry keys. Unfortunately, the logic in the rule above won’t capture this activity whatsoever and would require a separate rule. While both T1003/T1555 and T1552 share the same Sysmon process creation rule (61603) as a common starting point, T1552 involves entirely different behaviors, and therefore the rule written for this would have to be entirely separate, rather than a parent-child chain.

#2 Anti-Forensic Activity

Once I had credential dumping sorted out, my next target was anti-forensic activity. What happens when an adversary tries to erase their tracks by deleting generated logs? With default settings, Wazuh only flagged the action as a low-priority (level 5) event. In order to get these wipes to trigger SOC escalation, I needed to engineer custom rules to boost that severity.

During the initial Atomic test run, I noticed a major gap in the paper trail. The Domain Controller wasn’t natively looking out for Windows Event ID 1102 (Security Audit Log Cleared). Since the OS failed to generate the local event, Wazuh was none the wiser about anything happening, creating a complete blind spot to the tampering. To resolve this, I had to update the Advanced Audit Policy on ad01 via an auditpol command to enforce logging of security state modifications.

auditpol /set /subcategory:"Security State Change" /success:enable

Enter fullscreen mode Exit fullscreen mode

Once the OS knew that action was something to create a log for, I needed to deploy a custom rule to fix the earlier mentioned issue of Wazuh defaulting to level 5 on it.

<!-- T1070.001 Indicator Removal: Application or System Log Cleared -->
<rule id="100008" level="12">
  <if_sid>63104</if_sid>
  <description>CRITICAL: Windows Application or System log was cleared (T1070)</description>
  <mitre>
    <id>T1070.001</id>
  </mitre>
</rule>

<!-- T1070.001 Indicator Removal: Security Log Cleared -->
<rule id="100009" level="12">
  <if_sid>63103</if_sid>
  <description>CRITICAL: Windows Security Audit Log was cleared (T1070)</description>
  <mitre>
    <id>T1070.001</id>
  </mitre>
</rule>

Enter fullscreen mode Exit fullscreen mode

By nesting these rules under the parent SIDs, they target the structural OS events, rather than just looking at command-line inputs. That way, no matter what tool an adversary might use to wipe the logs, the action itself will trigger a critical alert.

Additionally, one should consider other anti-forensic techniques, such as timestomping and the deletion of malicious files post-exploit. While process creation logging (Sysmon Event ID 1) successfully recorded the execution of utilities like wevtutil.exe, relying on process execution leaves gaps if an adversary can use native APIs. Tracking timeline reconstructions effectively requires an expansion on the ruleset tree to monitor for Sysmon Event IDs 11 and 23 (file creation and deletion) to capture when staging folders are manipulated or purged.

#3 Persistence Mechanisms

The next phase of my lab modifications targeted endpoint visibility into persistence mechanisms. If malware was designed to survive a system reboot, like through Registry Run key additions, Startup folder shortcut drops, Explorer shell context menu hijacking, and rogue Windows Services, I wanted there to be an actionable alert for it. Luckily, my earlier configurations were already tuned so that Wazuh’s native engine did most of the heavy lifting here, successfully parsing structural changes without requiring me to come up with brand-new rules after running the Atomic modules for T1547.

The registry modifications and startup directory shortcuts lit up Sysmon Event IDs 1, 11, and 13, and RegistryEvent (ID 13) proved invaluable for capturing modifications to TargetObject paths. While most baseline monitoring focuses solely on standard Run keys, tracking subkeys under _Classes\...\shell successfully flagged living-off-the-land persistence attempts that traditional registry audits miss.

Additionally, creating a test service generated a correlation-ready audit trail by pairing Sysmon Event ID 13 with a native Windows System Event ID 7045 (New Service Installed). Finally, executing administrative configuration utilities like secedit to modify security templates and turn off persistence configurations still sent the necessary alerts needed to know something suspicious was brewing on ad01.

However, this round of testing brought a huge real-world realization to me: the biggest challenge with autostart persistence isn’t the lack of data, but separating it from routine administrative noise. Legitimate software constantly updates registry keys and installs services during normal operation. This experiment taught me that a hardened security architecture can’t simply look for service creation. High-fidelity detection means you have to dig into process lineage, specifically separating normal parent-child process relationships, and flagging when services attempt to execute out of unusual locations like AppData or temp directories.

#4 Stealthy Triggers

For my final experiment, I wanted to push the lab’s detection capabilities against stealthier, trigger-based persistence models that bypass startup folders and registry keys entirely. I focused on two techniques: Windows Management Instrumentation (WMI) Event Subscriptions and Image File Execution Options (IFEO) debugger hijacking. These mechanisms are particularly dangerous because they allow an adversary to remain completely dormant until a specific system condition is met or a standard Windows accessibility utility (like sethc.exe, a.k.a. Sticky Keys) is launched.

This one provided some of the best troubleshooting and learning moments of the whole project. Right out of the gate, the Atomic test scripts failed because modern PowerShell cmdlets faced an object type mismatch with the version of Windows ad01 was running. This forced me to switch to manual entry so I could use legacy WMI cmdlets (Set-WmiInstance) just to get the structure (__FilterToConsumerBinding) to register. After that, the Atomics ran better, but then I ran into a telemetry blind spot. While Wazuh was able to capture the __EventConsumer log generated by Sysmon, the other steps were completely missing.

While the native Wazuh engine was able to read the WMI alerts, it was blind to the rest of the process because the default Sysmon configuration was filtering out the rest of the simulated attack lifecycle. To fix this, I had to crack open sysmonconfig.xml to explicitly enable collection for Sysmon WMI Event IDs 19, 20, and 21. Once that change was saved and the system reloaded, I tried running the simulation again. This time the entire WMI structure registered!

For the IFEO hijacking, to my relief, Sysmon successfully captured modifications to the Debugger string value within the registry. I discovered this worked out of the box because tampering with accessibility binaries is a known privilege escalation shortcut, and auditing writes to the IFEO hive yielded an immediate high-fidelity alert. Ultimately, this portion taught me that a hardened detection setup isn’t just about writing rules or checking dashboards. It requires constantly auditing underlying configuration files so your tools actually see potential threats within your environment.

The Reality of Detection Engineering

The main concept I took away from this project was just how big of a gap there is between default log generation and legitimate threat detection. Before starting, I assumed high-risk actions like credential dumping, log clearing, and known persistence mechanisms would trigger high-severity alerts right out of the box.

Instead, default SIEM rules treated most of these actions like generic events, mostly to keep dashboard noise to a minimum. Credential extraction showed up as normal process execution, and log clearing didn’t generate anything useful at all. It wasn’t until after I modified local_rules.xml that the system realized these actions could be problematic. I also ran into blind spots I wasn’t expecting, such as ad01 not logging Windows Security Event 1102 until I instructed it to.

After a bit of digging, I learned that these default settings were never meant to be a one-size-fits-all option but just a baseline to start from. It showed me that effective detection engineering means actively tuning to what a specific environment needs, which requires knowing what counts as “normal business” versus something worth investigating. A smaller company with a handful of computers has wildly different needs compared to a massive corporation with thousands of potentially rotating employees.

If I were giving advice to someone tackling a project like this for the first time, I believe it boils down to three things:

1. Take notes (about EVERYTHING) in real time. While previous assignments were small enough to easily rely on memory, trying to remember what went wrong, when, and why after the fact is a lot harder when juggling multiple machines. No matter how much you might want to just get through this ASAP, taking detailed notes will help you far more than skipping it. Your thoughts, a stray error, or a command that didn’t work the way Google said it should might seem insignificant at the time, but you never know what will end up mattering down the line.
2. Be mindful of how your virtual machines are set up to behave. When testing VMs go to sleep, services like WazuhSvc might disconnect, requiring you to check on (and restart) the service every time you sign back in. Even something as mild as taking a lunch break while running Atomics can make it look like a new detection rule failed, when in reality, the log simply didn’t have a chance to send because the VM went idle. Always check active service states before trying to troubleshoot.
3. Verify existing local log settings before writing new SIEM rules. The SIEM cannot generate an alert for something if the OS isn’t actively logging it. Never assume Windows is already looking out for something you think is important. Before assuming a rule isn’t working, remember that the local Event Viewer is your best friend for making sure Advanced Audit Policies are enabled to capture events locally in the first place.

Special Thanks

Now that I’m at the end of this project, I have the benefit of hindsight to see just how important some of the sources I used were to this process. While I regret not writing down everything that was helpful, I want to give a special shout-out to:

• External Researchers & Tool Creators: I want to specifically thank the security researchers behind open-source frameworks like Atomic Red Team, including creators like Casey Smith and Matt Graeber at Red Canary, whose work makes hands-on detection engineering accessible for learners without requiring an advanced pentesting background. I would say this was the most impactful resource I used for this project. I do not (yet) know how to manually construct or execute all of the complex adversary techniques needed for experimenting like I wanted to, and thankfully, I didn’t need to. Having access to pre-packaged testing scripts allowed me to focus more on log analysis and anomaly detection with easy-to-use triggers for firing off events on demand.
• Arpine, my learning coach through TripleTen: Even though my own social awkwardness kept me from reaching out as much as I should have, the periodic check-in texts were like an anchor that kept me accountable and motivated to see this through, despite the periods when my mental attitude was sinking.

References

1. Atomic Red Team by Red Canary [n.d.]: An open-source library of simple, focused adversary emulation tests mapped directly to the MITRE ATT&CK framework. This was essential for executing on-demand attack techniques against the Active Directory endpoint to generate test telemetry.
2. MITRE ATT&CK Enterprise Matrix by The Critical Threat MITRE Corporation [2026]: A globally recognized knowledge base of adversary tactics and techniques based on real-world observations. Used throughout the project to map simulated attacks (like credential dumping and persistence) to standard industry taxonomy.
3. Cyber Analytics Repository (CAR) by The MITRE Corporation [2026]: A knowledge base of analytics focused on specific MITRE ATT&CK techniques. It provided helpful conceptual frameworks for understanding how specific telemetry data can detect adversary behaviors.
4. MITRE D3FEND by The MITRE Corporation [2026]: A knowledge graph of cybersecurity countermeasure capabilities. Referenced to understand defensive concepts and counter-measures mapped against active adversary techniques.
5. Microsoft Sysinternals Utilities (Sysmon) by Microsoft Corporation [2026]: Official documentation for advanced system monitoring tools. Used to configure and understand event logging behaviors—specifically Sysmon Event IDs tracking process creation, file modifications, and registry changes.