Behavioral Detection Playbook

This playbook is for detection engineers, SOC analysts, and incident responders. Malware that rewrites itself, or that attackers regenerate with AI for every target, defeats hash and static signature matching. It still has to do things: execute, persist, escalate, reach the network, and touch data. Behavioral detections target those actions, which change far less than the code does.

Use the first half to review detection coverage and the second half as an incident runbook.

How to use this playbook

1. For each detection strategy, confirm three things: you collect the telemetry, a detection exists, and it has been tested in a lab.
2. Record gaps against the MITRE ATT&CK technique IDs listed so coverage is measurable.
3. Retest after EDR, SIEM, or logging changes, which silently break detections more often than attackers do.

Telemetry prerequisites

• □ EDR on all endpoints and servers with tamper protection enabled.
• □ Process creation logging with full command lines, plus PowerShell script block logging.
• □ DNS, proxy, and firewall logs retained long enough to scope an intrusion.
• □ Synchronized time across all sources so timelines line up.

Detection strategies

• □ Environment and sandbox checks followed by payload execution (T1497, Virtualization/Sandbox Evasion). Adaptive malware probes before acting.
• □ Script interpreters spawned by office apps, browsers, or document viewers (T1059, Command and Scripting Interpreter).
• □ Process injection into unrelated processes (T1055).
• □ Access to credential stores or LSASS memory (T1003, OS Credential Dumping).
• □ Obfuscated or runtime-generated code: high-entropy scripts, repeated decode-and-execute chains, and on-the-fly compilation (T1027).
• □ Command and control over common web protocols and services (T1071, T1102), including calls to LLM APIs from processes with no business reason to make them.
• □ Mass file modification or encryption, and deletion of backups or shadow copies (T1486, T1490).
• □ Exfiltration to cloud storage or web services (T1567).

For attacks against your own AI systems, such as model evasion or poisoning, MITRE ATLAS provides a comparable technique catalog.

Response runbook

1. Triage: confirm the behavior chain and pull the process tree from EDR.
2. Contain: network-isolate the endpoint through EDR. Avoid powering it off, which destroys memory evidence.
3. Collect: memory image, relevant logs, the sample, and network data, with chain of custody.
4. Scope: hunt across the fleet for the same behaviors, not the same hash. Variants will hash differently.
5. Eradicate: reimage rather than clean, and reset every credential that was present on the host.
6. Improve: turn what you learned into new or tuned behavioral detections, and test them.
7. Communicate: brief the SOC on the behavior patterns and update this playbook.

For the wider incident lifecycle, align with NIST SP 800-61 Rev. 3.

Common pitfalls

• Hunting only by hash or IOC. It finds yesterday's sample, not today's variant.
• Untested detections. A rule that has never fired may be broken, not quiet.
• Overly broad behavior rules. Tune with allow-lists by parent process, signer, and path to control alert volume.
• Rebuilding without rotating credentials, which lets the attacker walk straight back in.