Threat Hunting with Sigma and Elastic: From Indicator to Detection Rule

Threat hunting fails when it stays a vibe. An analyst greps for a bad hash, finds nothing, and declares the environment clean, which proves only that one artifact is absent. The Basilisk team runs hunting as an engineering pipeline: a hypothesis expressed as a portable Sigma rule, compiled to an Elastic query, validated against real telemetry, tuned, and then promoted to a standing detection. This piece walks the full path from indicator to detection with concrete rules, queries and tuning steps you can reproduce in a lab tonight, and it is built to pair with an emulation source so you are hunting something real rather than staring at an empty index.

From indicator to detection: the pyramid of pain#

David Bianco's Pyramid of Pain is the mental model that makes hunting worth the effort. Hashes and IPs sit at the bottom: trivial for an attacker to change, so a hunt keyed on them expires in hours. Domains and network artifacts are a little harder. At the top sit tools and TTPs, the tactics, techniques and procedures an adversary would have to re-engineer to evade, which is expensive for them and durable for you. Good hunting targets behavior, not indicators: not 'find hash X' but 'find any process that spawns a script interpreter from an Office document and then makes an outbound connection'. Sigma exists precisely to express that behavioral logic once and run it across any backend, so your intellectual work is not locked to a single vendor query language.

Anatomy of a Sigma rule#

A Sigma rule is a small YAML document with a fixed skeleton. The logsource block declares what telemetry it needs, for example product: windows and category: process_creation, which maps to Sysmon Event ID 1 or Windows 4688. The detection block holds one or more named selections and a condition that combines them with boolean logic. A minimal LOLBin rule might define selection as Image|endswith: '\\certutil.exe' together with CommandLine|contains: '-urlcache', then set condition: selection. Add a falsepositives list and a level so triage knows how to weigh a hit, and tag it with the ATT&CK technique it covers. The discipline that matters: one rule expresses one testable behavior, with field names drawn from a schema you actually ship, not from a blog screenshot.

Setting up the Elastic stack#

You need telemetry before you need rules. Stand up Elasticsearch and Kibana, then enroll endpoints with the Elastic Agent managed through Fleet, shipping the System and Windows integrations so process creation, network, and authentication events land in indices normalized to the Elastic Common Schema (ECS). ECS is the linchpin: it renames vendor fields to stable names like process.command_line, process.parent.name and destination.ip, so a query written once keeps working as sources change. On Windows, deploy Sysmon with a curated configuration (a maintained community config is the sane default) because the native audit log is too coarse for behavioral hunting. Verify ingestion by confirming that a test certutil execution shows up in Discover with a populated process.command_line before you trust any rule.

Compiling Sigma to an Elastic backend#

Sigma is portable because a compiler translates it to each backend's dialect. The modern toolchain is sigma-cli driving pySigma with a plugin per target. Install the Elasticsearch plugin, then run sigma convert -t elasticsearch -p ecs_windows rules/certutil_urlcache.yml to emit an Elasticsearch query string, or target -t eql to get Event Query Language for correlation across events. A pipeline such as ecs_windows is what remaps the rule's generic field names onto your ECS fields, and skipping it is the single most common reason a converted rule returns zero results despite a real hit. For sequence logic, Elastic's EQL expresses 'process A then network B by the same process within N seconds', while the newer ES|QL is excellent for exploratory aggregation during the hunt itself.

A concrete hunt: living-off-the-land binaries#

Attackers avoid dropping malware by abusing signed system binaries: certutil to download, mshta and rundll32 to execute, regsvr32 for the Squiblydoo technique, bitsadmin for transfer. Start broad in ES|QL: FROM logs-* | WHERE process.name IN ("certutil.exe","mshta.exe","regsvr32.exe") | STATS count = COUNT(*) BY process.command_line, host.name | SORT count ASC, then read the rare command lines, because normal admin use is high-volume and repetitive while the attacker's invocation is a lonely outlier. Pivot on any hit to the parent process and the subsequent network connection to confirm intent. The deeper KQL patterns for each binary, with the benign baselines to subtract, are catalogued in Hunting Living-off-the-Land Binaries on Windows with KQL.

Hypothesis-driven hunting mapped to ATT&CK#

Do not hunt at random; hunt a hypothesis tied to a technique. Frame it as a sentence: 'If an adversary performed Kerberoasting (T1558.003), I would see a spike of TGS requests with RC4 encryption from a single account.' Then express that as a rule and test it against emulated activity, because a hunt you cannot trigger on demand is a hunt you cannot trust. Generate the behavior safely in a lab: run the Kerberoasting chain from Active Directory Pentest: Step-by-Step Kerberoasting in a GOAD Lab, or automate a whole ATT&CK matrix with Adversary Emulation with Caldera and MITRE ATT&CK in a Corporate Lab. For east-west movement, the telemetry and detections are in Lateral Movement in the Lab: SMB, WMI and WinRM with a Detection Focus.

Tuning and false positives#

A rule that fires on every backup job is noise that trains analysts to ignore alerts, which is worse than no rule. Tune with data, not guesswork: run the candidate over 30 days of history, read every hit, and characterize the benign clusters, a specific service account, a patch-management tool, a monitoring agent. Encode those as explicit exclusions in the rule's filter selection rather than broadening the match, so you subtract known-good without blinding yourself to the malicious variant. Track precision as a number and demand that a rule clear a threshold before promotion. Beware the opposite failure too: an over-tuned rule with so many exclusions that the attacker simply reuses an excluded account. Tuning is subtraction of the explained, never subtraction of the inconvenient.

From hunt to standing detection#

A successful hunt is a hypothesis that found something or proved a gap; either way it should not evaporate. Promote the validated Sigma rule into your detection-as-code repository, version it in Git with its ATT&CK tags and false-positive notes, and deploy it as a scheduled Elastic detection rule that raises an alert with the context an analyst needs to triage in one screen. Close the loop with the red team so every emulated technique yields either a detection or a documented blind spot; that feedback cycle is the whole point of Purple Team in Practice: Building a Red vs Blue Feedback Loop. When a detection does fire in anger, the responder needs host-level triage, which is where DFIR on Linux: Live Triage with UAC and Velociraptor takes over.

Pitfalls and checklist#

The recurring failures: converting a rule without the ECS pipeline and trusting the zero result; hunting on a data source you never actually ingested, so absence means nothing; keying on hashes and IPs that expire; writing a rule you cannot trigger on demand; and shipping a noisy rule that erodes trust in the whole pipeline. The checklist before any hunt: (1) hypothesis written as a sentence tied to an ATT&CK technique; (2) required log source confirmed present and populated in ECS; (3) Sigma rule expressing one behavior, with false positives listed; (4) compiled with the correct pipeline and the query manually sanity-checked; (5) triggered against emulated activity to prove it fires; (6) tuned over historical data with precision measured; (7) promoted to version-controlled detection-as-code; (8) linked back to the purple-team backlog.

FAQ#

Why Sigma instead of writing Elastic queries directly? Portability and review. A Sigma rule is vendor-neutral, so the same behavioral logic compiles to Elastic today and to another SIEM tomorrow, and it reviews as a small declarative document instead of a sprawling query. You still run native ES|QL for interactive exploration; Sigma is for the durable, shareable detections that outlive any one platform. The two are complementary, not competitors.

How is hunting different from alerting? Alerting runs known-bad detections continuously; hunting proactively tests hypotheses about activity no rule catches yet, and its output is often a new rule. A mature program feeds hunting results into alerting so today's manual hunt becomes tomorrow's automated detection. If a hunt never produces a durable artifact, a rule or a documented gap, it was entertainment, not engineering.

Conclusion#

Treat hunting as a pipeline, not a hunch. Aim at behavior high on the Pyramid of Pain, express each hypothesis as a single Sigma rule, compile it to Elastic with the right ECS pipeline, and prove it fires against emulated activity before you believe a clean result. Tune with real data, measure precision, and promote survivors into version-controlled detections wired back to a purple-team loop. Done this way, an empty result is meaningful evidence rather than false comfort, and every hunt leaves the environment with one more durable detection or one honestly documented blind spot. That accumulation, not any single clever query, is what actually moves an adversary up the pyramid and out of your reach.