Cybersecurity Encyclopedia

Guidebook

Detecting Encryption Behavior

Spot ransomware-style encryption early: entropy spikes, mass extension changes, high write rates, and the process context behind suspicious file activity.

Quick facts

Difficulty
Advanced
Duration
13 minutes
Updated
Calm cybersecurity illustration for Detecting Encryption Behavior, showing abstract ransomware and recovery evidence cards, connected systems, and defensive control checkpoints.

Encryption does something to files that is hard to hide: it makes them look like noise. A well-encrypted file has no repeating patterns, no readable structure, and a byte distribution close to random, which is measurably different from the ordinary documents, spreadsheets, and images it replaced. When encryption is applied to a few files, that difference is invisible in the aggregate. When it is applied to thousands of files in minutes, it produces a distinctive footprint across several telemetry sources at once: a jump in the randomness of file contents, a wave of renamed or new extensions, a write rate far above the baseline, and a single process or small set of processes doing the touching. Detecting encryption behavior is the craft of reading those signals together, quickly, without mistaking every legitimate burst of activity for a disaster.

This guide takes an advanced, evidence-first view of that craft. It treats each signal as a piece of a model that needs corroboration, and it spends as much time on why these signals lie as on why they work, because a detection that fires on every backup job and archive export is a detection nobody will trust when it finally matters.

Note
Defensive learning boundary
This guide is defensive education. It uses toy examples, observable evidence, and safe reasoning. It does not provide exploit instructions, malware code, credential theft steps, evasion playbooks, target scanning procedures, or operational offensive workflows. If you are handling an active incident, preserve evidence, follow your organization’s incident-response plan, and involve qualified responders and legal counsel where appropriate.

The four signals and why no single one is enough

The first signal is entropy, a measure of how random a file’s bytes are. Encrypted data pushes entropy toward its theoretical maximum, so a document that used to be full of readable text and now measures as near-random is suspicious. But entropy alone is a weak witness, because already-compressed formats such as archives, many images, and video are also high-entropy by nature. The randomness of file contents is explored in depth in File Entropy and Mass-Encryption Clues , and the key lesson is that high entropy is normal for a large slice of ordinary files, so a rising entropy reading is a prompt to look closer, not a verdict.

The second signal is mass extension change: files across a share suddenly carrying a new or uniform extension, or their original extensions being replaced. The third is write rate, an unusually high volume of file modifications per unit time, especially if it sweeps methodically through directories. The fourth, and the one that turns the others from noisy into meaningful, is process context: which process is doing the writing, what launched it, under which account, and whether it has any legitimate reason to touch thousands of files. Any one of these signals has a mundane explanation. It is their coincidence, on the same host, in the same short window, driven by an unexpected process, that constitutes real evidence. Reading the process side well draws on the same skills as Processes, Parents, and Command Lines and Suspicious Process Indicators .

Reading the process behind the writes

Behavioral detection lives or dies on process context, because it is what separates encryption-as-attack from encryption-as-normal-operation. A backup agent legitimately reads and writes enormous numbers of files at high entropy; so does disk-encryption software and a video-transcoding pipeline. What distinguishes malicious mass encryption is usually the combination of an unexpected parent-child chain, an account that has no operational reason to be rewriting a file share, and a process whose location or lineage does not match the known-good picture of that host. A defender who can answer “which process, launched by what, running as whom” has moved from an alarming graph to an actionable claim.

This is why baselining is inseparable from detection here. If you know what normally writes to a given file server, which scheduled jobs touch archives at night, and which agents legitimately produce high-entropy output, then a novel process doing the same thing stands out sharply. Without that baseline, every high-write process looks equally guilty and the detection drowns in noise. Establishing that reference picture is the subject of Known-Good Baselines , and it is the quiet prerequisite that makes entropy and write-rate signals usable rather than maddening.

What evidence matters?

The strongest evidence is corroboration across sources with tight timing. A single entropy reading proves little; an entropy jump, an extension sweep, and a write-rate spike that all land on the same host within the same few minutes, attributed to one unexpected process running under a particular account, is a claim a responder can act on with confidence. The timestamps matter as much as the values, because the near-simultaneity is itself the signal. Evidence is also stronger when it is preserved in a form that survives the event: exported file-system telemetry and process records outlast a screenshot of a dashboard, and they let a responder reconstruct the sweep rather than remember it.

Equally important is evidence about what the activity is not. Recording the legitimate high-entropy workloads on a host, the scheduled jobs and their windows, and the agents authorized to touch a share turns each of those into an exclusion you can apply quickly. When a burst appears, the question becomes whether it matches a known workload or falls outside all of them. That framing keeps the detection honest and connects directly to the earlier phases of the Ransomware Timeline : mass encryption is loud, but by the time it is visible the intrusion is old, so the corroborating evidence you preserve here is what lets responders look backward for the foothold.

Toy example

Consider a fictional architecture firm, Halvorsen Studio, with a large shared drive of drawings and project files. At 2:47 a.m., monitoring flags a rising entropy trend on that share, and within the same minute a write-rate alert fires for the same host. A junior analyst pulls the process context and finds that an unfamiliar process, not the backup agent and not the transcoding job that normally runs at night, is methodically rewriting files directory by directory, running under a service account that has never done this before. Extensions on completed files are all being replaced with a single uniform suffix.

The analyst does not need to be certain to act well. The claim is narrow and evidence-backed: one host, one unexpected process, one account, three corroborating signals within a two-minute window, sweeping a share that a known-good baseline says nothing should be mass-rewriting at that hour. That is enough to preserve the telemetry, isolate the host under the firm’s plan, and escalate, moving into the reasoning of Containment Decision Trees . Contrast that with a night three weeks earlier, when the same entropy and write-rate alerts fired but the process context showed the expected backup agent inside its scheduled window: same graph shape, entirely different meaning.

Common mistakes and false positives

The largest source of false positives is treating high entropy as guilt. Compressed archives, encrypted volumes, media files, and even some database operations are high-entropy by design, so a detector that alarms on entropy alone will cry wolf constantly. Closely behind it is ignoring scheduled work: backup jobs, archival compression, disk-encryption rollouts, and bulk media exports all produce write-rate spikes and high-entropy output that look, from a distance, exactly like an attack. A third mistake is anchoring on file extensions, since legitimate tools rename files in bulk and some malicious activity deliberately leaves extensions alone; extension change is a supporting signal, never a sole one.

The subtler failure is over-tuning in the other direction. A team burned by false positives may raise thresholds so high that a slow, deliberate encryption sweep, paced to stay under the radar, never trips the alert. The way through is not a single magic threshold but corroboration and context: require several signals together, attribute them to a process and account, and compare against a baseline of legitimate high-volume workloads. This is the same discipline as Detection Tuning and Signal Noise , where the goal is a signal precise enough that responders still believe it at three in the morning.

What to do next

When corroborating signals point at real mass encryption, the next steps belong to your response plan: preserve the file-system and process telemetry, decide on isolation deliberately rather than reflexively, and protect the backups before doing anything that might tip off an intruder still present elsewhere. Resist the urge to immediately reboot or reimage the affected host, because that can destroy the very process and timeline evidence that explains how far the intrusion reached. Because encryption is a late signal, treat its appearance as a prompt to look backward across the earlier phases, not as the whole story.

For learning, build the detection logic on a fictional environment: list the legitimate high-entropy workloads, name what should and should not write to each share, and then reason through which combination of signals would distinguish an attack from that baseline. Then read File Entropy and Mass-Encryption Clues for the measurement side, Backup Design for Recovery and Restore Drills for the recovery that detection buys time for, and Impact and Blast Radius to judge which host’s encryption threatens the most.

How this guide was made

This page is defensive education assembled from public detection and incident-response frameworks, illustrated with fictional organizations and invented processes. It describes the observable behavior of mass encryption so defenders can recognize it, and it deliberately contains no malware, no code, and no operational offensive detail. It makes no claim to certification, legal advice, or incident-response authority; for real events, follow your own plan and qualified responders.

Official references

For orientation rather than certification, the CISA StopRansomware Guide frames detection and response for encryption events, MITRE ATT&CK documents data-encrypted-for-impact as a technique with defensive context, and NIST SP 800-61 Rev. 3: Incident Response Recommendations describes the triage and escalation that follow once encryption behavior is suspected.

Read this alongside File Entropy and Mass-Encryption Clues for the measurement detail and the Ransomware Timeline for where this late signal sits in the whole incident, and pair it with Containment Decision Trees and Restore Drills for what to do the moment the signal is real.

Sources & further reading

Amazon Picks

Support defense habits with tangible tools

Advertisement 4 curated picks

Advertisement · As an Amazon Associate, TensorSpace earns from qualifying purchases.

Written By

JJ Ben-Joseph

Founder and CEO · TensorSpace

Founder and CEO of TensorSpace. JJ works across software, AI, and technical strategy, with prior work spanning national security, biosecurity, and startup development.

Keep Reading

Related guidebooks

Calm cybersecurity illustration for Backup Design for Recovery, showing abstract ransomware and recovery evidence cards, connected systems, and defensive control checkpoints.

Cybersecurity Encyclopedia

Backup Design for Recovery

Design backups that actually restore: offline and immutable copies, clear recovery objectives, and rehearsed restore …

Beginner 8 min read
Calm cybersecurity illustration for Containment Decision Trees, showing abstract ransomware and recovery evidence cards, connected systems, and defensive control checkpoints.

Cybersecurity Encyclopedia

Containment Decision Trees

A calm guide to containment decisions: when to isolate a machine, how to keep evidence intact, and who to inform before …

Intermediate 9 min read
Calm cybersecurity illustration for Ransomware Timeline, showing abstract ransomware and recovery evidence cards, connected systems, and defensive control checkpoints.

Cybersecurity Encyclopedia

Ransomware Timeline

Learn how a ransomware incident typically unfolds, from the first odd sign to restored systems, with calm defensive …

Beginner 8 min read