Don't Just Patch Cables, Program Reactions: Your Randori Feedback Loop

You've spent hours patching cables, configuring VMs, and making sure every sensor talks to every log collector. The network topology looks like a work of art. But when the randori exercise starts, something feels off. The red team moves in ways you didn't anticipate, alerts fire out of order, and your team freezes. Sound familiar?

The problem isn't your cables. It's that you're treating randori like a one-time setup instead of a continuous learning loop. The real power of adversary simulation comes not from the initial configuration but from how you program your reactions—how you capture what you observe, turn it into feedback, and adjust before the next round. This guide is for beginners who have run a few exercises and want to move from 'we survived' to 'we improved.' We'll show you how to build a feedback loop that makes each session sharper than the last.

Why the Feedback Loop Matters More Than the Initial Setup

Most beginners pour energy into the pre-exercise setup: choosing the right tools, mapping the network, scripting initial conditions. That work is necessary, but it's not sufficient. The difference between a randori that teaches you nothing and one that transforms your team is what happens after the red team goes home.

The Trap of Static Exercises

Without a feedback loop, you run the same exercise over and over. The red team uses the same TTPs, the blue team responds the same way, and you learn nothing new. The initial setup becomes a crutch—you blame the topology when things go wrong instead of examining your decision-making process.

Why Reactions Are Programmable

A reaction isn't just a gut feeling. It's a sequence: observe, classify, decide, act. In randori, you can program each step. For example, when an alert fires (observe), you tag it as high or low severity (classify), choose a response playbook (decide), and execute (act). The feedback loop lets you refine each step based on what actually happened. Did the classification rule miss something? Was the playbook too slow? You adjust, and next time the reaction is better.

The stakes are real. Industry surveys suggest that teams who debrief after every exercise improve their mean time to detect by 30-40% over five sessions compared to teams who don't. Without a loop, you're just practicing mistakes.

Core Idea: The Observe-Orient-Decide-Act (OODA) Loop for Randori

The feedback loop we're building is a specialized version of the OODA loop, adapted for randori. The key is to make each stage explicit and measurable.

Observe

During the exercise, capture everything: which alerts fired, who responded, what decisions were made, how long each step took. Use a shared document or a simple log. Don't rely on memory—write it down in real time.

Orient

After the exercise, classify each observation. Was this a known weakness? A new TTP? A procedural error? Group observations by type and severity. This is where you separate signal from noise.

Decide

Choose what to change. Not everything needs fixing. Prioritize based on frequency (how often did this happen?) and impact (how bad was the outcome?). For each item, decide on one concrete action: update a detection rule, rewrite a playbook, add a sensor, or train a specific skill.

Act

Implement the change before the next exercise. Then test it in the next round. The loop closes when you observe whether the change worked.

This framework turns randori from a one-off event into a continuous improvement engine. The initial setup matters, but the loop is what generates long-term value.

How It Works Under the Hood: Building Your Feedback Pipeline

Let's get specific about the mechanics. You'll need three components: a collection mechanism, a classification system, and a change tracker.

Collection Mechanism

During the exercise, assign one person (not the incident commander) to be the scribe. Their only job is to log events: timestamp, event type, who acted, what happened. Use a simple spreadsheet or a shared note. Don't try to analyze during the exercise—just collect raw data.

Classification System

After the exercise, sort each event into one of four buckets:

• Detection gap: an attack that generated no alert
• Alert fatigue: too many alerts, real signal missed
• Process failure: the team knew what to do but didn't execute
• Knowledge gap: the team didn't know the right response

Each bucket points to a different fix. Detection gaps mean you need new rules or sensors. Alert fatigue means you need to tune thresholds or aggregate alerts. Process failures mean you need better playbooks or training. Knowledge gaps mean you need to study the TTP.

Change Tracker

Create a simple table with columns: observation, bucket, action, owner, deadline, status. Review it before each exercise. This turns your feedback loop into a project management tool.

One team I read about used this system for three months. They started with 20 observations per exercise, mostly detection gaps. By the end, observations dropped to 5 per exercise, and most were process failures—a sign that their detection had improved and they were now fine-tuning response.

Worked Example: A Full Feedback Loop in Action

Let's walk through a composite scenario to see how this works in practice.

Setup

Your team runs a randori exercise with a simple network: a web server, a database, and a workstation. The red team uses a phishing email to gain access to the workstation, then moves laterally to the database.

Observation Phase

During the exercise, the scribe logs:

• 10:02 – Phishing email detected by gateway, alert fired
• 10:05 – User clicked link, no alert on endpoint
• 10:12 – Beacon detected on workstation (ETW alert)
• 10:20 – Analyst investigated beacon but didn't contain it
• 10:35 – Lateral movement to database detected (network alert)
• 10:40 – Team realized the workstation was still compromised
• 10:50 – Exercise ended, database data exfiltrated

Orientation Phase

After the exercise, the team classifies each event:

• Phishing detection: detection gap (user clicked despite alert—no endpoint alert for execution)
• Beacon detection: process failure (analyst didn't contain)
• Lateral movement: detection gap (network alert worked but was too late)
• Overall: two detection gaps, one process failure

Decision Phase

The team decides on actions:

• For the phishing execution gap: add a rule to detect process creation from email client (action, owner: engineer A, deadline: next week)
• For the containment failure: update the playbook to include immediate host isolation (action, owner: team lead, deadline: this week)
• For the lateral movement gap: research network segmentation options (action, owner: architect, deadline: next month)

Action Phase

The team implements the first two changes before the next exercise. In the next round, the red team tries the same phishing attack. This time, the endpoint alert fires, the analyst isolates the workstation, and lateral movement is blocked. The feedback loop worked.

Edge Cases and Exceptions

No feedback loop is perfect. Here are common edge cases and how to handle them.

When the Red Team Changes TTPs Between Rounds

If the red team shifts tactics, your previous observations may not apply. That's okay—the loop still works. You're not trying to predict every attack; you're improving your general detection and response. If the red team changes, you'll get new observations. The loop adapts.

When the Team Is Too Small to Have a Dedicated Scribe

In a small team, everyone is hands-on. Record the exercise (with permission) and review the recording afterward. Or use automated logging tools that capture alert timelines and analyst actions. The key is to collect data without overloading the team during the exercise.

When the Same Issue Keeps Appearing

If you see the same detection gap in three consecutive exercises, your fix isn't working. Revisit the action: maybe the rule is too narrow, or the sensor placement is wrong. Don't assume that implementing a change once is enough—the loop requires iteration.

When the Exercise Is Too Short to Gather Meaningful Data

Short exercises (under 30 minutes) may produce only a handful of observations. That's fine. Aggregate observations across multiple short exercises. After three or four, you'll have enough data to spot patterns.

Limits of the Approach

This feedback loop is powerful, but it has limits. Knowing them helps you use it wisely.

It Doesn't Replace Strategic Planning

The loop is tactical. It helps you improve specific detections and responses. It won't tell you whether your overall security strategy is sound. You still need to step back periodically and ask bigger questions: are we defending the right assets? Is our risk model accurate?

It Can Create a False Sense of Progress

If you fix every observation from the last exercise, you might feel like you're improving. But the red team can always find new angles. The loop measures improvement against past performance, not against all possible attacks. Stay humble.

It Requires Discipline to Close the Loop

The hardest part isn't collecting observations or deciding on actions—it's actually implementing the changes before the next exercise. Without discipline, the loop breaks. Assign owners and deadlines, and hold people accountable.

It's Not a Substitute for Real Incident Response

Randori exercises simulate attacks, but they're not the same as a real incident. The feedback loop prepares you, but real incidents bring pressure, uncertainty, and consequences that exercises can't fully replicate. Use the loop to build habits, not to certify readiness.

Reader FAQ

How often should I run the feedback loop?
After every exercise, no matter how short. Even a 15-minute exercise produces useful observations. The more frequently you close the loop, the faster you improve.

What if my team resists debriefs?
Make debriefs blame-free. Frame every observation as a system issue, not a person issue. Use language like 'the detection rule missed this' instead of 'you missed this.' Over time, the team will see debriefs as a tool for improvement, not criticism.

Should I involve the red team in the feedback loop?
Absolutely. The red team can provide valuable context about why they chose certain TTPs and what they observed about your defenses. Include them in the orientation phase, but keep the decision phase internal to the blue team.

How do I measure improvement?
Track metrics like time to detect, time to contain, and number of missed detections per exercise. Over several rounds, you should see trends. But don't obsess over numbers—qualitative improvements (the team communicates better, playbooks are clearer) matter too.

What if we don't have time for a full debrief?
Do a mini debrief: 10 minutes, three questions. What went well? What went wrong? What will we change? Even that short loop is better than nothing.

Can I automate parts of the loop?
Yes. Tools that log analyst actions and alert timelines can automate the observation phase. Some platforms even generate post-exercise reports with suggested improvements. But don't automate the decision phase—that requires human judgment.

Practical Takeaways

You don't need a perfect initial setup to run a great randori program. What you need is a feedback loop that turns each exercise into a learning opportunity. Here's your next move:

1. Assign a scribe for your next exercise. Their only job is to log events.
2. Run a 15-minute debrief immediately after the exercise. Use the four buckets to classify observations.
3. Choose one action to implement before the next exercise. Make it small and concrete.
4. Track it in a simple spreadsheet. Note the owner and deadline.
5. Repeat. After three exercises, review your tracker. You'll see a pattern of improvement.

The cables and topology are just the stage. The real show is how you react, learn, and adapt. Program your reactions, and your randori practice will never be static again.