Beyond Reactive IT

How a Single Connected Operating Model Closes the Seams That Reactive IT Leaves Open

A field study in business email compromise, the anatomy of a $180,000 loss, and what it teaches about the structure of modern IT

By: Chris Wall, Chief Information Officer

About this paper

This whitepaper draws on a real incident handled by Universal Systems Inc. The client organization, its operating brands, the impersonated vendor, and all individuals involved have been anonymized, and identifying figures have been generalized, to protect confidentiality and any matters that may be subject to legal privilege or active insurance claims. The technical sequence, the failure points, and the remediation described here are faithful to what actually happened. Nothing in this paper has been embellished to flatter the responder, including where the responder was us. The value of the case lies precisely in its honesty.

Executive summary

A mid-market acquisition holding company, operating several consumer-facing brands under one corporate umbrella, lost approximately sixty-six thousand dollars to a business email compromise. The money left through a fraudulent vendor banking change that looked, to the person who approved it, completely routine.

Every individual security control performed roughly as designed. Email filtering was in place. Endpoint protection was deployed. A managed IT provider was monitoring the environment and, in fact, detected and contained the compromised mailbox at the center of the fraud. Backups were intact. The help desk was responsive. By any conventional checklist, this was not an unprotected organization.

And yet the loss happened anyway, because the failure was not inside any single function. It happened in the seams between them. A mailbox compromise was detected and closed as a technical event by one team. A fraudulent payment was being worked as a finance problem by another. A vendor banking change was approved by a third. No one owned the relationships between those three facts, and the one piece of evidence that tied them together, a hidden inbox rule filtering the exact vendor domain involved in the fraud, sat in a closed ticket while the wire was being approved.

The core finding

This was not a technology failure. It was an operating-model failure. Four competent functions, leadership, operations, infrastructure, and lifecycle management, were running as separate disciplines that shared no common picture of reality. The loss occurred in the space between them, which is exactly where modern losses occur.

This paper reconstructs the incident in detail, identifies precisely where the structure failed, including where our own response fell short, and sets out the operating model that prevents this class of loss. The conclusion for any executive or owner is direct. You will not buy your way out of reactive IT with another tool. You engineer your way out of it by bringing leadership, operations, infrastructure, and lifecycle management into a single connected model where a fact known by one function is visible to the others before it becomes a loss.

Part one: anatomy of a routine-looking loss

The setup

The client is a holding company that acquires and operates regional service businesses, running several brands on a single shared Microsoft 365 tenant with one accounts payable function serving all of them. This is a common and sensible structure for an acquisition platform. It is also a structure that quietly concentrates risk, because a single compromised identity and a single payment process now sit upstream of multiple operating companies at once.

The organization had an ongoing vendor relationship with a national media company. Payments to that vendor were normal, recurring, and unremarkable. That ordinariness is important. Business email compromise does not succeed by being exotic. It succeeds by being boring enough to clear an approval queue without a second look.

The intrusion

Beginning in early April, an attacker gained access to the Microsoft 365 mailbox of an employee who handled vendor correspondence. The initial access vector was a stolen password. The tenant at that time was running on basic security defaults, applying the same rules to every user and adapting to nothing. Multi-factor authentication was not enforced on every sign-in, and there were no conditional access policies evaluating sign-in risk in real time.

Over roughly three weeks, log review later confirmed successful sign-ins to that single mailbox from multiple locations the employee had never been, including cities across the United States and, most tellingly, Cape Town, South Africa. None of those sessions were the employee. A modern identity policy would have challenged or blocked a successful authentication from an impossible location automatically. Nothing did.

Failure point one: identity

A stolen password was sufficient to fully own a mailbox. The single most effective control against this entire class of attack, phishing-resistant multi-factor authentication backed by risk-based conditional access, was not in force. This is an infrastructure decision that had never been connected to a leadership risk decision.

The patience

What the attacker did next is the part that separates a nuisance from a six-figure loss. Rather than blasting out spam, they went quiet and got surgical. They created a hidden inbox rule inside the compromised mailbox. Any incoming message containing the legitimate vendor's domain was automatically moved out of the inbox and marked as read. From that moment, the real employee was blind to the very conversation that was being hijacked in their name. The thread was being run by someone else.

The attacker had also pre-positioned a lookalike domain, a near-perfect visual twin of the real vendor's domain with a single character substituted, the kind of swap a busy human eye reads straight past. The fraudulent correspondence flowed from that lookalike domain, while the inbox rule suppressed any legitimate reply that might have exposed the deception. Evidence indicated read access on the vendor's side of the conversation as well, meaning the attacker could time messages against the real thread. This was not opportunistic. It was disciplined, two-sided, and rehearsed.

During the active window, more than two hundred outbound messages moved through the compromised account, concentrated on invoices and payment instructions. At the right moment, the attacker delivered a revised invoice and a request to update the vendor's banking details on file. It arrived inside an established, trusted thread, referencing a real relationship, in the middle of normal business.

The approval

The banking change was approved internally and processed by accounts payable. There was no verbal callback to a known-good number to confirm the change before money moved. That single missing control, a phone call to a number already on file rather than a number supplied in the email, is the proximate cause of the financial loss. Approximately sixty-six thousand dollars was wired to an account controlled by the attacker.

Failure point two: process

There was no written policy requiring out-of-band verbal verification of vendor banking changes before payment. The control that would have stopped the loss at the last possible moment costs nothing and takes ninety seconds. Its absence was a lifecycle and governance gap, not a technology gap.

Part two: where the structure actually failed

It would be easy, and wrong, to end the story at a stolen password and a missing phone call. Those were the mechanics. The reason the loss reached six figures is structural, and it is the entire point of this paper.

The detection that should have stopped everything

Here is the detail that makes this case worth writing about. The managed IT provider monitoring the environment detected the mailbox compromise and executed a clean technical containment. Within a tight window on a single evening, they reset the password, revoked active session tokens, removed the multi-factor registration to force secure re-enrollment, and deleted the malicious inbox rule. As a discrete piece of incident response, it was fast and competent.

But that inbox rule was not generic. It was scoped to the exact domain of the exact vendor that was, at that very moment, the subject of an active payment-fraud conversation. A precision rule filtering one specific vendor's email is not a routine cleanup artifact. It is a flare. The natural next questions for anyone who sees it are immediate and obvious: what is our financial exposure to this vendor, has anything been paid recently, and has there been any change to their banking details?

Those questions were not asked at the moment of detection. The mailbox compromise was triaged, contained, and closed as a self-contained technical event. The fraud was being worked separately as a finance and leadership problem. The two streams did not meet for roughly three weeks. By the time someone connected the inbox rule to the wire, the money was long gone.

Failure point three: the seam

The single most damaging failure was not a missing control. It was a missing connection. The fact that proved a fraud was in flight existed inside a closed technical ticket, while the people approving the payment never saw it. This is an information-flow failure between operations, infrastructure, leadership, and the finance function. It is the signature failure of siloed IT.

An honest accounting

Universal Systems Inc. handled the forensic investigation and remediation of this incident, and we will detail that work in the next section because it genuinely mattered to the outcome. But credibility requires saying plainly that the initial response, ours included, made the central mistake this paper is about. The mailbox compromise was closed without connecting it to the fraud it was evidence of. We name that not to flagellate, but because the lesson is the product. A responder who will not admit where the seam opened cannot credibly claim to close it.

The deeper truth is that no individual was negligent. The monitoring team did its job. The finance team did its job. Leadership did its job. Each function operated competently inside its own boundary. The loss lived in the boundaries themselves, in the handoffs that no one owned, in the absence of a single shared picture that would have made one team's flare visible to another team's decision. That is not a people problem you fix by trying harder. It is a structure problem you fix by changing the model. 

Part three: how Universal Systems closed it out

Once the streams were connected and the full scope was understood, the work fell into three layers: contain the technical compromise completely, reconstruct the truth to a standard that would hold up for insurers and counsel, and change the structure so the same seam could not open again. The first response had been fast but incomplete. The follow-through is where the actual recovery of control happened.

Technical containment and infrastructure hardening

Beyond the initial mailbox cleanup, the environment was hardened to remove the conditions that allowed the intrusion in the first place:

•     Enforced multi-factor authentication across the tenant, replacing the basic security defaults that had treated every user identically.

•     Deployed risk-based conditional access policies that evaluate sign-in risk in real time and challenge or block authentications from impossible or anomalous locations.

•     Blacklisted the attacker's lookalike domain at the tenant level, searched for additional lookalike domains positioned against the organization, and reported the malicious domain to its registrar.

•     Audited executive and finance-adjacent accounts for the same indicators of compromise, and reviewed the affected account for persistence mechanisms beyond the inbox rule, including consent grants, forwarding rules, and delegated permissions.

Forensic reconstruction

Speed of containment is only half of incident response. The other half is the ability to prove what happened, in order, with evidence, to people who will scrutinize it adversarially. We reconstructed the complete attack chronology from message trace data and preserved original email files, established the two-sided nature of the compromise, and confirmed which outbound messages were authored by the legitimate user versus the attacker. The result was a documentation package precise enough to support an insurance claim and any action by counsel, with confirmed facts clearly separated from inferences. When real money is gone, a defensible narrative is not paperwork. It is the difference between a recoverable claim and a denied one.

Process and lifecycle controls

The loss was ultimately enabled by a missing finance control, so technical fixes alone would have been a half-measure. The remediation reached into process:

•     A written vendor banking-change policy requiring out-of-band verbal verification to a known-good number on file before any payment instruction is altered.

•     Notification to the impersonated vendor, through a verified channel rather than the compromised thread, so they could investigate their own exposure.

•     Security awareness training tied directly to this incident, using it as the concrete, local example of why the controls exist, rather than an abstract threat from a stock slide.

The pivot

The same incident that exposed a siloed operating model became the catalyst for a connected one. Identity, monitoring, finance process, and leadership oversight were brought under a single coordinated program with shared visibility. The fix for a seam failure is not a better tool in one quadrant. It is a model that spans all four.

Part four: the connected operating model

Reactive IT is what you get when leadership, operations, infrastructure, and lifecycle management run as four separate disciplines that meet only when something breaks. Each optimizes for its own world. Each reports itself healthy. And the organization stays exposed, because no one is accountable for the relationships between them. The incident in this paper is that pattern rendered in money.

The alternative is to run the four as one model, where a decision or a discovery in any quadrant is visible and accountable in the other three. Defined plainly:

Map the incident onto this model and the failures line up one to one. Identity was an infrastructure decision never connected to a leadership risk decision. The missing callback was a lifecycle and governance gap. And the fatal delay was a seam between operations and the rest, a flare raised in one quadrant that never reached the others. Run as four programs, these gaps are invisible until a loss makes them visible. Run as one model, they surface as routine questions before money moves.

What changes when the model is connected

•     Onboarding a vendor automatically creates a lifecycle record, a named risk owner, and a review date, so an unverified banking change cannot quietly clear a queue.

•     A security detection is evaluated against business context, so a precision inbox rule targeting a specific vendor immediately triggers the question of financial exposure to that vendor.

•     A renewal or an end-of-life date feeds the budget conversation a quarter ahead, instead of arriving as a surprise invoice.

•     When something does go wrong, the response is fast because every function already shares one picture, instead of assembling one under pressure after the loss.

This is not a bigger team. It is a clearer one.

The common objection is that this sounds like enterprise overhead a lean company cannot afford. The opposite is true. A large organization can paper over a siloed structure with headcount at every seam. A fifty or five-hundred-person company cannot, and a shared-services acquisition platform least of all, because one identity and one payment process sit upstream of every brand it owns. For an organization without a deep bench, structure is the substitute for size. The connected model does not require more people. It requires deciding who owns each discipline, making the handoffs explicit, and maintaining one shared source of truth about what you have, what it costs, what protects it, and when it changes.

Part five: where to start on Monday

You do not fix this with a reorganization or a new platform. You fix it by making the seams visible and assigning them owners. Five concrete moves, in order:

1.     Build one inventory the whole organization trusts. Hardware, software, cloud tenants, identities, and vendors in one place. You cannot connect four disciplines around assets you cannot see.

2.     Enforce phishing-resistant multi-factor authentication and risk-based conditional access on every identity, today. This is the control that would have stopped the incident in this paper at the front door, before any of the rest was possible.

3.     Put a written out-of-band verification policy on every vendor banking or payment change. A verbal callback to a known-good number is the cheapest control in this entire paper and the one that would have stopped the loss at the last moment.

4.     Name an owner for every lifecycle and risk event. Every renewal, every end-of-life date, every vendor review, every open security finding needs a human name attached, not a department.

5.     Run one recurring review where all four disciplines sit at the same table. Not a status meeting. A working session where handoffs are inspected on purpose, before they fail on their own. This is the meeting that would have connected the inbox rule to the wire in three hours instead of three weeks.

The bottom line

Reactive IT is not a discipline problem or a budget problem. It is the predictable result of running four functions that never share a picture of reality. The wire fraud, the unauthorized change, the surprise renewal, the audit finding: in our experience these almost never come from a single failed control. They come from the unowned space between competent functions.

Bring leadership, operations, infrastructure, and lifecycle management together into one model, and that space stops being a blind spot. It becomes the part of the organization you can actually see and steer. A holding company lost sixty-six thousand dollars not because it lacked tools, but because the fact that could have saved the money lived in one silo while the decision that lost it lived in another. Close the seam, and you close the loss. That is the whole game.

About the author

Chris Wall is the Chief Information Officer at Universal Systems Inc., a full-stack IT solutions and managed services provider based in Salt Lake City, and serves as the acting security leader across a portfolio of operating companies. He writes about IT leadership, security, and building operating models that scale.