AI on Legacy Systems: You Don't Need to Rip and Replace to Win

If you run a bank, an insurer, a hospital network or a government department, you've probably heard some version of this: "We can't really do AI until we modernise our core systems." It sounds responsible. It's usually wrong — and it's expensive.

Legacy systems take the blame for most failed enterprise AI. Gartner has projected that over 40% of agentic-AI projects will be abandoned by the end of 2027, in large part because companies try to bolt modern autonomy onto decades-old infrastructure. Around 46% of enterprises name integration with existing systems as their number-one AI barrier. But the lesson isn't "replace everything first." It's "stop trying to make the old system behave like a new one."

Rip-and-replace is the slow, risky path

Replacing a core banking platform, an EHR or an ERP is a multi-year, multi-million programme with its own failure rate. If your AI strategy depends on finishing that first, you've effectively cancelled your AI strategy. Worse, you've tied a fast-moving capability (AI) to your slowest-moving asset (the core system).

The systems you already run are not the enemy. They're where your data, your processes and your institutional knowledge live. The goal isn't to remove them — it's to build a thin, well-governed AI layer on top of them.

The upper-layer approach

An "AI upper layer" sits above your systems of record and does three things: it reads from them, reasons over them, and writes results back into them. Done well, the underlying systems barely notice — and your users get AI inside the tools they already use.

Whether this is feasible comes down to one make-or-break question:

Can we reach the data — through an API, a data warehouse, a database connection or a reliable export — or is it trapped behind a vendor with no way out?This single answer should set your timeline, not the age of the system.

Plenty of "legacy" systems are perfectly reachable. A mainframe with a documented interface, an ERP with an API, a warehouse like Snowflake or Databricks, even a nightly export — all of these are workable foundations. The real blocker is rarely age; it's accessibility. A modern SaaS tool with no export can be a worse foundation than a 20-year-old database you fully control.

Read, reason, write — without disruption

A practical upper-layer build usually looks like this:

• Read: connect to the systems where the relevant data lives, using whatever access exists — API, warehouse, DB, or a governed export pipeline.
• Reason: apply the right model to the task (extraction, triage, drafting, classification), with retrieval grounded in your own data so answers are traceable.
• Write back: push the result into the system of record — a status, a draft, a routed case — so a human isn't re-keying it by hand. This step is where most ROI is won or lost.

Crucially, none of this requires touching the core system's logic. You're adding a capability, not performing surgery on the patient.

Where access is the project

Sometimes the honest answer is that the data isn't reachable yet — it's locked in a vendor platform, captured on paper, or never recorded. That's not a dead end; it just means the first phase is a small data-access or integration step, not the AI build. Naming that prerequisite up front is the difference between a project that ships and one that stalls six weeks in.

Build for your risk and compliance teams, too

For regulated and public-sector work, "runs on top of your systems" should also mean "runs inside your perimeter." Deploy in your own cloud, VPC or on-prem; keep your data in your environment; add human-in-the-loop on people-affecting decisions; and design to GDPR, the EU AI Act and your sector's rules from the start. The integration approach and the governance approach are the same decision made twice.

The takeaway is simple: you almost certainly don't need to replace your stack to start winning with AI. You need to know what's reachable, build a layer on top of it, and write the results back where the work happens. That's the entire game.