Technology and data integration workshop

For those of you who have delivered a system integration — whether it's two applications talking to each other or a full ERP landscape with dozens of touchpoints — you will know that integration is where most transformation programmes actually live or die. The individual systems are rarely the problem. It's the joins between them.

Let me explain a little further. An integration project, at its simplest, is a promise that data or process will flow correctly from one system to another, in the right format, at the right time, without anyone in the business noticing the join at all. The moment someone does notice — a duplicated record, a failed batch, a silent drop — that's the moment confidence in the whole programme starts to erode.

So why do integration projects go wrong so often? Put simply, the integration points are usually the last thing properly scoped and the first thing to slip. Individual system owners will confidently tell you their component works, and they're often right — in isolation. What nobody owns end to end is the handoff between them, and that's exactly where I've seen otherwise well-run programmes come unstuck.

Four things to get right

To manage this, I recommend four things:

  • Map every integration point before you build anything. On the Barclays Wealth Direct programme there were 95 separate integration points across multiple systems, and every one of them had to be planned individually — not assumed to work because the two systems either side of it worked.
  • Assign a single owner to the integration layer itself. Not the sending system, not the receiving system — someone accountable for the join, with the authority to hold both sides to the same standard.
  • Test integration early and continuously, not as a phase that happens once development is "done." Integration defects found in isolation are cheap; integration defects found in end-to-end testing are expensive; integration defects found in production are the ones that end up in a board pack.
  • Build in data integrity checks, not just connectivity checks. A successful connection between two systems tells you nothing about whether the data crossing it is actually correct — I've seen migrations pass every technical test and still hand over broken data, because nobody checked the numbers matched on both sides.

In reality, most integration failures I've dealt with weren't technical at all — they were governance failures. Nobody owned the join, so nobody noticed it was broken until the business did.

I'd also add: don't underestimate the vendor management dimension. Where multiple suppliers each own a piece of the landscape, you need one party — usually the client, sometimes a delivery partner — holding the overall integration architecture, or you end up with several vendors each correctly delivering their piece of a picture nobody has actually agreed on.

As ever, the right people matter as much as the right process. Whoever leads integration testing needs to understand both the technical mechanics of the interfaces and the business outcome those interfaces are supposed to protect — lose either half of that, and integration becomes a checkbox exercise rather than the thing actually holding your transformation together.

Written by James Brodie. Talk to James about an integration or migration programme →