How to Choose the Right Integration Method

Almost every integration mistake I have been called in to unwind started the same way: someone picked the method first and matched it to the problem afterwards. A team standardised on real-time APIs because that felt modern, then spent a year fighting nightly reconciliation jobs that a simple batch file would have handled in ten lines. Another queued every event through a message broker, then discovered their trading partner only accepted a flat file over SFTP. The methods were not wrong in the abstract. They were wrong for the job. Choosing the right integration method is less about technology depth and more about asking the right five questions before you write a single line of code. This guide gives you those questions, a decision flow to walk them through, and a selection matrix that maps common needs to the method that fits.

Start with the pillar: this article assumes you already understand the landscape of integration patterns and terminology. If you want the full map first, read Enterprise System Integration Explained, then come back here to make the actual choice. The pillar tells you what the methods are; this guide tells you how to pick between them.

1. There is no single right method

The first thing to unlearn is the idea that one integration style is superior. Consultants and vendors have strong incentives to sell a house style, whether that is an API-first mandate, an event-driven architecture, or an integration platform subscription. In real environments, a healthy landscape almost always runs several methods at once. The finance team pulls a nightly general-ledger extract as a batch file. The web storefront calls an inventory API in real time. The warehouse emits an event when a shipment leaves the dock. A B2B partner exchanges purchase orders over EDI because that is the only thing their fifteen-year-old ERP speaks. All four are correct, because each is matched to a different combination of timing, volume, direction and partner constraint.

The reason this matters is cost and fragility. Force a real-time API onto a bulk data-migration job and you get a chatty, slow, rate-limited nightmare that hammers the source system. Force a nightly batch onto a customer-facing stock check and you get sold-out items still showing as available for twenty hours. The method is not good or bad on its own; it is good or bad for a specific shape of problem. So the useful skill is not memorising every pattern. It is learning to read the shape of the problem and reach for the method that fits it. That reading comes down to a small set of questions.

2. The questions that decide it

Every method choice I make runs through the same five questions, in roughly this order. None of them require deep technical knowledge to answer. They require you to be honest about what the business actually needs, which is usually harder.

• Timing: how fresh does the data need to be? Does the receiving system need the data within seconds, or is within hours acceptable? A fraud check needs the transaction now. A monthly management report does not care whether the numbers arrived at 2am or 4am. Timing is the single biggest determinant of method, because it separates the real-time family (APIs, webhooks, streaming) from the scheduled family (batch, ETL, file transfer). Be ruthlessly honest here, because "real time" is the most over-specified requirement in integration. Most things people call real time are comfortably served by a five-minute or hourly cycle.
• Volume: how much data moves, and how often? Are you moving one record when a customer clicks save, or five million rows every night? Small, frequent, transactional payloads favour APIs and events. Large, periodic bulk movement favours batch and ETL. Trying to push a five-million-row load through a per-record API is one of the most common and most painful anti-patterns I see, and it usually surfaces as a source system falling over under the request load.
• Direction: who initiates, and does data flow one way or both? Does the source push when something changes, or does the target pull on a schedule? Is it a one-way feed or a two-way sync? Push-on-change points you at webhooks and events. Pull-on-schedule points you at batch and polling APIs. Two-way sync, where both systems can change the same record, is the hardest case and needs deliberate conflict handling regardless of method.
• Partner: is the other side internal or an external trading partner? Integrating two systems you control gives you free choice of method. Integrating with an external partner, a bank, a supplier, a government portal, a logistics provider, means you take whatever they support, and that is frequently EDI, a fixed file format, or a specific API contract you cannot change. The partner constraint often overrides every other consideration, so establish it early before you design anything.
• Skills: what can your team actually operate? The most elegant streaming architecture is worthless if nobody on the team can run a broker at 3am when it stops consuming. Match the method to the skills you have or can realistically acquire. A small team with strong SQL and no platform engineers is better served by well-built batch ETL than by a distributed event mesh they cannot support. Sustainability beats sophistication every time.

Answer those five and you have narrowed a dozen options down to two or three. The decision flow below turns the answers into a recommendation. For the deeper trade-off between the two big timing families, the companion guide on batch versus real-time integration covers that split in detail.

3. A decision framework

Here is the same logic as a flow. Start at the top, answer each question about the specific interface you are designing, and follow the branch to a recommended method. Real integrations sometimes land between two boxes, and that is fine; the flow is meant to get you to the right neighbourhood, not to remove judgement.

Notice that the flow never asks "which technology is best." It asks about the shape of the requirement and lets the shape choose. That is the whole discipline. When you find yourself between two boxes, fall back to the two tie-breakers that matter most in operations: which method is cheaper to run, and which one your team can support at 3am. Those two questions resolve most genuine ties.

4. The selection matrix

The flow is good for a single interface. When you are planning a whole landscape, a matrix is faster: identify the business need, read across to the recommended method and the reason. These are the five needs that cover the large majority of integration work I encounter.

The matrix and the flow will disagree occasionally, and when they do it is usually because a real requirement carries two needs at once. A customer order both needs a real-time confirmation (API) and needs to trigger a downstream warehouse event (webhook or message). That is not a contradiction; it is two interfaces on the same business process, each choosing its own method. Splitting a process into its distinct interfaces and picking a method per interface is the move that keeps large landscapes clean.

5. When to use APIs and webhooks

APIs and webhooks are the two halves of real-time integration, and confusing them is a common source of over-engineering. An API is a request-response contract: the caller asks, the provider answers immediately. Reach for a synchronous API when the caller needs the answer to proceed, when the data is a single current record or a small set, and when the interaction is genuinely on demand rather than continuous. A checkout validating a payment, a portal fetching an account balance, a mobile app submitting a form: all classic API territory. The cost to watch is chattiness. If a client has to call an API in a loop to find out whether anything changed, you have chosen the wrong method; that is polling, and it wastes calls and adds latency.

A webhook inverts the direction. Instead of the consumer asking repeatedly, the producer calls the consumer when something happens. Reach for a webhook when the source knows when an event occurs and the consumer only cares at that moment: payment confirmed, subscription cancelled, document signed. Webhooks eliminate wasteful polling and deliver events promptly, which is why they dominate modern SaaS integration. Their weakness is delivery reliability: the receiver must be online, and you need retry and idempotency handling because the same event can arrive twice. When event volume climbs or guaranteed delivery becomes non-negotiable, that is the signal to graduate from raw webhooks to a proper messaging layer.

The trap to avoid: do not use polling APIs as a substitute for events. A job that calls an API every minute asking "has anything changed" is a webhook you have not built yet. It burns rate limits, adds up to a minute of latency, and scales badly as you add sources. If the source can push, let it push. Polling is a fallback for when the source genuinely cannot emit events, not a default.

There is a useful middle path worth naming, because teams often miss it. Many modern APIs expose both a request-response endpoint and a webhook mechanism from the same product, and the right design frequently uses both together: the webhook tells you something happened, and an API call fetches the full, authoritative detail. The webhook payload stays small and acts as a trigger, and the follow-up API read guarantees you have the current state even if an event was delayed or delivered out of order. That pairing gives you the promptness of push with the correctness of pull, and it sidesteps the classic problem of trusting an event payload that may be stale by the time you process it. When someone asks whether to use an API or a webhook, the honest answer is often both, each doing the part it is good at.

6. When to use batch, ETL and files

The scheduled family is unfashionable and quietly indispensable. Batch, ETL and file transfer move data on a timetable rather than on demand, and for a large share of real-world integration that is exactly right. Reach for batch or ETL when the volume is high, when freshness within hours is acceptable, and when the receiving system benefits from getting a validated, transformed set all at once rather than a trickle of individual records. Nightly finance consolidations, data-warehouse loads, master-data synchronisation, system migrations: these are batch problems, and trying to force them into real-time interfaces is how you overload source systems and complicate what should be simple.

Plain file transfer, a CSV or XML dropped onto SFTP on a schedule, is the humblest method on the list and still one of the most widely used in enterprise reality, especially with external parties and legacy systems that cannot expose an API. It is transparent, easy to audit, trivial to reprocess when something goes wrong, and understood by every team. Do not be embarrassed to recommend a file when a file is what fits. The sophistication of a method is not a measure of its correctness. I have replaced brittle bespoke API integrations with a nightly file more than once, and the file was more reliable, cheaper to run and easier for the client to support. The batch versus real-time comparison goes deeper on where the scheduled family beats the real-time family and why the answer surprises people.

7. When to use messaging and streaming

Messaging and streaming sit between the request-response world and the batch world, and they solve a specific problem: decoupling. When you put a message broker or an event stream between two systems, the producer no longer needs the consumer to be available, and one producer can feed many consumers without knowing who they are. Reach for messaging when events must not be lost, when the event rate is high enough that direct webhooks become fragile, when you need to buffer bursts so a slow consumer does not fall over, and when several downstream systems care about the same event. An order placed might need to reach inventory, billing, fulfilment and analytics; a broker lets each subscribe independently instead of the order system calling four APIs and owning four failure modes.

Streaming is the high-throughput, continuous cousin, suited to telemetry, clickstreams, sensor data and any firehose of events where you also want to process the flow in near real time. The power is real, and so is the operational weight. A broker or stream platform is infrastructure you must run, monitor, secure and recover, and it introduces genuinely harder reasoning: ordering, exactly-once versus at-least-once delivery, consumer lag, replay. This is precisely where the skills question earns its place in the framework. Messaging is the right answer for high-volume, decoupled, guaranteed-delivery event flows, and the wrong answer for a team that cannot operate it or a problem that a webhook would have solved. Match the ambition of the method to the capability of the people who will carry it at 3am.

8. When to use iPaaS versus building it yourself

Every method above can be hand-built with code, or assembled on an integration platform as a service (iPaaS) that provides pre-built connectors, visual flow design, hosting and monitoring out of the box. This is an orthogonal choice: iPaaS is not a method, it is a way to deliver the methods. The decision hinges on connector coverage, team capacity and the number of integrations you expect to run. Lean toward iPaaS when you have many integrations to build across popular SaaS applications, when a small team needs to move quickly without owning integration infrastructure, and when the platform already has connectors for your systems so you assemble rather than code. The productivity gain is real, and so is the reduction in operational burden.

Lean toward building it yourself when the integration is few in number but deep and specific, when it sits in a performance-critical or highly regulated path, when your systems are bespoke enough that no connector exists, or when platform subscription and per-transaction costs would outweigh the engineering saved. Many mature landscapes end up with both: an iPaaS handling the long tail of SaaS-to-SaaS connections, and hand-built integrations for the few core, high-volume, differentiated flows. For a full treatment of what iPaaS is and where it fits, see the iPaaS guide. And once you have chosen methods for each interface, the discipline of assembling them into a coherent whole is its own subject, covered in how to design integration architecture. If your landscape centres on Microsoft Dynamics 365 Business Central, the specifics of its APIs and integration options are worth reading in the Business Central integrations guide, which grounds all of this in one concrete ERP.

9. References

The framework above reflects widely accepted integration practice. For the underlying concepts and standards, these are the canonical sources worth knowing:

• Hohpe, G. and Woolf, B., Enterprise Integration Patterns (Addison-Wesley). The reference catalogue for messaging patterns, still the standard vocabulary for message-based integration.
• Roy Fielding's doctoral dissertation defining REST, the architectural style behind most modern request-response APIs.
• The X12 and UN/EDIFACT standards bodies, which define the EDI document formats used for B2B trading document exchange.
• Gartner's market definition for Integration Platform as a Service (iPaaS), which frames the build-versus-platform decision covered above.
• For the surrounding landscape and terminology, the MAbbaz pillar Enterprise System Integration Explained.

Final thoughts

Choosing an integration method is not a test of how many patterns you can name. It is a test of how honestly you can read a requirement. Ask how fresh the data must be, how much of it moves and how often, who initiates and in which direction, whether the other side is a partner you must conform to, and what your team can actually operate. Those five answers narrow a dozen options to a clear favourite almost every time, and the decision flow and matrix above are just those questions arranged for speed. The methods themselves are all correct somewhere; the skill is putting each one where it fits and resisting the pull to standardise everything onto whichever style is currently fashionable.

When two methods still tie, let operations break the tie: pick the one that is cheaper to run and easier to support when it fails at night. A landscape built that way tends to be quiet, and quiet is the real mark of good integration. If you are staring at a decision like this on a specific interface and want a second opinion grounded in having made the choice many times, that is exactly the kind of conversation I am glad to have.

Related reading: Enterprise System Integration Explained, Batch vs real-time integration, What is iPaaS, How to design integration architecture, Business Central APIs and integrations.