What Is Data Synchronization?

Almost every integration problem, once you strip away the vocabulary, is a data synchronization problem. Your CRM and your ERP both hold customer records, and they need to agree. Your CAFM system and your finance ledger both track cost centres, and they need to agree. A field application collects readings offline and later reconciles them with the server, and the two copies need to agree. Data synchronization is the discipline of keeping copies of the same information consistent across systems as that information changes over time. This guide sits underneath the broader integration story told in the enterprise system integration pillar, and it drills into the one mechanism that most integrations ultimately depend on.

The message up front: synchronization is easy to demonstrate and hard to make correct. Copying a row from A to B works in a demo. Keeping A and B in agreement forever, through concurrent edits, out-of-order updates, deletes, and network failures, is a design problem with a small number of well-understood answers. Choosing the right pattern before you build is the entire difference between a sync that you can trust and one that silently drifts.

1. What data synchronization is

Data synchronization is the ongoing process of reconciling two or more copies of a dataset so that they reflect the same information, applying changes made in one copy to the others according to a defined set of rules. The key word is ongoing. A one-time export is not synchronization; it is a migration. Synchronization is what you build when both copies keep changing and you need them to stay in agreement over time, not just at a single moment.

A useful way to picture it: you have a source of truth, or several sources, and you have replicas or peers that must reflect it. A change happens somewhere. Synchronization is the machinery that notices the change, decides where it must be propagated, transforms it into the shape each target expects, applies it, and confirms the copies now agree. Every part of that sentence hides a design decision. How do you notice the change? How do you decide what propagates? What happens when two changes collide?

Synchronization shows up everywhere in enterprise systems. Master data such as customers, vendors, employees and assets is maintained in one system and synchronized to the others that consume it. Transactional data such as invoices, work orders and orders flows between operational systems and the finance ledger. Reference data such as tax codes and cost centres is distributed from a governing system outward. In every case the same problem repeats: the same logical record lives in more than one place, and those places must not disagree.

2. Why sync is harder than it looks

The demo version of synchronization is a loop that reads new rows from A and writes them to B. It works perfectly right up to the moment reality intrudes, and reality always intrudes in the same handful of ways.

Concurrent edits. The same record is changed in both systems before either change has propagated. A customer's phone number is updated in the CRM at the same time the address is updated in the ERP. Now both systems hold a version the other has not seen. Which one wins? If you naively let the last writer overwrite, you lose one of the two edits. This is the conflict problem, and it is the single hardest part of two-way synchronization.

Ordering. Changes do not always arrive in the order they were made. A record is created, then updated, then the update arrives at the target before the create because it took a faster path. Apply them in arrival order and you try to update a record that does not yet exist, or you apply an old value on top of a newer one. Correct synchronization has to reason about the order of changes, not just their content, usually with timestamps, version numbers or sequence identifiers.

Deletes. Creates and updates carry their own evidence: the row is there, with values. A delete is the absence of a row, and absence is hard to detect. If you sync by comparing what exists, a deleted record simply stops appearing, which is indistinguishable from a record that was filtered out or a query that failed. Handling deletes reliably usually means soft deletes, tombstone markers, or an explicit change log rather than snapshot comparison.

Partial failure. The network drops after you have written half a batch. Did the target receive the records or not? If you retry, will you duplicate the ones that did land? Robust synchronization has to be idempotent, meaning applying the same change twice has the same effect as applying it once, so that retries after failure are safe. Without idempotency, every network hiccup risks double-counting.

The honest caution: most synchronization bugs are not visible on the day you ship. They accumulate. A conflict resolved the wrong way here, a missed delete there, and six months later the two systems disagree in ways nobody can fully explain. By then the drift is spread across thousands of records and untangling it is forensic work. Design for correctness at the start, because retrofitting it onto a running sync is far more painful than building it in.

3. One-way versus two-way sync

The first and most consequential decision in any synchronization design is direction. Does data flow one way, from a source to a destination, or both ways between peers that can each originate changes? The two cases are not variations of one design; they are genuinely different problems with different difficulty levels.

In a one-way synchronization, one system is authoritative and the other simply follows. Changes flow in a single direction. The follower never originates a change that must propagate back, so there is nothing to collide with. There are no conflicts to resolve because the source always wins by definition. This is by far the simpler pattern, and where the business allows it, it is the one to choose.

In a two-way synchronization, both systems can originate changes, and each must reflect the other's. Now the conflict problem is unavoidable: the same record can change on both sides between sync cycles, and you need explicit rules to decide the outcome. Two-way sync is powerful because it lets users work naturally in either system, but it is significantly harder to build correctly, and it is where most synchronization projects underestimate the effort. The diagram below shows the essential difference, including the moment where a two-way sync must detect and resolve a conflict.

The practical guidance I give clients is to prefer one-way wherever the business genuinely has a single authoritative source for a given entity, and only accept two-way when both systems must legitimately originate changes to the same data. Two-way is not a badge of sophistication; it is a cost you take on when the workflow truly requires it. For the deeper treatment of this specific choice, see the dedicated one-way versus two-way sync guide.

4. Synchronization approaches

Beyond the basic one-way and two-way split, real designs fall into a few named topologies. The distinction is about where authority lives and how many systems can write. The table below sets out the common approaches, the direction of data flow, the conflict risk each carries, and where each typically fits.

Read the table as a difficulty ladder. Conflict risk rises as more nodes gain the right to write the same data. One-way and master-to-replica push authority to a single writer and are correspondingly simpler and safer. Two-way and multi-master distribute write authority and buy flexibility at the price of a conflict-resolution problem you must solve explicitly. The engineering effort is not proportional to how impressive the topology sounds; it is proportional to how many places can change the same record.

5. Conflict detection and resolution

Once more than one system can write the same record, conflicts are not an exception to handle someday; they are a certainty to design for now. A conflict occurs when the same logical record has been changed independently on two sides since they last agreed. Handling it has two parts: detecting that a conflict happened, and resolving it to a single agreed outcome.

Detection depends on knowing what each side last saw. The common mechanisms are version numbers that increment on every change, timestamps that record when a change was made, or content hashes that fingerprint the record's state. When a change arrives whose expected base version does not match the target's current version, you know the target has moved on since the source last saw it, and you have a conflict. Detection without one of these markers is guesswork.

Resolution is a policy decision, and there is no universally correct answer. The common strategies are:

• Last write wins. The change with the latest timestamp is kept, the other discarded. Simple and popular, but it silently loses the discarded edit, and it depends on clocks being trustworthy across systems, which they often are not.
• Source priority. One system is declared authoritative for a given field or entity, and its value always wins the conflict. Predictable and easy to reason about, which is why it is the workhorse of most enterprise integrations.
• Field-level merge. If the two sides changed different fields of the same record, keep both changes and merge them, only escalating to a true conflict when they touched the same field. More work, but it preserves both legitimate edits.
• Manual resolution. Flag the conflict and route it to a human to decide. Necessary for high-value records where an automated wrong choice is worse than a short delay, but it does not scale to high volumes.

The point I stress in design reviews is that the resolution rule must be an explicit, documented decision, not an accident of whichever record happened to be processed last. A synchronization with no defined conflict policy still has one; it is just an undocumented and non-deterministic policy, which is the worst kind. Decide the rule per entity, write it down, and make the sync enforce it consistently.

6. Full versus incremental sync

Separate from direction and topology is the question of how much data you move on each cycle. There are two broad answers, and the difference matters enormously as data volumes grow.

Full synchronization compares or transfers the entire dataset every cycle. Every record is examined, and the target is reconciled against the complete source. It is simple, self-correcting, and forgiving: because it looks at everything, it naturally heals drift and never misses a change, since it does not rely on detecting individual changes at all. Its weakness is cost. On a large dataset, moving and comparing everything on every cycle is slow and expensive, and it cannot run often enough to keep data fresh in real time.

Incremental synchronization moves only what has changed since the last cycle. It identifies the delta, the set of records created, updated or deleted since the previous run, and propagates just that. It is far cheaper and can run frequently, even continuously, which is what makes near-real-time sync feasible. Its weakness is that it depends entirely on reliably detecting changes, and if the change-detection mechanism ever misses something, that record silently drifts and stays drifted, because incremental sync never looks at unchanged records again.

Detecting the delta for incremental sync is its own small discipline. Approaches range from a modified-timestamp column you query for recent changes, to a dedicated change log, to change data capture reading the database transaction log directly. Change data capture is the most robust of these because it observes every committed change at the source without relying on application code to record it. It is important enough to warrant its own treatment, covered in the change data capture guide.

The pattern that works in practice: run incremental sync continuously for freshness, and schedule a periodic full sync, perhaps nightly or weekly, as a reconciliation safety net. The incremental pass keeps data current cheaply; the occasional full pass catches and heals any drift the incremental mechanism missed. This belt-and-braces combination gives you both freshness and correctness, and it is the design I recommend for any sync that matters.

7. Sync versus replication versus integration

These three terms are used loosely and often interchangeably, which causes real confusion in requirements conversations. They are related but distinct, and being precise about them sharpens design.

Replication is the narrowest. It copies data from one store to identical or near-identical copies, usually within the same technology, to serve read scaling, high availability or disaster recovery. Database replication makes a standby that mirrors the primary. There is typically no transformation and no business logic; the copy is a faithful reproduction. Replication is a special, tightly-constrained case of synchronization where the schema is identical and one side is authoritative. It has its own full treatment in the data replication guide.

Synchronization is broader. It keeps copies consistent across systems that may have different schemas, requiring transformation, and it deals with the full complement of concurrent edits, conflicts and bidirectional flow that replication usually avoids. Where replication mirrors, synchronization reconciles.

Integration is broader still. It is the whole discipline of making separate systems work together, of which synchronization is one important mechanism. Integration also covers request-response service calls, event-driven messaging, orchestration of multi-step processes, and the transformation and routing that connect fundamentally different applications. Synchronization answers "how do we keep this data consistent"; integration answers the wider "how do these systems cooperate at all". A concrete example is connecting a finance platform to surrounding systems, where synchronization of master data sits alongside transactional and event flows, as explored in the Business Central APIs and integrations guide.

8. Best practices

Across the synchronization work I have designed and reviewed, the same principles separate the reliable implementations from the ones that quietly rot:

• Pick a single source of truth per entity. Decide, for each kind of record, which system owns it. Ambiguous ownership is the root cause of most conflict pain. One-way sync from a clear owner is preferable to two-way sync between rivals.
• Prefer one-way where the business allows it. Do not take on two-way or multi-master complexity for the appearance of flexibility. Accept it only when the workflow genuinely requires edits to originate on multiple sides.
• Make every operation idempotent. Applying the same change twice must have the same result as applying it once, so that retries after a failure are always safe. Use stable identifiers and upsert semantics rather than blind inserts.
• Track versions or timestamps on every record. You cannot detect conflicts or order changes correctly without a per-record marker of what changed and when. Build this in from the start; retrofitting it is painful.
• Define the conflict rule explicitly, per entity. Write down what wins when the same record changes on both sides, and make the sync enforce it deterministically. An undocumented rule is still a rule, just an unpredictable one.
• Handle deletes deliberately. Decide whether deletes propagate, and if so use soft deletes or tombstones so absence is detectable rather than inferred from a missing row.
• Combine incremental freshness with periodic full reconciliation. Run incremental for currency and a scheduled full pass to heal any drift the incremental mechanism missed.
• Log and monitor everything. Record what synced, what conflicted, what failed, and how many records each side holds. Silent sync is sync you cannot trust; you need to be able to prove agreement, not assume it.
• Test the failure paths, not just the happy path. Kill the network mid-batch, replay out-of-order changes, edit both sides at once, and confirm the sync recovers correctly. The happy path always works; the value is in surviving the rest.

None of these is exotic. They are the accumulated scar tissue of systems that drifted, corrupted, or double-counted because one of them was skipped. Build them in from the design phase and synchronization becomes a dependable foundation rather than a recurring source of mysterious data discrepancies.

9. References

The concepts in this guide are standard across distributed systems and data engineering literature. For readers who want to go deeper into the underlying theory and established practice, these are solid, vendor-neutral starting points:

• Martin Kleppmann, Designing Data-Intensive Applications (O'Reilly). The chapters on replication and on the trouble with distributed systems cover ordering, conflicts and consistency in depth.
• Gregor Hohpe and Bobby Woolf, Enterprise Integration Patterns (Addison-Wesley). The canonical catalogue of messaging and integration patterns that underpin how synchronization is implemented in practice.
• The concept of conflict-free replicated data types (CRDTs) in the academic and engineering literature, for the formal treatment of automatic conflict resolution in multi-master and offline-first systems.
• Vendor documentation on change data capture and log-based replication from major database platforms, for the concrete mechanics of reliable incremental change detection.

Final thoughts

Data synchronization is one of those topics that looks trivial from a distance and reveals real depth the moment you build it for keeps. The core idea is simple: keep copies of the same data in agreement as it changes. The difficulty is entirely in the corners, in concurrent edits, out-of-order arrival, deletes, partial failures, and the conflict resolution that two-way and multi-master designs force upon you. Get the direction right, pick a clear source of truth, make operations idempotent, track versions, define your conflict rule explicitly, and combine incremental freshness with periodic full reconciliation, and you have a synchronization you can trust.

Zoom back out and synchronization is one mechanism within the wider integration discipline, sitting alongside service calls, events and orchestration. If this is part of a larger effort to make your systems cooperate, the enterprise system integration pillar puts it in context, and the related guides on one-way versus two-way sync, data replication and change data capture go deeper on the pieces this article introduced.

Related reading: Enterprise system integration explained, One-way vs two-way sync, What is data replication, What is change data capture, Business Central APIs and integrations.