Telling Communication Failures Apart From Equipment Anomalies

A mechanical technician is sent to a plant room because a reading went flat. They find the equipment running normally and nothing to do. The actual event was a gateway that stopped forwarding, and the person who could have fixed it in ten minutes never heard about it. This happens on almost every estate with automated alerting, it is expensive in a quiet way, and it is almost entirely preventable with information the system already has.

The message up front: classify by where the evidence points before deciding who to send, and the strongest evidence is correlation across points that share infrastructure. A departure affecting every point behind one device, controller or link is a data path event. A departure confined to one point while its neighbours behave normally is a candidate plant or instrument event. Carry the device's own quality flags through so the system can say when a value is known-bad, and route the two classes to different owners with different urgency.

1. Why they look the same

The confusion is structural rather than careless. Most consumers of building data see a series of values and nothing about how those values arrived.

• A stopped feed produces a flat line, and so does a sensor that failed, and so does a genuinely stable condition. All three render identically.
• A buffered gateway releasing its backlog produces a burst, which looks like rapid change and is actually delayed delivery, as described in gateway store-and-forward.
• A missing interval looks like a step when the chart joins across it, which is why gaps should render as gaps.
• A stale value repeated looks like a steady reading, and some collection paths repeat the last known value rather than reporting nothing.
• A failed sensor often reports a plausible number. Not an obviously wrong one, which is why the device's own assessment in quality flags is worth more than any plausibility check.

2. Correlation across shared infrastructure is the strongest signal

The single most useful discriminator is not anything about the value. It is whether other points sharing the same path are affected at the same moment.

Physical plant and communications infrastructure have different blast radii. A piece of equipment misbehaving affects the points that measure it. A network segment, a controller or a gateway failing affects every point behind it regardless of what they measure, and that is a pattern no plant fault produces. A simultaneous departure across a temperature, a pressure, a status and a meter reading that happen to share a controller is a communications event with very high confidence, and it needs no interpretation of what any of those values mean.

Using this requires knowing the topology: which points sit behind which device, which devices behind which gateway, which gateways on which link. That information comes from the discovery and register work in device discovery and the point dictionary, and it is frequently the missing piece rather than the logic.

3. The discriminating checks

No single row is decisive. Several together usually are, and the combination of scope plus arrival pattern is sufficient in the large majority of real cases.

4. Routing the two classes

The classification only earns anything if it changes what happens next.

• Data path events go to controls, integration or IT, depending on where the boundary sits, and with the urgency appropriate to losing visibility rather than to losing plant. The escalation design for this is in the connectivity loss playbook.
• Instrument events go to instrumentation, against the sensor rather than the plant, so the asset history stays honest.
• Plant events go to the maintenance trade, with the detected condition and the surrounding context attached.
• Ambiguous cases go to a triage step, not to a trade. A short review by somebody with access to both the topology and the plant costs far less than a wasted attendance, and a quarter of an hour resolves most of them.
• Record the classification on the work order. Otherwise a run of data path events appears in the failure analysis as equipment problems, which corrupts exactly the figures in fault classification.

5. The genuinely hard cases

Three situations resist the checks above and are worth recognising rather than forcing.

A single device failing. A controller that stops responding affects all its points, which looks like a data path event, and it is, but the device itself is a physical item that may need replacement. The routing answer is usually controls rather than a mechanical trade, and the point is that the first responder should be whoever owns the controller.

A plant event that causes a communications event. A power interruption to a plant room takes out both the equipment and its controllers. Here the correlation test gives the right structural answer, a shared-infrastructure event, and the underlying cause is physical. Treating the loss of visibility as the first thing to restore is still correct.

Intermittent degradation. A link that drops occasionally produces scattered short gaps that never look like a clear event individually, and in aggregate they quietly depress completeness and generate unexplained findings. This one is not caught by per-event triage at all; it is caught by trending completeness per collection path, which is why the monitoring in ingestion monitoring is a separate discipline from alert triage.

6. Measuring how often you get it wrong

The value of this is measurable, and measuring it is what keeps it funded.

• Count attendances closed with no fault found where the underlying cause was later identified as a collection event. This is the cost being avoided and it is usually larger than people expect.
• Count work orders reassigned between trades, which is a direct measure of misrouting and easy to extract.
• Track the data artefact share of detection findings, which is the same measure seen from the analytics side, per pilot evaluation.
• Review misclassifications monthly and adjust the rules. Most errors cluster on a few alert sources with incomplete topology data, and fixing the register fixes the class.

The idea to walk away with

Data path failures and plant events are indistinguishable in a value series and distinguishable in their blast radius, and the blast radius is information most pipelines have and most alerting setups discard.

Record the collection path with every reading, classify on correlation across shared infrastructure plus the device's own quality flags, route the two classes to different owners with different urgency, send ambiguous cases to a short triage rather than to a trade, and record the classification so the failure analysis stays honest.

Final thoughts

Pull last quarter's work orders closed as no fault found on alert-driven jobs, and check how many fall on the same day as a collection gap on the same controller.

That query takes minutes if the collection path is recorded and an afternoon if it is not, and either way the answer is the business case. If the overlap is substantial, you have found a recurring cost that a classification rule removes, and the effort is in the register rather than in the logic.

Related reading: Connectivity loss playbook, Data quality flags, Gateway store-and-forward, Ingestion monitoring, Signal consistency checks, Faults that clear before arrival.