BMS Data Retention Policy for Facilities Teams

Retention only becomes visible at two moments, and both are bad ones. Either somebody needs data from eighteen months ago to settle a dispute or justify a capital case, and discovers the trends were thinned to hourly averages after ninety days. Or storage costs appear on a budget line and somebody proposes deleting everything older than a year without asking what depends on it. Deciding in advance is cheap. Deciding at either of those moments is not.

The message up front: there is no single retention period, because building data is not one thing. Classify it, because an alarm record and a one-second trend have completely different value profiles, sizes and obligations. Then set duration and resolution separately for each class, since the usual answer is to keep a long history at reduced resolution rather than to choose between long and detailed. Record the reason each period was chosen, because an undocumented period cannot be defended or sensibly changed. And build an exception that pins evidence out of the volume-managed store when something interesting happens.

1. Why the default is a decision nobody made

Without a policy you still have retention behaviour, inherited from three places.

• Controller trend buffers are small and circular. They hold a fixed number of samples and overwrite the oldest. Depending on interval, that can be hours. Anything not collected upward before it wraps is gone, and nothing reports that it happened.
• The head end or historian arrives with a vendor default, often a modest period, often never reviewed. Sometimes it is effectively infinite, which looks generous until a disk fills on a Sunday.
• The integration or analytics layer keeps whatever its designer assumed, which is usually whatever the development environment did.

The result is three different periods at three layers, none chosen deliberately, and nobody able to answer how far back the data actually goes. That question is the first one to ask, and in most estates the answer takes a surprising amount of digging.

2. Classify before you decide anything

The pattern worth noticing: the class that consumes nearly all the storage is the one with the shortest useful life, and several of the classes worth keeping for years are small enough that their cost is irrelevant. A policy that applies one period across all of them gets both halves wrong at once, which is the most common outcome.

3. The four things that should set the period

• Operational need. How far back does anyone look during normal work? For high-resolution telemetry this is usually days to weeks. Diagnosing a fault rarely needs last March at one-minute resolution.
• Analytical need. Seasonal comparison needs more than a year, so that like is compared with like. This is the argument for a long tail of aggregated data and it is usually satisfied at hourly resolution, which is a fraction of the volume.
• Obligation. Contractual commitments, reporting schemes, warranty conditions, and whatever legal retention duties apply where you operate. These are the only genuinely non-negotiable drivers and they need to be established rather than assumed, with your own legal or compliance advice.
• Investigation. Disputes and incidents surface long after the event. This is what drives alarm records, operator actions and the audit trail described in the alert-to-work-order audit trail, and it is why those classes should outlive the telemetry that produced them.

Write the driver against each class in the policy. A period with a reason attached can be revisited sensibly when circumstances change. A bare number cannot, and will either be defended beyond its usefulness or deleted by someone who did not know why it was there.

4. Resolution and duration are separate decisions

The question is almost never "keep or delete". It is "keep at what resolution", and treating it that way usually dissolves the conflict between the operational and analytical drivers.

A tiered shape works for most estates: full resolution for a short recent window, a coarser rollup for a medium period, and a daily or monthly summary kept effectively indefinitely because it costs almost nothing. The specific intervals depend on your collection rates, point count and what your analytics actually need, and the sizing arithmetic behind that is set out in sizing a process historian.

What matters more than the numbers is doing the downsampling correctly, because this is where retention policies quietly corrupt data:

• Averaging destroys peaks. An hourly mean hides the excursion that mattered. If peaks are analytically important, keep minimum and maximum alongside the average; it triples a very small number and preserves what you actually need.
• Counters must not be averaged. A cumulative meter reading averaged over an hour is meaningless. Counters need their last value in the interval, or the delta across it, and getting this wrong produces energy reports that are confidently wrong rather than obviously broken. The related failure is covered in energy meter reset and rollover.
• Binary and multi-state points need state changes, not samples. Averaging a run status produces a duty percentage, which may be useful and is not the same thing. Keep the transitions.
• Record which rows are raw and which are derived. Otherwise a later analysis treats an hourly average as a reading and draws conclusions from a resolution that does not exist.
• Downsample once, from raw. Rolling up an already rolled-up series compounds the distortion, and several platforms will happily do it.

5. The exception that makes short retention safe

A short retention period on high-resolution telemetry is correct and also uncomfortable, because the one time you want second-by-second data is the one time something odd happened, and by the time anyone is curious the window has closed.

The resolution is to pin evidence rather than to extend retention across the board. When something notable occurs, an alarm, an intermittent fault, an incident, a dispute, the surrounding high-resolution data is copied out of the volume-managed store and attached to the record that describes the event. A few kilobytes per event, kept for as long as the record is kept.

• Trigger it automatically on alarm clearing, on work order creation from a building event, and on any incident record. Manual pinning is the same as no pinning.
• Include related points, not only the one that alarmed. The value of a snapshot is in the context around the event. The reasoning is set out in what to do when a BMS fault clears before a technician arrives.
• Store it with the record, not with the telemetry. The point is to move it onto a different retention clock, and leaving a pointer back into the telemetry store achieves nothing.
• Allow manual pinning as well, for the engineer who notices something interesting and wants it kept.

This single mechanism is what lets you set an aggressive period on the biggest class without losing the data that actually matters, and it is the part of a retention policy most often missing.

6. The parts that are personal data

Building telemetry is generally not personal data. Several adjacent classes are, and a policy that ignores this is incomplete.

Operator actions name individuals. The audit trail names who was notified, who acknowledged and who attended. Access control integration, where it exists, records movement. Fine-grained occupancy data can become personal in a small space even when no individual is identified by the sensor, which is discussed in occupancy sensing for building systems.

Whatever data protection regime applies where you operate will have something to say about how long such records are kept and on what basis. Establish which applies and take proper advice; I am not going to summarise anyone's law here, because the duties differ by country and a confident generalisation would be wrong somewhere. What is safe to say generally is that these classes should be identified explicitly in the policy, given their own periods rather than inheriting the telemetry period, and have access restricted by role as described in RBAC design for CAFM systems.

7. Where it lives, and who can delete it

A policy that names periods but not locations is not applicable, because building data is held in several places at once with different owners.

• Controller buffers. Short, circular, usually not configurable without the controls party. Treat as a staging area, never as a record.
• The head end or supervisory database. Often where the de facto history lives, frequently sized for a period nobody chose.
• A dedicated historian or time-series store. Where a deliberate policy is easiest to apply, and where the tiering in section 4 belongs. Integration patterns in SCADA and historian integration apply equally here.
• The analytics or reporting layer, which often holds its own copy with its own lifecycle, and is the place most likely to be forgotten entirely. Patterns in data lakes and lakehouses for maintenance telemetry.
• The maintenance system, holding work orders, cases and pinned evidence, usually on a much longer clock.
• Backups, which are the reason "we deleted it" is often untrue. A retention policy that does not address backup rotation does not actually delete anything.

For each location the policy should name who can change the retention setting and who can delete data outside the normal schedule. Where a supplier holds data on their platform, the policy also needs to say what happens at contract end, which is a question worth settling before signing rather than after, as discussed in cloud BMS and building management as a service.

8. Writing the policy

Short is better. A two-page document people follow beats a twenty-page one nobody opens. What it needs:

• A table of classes, with retention period, resolution at each tier, storage location and owner. This is the policy; everything else is supporting text.
• The driver and justification per class, in one line each.
• The evidence pinning rule, including what triggers it and where pinned data goes.
• Legal hold. A stated mechanism for suspending deletion on specified data when a dispute or investigation is live, who can invoke it, and how it is released. Without this, the policy will delete the one thing somebody needed, on schedule, correctly.
• Deletion method and backup treatment, so that deletion means deletion.
• Review cadence and owner. Annually is usually enough, and sooner after any change to collection rates or point count.
• Exceptions register, because there will be exceptions and they should be visible rather than informal.

9. The part nobody does, which is deleting

Most retention policies are written, approved, and then not enforced, because deletion is irreversible and nobody wants to be the person who signed it off. The policy becomes a statement of intent while storage grows indefinitely.

• Automate it. Deletion that requires a human decision each cycle will not happen. The decision was made when the policy was approved; execution should follow automatically.
• Run a dry run first, reporting what would be deleted, for a full cycle before enabling live deletion. This finds the dependencies nobody mentioned.
• Notify before the first live run, with enough warning for anyone relying on the data to object. Objections are useful information and usually arrive from somewhere unexpected.
• Log every deletion: what class, what period, how many records, when. Without this you cannot prove the policy is being applied, which matters if the policy exists for a compliance reason.
• Check the backups actually rotate. The most common reason data survives its retention period is a backup regime nobody included in the policy.

10. Measuring

• Actual oldest data per class per location. Compare against the policy. Divergence in either direction is a finding.
• Storage growth rate, and the projection against capacity. The number that tells you when this becomes urgent.
• Requests for data older than the retention period. The evidence that a period is set too short, and much better than arguing about it in the abstract.
• Pinned evidence volume. Should be small. If it is large, the trigger is too broad and the exception is becoming the policy.
• Deletion runs completed on schedule. The simplest measure of whether the policy is real.

The idea to walk away with

The instinct that retention is a trade-off between cost and usefulness is nearly backwards. The data that costs the most is the data with the shortest useful life, and most of the data worth keeping for years is small enough that its cost does not matter. Separating the classes turns an argument into an easy set of decisions.

Then separate resolution from duration, so you keep a long history cheaply rather than choosing between long and detailed. Add evidence pinning so a short period on raw telemetry stops being risky. Write the reason beside every period. And automate the deletion, because a policy that is never executed is a document, not a control.

Final thoughts

Start by finding out what you actually have. For each storage location, ask what the oldest record is, what the configured period is, and who can change it. In most estates those three answers do not agree anywhere, and several locations turn out to be holding data nobody knew about.

That inventory is usually more revealing than the policy that follows it. It is common to find high-resolution data being discarded after days while a forgotten copy of the same data sits in an analytics environment growing indefinitely, which is precisely the wrong way round and costs money at both ends.

Related reading: The alert-to-work-order audit trail, Energy meter reset and rollover, When a fault clears before the technician arrives, Sizing a process historian, Data lakes for maintenance telemetry, RBAC design for CAFM systems.