There is a version of the predictive maintenance conversation that happens in the boardroom, and a different one six months later in a room with a reliability engineer, a data engineer and an increasingly quiet vendor consultant. The first is about artificial intelligence predicting failures. The second is about why the model fires four alerts a day, three of which are nothing, and why nobody can explain the fourth. The gap between those rooms is not ambition. It is machine learning engineering: problem framing, data volume, class imbalance, the right evaluation metric, drift, explainability and the operational burden of keeping a model alive.
The message up front: the binding constraint on machine learning for maintenance is almost never the algorithm. It is labelled failure history. Most sites have nowhere near enough run-to-failure examples to train a supervised failure predictor, and no amount of model sophistication substitutes for examples that do not exist. For the large majority of assets the realistic answer is unsupervised anomaly detection layered on well-chosen engineering thresholds, with a hybrid physics-plus-statistics approach on the handful of assets that justify the effort. Less exciting than the vendor deck, and the version that survives contact with a plant.
If the strategic questions are not yet settled, which assets deserve prediction at all, whether their failure modes even have a detectable warning period, and how the output reaches a technician, start with the predictive maintenance and failure prediction pillar. That piece covers the P-F curve, remaining useful life and the strategy layer. This one assumes that groundwork and goes down a level, into the modelling.
1. Problem framing: the choice that decides everything else
The most consequential decision in a maintenance ML project is made in the first week, usually without much deliberation, and rarely revisited: what question are you asking the model to answer? Get it wrong and every later step, data collection, labelling, metric selection, deployment, inherits the mistake. Four framings show up in practice, with radically different data requirements.
Anomaly detection asks: does this asset look different from how it normally looks? You train only on healthy operating data and score deviations. No failure labels required. It is available to almost everyone and consistently undersold, because it does not sound like prediction.
Classification asks: which known fault type is developing? That needs a labelled history where somebody recorded, consistently, what the fault actually was. If your work orders close with "fixed" in the comments field, you do not have a classification dataset and you will not have one for another three years.
Regression to remaining useful life asks: how many days of useful life are left? The framing everyone wants and almost nobody can support. It requires many complete degradation trajectories, assets instrumented from healthy through to functional failure, for the same asset type under comparable duty. Two hundred near-identical pumps with five years of streaming data can support it. Three bespoke chillers cannot.
Survival analysis asks: what is the probability this asset fails within the next N days, given age, duty and condition? It is the framing most honest about censored data, meaning assets that have not failed yet but are still informative, and it is badly underused. If you have failure dates and install dates but no high-frequency condition data, survival models are often the best available option and far more defensible than forcing an RUL regression onto thin data.
| Framing | What it needs | When to use it | Characteristic failure mode |
|---|---|---|---|
| Anomaly detection (unsupervised) | A clean window of healthy operating data. No failure labels. | Almost always the right start. Essential when failure history is thin. | Flags benign changes too: setpoints, seasons, a new operating regime. Alert fatigue. |
| Binary classification (failure in horizon) | Labelled events with reliable timestamps, plus condition data across the horizon before each. | Tens of labelled events per asset class and a defined, useful lead time. | Severe imbalance, plus label leakage from features that exist only because work was already planned. |
| Multi-class (fault type) | Consistently coded failure modes, ideally Problem-Cause-Action, with enough per class. | Diagnostic support on instrumented rotating equipment with known signatures. | Rare classes ignored entirely. Inconsistent coding teaches the coder's habits, not the physics. |
| Regression to RUL | Many complete run-to-failure trajectories, comparable assets, comparable duty. | Large homogeneous fleets with continuous data. Rare outside heavy industry and aerospace. | False precision: a confident day count from data that cannot support one. |
| Survival / hazard | Install dates, failure dates, censored survivors, optional duty and condition covariates. | Fleet risk ranking and interval optimisation with no high-frequency data. | Population answers mistaken for individual-asset predictions. |
| Hybrid (physics plus data) | An engineering degradation model or known fault frequencies, plus a residual model. | High-value critical assets where engineers must defend the output. | Higher build cost; needs domain and data skills in the same room. |
What I would recommend to any team starting out: write the framing as one sentence with a time horizon in it, before anyone opens a notebook. "Flag pumps whose vibration signature deviates from their own healthy baseline, within 24 hours of the deviation starting" is a framing you can build, test and act on. "Use AI to predict pump failures" is an aspiration, and it is how projects drift for a year.
2. Labelled failure data is the binding constraint
Here is the arithmetic vendors tend not to walk through. Suppose you want a supervised model that predicts failures on a class of centrifugal pumps with a two-week lead time. As a rough working rule, that needs on the order of dozens of labelled failure events to learn a signature that generalises, not three or four. Now count what a typical site has: perhaps forty pumps of that class, a mean time between failures of several years, and condition monitoring installed eighteen months ago. Failures before the sensors went in have no condition data attached, so they are unusable. The ones since may number five, two of them closed with a failure code somebody picked from the top of the dropdown.
That is not a data science problem, it is an arithmetic problem, and it is the single most common reason maintenance ML projects quietly stall. A model cannot learn from failures it has never seen.
There are legitimate moves when labelled data is scarce, and they are what competent practitioners actually reach for:
- Reframe to unsupervised. Anomaly detection needs healthy data, which you have in abundance, rather than failure data, which you do not. The honest answer for most sites, covered in depth in the anomaly detection and early fault warning pillar.
- Pool across the fleet, carefully. Twenty sites with three failures each is sixty labelled events, if the assets are genuinely comparable and duty cycles are similar. This is a real structural advantage for a multi-site operator, and one of the few genuine arguments for a centralised data platform.
- Widen the label. Rather than only functional failures, label interventions that found a genuine developing fault. There are far more of those, and for early-warning purposes they are the event you care about. Needs decent work-order narrative quality, but it multiplies your usable label count.
- Use rig or vendor data, with scepticism. Public bearing degradation datasets and vendor fleet models can bootstrap a signature. Treat any transferred model as a hypothesis to validate on your assets, never a finished product, because duty, mounting and instrumentation differ.
- Fix the coding now so the dataset exists in three years. Unglamorous and the highest-return action available. Nobody funds it, which is why the same organisation has the same conversation again three years later.
The question to ask in the first vendor meeting
Ask how many labelled failure events of your asset class their approach requires, and what it does when you have fewer. A credible answer names a number and describes an unsupervised fallback. An answer about the platform learning your plant automatically is telling you the vendor has not yet counted your failures either.
3. Physics, statistics and machine learning: why hybrids usually win
There are three ways to reason about a degrading machine, and the industry habit of treating them as a progression with machine learning at the top is a mistake. They are complementary, and the best deployments blend them.
Physics-based models encode the mechanism: bearing defect frequencies from geometry and shaft speed, fouling models, crack-growth relationships, pump affinity laws. They need almost no historical data and are fully explainable, which matters enormously for adoption. Their weakness is build cost per asset type and brittleness when the operating envelope departs from the assumptions.
Statistical models sit in the middle: control charts, exponentially weighted moving averages, Weibull analysis, hazard models, principal component residuals. Deeply underrated. A control chart on a physically meaningful feature catches a large share of what a neural network would catch, is defensible in a meeting, and takes days rather than quarters to build.
Machine learning models learn the mapping without being told the mechanism. Gradient-boosted trees on engineered features remain the right default for tabular condition and process data. Deep learning has a genuine place on raw high-frequency signals, spectra and images, where feature engineering is itself the hard part, which is the subject of the deep learning for maintenance pattern recognition pillar. On a few thousand rows of tabular data it is usually the wrong tool.
The pattern that works is a hybrid: physics carries the structure, the data model carries the residual. Use the engineering model or known fault frequencies to compute what the signal should look like under current load, speed and ambient conditions, then let a statistical or ML model watch the gap between expected and observed. That strips out the load and seasonality variation that makes raw anomaly detection noisy, sharply reduces the data the learned component needs, and produces an output an engineer can interrogate. "Discharge pressure is four percent below the affinity-law expectation at this speed and has been drifting for eleven days" gets acted on. "Anomaly score 0.83" gets ignored.
4. Feature engineering: where the domain knowledge actually lives
For tabular maintenance data, feature engineering matters more than model selection, and it is where reliability engineering earns its place in the project. A modest model on good features beats a sophisticated model on raw columns almost every time.
From vibration, the useful features are physically motivated rather than generic. Overall RMS velocity and peak acceleration give the broad trend. Crest factor and kurtosis indicate impacting, meaning developing bearing defects, and often move before overall amplitude does. Band energies around the calculated bearing defect frequencies, and around one, two and three times running speed, separate bearing wear from imbalance from misalignment. Envelope spectra pull weak repetitive impacts out of broadband noise. All of these need shaft speed to be meaningful, which is why variable-speed drives complicate vibration analytics: a feature computed without normalising for speed is largely measuring duty cycle.
From process data, the highest-value features are relative rather than absolute. Deviation from expected performance at current load. Efficiency from parameters you already log. Differential pressure across a filter or exchanger. Motor current normalised by delivered flow. Rate of change over a rolling window, and the rate of change of that rate, since acceleration in a degradation trend is often the real signal. Time since last intervention. Running hours and start-stop counts, since start transients do disproportionate damage.
From the CMMS itself: corrective work orders in the trailing twelve months, time since last PM, parts replaced recently, whether the asset has a repeat-failure pattern. On thin-data problems this maintenance-history context sometimes carries more predictive weight than the sensors, which is uncomfortable for a sensor-led business case but useful to you.
The leakage trap that inflates every offline result
A model that looks excellent in validation and useless in production has usually learned something it will not have at prediction time. Classic maintenance examples: a work-order status that only changed because work was already planned, a sensor that goes flat because the asset was isolated for repair, a parts-issued record dated before the failure but entered after it. Ask of every feature: would this value have been available, with this content, at the moment the prediction was needed? Random train-test splits on time-series data leak by construction. Split by time, and preferably by asset too.
5. The class imbalance problem, stated plainly
Failures are rare. That is the point of a maintenance programme, and it is what makes supervised failure prediction statistically awkward. Frame the problem as "will this asset fail in the next fourteen days" and sample daily, and a fleet with reasonable reliability produces something on the order of one positive day in several hundred or several thousand negative days.
The consequence is the most quoted and most misunderstood fact in this field: a model that predicts "no failure" every time, for every asset, on every day, scores extremely high accuracy. It also never once catches a failure. Accuracy on an imbalanced problem measures the imbalance, not the model. A vendor or internal team leading with an accuracy figure on a failure-prediction model is either not thinking clearly or hoping you are not.
What genuinely helps, in rough order of how often I would reach for it:
- Widen the prediction window. A thirty-day horizon has many more positive labels than a three-day one. Choose the horizon from operations, how much notice the planner needs to get parts and a slot, and accept the precision consequences, rather than picking a horizon that flatters the model.
- Class weighting over resampling. Weighting the loss function is cleaner than synthetic oversampling on time-series sensor data, where interpolated examples can be physically meaningless.
- Treat it as anomaly detection instead. If positives are that rare you may not have a classification problem at all. One-class methods sidestep imbalance by never asking for negatives.
- Cost-sensitive thresholding. The model outputs a score; the threshold is a business decision, set from the relative cost of a missed failure versus an unnecessary inspection. That differs by an order of magnitude between a redundant car park fan and a single-point-of-failure transformer, which is where asset criticality classification earns its keep: criticality sets the threshold.
- Aggregate to the asset, not the day. Per-asset-per-episode performance is both more honest and more useful to operations than per-day performance.
6. The metrics that matter: precision, recall and lead time
Choosing the evaluation metric is a governance decision disguised as a technical one, because whichever metric you report becomes the thing the team optimises. For maintenance, accuracy is nearly worthless. The pair that matters is precision and recall reported together, with lead time as the third dimension nobody includes and everybody needs.
| Metric | What it measures | In maintenance terms | Why it can mislead |
|---|---|---|---|
| Accuracy | Share of all predictions that were correct. | Meaningless here. Always predicting "healthy" scores very high when failures are rare. | Dominated by the majority class. Never report it alone. |
| Recall (sensitivity) | Of the real failures, how many did the model flag? | How many breakdowns you caught. Misses still happened, unplanned. | Trivially driven to 100% by flagging everything. Worthless without precision beside it. |
| Precision | Of the flags raised, how many were real? | How well your crew's inspection time was spent. Low precision is wasted trips and lost credibility. | Raised by flagging only obvious late-stage cases, which destroys lead time. |
| F1 / F-beta | Harmonic mean of precision and recall; beta weights recall higher. | One number for tuning. Use F2 when a missed failure costs far more than a wasted inspection. | Hides which side is failing. Always inspect both components. |
| PR-AUC | Area under the precision-recall curve across all thresholds. | The right headline for comparing models on imbalanced maintenance data. | Not a deployment number. You still choose one threshold and live with it. |
| ROC-AUC | Area under the true-positive vs false-positive curve. | Widely reported, comfortingly high, optimistic under heavy imbalance. | Insensitive to the false-positive volume that burdens your crew. Prefer PR-AUC. |
| Lead time | How far ahead of failure the flag was raised. | What operations cares about most. A correct flag two hours out is a breakdown with paperwork. | Almost never reported. A model can improve everywhere else while becoming useless. |
| Alerts per crew per week | Raw operational load created by the model. | Determines whether the programme survives its first quarter. | Not a model metric, which is why it gets left out of the report. |
The way I would frame acceptance criteria before any model is built: agree, with the maintenance manager in the room, a minimum useful lead time, a maximum tolerable number of false alerts per crew per week, and the recall level below which the programme is not worth running. Those three numbers define a box. The model either lands inside it or it does not, which is a far more productive conversation than arguing about model families, and it prevents a model being declared successful on a metric nobody in operations recognises.
Precision is a trust budget
Technicians spend trust on your model the first few times they are sent to inspect a healthy machine. Once that budget is gone, alerts get closed without investigation and recall becomes irrelevant, because nobody acts on the true positives either. Tune conservatively at launch, build a track record, then loosen the threshold. A model that catches sixty percent of failures and is believed beats one that catches eighty percent and is ignored.
7. Model drift and retraining: the model is not the deliverable
A maintenance model degrades from the day it is deployed, mostly for reasons nothing to do with the model. Plants change. The kinds of drift need different responses.
- Data drift: a sensor is replaced with a different make, a gateway resamples at a different rate, a recalibration shifts a baseline, a historian tag is renamed. Feature distributions move and the inputs no longer mean what they meant in training. The most common cause of a model silently going wrong, and a data engineering problem.
- Concept drift: the relationship changes. An impeller is upgraded, the control strategy retuned, the plant moves from baseload to cycling duty, a lubricant is changed. The old signature no longer maps to the new failure behaviour.
- Intervention drift: the subtle one, specific to this field. The model works, so maintenance intervenes earlier, so the failures it was trained to recognise stop occurring in that form. A successful predictive model destroys its own training distribution. Keep recording near-misses and what was found at inspection, or you lose the ability to evaluate it at all.
- Seasonal and operational cycling: not drift, but routinely mistaken for it. A model trained through a Gulf winter and deployed into August will flood you with alerts unless ambient conditions and load are features, or the expectation is normalised for them.
The practical response is monitoring, not scheduled retraining. Blind monthly retraining is a habit, not a strategy, and on intervention-drifted data it can make things worse. Instead: track the input distribution of every feature and alarm on shift; track the alert rate per asset class, because a rate that doubles overnight is a data problem rather than an epidemic; track precision from technician feedback on closed alerts; and hold a fixed reference test set so a retrained model can be compared with the incumbent before it replaces it. Retrain when a trigger fires or the plant changes materially, and always shadow the new model against the old before switching.
This is where the feedback loop earns its name. Every alert a technician closes as confirmed, not found, or already known is a label. If the CMMS captures that disposition, the dataset grows every week and the evaluation stays honest. If alerts arrive by email and are closed by being deleted, you will never know whether the model works.
8. Explainability, or why maintenance engineers reject black boxes
Data science circles tend to treat demands for explainability as a soft preference, a matter of stakeholder comfort. In maintenance it is nothing of the kind. An engineer who takes a critical asset offline on a model's recommendation is putting their professional judgement, and sometimes a safety case, behind that decision. They will be asked why. "The model said so" is not an answer that survives an incident review, and it is not an answer that should.
So the practical bar is higher than a global feature-importance chart. An alert needs enough context that an engineer can decide within a couple of minutes whether to act:
- Which signals drove this alert, ranked, with recent values against their normal range. Per-prediction attribution, not model-level averages.
- The trend, plotted. A chart of the driving feature over recent weeks does more for adoption than any attribution method. Engineers read trends fluently, and a visible trend converts a score into a diagnosis.
- A physical hypothesis where one exists. Energy rising in a bearing defect frequency band is a candidate diagnosis, not just an anomaly. Another argument for physically meaningful features: they make the explanation write itself.
- Similar past cases. "Three previous alerts with this pattern on this asset class: two confirmed bearing wear, one found nothing." Intuitive, and requires no interpretability machinery.
- A stated confidence and a recommended action. Monitor, inspect at next opportunity, or intervene now. Three tiers map onto how planners already think.
This bears directly on model choice. A gradient-boosted tree on twenty engineered features, with per-prediction attributions and the driving trend plotted, is a defensible system. A deep network on raw waveforms may perform better on paper and will be far harder to justify to an engineer at two in the morning. Where interpretability is cheap, buy it; where you genuinely need the deep model, budget explicitly for the explanation layer rather than assuming the accuracy will speak for itself.
There is a governance dimension too, and it is becoming less optional. Any model influencing operational or safety-relevant decisions needs documented ownership, a record of what version was live when, and a defined path for challenging an output. I have written about that framework separately in the AI governance for enterprise operators pillar, and the risk-management structure in the NIST AI Risk Management Framework is a reasonable starting point for operators who need something structured to point at.
9. The MLOps burden nobody budgets for
The proof of concept is the cheap part. That is the sentence I would most want a steering committee to absorb, because the cost profile of a maintenance ML programme is weighted heavily towards everything after the model first works. What a production model requires, continuously:
- A data pipeline that does not silently break. Sensor dropouts, gateway outages, tag renames, timezone handling, late-arriving data. A model scoring on stale or partially missing inputs produces confident nonsense rather than an error, which is worse.
- Feature consistency between training and serving. Features computed one way in the training notebook and another in the production job is among the most common and most quietly damaging defects in applied ML.
- Versioning of data, features, model and threshold. When someone asks why the model behaved differently in March, you need an answer. The threshold is part of the model and must be versioned with it.
- Monitoring of the model, not just the servers. Uptime says nothing about whether predictions are still valid.
- Integration into the CMMS or EAM, maintained. Work-order creation from an alert, disposition captured on closure, and that integration surviving every upgrade of Maximo, SAP PM, Hexagon EAM, Planon or Infor EAM. Upgrades break integrations, and someone has to own that.
- Named ownership. Not a project. A running service with an owner, an on-call path and a budget line.
Where this approach does not work, stated plainly
Do not start a custom ML programme if any of these is true: fewer than roughly a dozen comparable assets; condition history shorter than a couple of times your mean time between failures; unreliable failure coding with no project to fix it; no named owner who will still be accountable in two years; a manual CMMS integration; or assets not critical enough that a caught failure pays for the effort. The answer then is not a smaller model, it is engineering thresholds, condition-based alarms and better failure coding, which is good maintenance practice and considerably cheaper. The condition-based versus predictive maintenance pillar sets out that boundary.
There is a build-versus-buy judgement embedded here. For most operators outside heavy manufacturing, the sensible route is the analytics already bundled with the condition-monitoring hardware or the EAM platform, rather than standing up a data science function. Bundled models are often unremarkable, but somebody else carries the pipeline, the versioning and the upgrade compatibility, and that burden is the real cost. The platform comparison pillar covers that landscape, and the sensing layer underneath it is in the IoT sensors pillar.
10. The realistic path: thresholds first, anomaly detection second, supervised last
Putting the preceding sections together gives a sequence, close to the reverse of how most programmes are run.
- Stage 1: engineering thresholds and alarm rationalisation. Choose limits from the standards and the manufacturer's data rather than whatever the installer left configured. For rotating equipment the ISO 20816 vibration evaluation series gives defensible zone boundaries by machine class. A large share of the value attributed to AI programmes is available at this stage, at a fraction of the cost.
- Stage 2: trend and residual monitoring. Normalise for load and speed, compute a small set of physically meaningful features, and monitor their rate of change against the asset's own history. Transparent, and it handles the variable-duty problem that defeats naive thresholds.
- Stage 3: unsupervised anomaly detection. Learn each asset's normal envelope from its healthy data and score deviations. No labels needed. For most sites this is the ceiling of what the data honestly supports, and a respectable place to stop.
- Stage 4: supervised models, on the narrow set of asset classes that qualify. Large homogeneous fleets, real failure history, continuous data, a named owner. Where the classification and RUL framings belong, and where the manufacturing context tends to qualify while a mixed building portfolio does not.
- Throughout: capture dispositions. Every closed alert labelled confirmed, not found, or already known. This makes stage 4 possible in three years and keeps stages 1 to 3 honest meanwhile.
The uncomfortable implication is that a well-run programme may never reach stage 4, and should not treat that as failure. Stages 1 to 3 catch a great deal, are explainable, are cheap to maintain, and need no data science team. Stage 4 is a genuine uplift on the asset classes that can support it, and an expensive way to generate noise everywhere else.
11. A readiness checklist you can use before committing budget
Before approving a machine learning initiative, work through this. A "no" is not fatal, but it tells you what the first project actually is.
• Is the question one sentence with an explicit time horizon?
• Does that horizon match the notice a planner needs?
• Can our data support the framing we picked?
Data
• How many labelled failures of this asset class, with condition data attached? Count them.
• Is the condition history longer than one MTBF?
• Is failure coding consistent enough to trust as labels?
• Can we normalise for load, speed and ambient conditions?
Evaluation
• Agreed minimum useful lead time?
• Agreed maximum false-alert rate per crew per week?
• Precision and recall reported, accuracy excluded?
• Test split by time and by asset, not randomly?
Operations
• Does an alert become a work order in the system technicians already use?
• Is the disposition captured on closure as a future label?
• Can an engineer see why an alert fired, in under two minutes?
• Named owner accountable for the running model in two years?
• Who re-tests the integration after the next EAM upgrade?
The idea to walk away with
Machine learning in maintenance is an engineering discipline, not a purchase, and the constraint is your failure history rather than the algorithm. Frame the question narrowly, with a horizon. Count your labelled failures honestly and pick a framing that number supports, which for most sites means unsupervised anomaly detection on well-chosen thresholds rather than a supervised failure predictor. Build features a reliability engineer recognises, because they make the model both smaller and explainable. Report precision, recall and lead time, never accuracy. Assume drift and monitor for it rather than retraining on a calendar. Budget for the operational life of the model, not the proof of concept.
Do that and you get a system engineers trust and act on. Skip it and you get what many organisations have: a platform, a dashboard, an alert queue nobody opens, and a failure rate that has not moved.
Final thoughts
My contribution in these conversations is usually deflationary. When a team wants to discuss model architectures, the more productive questions are how many labelled failures exist, what lead time operations needs, and who owns the pipeline in year three. Those decide the outcome; the model family barely does.
None of which makes the AI framing empty. Anomaly detection genuinely catches developing faults that fixed thresholds miss, particularly on variable-duty equipment where a single limit cannot work. Pattern recognition on spectra and thermal images genuinely reduces specialist review time. Survival models genuinely improve interval decisions across a fleet. Real gains, worth pursuing. They are simply smaller, more specific and more conditional than the deck suggested, and they arrive on the back of clean data, sound feature engineering and honest evaluation rather than in place of them.
Assessing a predictive maintenance AI proposal?
Independent review of whether your data can support the model being proposed, what framing fits, how to evaluate it honestly, and what the CMMS/EAM integration will really cost to keep running. 22+ years across enterprise CMMS, EAM, CAFM and ERP implementations. No platform vendor margins, no reseller arrangements.
Book a conversationRelated reading: Predictive maintenance: a practitioner's guide, Predictive maintenance and failure prediction, AI anomaly detection and early fault warning, Deep learning for maintenance pattern recognition, Condition-based vs predictive maintenance, Predictive maintenance platforms compared.
Muhammad Abbas
CMMS / CAFM Manager & Independent Advisor · 22+ years across enterprise CMMS, EAM, CAFM and ERP implementations in utilities, oil and gas, manufacturing, government and facility operations.
Work with me