Getting Technician Feedback Back to a Detection Service

A detection service that never learns what its findings turned out to be will not improve, and the knowledge it needs already exists: the technician who attended knows whether there was anything there. The usual attempt to capture it is a feedback control on the alert, a thumbs up or a was-this-useful prompt. Response rates start low and fall, the responses that do arrive are systematically biased towards the annoying alerts, and after a few months the feature is quietly removed and the conclusion drawn is that technicians will not give feedback.

The message up front: derive feedback from work the technician was already going to record rather than asking for an extra judgement. The closing fields on a work order, the fault classification, the parts used and the time spent together say far more about whether a finding was real than any rating would. Where an explicit input is genuinely needed, make it one tap inside the existing closing flow, never a separate system, and make it about what they observed rather than about the quality of the alert.

1. Why rating prompts fail

The failure is predictable and it is not about willingness.

• It is extra work with no return to the person doing it. The benefit accrues to a system and to an analyst, and the cost falls on somebody at the end of a job.
• It asks the wrong question. Whether an alert was useful conflates several things, and the technician is being asked to judge a system rather than report an observation, which is both harder and less reliable. The distinctions are set out in labelling BMS anomalies.
• The responses are biased. People respond when irritated or when something was notably good, so the collected feedback over-represents both tails and says little about the ordinary middle.
• It usually lives in the wrong place. A prompt in a different application from the one used to close jobs will not be reached on a busy day.
• Nothing visibly changes. Feedback that produces no observable effect trains people to stop giving it, and faster than any incentive can counteract.

2. Derive it from the record instead

A closed work order already contains most of a label, if the fields are structured. This is the highest-return observation in the whole area.

3. When you do need to ask, ask once and ask narrowly

Some information genuinely cannot be derived, principally the distinction between nothing was wrong and something was wrong but had passed. That distinction matters and the record rarely carries it.

• One field, inside the existing closing screen. Not a separate form, not an email, not a different application. If it is not where the job is closed, it will not be filled.
• Ask what they observed, not what they thought of the alert. Options along the lines of found and fixed, found but not resolved, condition had cleared, nothing apparent, wrong equipment. These are observations, which people answer accurately and quickly.
• Keep it to a few options with a clear none-of-these. A long list produces the first plausible option rather than the right one.
• Make it optional and measure the response rate. A mandatory field produces a filled field rather than information, and the response rate is itself diagnostic.
• Never ask twice about the same job. A reminder converts a mild cost into resentment and takes the rest of the loop with it.

4. Close the loop visibly

The single thing that sustains feedback is evidence that it changed something, and this is where most implementations stop.

Three mechanisms work. Suppress or retune the specific source that repeatedly produced nothing, and tell the people who reported it that you did, naming the alert. Report back periodically on what the feedback changed, in concrete terms, as a short note rather than a dashboard. And give the technicians something in return in the same interface: showing whether this is the first occurrence on this asset, drawn from the joined history, is genuinely useful on arrival and costs nothing to display.

That last one matters more than it appears. A feedback loop where information flows only outward is an obligation; one where the system also tells the technician something they wanted to know is an exchange, and exchanges persist.

5. Using the feedback without over-reading it

Feedback collected this way is useful and it is not a clean labelled dataset, and the difference needs holding onto.

• It is incomplete and non-random. Jobs that were closed carefully are over-represented, which correlates with the seriousness of what was found.
• No fault found is not a false positive. It means nothing was apparent at the time of attendance, which is a different statement and a common source of unfair conclusions about both the method and the technician.
• Use it for routing before using it for tuning. The clearest early return is correcting which trade an alert class goes to, which is a mapping fix from alarm code mapping and improves outcomes immediately.
• Treat retuning from feedback as a change with a record. Adjusting sensitivity because of a run of unproductive alerts is reasonable; doing it without recording when and why makes later evaluation impossible, which is the versioning point in monitoring drift.
• Watch for feedback that is really about the data. A pattern of wrong equipment responses is a mapping error, not a detection error, and chasing it as the latter wastes everybody's effort.

6. The organisational conditions

A feedback loop is a process that needs an owner and a cadence, and it fails in organisational ways rather than technical ones.

Somebody has to be accountable for reading the feedback and acting on it, and it should be the same person who owns the detection service, because otherwise the loop ends at a report. The cadence should be regular and modest, because a monthly half hour that consistently produces one or two changes beats a quarterly review that produces a list.

It also has to be clear that feedback is not a performance measure on anybody. The moment a technician suspects that recording no fault found reflects on them, the field becomes unreliable in a way that cannot be detected or recovered, and the same applies if a high false-finding rate is treated as a failure by whoever configured the detection. The loop depends on both parties being able to report an unwelcome result without cost, which is a management choice rather than a system design one. Where a contractor is involved, the same principle has to survive the commercial relationship, which is harder and worth addressing explicitly in how the SLA is framed, per alert SLA design.

The idea to walk away with

The feedback a detection service needs is mostly already being recorded, in the closing fields of work orders, and the usual mistake is to add a rating mechanism instead of using what exists.

Derive the signal from cause codes, parts, reassignments and recurrence; add at most one narrow observational field inside the existing closing flow; close the loop visibly and give the technician something useful back; use the result for routing before tuning; and make sure nobody is measured on what they report.

Final thoughts

Before building anything, check what proportion of last quarter's corrective work orders were closed with a cause code.

If it is high, the feedback loop is largely already there and the work is to read it. If it is low, that is the project, and it pays back across failure analysis, warranty and reliability reporting as well, which makes it a much easier thing to fund than a feedback feature for a system some of your colleagues are still sceptical about.

Related reading: Labelling BMS anomalies, Failure codes, Alarm code mapping, Faults that clear before arrival, Monitoring drift, Alarm ownership.