Teaching Sensors to Say "I Can't See"

Motivation

Every sensor is blind somewhere, by its nature rather than by a defect. A microphone needs someone to speak, a camera needs line of sight, a WiFi link only sees motion inside its own geometry. So we combine sensors, on the premise that they cover each other’s gaps.

The trouble starts when the world itself silences the evidence. A person asleep on the sofa does not move, so the motion sensor reports an empty room. A pump-turbine that is switched off makes no sound, so the acoustic monitor reports a healthy machine. Nothing is broken: the sensor is working exactly as designed, and its readings look perfectly normal. Adding a second sensor of the same kind does not help, because it is blind in the same places, and three identical wrong readings only raise the confidence the pipeline reports.

Today the output looks the same either way: a confident “empty”, a confident “healthy”. The question nobody passes along is the one that matters: could this sensor, under the conditions of this interval, have told the answers apart at all?

Problem statement

Many fields already compute this question. Target tracking treats a missed detection as evidence, sensor management scores a sensor before tasking it, and active hypothesis testing picks the next measurement by how well it would separate the open answers [1], [2]. Model-based diagnosis asks which assumption to question when evidence conflicts [3], [4]. But each of them computes it to decide what to measure next, and then throws it away. It never travels with the claim.

Weighting schemes cannot fill the gap, because they learn trust from the readings, and the readings are exactly what the silenced state suppressed. A blind sensor in a quiet place looks precise. Sensors blinded together agree. In our own deployments we see the consequences: a day resting on a single observed window is reported as fully observed, a missing indicator is averaged in as if it were a neutral reading, and a monitor that declines to score a window emits the same record as a healthy one.

This thesis closes that gap for one real system: compute, for each sensor view and each interval, whether it could have discriminated between the answers in play, and carry that verdict up to the decision, so the system says “cannot assess” instead of “nothing is happening”.

Research questions

For a given sensing question (is anyone in the room? is the machine healthy?), how can a forward model and the measured conditions of an interval tell whether a sensor view could have separated the possible answers, without looking at its readings?
How often, in a real deployment, does a confident output rest on views that could not have seen the answer, and which states (sleep, stillness, a machine switched off) cause it?
When views with no discriminating power are withheld and “cannot assess” is reported explicitly, how do false negatives and disagreements between views change?

Suggested methodology

Pick one deployment from the group: the WiFi sensing mesh in our lab rooms, the in-home voice array, or the hydropower monitoring pipeline. All three have logged data and known blind spots.
Model what each view could see. Write a simple forward model per view (e.g. Fresnel-zone geometry for a WiFi link, speech presence for a microphone, the operating mode for the turbine) and measure the conditions of each interval that do not come from the claim itself: link liveness, gain state, operating mode.
Compute discriminability per interval: how different the predicted readings would be under the competing answers. Zero means the view is not a witness for that question in that interval.
Carry it to the output. Extend the pipeline so every claim leaves with the views it rests on and an explicit “cannot assess” when none could discriminate.
Evaluate on scripted episodes with known ground truth (a still occupant vs an empty room, a machine off vs running quietly), comparing the original and the extended pipeline on false negatives, abstentions, and view disagreements.

Previous theses in this line

Explainable Multimodal-based Depression Awareness at the Edge, Tibor Haller (CML).
Self-Supervised Transfer Learning for Acoustic Anomaly Detection in Pumped-Storage Hydropower Plants, Stefan Krummenacher (label-free sentinels that detect when a calibration has expired).
Multimodal Sensing for Acoustic–Vibration Based Fault Localization in Hydropower, Zakaria Omarar.

References

[1] H. Chernoff. “Sequential Design of Experiments.” The Annals of Mathematical Statistics, 30(3):755-770, 1959. https://doi.org/10.1214/aoms/1177706205

[2] M. Naghshvar and T. Javidi. “Active sequential hypothesis testing.” The Annals of Statistics, 41(6):2703-2738, 2013. https://doi.org/10.1214/13-aos1144

[3] R. Reiter. “A theory of diagnosis from first principles.” Artificial Intelligence, 32(1):57-95, 1987. https://doi.org/10.1016/0004-3702(87)90062-2

[4] J. de Kleer and B. C. Williams. “Diagnosing multiple faults.” Artificial Intelligence, 32(1):97-130, 1987. https://doi.org/10.1016/0004-3702(87)90063-4