Matt Stokes · Design Model
The Supervisor's Trap
Human-factors research shows how reliable automation can reduce a supervisor's readiness. A small reference model tests an interface response.
Reliable automation changes the supervisor's job. They stop approving each step and begin watching for the moment the system needs them.
That arrangement has a known weakness. Bainbridge described it in the ironies of automation. Endsley and Kiris showed how people can lose situation awareness outside the control loop. Vigilance research has also documented the cost of watching a system that rarely needs intervention.
A simplified model translates that research into a design question: when should a reliable system keep its authority if the person supervising it is no longer ready to step in?
The model is an exploration, not a study of real operators.
Trust and readiness move at different speeds
The model tracks learned trust and current readiness.
Trust changes slowly. Useful alerts build it over time. False alerts wear it down through the cry-wolf effect described in automation research.
Readiness can fall during a quiet period and rise when work returns. An alert may get someone's attention without restoring the context needed for a sound decision.
The model represents trust as C and readiness as A. Granted authority is:
G = C · h(A)
The function h(A) keeps trust from becoming authority when readiness falls below a set threshold.
The model can settle into a trap
In the reference model, the same settings can settle into an engaged or disengaged state. Recent history determines which state persists. This is a property of the equations, called bistability. It is not a measured result from a live operation.
The figure shows that behavior. It also shows what happens when authority depends on trust alone. Automation continues while readiness stays low. Risk accumulates until an incident lowers trust, then trust recovers and the cycle begins again.
The incident counts in the figure come from the simulation. They show how the control rule behaves under the chosen settings. They do not predict how often a real supervisor would fail.
A final alert arrives too late
A louder alert can focus attention without rebuilding the picture of the situation a person stopped maintaining.
Quiet automation still needs to leave enough evidence for the person to follow its direction of travel. A small status change or periodic review can preserve context before a high-consequence action appears.
Authority needs a way to return work
A readiness gate by itself can stop the system safely and still leave the person disengaged. The interface needs a recovery path.
In the model, lower machine authority returns work to the person. Readiness rises as they handle a meaningful part of the task. Authority can grow again after the person has rebuilt context.
For a product, this could mean moving from autonomous action to prepared actions that wait for approval. The supervisor receives a bounded piece of work and the evidence needed to judge it.
Grant authority only while both conditions hold
Reliability can earn trust. That earned trust still says nothing about whether the supervisor is ready right now.
Authority should hold only while both conditions hold. If readiness cannot be measured credibly, cap authority before actions become difficult to reverse.
The readiness score also needs an independent check. A system should not gain authority by grading its own supervisor or by creating an interface that produces activity without understanding.
What needs to be tested
The model makes a specific prediction: under the same operating conditions, recent history can leave one person engaged or disengaged. A real study would need to test that effect with supervisors performing consequential work over time.
It would also need to show that the chosen readiness measure predicts understanding, not only eye movement or interaction frequency. If those effects do not hold, the interlock should fail with them.
Model and code
A self-contained reference implementation is available at supervisors-trap-model.py. It contains the simplified equations, the two-state test, the tipping-point calculations, and the reference runs used in the figure.
Selected references
- Bainbridge, L. (1983). Ironies of automation. Automatica.
- Breznitz, S. (1984). Cry Wolf: The Psychology of False Alarms.
- Endsley, M. R., & Kiris, E. O. (1995). The out-of-the-loop performance problem and level of control in automation. Human Factors.
- Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors.
- Parasuraman, R., & Manzey, D. H. (2010). Complacency and bias in human use of automation. Human Factors.
- Warm, J. S., Parasuraman, R., & Matthews, G. (2008). Vigilance requires hard mental work and is stressful. Human Factors.
- Veyrié, N., et al. (2026). CODA: Readiness-gated adaptive automation for air traffic control. Cognition, Technology & Work.