Accountability needs a name on it
A generated answer does not carry responsibility, so something else has to. The last section of a requirements document should say who reviews, what they are checking and what happens when they reject it. Without that, every escalation ends in an argument about whether the system was right.
A workable review process
- A defined set of cases that are always reviewed
- A sampling rate for everything else
- A recorded outcome, so the model is measured over time
None of this is novel process. What is new is that the system cannot be signed off once and then left alone.