“A human reviewed it.” A reassuring sentence in a conversation about the risks of AI. However, the human reviewer is often the weakest link in the chain, not the safety net many assume.
Many AI policies rely on the same safety measure: AI prepares the output and a person checks it. Think about how you review a young colleague’s draft. You expect to find mistakes, so you read critically, challenge the reasoning and check the numbers. With AI output, your critical approach can gradually fade.
The better AI performs, the more responsibility we give it, but the less critically we tend to review its output. This creates a dangerous gap: the system’s impact grows while the control designed to catch its mistakes becomes weaker.
A human in the loop sounds reassuring. But a review of roughly 60 academic studies shows a recurring problem: humans tend to accept convincing yet incorrect AI recommendations instead of challenging them. A human in the loop is only effective if that person is still attentive enough to catch errors.
Many AI policies rely on the same safety measure: AI prepares the output and a person checks it. Think about how you review a young colleague’s draft. You expect to find mistakes, so you read critically, challenge the reasoning and check the numbers. With AI output, your critical approach can gradually fade.
The better AI performs, the more responsibility we give it, but the less critically we tend to review its output. This creates a dangerous gap: the system’s impact grows while the control designed to catch its mistakes becomes weaker.
A human in the loop sounds reassuring. But a review of roughly 60 academic studies shows a recurring problem: humans tend to accept convincing yet incorrect AI recommendations instead of challenging them. A human in the loop is only effective if that person is still attentive enough to catch errors.
