Key Takeaways
- True AI augmentation requires deliberate strategic friction to prevent human cognitive atrophy and build compounding institutional memory.
- Having “humans-in-the-loop” in your company's AI adoption becomes meaningless if employees passively approve AI drafts without applying critical thinking or lived experience.
- Cognitive debt accumulates when tasks are completed efficiently through AI without being integrated into a worker's memory or understanding.
- Documenting and logging human overrides of AI outputs creates the essential feedback loop needed to make AI systems smarter over time.
- The future of recruitment will test for both AI fluency and unassisted critical thinking to ensure workers are fluent rather than dependent.
The real shift in today’s world of work is subtle but profound: Decisions are no longer made by an individual, but by a human+AI system. Drafts, analyses, product ideas, and even risk assessments now start with an AI output that feels authoritative and complete.
Most organisations tell themselves a comforting story: We have humans in the loop. AI drafts, humans decide. Partnership achieved.
The research tells a different story.
I've watched this happen in teams I've worked with. In one client, a team proudly explains their AI-augmented UX research workflow. A researcher pastes sanitised transcripts into an LLM tool, asking it to tag each transcript and cluster themes. Then a researcher scans the output, makes minor edits, and shares the insights with stakeholders.
I asked one question that silenced the room: “How do you know the AI tagged the transcript correctly and completely? Did it pick up on the subtle cues that only lived experience as a researcher can notice?”
The work got done. The thinking didn’t.
Recent work from MIT makes this more than an anecdote. Researchers divided university students into three groups: 1. Library-access only, 2. Use of Google, and 3. ChatGPT. Each group wrote essays over four months.
The ChatGPT group showed significantly lower brain-wide connectivity, weaker engagement of executive networks, and produced essays that outside judges described as “soulless.” When asked to revise without AI, many struggled to recall what they had written. The researchers called it “cognitive debt”: tasks completed efficiently but never integrated into memory or understanding. Sequence matters too: students who wrote unaided first, then refined with AI, showed stronger neural engagement throughout.
For leaders, the implication is clear: AI is not just changing what your people produce; it is changing how much genuine cognitive effort they invest. If the machine gets faster and the human brain gets lazier, the partnership becomes fragile.
Why “human-in-the-loop” is not enough with AI adoption
Most organisations comfort themselves with that familiar phrase. In reality, it often means a tired employee skimming an AI draft at 4 p.m. and clicking approve.
We already know how that story ends. In the COMPAS recidivism case, a risk-scoring algorithm was introduced as decision support for US judges, not automation. On paper, humans remained in charge. Judges had everything they needed: the nature of the crime, behaviour in custody, personal context, victim impact, and prior record. COMPAS was meant to add just one more input: a statistical prediction of reoffending likelihood.
In practice, many deferred to it. One data point displaced five. Corporate AI is quietly reproducing the same dynamic without the public scrutiny, and without the Royal Commission.
A passive human-in-the-loop contributes nothing to the thinking the organisation needs, and nothing to the institutional memory that would make the AI smarter over time. No captured reasoning, no documented overrides, no record of where the system got it wrong. The loop never closes.
In the case of the research team I was working with, AI can tag transcripts, but when 70% of human communication is non-verbal, it requires lived experience, emotional intelligence, or simply seeing the way someone interacts with a prototype. The question I posed to the room was: can we rely on its tagging of transcripts?
Removing that cognitive friction and emotional intelligence of a researcher makes the insights fragile, incomplete, and the downstream decision-making of product teams and business stakeholders easily fragile.
“Just because AI CAN do a task. It doesn't mean it SHOULD do the task.”
Designing the loop
What smart organisations are building now isn't a handbrake on AI. It's the architecture of a three-way compound.
AI delivers speed and scale. Humans who are genuinely thinking scope, steer, question, override, and get stronger at exactly the judgements AI cannot make. And when that thinking is captured and logged, it becomes the institutional memory that can be fed back into the AI to make it smarter over time about your specific context, customers, and edge cases.
- AI's speed and scale
- Stronger human judgement
- More intelligent AI
All three, compounding—not because of the AI alone, but because of how the human and AI halves are deliberately designed to work together.
The most undervalued pattern for making this real is captured overrides: logging human disagreements with AI outputs and the reasons behind them. Every documented override becomes part of the organisation's institutional memory.
Feed it back, and the AI gets smarter about your specific reality. The customers it keeps misreading, the edge cases it keeps missing, the ethical lines your organisation draws that no training set could anticipate. Without this, the AI never learns from experience. Mistakes keep happening.
Good AI design is not about obsessing about efficiency.
Making everything as frictionless as possible is actually one of the riskiest things you can do in the AI era.
It is about inserting strategic friction – pause at the right moment so humans do the work only they can do, and so that work has somewhere to go.
The signal is already in hiring
The evidence that something is shifting isn't coming from think tanks. It's showing up in how the most sophisticated organisations recruit.
Gartner's top strategic predictions forecast that atrophy of critical-thinking skills, driven by constant GenAI use, will push around 50% of global organisations to require AI-free skills assessments, reintroducing in-person whiteboard challenges, air-gapped coding assessments, and unassisted case-study analysis. Not as an anti-AI statement, but as a direct response to AI obscuring professional capabilities and watching capable people become dependent on a tool they can't override.
By 2027, Gartner predicts 75% of hiring processes will include AI proficiency assessments too. Both. In the same process.
That's not a contradiction. It's the clearest signal of what these organisations now value: people who can use AI well, and think without it when it matters. AI-fluent, not AI-dependent. The loop only works if the human-half is genuinely strong.
The real question for leaders
Over the next year, many enterprises will move from AI assistants to AI agents—autonomous systems that initiate actions, orchestrate workflows, and interact with other tools with minimal human touch. The value they create and the harm they can cause will depend less on the model and more on the humans who design, deploy, and govern them.
The organisations that define this next decade won't be the ones who moved fastest. They'll be the ones who moved fast and kept their people thinking. Who built the deliberate architecture that kept human judgement active, exercised, and fed back into the system.
AI gets faster execution. Humans get stronger judgement. The AI gets smarter from what humans notice, question, and record. Each makes the other more valuable over time.
That's why I called my company AI Flywheel. That's what winning looks like.
The question isn't “How quickly can we deploy AI?” It's whether you're building the human half of this equation with the same deliberate investment as the AI half.
Because without it, you don't have a compounding effect.
You have a machine running without a steering wheel, and nobody who remembers how to drive.
References
- Kosmyna, N., et al. (2025). Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task. MIT Media Lab / arXiv preprint (not yet peer-reviewed). media.mit.edu · arxiv.org/abs/2506.08872
- Angwin, J., et al. (2016). Machine Bias. ProPublica. propublica.org
- Angwin, J., et al. (2016). How We Analyzed the COMPAS Recidivism Algorithm. ProPublica. propublica.org
- Gartner, Inc. (2025). Gartner Unveils Top Predictions for IT Organizations and Users in 2026 and Beyond. Press release, 21 October 2025. gartner.com
- Gartner, Inc. (2025). Gartner Says AI Revolution and Cost Pressures Are Two Forces Driving the Top Four Trends for Talent Acquisition in 2026. Press release, 7 October 2025. gartner.com
About the author
Riley Coleman (they/them) is a human-centred design veteran and AI adoption coach with 15+ years transforming how people interact with technology. Riley's impact-driven career spans award-winning harm minimisation programmes (recognised by the International Red Cross and United Nations) to leading enterprise design transformations at companies like Telstra or building design teams from the ground up in the EU at Epidemic Sound and Taxfix, where they scaled teams 400% whilst pioneering AI-enhanced user research methodologies. Now, as founder of AI Flywheel, Riley democratises AI expertise through human-centric programmes that rapidly build AI literacy and make trustworthy AI implementation practical and approachable. Drawing from extensive experience in harm minimisation and service transformation, and AI ethics; they're building a community of practice around ethical AI adoption. Helping professionals develop confidence to work alongside AI whilst maintaining human agency. Riley's core conviction: the best innovation happens when we combine technical capability with human insight. We all have the chance to tip the scales towards a future where humanity thrives alongside trustworthy AI systems.
Latest.
What future marketing teams look like in the age of AI
Hiring Insights, Marketing
Your complete guide to salary negotiation
Job Seeker
A new job… in this economy?
Job Seeker
