Judgement: The skill AI can't replace
Free masterclass page 2 of 3
Decision traps, two types.
You might wonder why this masterclass doesn't focus on cognitive biases, the way most decision courses do.
There's a reason.
Biases are not defects. They are features of a brain that was built for speed. Fast, automatic, effortless thinking is what allowed our ancestors to spot a predator in the grass and tell a poisonous berry from a safe one. It still helps you today, navigating a crowded airport, reading a client's tone in the first thirty seconds of a call, deciding what to eat without paralysing yourself with analysis. The system works. The problem is that it now operates in a world of overwhelming complexity, information overload, and consequential choices it was never designed for. We trip not because the machinery is broken, but because the terrain has changed.
And this leads to the real issue: knowing about these traps is rarely enough to stop them.
I teach this for a living. I've spent fifteen years studying cognitive biases, training professionals to recognise them, building frameworks to address them. And not long ago, I caught myself deep inside a trap I warn others about.
I had become convinced that a particular type of client was the right strategic direction for Meta-decisions. The early signals were ambiguous, some conversations went well, others went nowhere, but I kept reading them as confirmation. A polite "let's stay in touch" became evidence of interest. A rejected proposal was bad timing. When a colleague pointed out that the clients actually signing up came from a completely different profile, I found reasons why that was temporary.
Weeks later, when I stepped back and looked at the pattern as a whole, I recognised what had been happening. I had wanted that strategic direction to be right. And because I wanted it, my reasoning had bent itself around that conclusion, selecting the evidence that supported it, discounting what didn't, all below the level of conscious awareness. I wasn't lying to myself. I was reasoning carefully. The reasoning just happened to be in service of a conclusion I'd already reached.
This is motivated reasoning. Knowing its name did not protect me from it.
Now consider what motivated reasoning looks like when you have AI at your fingertips. You feel a hunch. You ask the AI. The AI, which has no judgement about your situation, only pattern-matching across its training data, surfaces three articles, two case studies, and a framework that all support your hunch. You feel vindicated. You were already wrong before you asked. Now you're wrong with citations.
The question becomes: what would have caught me?
Not more knowledge. Not awareness. What I needed were changes to my process, my environment, and my habits, changes that would have made the trap harder to fall into. That's what the rest of this masterclass is about.
Research in decision science draws a useful distinction between two types of traps: errors that happen because we use the wrong mental strategy (let's call them type I), and errors that run deeper, rooted in automatic associations or driven by our motivations (type II).
Type I: Strategy-based errors
These are structural errors in how we reason, caused by using the wrong mental model or the wrong logic. The good news: there are direct ways to fix them. Point out the mistake, give the right framework, and the fix holds. With some adjustments in how we were taught to think from a young age, many could have been avoided entirely.
There are several such errors, each a place where thinking reliably goes wrong. Each has a specific cognitive or behavioural fix, often a single question you need to ask yourself. As you go through them, notice how quickly the fix becomes obvious once the trap is named.
These errors don't disappear when AI enters the picture. They get a new delivery mechanism. An AI tool running a pilot analysis on a small dataset will present the results with the same false confidence that insensitivity to sample size produces in your own thinking. A detailed, polished AI output makes base rate neglect easier, not harder, because the specifics are so compelling. An AI-generated scenario triggers the conjunction fallacy just as effectively as a human pitch. The traps are the same. The packaging is better.
Type II: Automatic associations and motivated reasoning
The second type is more stubborn.
Knowing about the automatic associations our minds make, or how motivated reasoning affects us, doesn't stop us from falling into them. Motivated reasoning is essentially the psychological mechanism behind motivational biases: distortions in judgement caused by a desire for a particular outcome, wanting something to be true, or wanting to avoid something being true. It operates throughout the thinking process, shaping which information gets noticed, how it gets weighted, which conclusions feel satisfying. Because it's not always conscious, you cannot easily fix a process you don't know you're running.
And AI doesn't help. These traps don't come from a lack of information or processing power. They come from automatic thinking, motivation, emotion, and social dynamics. Giving a faster engine to a driver who doesn't know they're steering in the wrong direction doesn't fix the steering. It gets them there faster.
My story above was motivated reasoning at work. I wanted the strategic direction to succeed. So the evidence that said otherwise got filtered, minimised, reinterpreted. Not because I chose to ignore it, but because the motivation was doing the filtering before my conscious reasoning even got involved.
There are more than 200 documented traps like this. You could learn every one of them and still fall into the next one that matters. That's the point. The ones that shape your working life most, that cost you and your team the most, number closer to 30. Here are nine of them. As you read, notice something: recognising the pattern won't be enough to stop it. These operate below the level where awareness helps.
You'll notice this list doesn't come with direct fixes. That's not an oversight. These traps don't respond to single-tool solutions the way the strategy-based ones do. But they do respond to something.
What follows are three organisations that faced these traps at scale. Their people didn't get smarter. The conditions around their judgement changed.
Nineteen items
A surgeon is about to make the first incision. The patient is anaesthetised, the team is assembled, the lights are bright, the instruments laid out. Everyone in the room is highly trained, highly focused, and highly confident.
No one has confirmed which leg they're operating on.
This is not hypothetical. Wrong-site surgery, operating on the wrong limb, the wrong organ, the wrong patient, happens in real hospitals, performed by competent surgeons, more often than anyone in the profession would like to admit. It is not caused by incompetence. It is caused by skilled professionals, working under pressure, routinely skipping steps they consider obvious.
In 2007, surgeon Atul Gawande and a team at the World Health Organisation set out to measure how often things actually went wrong. Not dramatic headline errors. The routine, preventable kind. Across eight hospitals in eight countries, major complications occurred in 11% of surgical procedures. One in nine.
Most individual surgeons estimated their own complication rate as far below that number. This is overconfidence in its most dangerous form: not arrogance, but the quiet certainty that expertise produces. That certainty is precisely what allows the error to go unchecked. Each surgeon believed they were the exception. The base rate said otherwise.
The errors Gawande's team found were not failures of skill. They were failures of omission: antibiotics not given before incision, allergies not confirmed, the surgical team not introduced to each other by name. Each omission individually trivial. Collectively, they were producing harm in one of every nine surgeries. And nobody noticed the drift, because each time a shortcut produced no visible harm, it felt more acceptable the next time. The standard had moved. Not by decision. By erosion.
The surgeons' instinctive response was revealing. Most framed the question as: am I skilled enough to not need a checklist? Put that way, the answer was obviously yes. And the checklist was impossible.
Gawande reframed it. The issue was not whether any individual surgeon was good enough. It was whether the process surrounding the surgery was reliable enough to catch the errors that any individual, however skilled, would inevitably make under pressure. Not a question about competence. A question about system design.
With that reframe, the checklist became what it actually was: a process tool, the same kind aviation had used for decades. Not because pilots are incompetent. Because the consequences of omission are too high to leave to individual memory.
The checklist was almost absurdly simple. Nineteen items on a single sheet, completed in a few minutes at three points during every operation. Confirm identity. Confirm the site. Confirm the procedure. Has the anaesthesia equipment been checked? Are antibiotics on board? And one item that changed more than any other: does anyone in the room have concerns they haven't voiced?
That last question is a structural intervention on psychological safety. It doesn't rely on a junior nurse finding the courage to challenge the senior surgeon. It builds the invitation into the process. The hierarchy is still there. But the checklist gives everyone in the room explicit permission to speak.
The results, published in the New England Journal of Medicine in 2009, were unambiguous. Across 7,688 patients in eight countries, complications fell from 11% to 7%. Deaths fell from 1.5% to 0.8%, a reduction of more than 40%. The results held in high-income and low-income settings alike.
Later research revealed that the checklist's effectiveness depends on how it's used. When teams treat it as bureaucratic box-ticking, it loses its power. Gawande acknowledged this: the checklist is not the solution. It's a catalyst for the behavioural and cultural changes that produce the solution.
The same hospitals that adopted Gawande's checklist are now integrating diagnostic algorithms that outperform individual doctors at pattern recognition in imaging, pathology, and differential diagnosis. The technology is impressive. And it is introducing a new form of the same old trap. When the AI says the scan is clear, the radiologist's vigilance drops. When the algorithm recommends a course of treatment, the surgeon's independent judgement gets quieter. Not because they're lazy. Because the system feels reliable, the same way their own expertise once felt. The feeling of reliability is where the next generation of errors will live. The checklist was built for the overconfidence of human expertise. The next one will need to be built for the overconfidence of algorithmic assistance. Same trap. New source.
The checklist changed what happened inside a single room: one team, one procedure, one set of judgements made under pressure. But what happens when the problem isn't one person's judgement? What happens when a whole team is too close to its own work to see what's wrong with it?
The screening room
In the middle of production, the senior creative team at Pixar sat in a darkened screening room and watched the latest cut of a film that was not working.
This was not unusual. What was unusual was the film: Toy Story 2, the sequel to the movie that had launched the studio. The cut was broken, the story incoherent, the emotional core missing, the characters flat. The entire production would have to be restarted. The team in the room knew it. The director knew it. Nobody had said it clearly until this meeting.
Ed Catmull, Pixar's president, later made a statement that became famous inside the company: "Early on, all our movies suck." It wasn't false modesty. It was a description of how creative work actually develops, and an acknowledgment that the people closest to a project are always the last to see its problems clearly.
The reason is simple and human. A director who has spent eighteen months building a world, developing characters, and shaping a story does not hear feedback the same way an outsider does. When someone says "the second act drags," the director hears it through everything they know about why the second act is the way it is, the backstory that justifies it, the payoff it sets up, the constraint it works around. The feedback is absorbed into the existing vision rather than allowed to challenge it. This isn't stubbornness. It's the natural behaviour of a mind that has committed deeply to a hypothesis. Confirmation bias doesn't require bad intentions. It requires investment.
There's a second problem, subtler and equally dangerous. Pixar's track record, a string of critical and commercial successes unmatched in the history of animation, creates a feeling inside the building that things will work out. Every previous film went through a broken phase and emerged brilliantly. That history is reassuring. And reassurance is the enemy of urgency.
And then groupthink. The production team has been working together for months, sharing the same assumptions, the same references, the same language. Doubts about the director's vision feel disloyal to voice. The team converges on "it's coming together," an assessment that reflects social harmony more than honest evaluation.
Pixar's answer to all three is a practice called the Braintrust.
Every few months, a group of senior creative people, directors, writers, and story leads from other projects, gathers to watch the current state of a film in production and give the director candid feedback. Their job is to say what is not working and why. Not to be polite. Not to hedge. Not to suggest fixes, that's the director's job. The Braintrust diagnoses. The director treats.
The design choice that makes this work is counterintuitive: the Braintrust has no authority. None. The director doesn't have to follow a single suggestion.
It sounds like it would make the feedback toothless. It does the opposite.
When feedback comes from someone who can kill your project, you stop listening to the content and start managing the relationship. You defend, you explain, you perform confidence. When feedback comes from a peer with no power over you, the defensiveness drops. You can actually hear what they're saying, because hearing it costs you nothing. The absence of authority is what makes honesty possible.
Toy Story 2 was restarted after such a session and became one of Pixar's most acclaimed films. Ratatouille, Inside Out, and numerous others were substantially reshaped by the same process.
What Pixar understood, long before the current moment, is that evaluative judgement, the capacity to look at something and say whether it works, is the faculty most easily dulled by proximity and most easily protected by structure. As AI tools generate more of the raw material in every industry, drafts, options, analyses, designs, that evaluative capacity is becoming the entire human contribution. The Braintrust was built to protect exactly this. Not by making people smarter, but by making sure the right people were in the room with the right distance and the right safety to say what they actually saw.
The Braintrust solved the problem within a creative team. One group, one project, one set of blind spots corrected by people who weren't too close. But what happens when the trap isn't inside a team? What happens when the entire organisation, its incentives, its culture, its definition of what good performance looks like, is the thing suppressing good judgement?
Know-it-all
In 2011, a cartoonist named Manu Cornet posted a series of satirical org charts of the major tech companies. Google's was a tangled but functional web. Apple's was a neat wheel with one man at the centre. Microsoft's was a set of boxes pointing guns at each other.
The cartoon went viral. Inside Microsoft, it stung. Because it was accurate.
Under CEO Steve Ballmer, Microsoft ran a performance review system called stack ranking. Every team, regardless of how well it performed collectively, was required to sort its members into a fixed distribution: a small percentage rated top performers, a larger middle, and a mandatory bottom tier facing demotion or termination. Ratings were relative. Your performance was measured not against your goals but against your teammates.
The system was designed to create a meritocracy. What it created was a war of all against all. Engineers avoided strong teams because being ranked against excellent colleagues was dangerous. Helping a peer meant helping someone who might outrank you. Sharing knowledge across teams was irrational, because the team next door wasn't an ally but a competitor for the same resources and recognition. Information did not flow between groups because there was no incentive for it to flow and every incentive for it not to.
The result was a company with extraordinary technical talent that could not collaborate. Microsoft missed mobile. Windows Phone was a catastrophic failure. It was late to cloud, ceding the lead to Amazon. Between 2000 and 2014, while Apple and Google reshaped the technology industry, Microsoft's stock went essentially nowhere. The technical capabilities were never the problem. The system in which those capabilities operated was the problem.
Satya Nadella became CEO in February 2014. He later described what he inherited with unusual candour: "We'd lost our soul."
His diagnosis was not that Microsoft needed a better strategy. It needed a different decision environment, different conditions for how information was shared, how judgement was exercised, how people related to each other and to their work. Strategy would follow from culture. Culture would not follow from strategy.
The intervention started with a reframe so simple it almost sounded like a platitude. Microsoft needed to shift from a "know-it-all" culture to a "learn-it-all" culture. The phrase came from Carol Dweck's research on growth mindset.
In a know-it-all culture, the highest-status behaviour is demonstrating that you're right. Being right means you're smart. Being smart means you're safe. In this environment, admitting uncertainty is weakness, asking for help is failure, changing your mind is defeat. These are precisely the conditions that prevent an organisation from learning: from updating beliefs in response to evidence, hearing bad news early, catching errors before they compound.
In a learn-it-all culture, the highest-status behaviour is curiosity. Admitting what you don't know is a starting point. Asking questions is valued. Changing your mind in response to evidence is strength, not inconsistency.
Nadella didn't just announce the shift. He modelled it. He listened more than he spoke in meetings. He shared personal stories about mistakes. He asked questions he didn't know the answer to in public, in front of senior leaders, and treated the answers as genuinely informative rather than as challenges to his authority.
Then he changed the structures. Stack ranking was gone. Nadella replaced it with a performance framework that rewarded collaboration, learning, and team impact. The question was no longer "who should be ranked lowest?" but "how did this team create impact together?" The incentive system, which had been actively rewarding the behaviours destroying the company, was realigned with the behaviours the company needed. Microsoft shifted from a Windows-centric structure, where every team defended the franchise, to a functional structure organised around capabilities: cloud, AI, productivity. Teams that had been walled off from each other were reconnected.
The results took time, but when they came, they were staggering. Azure became the world's second-largest cloud platform. Teams became the dominant workplace collaboration tool. Microsoft embraced former rivals, making Office available on iOS and Android, partnering with Linux, integrating with open-source communities it had spent decades opposing. Market value went from roughly $300 billion to more than $3 trillion within a decade.
The temptation is to tell this as a leadership story about one brilliant CEO. It's more accurate, and more useful, to tell it as a decision environment story. The people inside Microsoft in 2013 were substantially the same people inside Microsoft in 2016. Their individual abilities didn't change. What changed was the system around them: what was rewarded, what was safe, what information flowed where, and what the highest-status behaviour in a room looked like.
Microsoft's transformation was, in essence, a shift from a culture that optimised for having answers to one that optimised for asking questions. A know-it-all culture treats certainty as the highest virtue. A learn-it-all culture treats curiosity as the highest virtue. That distinction mattered when the competition was Google and Apple. It matters far more now, when the most powerful tool in your organisation can produce confident, polished, data-supported answers to virtually any question in seconds. An organisation that rewards having answers will defer to that tool. An organisation that rewards asking questions will use it, and then exercise judgement over what it produces. The first outsources the very faculty that makes human leadership valuable. The second strengthens it.
The people don't change. The conditions around their judgement do.