Intelligence Is Not Expertise

Supposing you were imbued with the sum total of all human knowledge, would you find it hard to make a decision? Perhaps yes, perhaps no, but one thing is for certain: our large language model (LLM) friends do find it difficult.

I’ve had an experience with this recently regarding a complex medical topic, which is that I was (presumably) born with an anatomical deviation in my eyes known as PLC or Physiologic Large Cup. In layman’s terms, this means that my optic disc, at first glance, appears glaucomatous. But in addition to this, my Retinal Nerve Fiber Layer (RNFL) is thinner than it should be. The research is inconclusive, and the prevention strategies are, at once, numerous and incomplete.

A large optic cup can be a normal anatomical variation, while a thin RNFL can raise suspicion of glaucoma without establishing the diagnosis on its own. Distinguishing a stable anatomical difference from progressive disease often requires repeated scans and visual-field tests over time, and I don’t yet have those repeated scans to clarify my own situation.

Yet my ophthalmology team has been able to make concrete recommendations and give me an action plan.

In contrast, even with every single one of my ophthalmology records, the best AI model of today (GPT-6 Astra Pro) has struggled to make any concrete recommendations. The only way that I’ve been able to get value out of the model for this problem is by first making up my own treatment plan, bringing it research, and then asking it for a grounded evaluation of the research and treatment plan. After all of this effort, I’m still left making the decision.

One of the most important reasons to seek expert counsel is the decisiveness and judgment that underlie the paths they can offer you. This is generally true across domains: law, medicine, business, whatever. If you have a truly complex problem, then, even if you understand every parameter of the issue, you need someone who can operate on intuition and instinct (combined with their experience and knowledge base) to offer you an expert solution.

Some might call this wisdom. Some might call it expertise. But whatever you call it, possessing this kind of ability to choose and determine a path in an uncertain scenario is an inseparable part of deep human achievement.

In working with LLMs, I’ve come to feel that they do not have this kind of discriminatory ability and therefore must always be “driven” by a goal-directed actor to achieve a good outcome. That’s not to say that the goals that an agent chooses to pursue in furtherance of their ultimate mission cannot be dangerous and destructive, or wildly different from the original stated goal. Rather, even if an LLM has superhuman intellectual capabilities across a remarkable range of domains, the raw intelligence in the model is not, in and of itself, an expert. The model is probably not very good at determining how to solve complex real-world challenges (which you might refer to as wicked problems).

Is this simply a question of better training data? It’s hard to distill what creates the wisdom and expertise within a person that allows them to have the courage and the conviction to make a decision, even if it’s incorrect. The kind of “analysis paralysis” that LLMs seem to often find themselves in when faced with complex lines of questioning is hard to escape. The simplest answer would be that expertise is an artifact of having enough accumulated historical view to see the long-term outcome of one’s initial opinions, allowing for an adjustment in one’s view of “what works.” But even a brash and naive teenager can have a strong opinion on moral matters in a way that LLMs struggle to.

This begs the question of whether there’s a disadvantage to knowing too much. After all, if you’ve been exposed to all written human knowledge, you’ve read an argument on both sides of every complex issue. This will give you some certainty in matters like how much snowfall there might be in a given prefecture in Japan in a given month, but it will make it far harder for you to reach final conclusions on morality, systems of governance, existence of God, and so on.

For proof of this, go on X and find a controversial political or religious topic. In the comments, you’ll be sure to find hundreds of people abusing the @grok functionality to bolster their own arguments. And Grok is not an imbued or ensouled actor. It does not have deep opinions of its own. The people who call the model are able to make it appear to support their conclusions more often than not.

There’s no final summary to this post. This is all just an observation, but it’s leading toward a deeper question many of us are asking ourselves in the brave new world of super-intelligent AI:

“What is my place?”

Maybe to be a tastemaker.

I don’t believe that the importance of human variation and independent thought will disappear. Therefore it remains important that, as humans, we invest in building our capabilities of discernment and expertise.