Experts and the Focusing illusion

Inspired by a recent-ish episode of Rationally Speaking I figured I'd blog about an idea I've had noodling around for a bit.  Stated simply, trust experts much more on matters of "is" than on matters of "ought".  An expert will tend to know much more about their area than you do but the act of acquiring that expertise and the norms of their profession might cause them to have different priorities and values than you and be conscious of that fact.

I think the first thing to consider is the focusing illusion or the fact that, as Daniel Kahneman put it, nothing is as important as you think it is when you're thinking about it.  When you spend careful thought on some topic you will often find myriad ways in which it affects your life.  If you don't spend equal careful thought on other topics you don't see how those others also have myriad impacts.  And hence, you overestimate the relative importance of the first compared to all the others.

Ask someone to estimate the size of any aspect of the Federal budget in comparison to the whole and they'll almost certainly give a gross overestimate.  Foreign aid?  Maybe 10%.  The military?  Maybe half.  Welfare?  Maybe half.  Interest on debt?  Maybe a quarter.  And so on.  If you ask someone about all of the in sequence they might do an OK job but asked in isolation they'll do terribly.

People mostly talk about focusing illusion in relation to quick judgements but it seems to me that prolonged exposure to a single thing can have similar effects.  Health research suggests that people who sunbathe live a bit longer than people who don't.  This is correlation rather than causation but there's a pretty credible causal mechanism in vitamin D and other processes and it suggests at least that sunbathing at least can't be very bad.  But dermatologists not only recommend never sunbathing, they recommend always covering up with long sleeves and broad brimmed hats whenever outdoors.  Why the disconnect?  Well, people who die of skin cancer are central to dermatologists' work and get a lot of attention from them.  People who get heart disease from vitamin D deficiency don't.  So even if the American Association of Dermatologists' advice kills more people on net it saves lives among the people that dermatologists are more focused on.  It's not that dermatologists are bad people, it's just that they've been shaped by the process of acquiring their expertise to focus on the particular area of their expertise and do badly making tradeoffs outside it.

And then you have cases which are less the focusing illusion and more how exposure to certain situations can change people's tastes.  

If you study sculpture for a living you will probably grow bored with attractive representation of human figures eventually, that's what everyone does!  I might enjoy Baja fish tacos but if I had to eat them every meal I'd get sick of them pretty quickly.  The process isn't as fast or thorough with art but I have no reason to believe that it doesn't still happen to some extent.  And as you become a connoisseur of some art form you'll start to appreciate subtleties of it and what ti's expressing that others just won't get.  Finding greater joy in objects is a noble thing overall and I'd actually like to encourage it.  But at the same time I shouldn't go to a sculpture expert for help in finding something for my garden unless they also have experience putting aside their own preferences and helping art rubes like myself.

And then you have cases where a professional organization inculcates particular ethical notions in its members.

Sometimes this is straightforwardly functional.  While I think the conditions enabling scientific advances are interesting I generally find disputes over who was the first to come up with some idea nothing but boring tedium.  But for scientists whose careers depend on being first a concern for the abstract truth of these matters is almost necessary if science as an institution is going to function.  And there are many other aspects of scientific culture that might be odd in the micro scale but also necessary if a solid edifice of truth is going to be built out of the crooked timber of humanity.

And then there are less functional adaptations that come out of professional norms.  Early in the pandemic there was a letter signed by a who's who of famous academic philosophers asking that people be allowed to volunteer to expose themselves to coronavirus in order to help us learn about it.  We let soldiers go to war to protect their homes, we let firefighters enter burning buildings to save those inside, and it would be odd if we didn't extend the same principle here.  But while ethics was firmly on the side of challenge trials medical ethics was just as firmly convinced that they were inconceivable.  Why?  A cynic would say that all professional ethics organizations are ultimately about letting institutions avoid blame rather than letting people do good.  Those can be surprisingly close in many circumstances but not in all.

This idea that expertise changes a person's preferences and values is just one story to use to explain the world.  We humans are a complicated bunch and there are many other stories that might bear on any particular incident.  People are also selfish in a way that transcends their procession.  People are also idealistic in a way that transcends their profession.  People who are already different are attracted to different fields rather than being changed by their fields.  People are also just different from each other in ways that defy stereotyping.  The above isn't anything like the whole story, just a lens to examine the world with.

But if an expert tells you what the effect of some policy is then you should take that much more seriously than when they tell you we must adopt that policy.