August 2012

                                                                  EXPERTS THAT AREN'T

Me for example.

Bankers for another.

And don't let me start in on weather forecasters!

I should have realised how little I really know about single line kite stability years ago, the evidence was clear.  Like that I approximately doubled my effectiveness at taming errant kites by a simple expedient:  Making whatever change is indicated by the theory I was currently enamoured of, and then trying exactly the opposite as well (which works almost as often).  So much for my fancy theories therefore.

And likewise, bankers (and money managers, and economists) should have realised years ago that they didn't know much about what was actually going on in the world's economy, but it's taken the ongoing meltdown to reveal just how little they do understand.

So how is it that experts in complex fields like kite design and managing economies don't know what they're doing?

I've just found out, courtesy of one Daniel Kahneman*.

Kahneman is a behavioural economist.  Actually his doctorate is in psychology, but the field of study he has made his own (with Amos Tversky) is behavioural economics; the study of how people actually behave rather than how they 'should' behave according to economic theories propounded by various luminaries from Adam Smith thru Marx and Keynes to the 'rational man' model of the Chicago School.

Kahneman has turned the world of economics upside down (and received a Nobel prize for it).  And not just economics either, his findings have wide application- though I expect he may be a little surprised to find them being applied to kite designing.

Surprise surprise, we aren't logical or rational in our decision making.  Asked the same question phrased in subtly different ways, we give opposite answers.

But amongst Kahneman's 40 years of evidence-based insights into the human condition, there is one that strikes me as particularly significant- for those of us who make kites, for the world's financial health and for lots of other specialist fields as well.

It's that experts aren't.

After exhaustive controlled studies, Kahneman et al found that in fields that have embedded feedback effects (like the stock market and kite flying) or that involve many related variables (such as first-visit medical diagnosis), the more 'expert' someone became, the less successful they were at predicting cause and effect and trends, relative to an informed 'man or woman in the street'.

The explanation he uses to explain this finding is the "framing" effect; that the more you know about something (or the more recently you have been exposed to a particular influence) the more weight you give to that specialist knowledge or recent contact relative to all the other factors that are likely to be in play.  For example, a kidney specialist (or a GP who has just seen four patients in a row with kidney disease) will tend to frame the next patient's symptoms in terms of kidney disease-even if they have something else entirely.

In one particular study, Kahneman's team analysed the performance of a leading money management firm (at their request)  The conclusions were unequivocal- the traders predictions of various indices and future trends in particular stocks and bonds were less accurate than predictions by reasonably informed outsiders- and were often not even as good as a coin toss.  They were destroying value for their clients rather than enhancing it.

Which is startling enough, but then it gets much worse:

In the money managers case above, Kahneman presented this information to them in a form (numbers and stats) they were familiar with (these things are tools of trade in the money game)- and wasn't pulled up, then or later, for any errors in the base data, methodology, or conclusions.

Astonishingly, this made absolutely no dent at all in the trader's supreme confidence.  When surveyed afterwards, their belief in their ability to pick stock prices and trends remained at the very highest level compared to that of outsiders who had actually achieved consistently better predictive success (used by Kahneman as his 'control' in this study).

Faced with irrefutable evidence that what we think we know is completely wrong, rather than changing their view, experts (and indeed all of us) are somehow able to completely ignore this evidence and carry on as though we'd never heard it.

Now I'm sure I'm not this stupid- and I'm sure you aren't either.

But it appears that we are.  Astonishing.

So bankers not only destroyed their own businesses (requiring public bail-outs), they had a serious go at taking the world down with them- but woke up the next morning still believing in their own competence , high ethical standards, and that they deserve eye-watering salaries and bonuses.

And I can design a new kite that's built on complex hard-won theories developed over 50 something years of doing this, and when it doesn't fly, get up the next day and start on a new idea, full of self belief. 

Kahneman is by no means the first person to have noticed how easily taken in by expertise we are (the experts themselves most of all):

Writer H.L Mencken (1880-1956) said; "For every subtle and complicated question, there is a perfectly simple and straightforward answer, which is wrong".

Kahneman is just the first to have accumulated enough evidence to take this from an interesting observation into the realm of hard science (or as close to as behavioural psychology has ever achieved).

So what happens now?

Bankers should all be shot- not for incompetence (there's plenty of that going around) or even for criminal behaviour (this too), but for deluding themselves.

Kite makers?  We should be shot also, same reason; do any of us know what we're doing?  But in the case that no-one bothers to, I think I'll stop trying to be an engineer and become an artist- they're expected to be delusional- plus some of them get paid heaps.

Just like bankers.

Hmmm.

Peter Lynn, Ashburton, NZ August 1 '12

*"Thinking Slow and Fast", Daniel Kahneman, Farrar, Straus and Giroux, 2011.