Sunday, September 7, 2008

The value of predictions

Many organizations and individuals make predictions about all sorts of things. Some of these turn out to be on target and many do not. We have become quite good at predicting things we can make good mathematical models for such as the weather. By quite good, I mean that we can forecast a few days into the future. When it comes to forecasting the outcomes of complex systems (which covers nearly everything else we care about) then the results are typically poor. We tend to hear about the successful predictions and not the ones that are wrong so our perspective is focused on the fact that some people do get it right. Unfortunately we don’t know who those people are and we also don’t know which predictions they will get right and which they will get wrong. Most forecasters get lucky once in a while but the majority of them do not so it's hard to pick someone to follow with any confidence.
The bottom line is that when we make a prediction we are making a guess based on whatever facts we have at hand, which models we are using, our personal intuition and the exact distribution of some tea leaves in the bottom of a cup. The scientific and engineering professions have made some fantastic leaps over the last century or so but few of the significant ones were predicted. We just don’t know what we'll be able to do in five or ten years. Only time will tell.

6 comments:

TM said...

Nigel

I echo your sentiments and in regard to the example you used no argument will come from me, but with technology (new fuel sources) the socio/political climate can come into play and make or break a prediction and that is that is the sad truth of the matter. If the powers that be do not want something to happen, then odds are it will not. That is a sad commentary to teh populace as a whole and really can hamstring the engineering and scientific professions.

askill said...

Nigel: What role do you think $ plays in the whole realm of prediction. Those who have probably don't want change unless they can control for their own profit. The reverse is if the future can be laid out by currently empowered their is a they can also benefit. Pure research to develop the future best thing "since sliced bread" is generally more expensive and less likely to come out of a garage.

wincoder said...

TM: So I agree that someone can scuttle an idea but the prediction needs to take that into account and that makes the prediciton harder to do. You can't say - I predict that . . . unless Joe loses his watch. Otherwise you'd need to include thousands of conditional terms.

Al: Same answer. Money plays a big role but if you predict something you need to take that into account - otherwise it's not a valid prediction.

asdasd said...

>>Most forecasters get lucky once in a while but the majority of them do not so it's hard to pick someone to follow with any confidence.

Do you have the stats for that one way or the other? I'd be curious to see what the actually success rates are for folks like Halal and his fellow snake-oil salesmen.

wincoder said...

My info comes only from Sherden's "The Fortune Sellers" and the application of his ideas to what I see in practice. I.e. people offering to sell you advice on (say) stocks don;t seem to be so rich that they don't need you to buy their predictions.

Adolf Reich said...

I think that as a society we're especially susceptible to the deception peddled by the prognosticating hucksters in financial & investing arena. An honest, accurate prediction is the last thing you can expect from them. They're purpose is to entertain, generate advertising dollars, and deceive the masses for insider profit. So yes, I think this concept is endemic throughout our society - much wealth cleverly siphoned off our economy with worthless predictive advice.