[[ Check out my Wordpress blog Context/Earth for environmental and energy topics tied together in a semantic web framework ]]

Saturday, March 20, 2010

Fake Looking Data versus Monte Carlo?

Over the several years I have blogged about technical analysis of oil depletion and other topics, I noticed that I get more feedback whenever I post a Monte Carlo analysis. Typically any raw Monte Carlo results when plotted has the feel of "real" data. The realistic look of the data has to do with the appearance of statistical fluctuations in the output. For some innate reason, I think that noisy profile gives people added confidence in the authenticity of the data.

Yet, for most of my results, I also have a pure analytic result solved strictly by equations of probability. Of course these do not show noise because they provide the most likely outcome, essentially evaluated over an infinite number of samples. Yet, these do not seem to generate as much interest, perhaps because they appear to look "phony" : as in, no data in real-life can look that smooth.

Little do most people realize, but the Monte Carlo simulation results from an inversion of the analytical function, simply run through a random number variate generator. I usually do a Monte Carlo analysis to check my work and for generating statistical margins, but I also think having a bit of realistic noisy-looking output helps to reassure the reader that the results have some perceived greater "authenticity".

So people like to see statistical noise and spiky conditions, yet these same fluctuations make the underlying trends harder to understand. By the same token, other people will dispose of "outlier" data as unimportant. Yet most outliers have great significance as they can reveal important fat-tail behaviors. Morever, often these outliers do not show up in Monte Carlo runs unless the rest of the histogram gets sufficiently smoothed out by executing a large sample space. But then you run the risk that people will say that the output looks faked.

Like gambling, you never win. Go figure.


Post a Comment

<< Home

"Like strange bulldogs sniffing each other's butts, you could sense wariness from both sides"