Power Laws and Entrophic Systems

Every time I look at a distribution of some new set of data I come across, I can't help but notice a power law describing the trend. Also known as fat-tail distributions, NN Taleb popularized these curves with his book The Black Swan. Curiously, both he and Benoit Mandelbrot, his mentor, never spent much time pursuing the rather simple explanation for the classic 1/X cumulative distribution, often known as Zipf's Law or the Mandelbrot/Zipf variant.

That drives me crazy, as I don't consider making the assertion of the fat-tail's origin a huge academic risk. To me, I find that a power law invariably comes about when you invert a measure that follows a Poisson or exponential distribution with respect to its dimension. For example, a velocity distribution that shows a damped exponential profile, when inverted to the time dimension will give a fat-tail power law. I call that entropic velocity dispersion. See the last post on train scheduling delays to appreciate this.

So why doesn't anyone understand this rather simple transform and the general concept of power laws? Let us go through the various disciplines.

The Statistician: Hopes that everything will fall into a normal Gaussian distribution, so they can do their classical frequentist statistical analysis. Many show wariness of Bayesian analysis, of which the Maximum Entropy Principle falls into. This pollutes their pure statistics with the possibility of belief systems or even physics. Likes to think in terms of random walk, which almost always puts them in the time-domain and they can then derive their Poisson or Gaussian profiles with little effort. They need to usually explain fat-tails by Levy walks, which curiously assumes a power law as a first premise. So they never get the obviousness of the natural power law.

The Physicist: Wants to discover new laws of nature and thus directs research to meet this goal. Over the years, physicists have acquired the idea that power-laws somehow connect to critical phenomena, and this has gotten pounded into their skulls. Observing critical phenomena usually holds out hope that some new state of matter or behavior would reveal itself. So associating a power law within a controlled experiment of some phase transition offers up a possible Nobel Prize. They thus become experts at formulating an expression and simplifying it so that a power-law trend shows up in the tail. They want to discover the wondrous when the mundane explanation will just as easily do.

The Mathematician: Doesn't like probability too much because it messes up their idealized universe. If anything, they need problems that can challenge their mind without needing to interface to the real world. Something like Random Matrix Theory, which may explain velocity dispersion but within an N-dimensional state space. Let the Applied Mathematician do the practical stuff.

The Applied Mathematician: Unfortunately, no one ever reads what Applied Mathematicians have to say because they write in journals called Applied Mathematics, where you will find an article on digital image reconstruction calculation next to a calculation of non-linear beam dynamics. How would anyone searching for a particular solution find this stuff in the first place? (outside of a Google search that is)

The Engineer: Treats everything as noise and turbulence which gets in their way. They see 1/f noise and only want to get rid of it. Why explain something if you can't harness its potential? Anyways, if they want to explain something they can just as easily use a heuristic. Its just fuzzy logic at its core. For the computational engineer, anything else that doesn't derive from Fokker-Planck, Poisson's Equation, or Navier-Stokes is suspect. For the ordinary engineer, if it doesn't show up in the Matlab Toolbox it probably doesn't exist.

The Economist and The Quantitative Analyst: Will invoke things like negative probabilities before they use anything remotely practical. Math only serves to promote their economic theory predicting trends or stability, with guidance from the ideas of econometrics. Besides, applying a new approach will force them to sink the costs of years of using normal statistics. And they should know the expense of this because they invented the theory of sunk costs. The economist is willing to allow the econophysicist free reign to explore all the interesting aspects of their field.

I wrote this list at least partly tongue-in-cheek because I get frustrated by the amount of dead-end trails that I see scientists engaged in. They get tantalizingly close with ideas such as superstatistics but don't quite bridge the gap. If I couldn't rationalize why they can't get there, I would end up banging my head on the wall, thinking that I have made some egregious mistake or typo somewhere in my own derivation.

Lastly, many of the researchers that investigate fat-tail distributions resort to invoking ideas like Tsallis Entropy, which kind of looks like entropy, and q-exponentials, which kind of look like exponentials. The ideas of chaos theory, fractals, non-linear dynamics, and complexity also get lumped together with what may amount to plain disorder.

Unfortunately this just gets everyone deeper in the weeds, IMO.

---

Entrophic

Over at the Oil Drum, somebody accidentally used the non-existent term "entrophic" in a comment. I asked if they meant to say entropic or eutrophic, of which the suffix trophic derives from the Greek for food or feeding.

Yet I think it has a nice ring to it, seeing as it figuratively describes how disorder/ dispersion play in depletion of resources (i.e. eating). So entrophic phenomena describe the smeared out and often fat-tail depletion dynamics that we observe, but that few scientists have tried to explain.