There is a 15% chance you will read this to the end

Naveen hasn't posted anything on probability or statistics for a while, so I figured I would give it a shot...

What does it mean to say there is a 10% chance of rain tomorrow? Or that there is a 7% chance Iraq will be a peaceful democracy in two years? These numbers appear silly: these are things that either will happen or won't.

In reality, all probability is like this. For instance, if you flip a coin and while it is in mid-air, I say there is a 50% it will land heads up, no one questions that this is a perfectly reasonable statement. Yet, as far as physics is concerned, what side the coin lands on is already predetermined by many variables, such as the coin's radial velocity, time to landing, etc. The traditional definition of probability (from the so-called frequentist school of thought) is that probability measures observed outcomes with a large enough number of observations. So if I flip the same coin 100 times, it should come up 50 times heads and 50 times tails. But this is only a statement of expectation about the likely outcome. It is perfectly possible the coin lands heads up 49 times or 48 times or even 0 times. Each of these outcomes itself has a probability associated with it.

That's the main problem with the traditional definition of probability. It is ultimately a circular definition since probability is supposed to measure outcomes, but outcomes of repeated experiments are themselves subject to probability. In pure theory, the resolution to this paradox is to say that with an infinite number of coin tosses, exactly half the time the coin will end up heads and the other half it will end up tails [more formally, as N goes to infinite, the coin will land heads up N/2 times and tails N/2 times]. But of course, talking about infinite does us little good in analyzing the real world.

To me, the traditional definition simply does not make sense. Instead, I think of probability just as a way of being systematic about our ignorance of the world. Except maybe in quantam mechanics, probability doesn't really exist. Theoretically, I could know whether the coin will land heads or tails but without that information, I say there is a 50% chance of each as a way of stating my beliefs about the possible outcomes. To put it another way, probabilities give the betting odds I would be willing to accept (if I am a gambler) for a given outcome.

In the end, I think probability is not something that can be concretely defined. It is a useful concept simply because it allows us to make sense of and systematize a complex world.