# The Ballot Theorem

Suppose in an election with two candidates, we are certain of the final tally. However, as the votes are counted sequentially, the lead may switch between the candidates during the process. This was certainly the case in the 2020 U.S. presidential election, where in several swing states, different types of votes (mail, early-voting, in-person) were counted at different stages. While the final tally does not change in a recount, the lead change may. One may ask: How likely will there be lead changes in a recount? Can the winning candidate always lead from start to finish? The Ballot Theorem provides a quantitative answer to this question.

(Informal) Suppose candidates and each have and votes, respectively, with . If the ballots are sorted uniformly at random and counted sequentially, the probability that always leads in this sequential count is .

The theorem matches our intuition that if the election is close, i.e., the count difference is small, then it is very likely that in a recount, will be winning at some point, even though will eventually overtake ‘s lead and win it all at the end. Let us formalize the statement in terms of random variables.

If are random variables, is the average of
(Ballot Theorem) Suppose are i.i.d. random variables with , where . Then

To see that the formal statement captures the semantics of ballot counting, imagine that all votes are in a box, and we randomly sample a ballot from a box without replacement. At time , if the -th sample is for Candidate and if the -th sample is for Candidate . Then . In the rest of this post, we will prove the Ballot Theorem using two commonly used techniques in probability, the reflection principle and martingale.

## Reflection Principle

Reflection Principle is a common technique used in Brownian motion, which is a continuous stochastic process that can model the movement of particles or the trajectory of stock prices over time. The idea is that if at any time, the stock price goes up or down equally likely, then any upward trajectory of the stock price has a corresponding downward trajectory (“reflected” along the x-axis). This symmetry allows us to reduce certain questions about a sequence of random variables to just one. For instance, the Ballot Theorem shows the probability that all random variables are positive is reduced to the single constant random variable .

Before writing the proof formally, let us first describe the Ballot Theorem at a high level. Imagine the ballots are sorted, and we start counting the votes one by one. Suppose the first vote belongs to Candidate . Since , at some point during counting, must relinquish its lead and tie with , say with votes each. Now pause the counting (temporarily!), and pair up each of the counted votes for with a unique counted vote for . With this pairing, we can re-create another recount by just swapping every counted vote with its paired vote. Then in this other recount, the first vote belongs to Candidate and afer votes, is tied with . In other words, whenever a recount has losing its initial lead, there is another recount where initially loses its lead as well!

Imagine plotting the vote counts. The scenario that has an initial lead is represented by a curve that is below the x-axis and eventually hits the x-axis. The curve for the imagined recount described above is simply obtained by reflecting the original curve along the x-axis.

The probability that the first vote is for and relinquish this lead is (since does not have enough votes to win in the final tally). By reflection, the probability that the first vote is for and relinquishes this lead is also . So the probability that never falls behind is .

Let be the stopping time when first becomes , i.e.,

Note that , so it suffices to compute . To this end, note that if , by the minimality of , either or .

Now here is the reflection principle at work.

(of Claim) Consider such that . Consider its reflection . Since the are i.i.d. and and are in one-to-one correspondence, the two probabilities are equal.

Again by the minimality of ,

Note that . To see this, if is such that , then . Conversely, if and , there must be some such that .

Thus, putting the above together, we have

And finally, since the random variables are i.i.d.,

finishing the proof.

The proof used a subtle property of the stopping time . When always leads , is defined to be , and so by assumption. At first glance, the definition that if always leads seems arbitrary. However, the semantic of stopping time requires that the event must be “observable” at time . The information that always leads is only determined at time , so must be defined to be . In the language of measure theory, . See the next section where stopping time is formally defined in terms of -algebra.

## Martingale

In a betting system, a gambler often formulates a strategy at time based on previous observations, say the first card draws. For example, if we see a sequence of coin tosses being head, it is tempting to bet big on the 11th toss being a tail. A martingale is essentially a strategy where the expectation valuation at time given the observations up to does not differ from previous valuation. Continuing the coin example, if the coin tosses are fair, then the expected winning on the 11th toss is the same as previous coin tosses.

A sequence of integrable random variables is a martingale if .

At time , a gambler may have access to other information computed from , not just the values of these previous random variables. Thus, we use a larger set, a -algebra that semantically captures all the information available up to time (as opposed to just . More generally,

A sequence of integrable random variables is a martingale with respect to the -algebras if

• is a filtration, i.e., ,,
• is -measurable,
• ,.
• Semantically, the first criterion just says that information at time is cumulative: it includes information available previous time. For the second criterion, by definition , , so it guarantees that actually captures the information at time . Lastly, the third criterion enforces that the betting system is fair as discussed.

In probability, many problems have a hidden martingale, and finding it often illuminates the problem more. Here is roughly how martingale is embedded inside the Ballot Theorem. Imagine we have randomly sorted ballots for a recount. At time (the negative indices will be explained soon), we have ballots uncounted. Since we knew the final tally, and we have just counted ballots, we know how many of the remaining ballots are for Candidates and , just not the specific order that they will come out. Thus, if we draw a random ballot from the remaining ballots, its distribution is exactly , implying that is a martingale (precise calculation within the proof below).

We make the curious choice to index our sample average with negative indices, even though we could have shifted the entire index sequence by to the right. However, in theory, the number of random variables may be infinite (the setting for the Law of Large Numbers). In this case, when the average is taken over just one random variable or zero variable, the martingale stops. It is simply more instructive to let or be the last random variable in this martingale. In fact, martingale with negative indices are also called backward martingale. Backward martingales are easier to work with as they have stronger convergence guarantees. The reason is that a backward martingale has a last random variable, which by definition is bounded and therefore bounds the entire sequence of random variables via the martingale property.

We do not need the full machinery of backward martingale since in our application, , the number of ballots, is bounded. But for pedagogical purposes we will retain the use of negative indices to remind ourselves that is actually a backward martingale with stronger tools should we need to avail ourselves to them.

Similar to the reflection principle, we will also make use of stopping time, but this time we will use it more extensively, so we define it carefully now.

A random variable is a stopping time with respect to if for all , , where is a filtration.

Semantically, when represents the cumulative information at time , any stopping time criterion (bet after five heads are tossed, fold when money is below a threshold, etc) can only depend on the information available up to time . Stopping time is often used in conjunction with martingales. Often it is useful to stop a sequence of random variables, say the first time that hits . If the original sequence forms a martingale, then the sequence up to the stopping time — which itself is a random variable — remains a martingale if is bounded. There are more general statements, but the following suffices for our purpose.

(Martingale Optional Stopping Time) Let be a martingale and a stopping time with respect to the filtration , where . Then .
By the definition of conditonal expectation, it suffices to show that

• is -measurable and
• for all , .
• The first condition is satisfied by definition. For the second, by telescoping, we have

For each term, we have

since and is a martingale with respect to .

(Of Ballot Theorem) For , Define . Let denote the -algebra . We first show that the sample averages form a martingale.

.
(Of Claim) Since the random variables are i.i.d., we have

On the other hand, by i.i.d. again, we have

Now let be the stopping time when first becomes , i.e.,

Since is a stopping time, in the else-case above, must be since . Since form a martingale, by the Martingale Optional Stopping Time Theorem,
we have

Since , by the Law of Total Expectation, we have

Evaluating both sides, and writing , we have

where the third through fifth line follow from if , the definition of , and that if , respectively.

# Normal numbers

Imagine if we sample numbers from the unit interval and count the occurrences of each digit in every number’s infinite decimal representation. For instance, in the number , every non-zero digit appears exactly once, but appears infinitely often (trailing zeros). On the other hand, for the number , the digit occurrences (after the “dot”) is equal for every digit. As the number of samples increases, one may expect that the number of occurrences for each digit in to be close to one another. In fact, the Normal Number Theorem, first formulated by Borel, not only formalizes this intuition but goes a step further: for almost every number in , each digit appears equally likely. Below we discuss this theorem in detail.

We first recall a few basic facts about the representation of a number. Rather than working entirely in base , we work with an arbitrary base .

Let with . We say that has a base- representation if there is some such that the series converges to , with .

In other words, is the integer component of and the fractional component. With , our familiar decimal representation, we know that every real number has a decimal representation, and by identifying as , the representation is unique. Analogously, for every , every real number has a base- representation as well.

(Normal numbers for base b) Let with . We say that is normal with respect to base and digit if , and is normal with respect to base if for every , it is normal with respect to base and digit .

Note that in the definition above, , the digits in the integer component of , are completely ignored. The reason is that for every , its integer component is bounded, so in the above definition is fixed and does not affect the limiting value of digit frequencies.

We say that is normal if for every , is normal with respect to .

We now state the Normal Number Theorem formally:

There is a null set such that every , is normal.
The theorem is non-constructive — while one is almost guaranteed to find a normal number, it does not construct specific normal numbers. Furthermore, it is not known if famous irrational numbers such as and are normal.
The power of the theorem lies in aggregation by 1) non-constructively examining all numbers at once instead of a individual number and 2) considering all digit precision instead of a specific precision. For instance, the frequency of the leading digit often obeys Benford’s Law, where the digits and in a decimal representation tend to appear more frequently than and .

Let us first consider a truncated version of the theorem. Consider tossing a fair coin independently times. From a combinatorial argument, most sequences of coin tosses have about heads and tails. By interpreting a head as and a tail as , most -bit numbers are normal. The passage from bits to infinite bits is the crux of the theorem, since the combinatorial process of selecting a random -bit number no longer extends. The proof illustrates how a measure-theoretic abstraction can circumvent this difficulty. In particular, we will apply the following “-trick” repeatedly.

Let be any probability space. Let be a countable collection of events with each . Then .
Let . By assumption, can be bounded above by . Then

Since was arbitrary, must equal .

(Normal Number Theorem)
First consider the interval , where . By the previous proposition, it suffices to show that the set of non-normal numbers in is null. Now fix . By the proposition again, it suffices to show that the set of numbers in that are not normal with respect to is null. Now fix . By the proposition again, it suffices to show that the set of numbers in that are not normal with respect to base and digit is null. By definition, for , is normal with respect to base and digit iff is normal with respect to base and digit . Thus, without loss of generality, we may further assume that .

Now consider the probability space (, where is the set of Borel sets on , and the Lebesgue (probability) measure. For , define to be , where is the -th digit of the base expansion .

is a sequence of i.i.d. Bernoulli random variable with probability .
(of Claim)
As defined, is an indicator function, and for , the set is just a union of intervals:

which is a Borel set, implying that is a Bernoulli random variable with .

To show independence, it suffices to show that for every , the family of random variables is independent. This follows directly from the observation that (since the interval has length ) and the previous observation that .

Given our setup, we now apply the Strong Law of Large Numbers to conclude: there exists a null set so that for every , converges to , i.e., is normal with respect to base and digit .

As a corollary, we conclude that for every continuous random variable, its output is most likely normal.

Let be a probability space, and a continuous random variable with a probability density function. Then , where is the set of non-normal numbers.
By the Normal Number Theorem, the set of non-normal numbers, denoted , is null, so for every , there is some open set with . Let be the probability density function of , then

finishing the proof.

A continuous random variable having a probability density function is equivalent to the cumulative distribution function of being absolutely continuous. (And recall the Cantor function as an example where may not even have a probability density function!) Alternatively, the above corollary also follows by observing that can be approximated by a finite number of small intervals in (e.g., Littlewood’s First Principle as discussed in a previous blog post), which being absolutely continuous must map into another null set.