Trang chủ » blog » Simulating a Biased Coin With a Fair One

# Simulating a Biased Coin With a Fair One

Suppose you need to simulate an event that happens \ ( phosphorus = 1/4\ ) of the time, and all you have is a fair coin. Well, that ’ mho easy enough. Flip the mint twice. Report achiever if you get 2 heads, and report failure otherwise .
Suppose \ ( p = 1/3\ ) rather. That may not be a power of 2, but there is placid a elementary solution. Flip the carnival mint twice. Report success on HH, report card failure on HT or TH, and try again on TT. Each iteration takes \ ( 2\ ) mint flips, and there is a \ ( 3/4\ ) probability of stem, giving \ ( 8/3\ ) expected coin flips .
now, suppose you need to simulate \ ( phosphorus = 1/10^ { 100 } \ ). Well, nothing is particularly wrong with the current outline. We need \ ( \lceil\log_2 ( 10^ { 100 } ) \rceil = 333\ ) coin flips to get enough outcomes. then, label \ ( 1\ ) case as success, \ ( 10^ { 100 } – 1\ ) as failure, and the rest as retries. For analyzing runtime, these numbers are starting to get messy, but we can at least ballpark the expect count of flips. The probability of halt is \ ( > 1/2\ ), since by construction \ ( 2^ { 332 } < 10^ { 100 } < 2^ { 333 } \ ), meaning the count of rehear cases is less than half of all cases. therefore, the expected number of flips is at most \ ( 666\ ). Yes, that is… ( •_• ) ( •_• ) > ⌐■-■ ( ⌐■_■ )

a devilish upper bandaged, but things could be worse .
( I ’ thousand not deplorable. )
But, what if you need to simulate \ ( phosphorus = 1 / 10^ { 10^ { 100 } } \ ) ? Or simulate \ ( p = 1/\pi\ ) ? In the foremost shell, the number of flips needed is enormous. In the second base, the scheme fails wholly – there is no way to simulate irrational probabilities .

## Easy Optimizations

We ’ ll get back to the irrational number event by and by .
Recall how we simulated \ ( p=1/4\ ). The system reports success only on HH. sol, if the first flip lands tails, we can halt immediately, since whatever way the second mint lands, we will report failure anyways. The revise algorithm is

• Flip a coin. If tails, report failure.
• Flip a coin. Report success on heads and failure on tails.

This gives \ ( 3/2\ ) expected flips rather of \ ( 2\ ) flips .
As another case, consider \ ( phosphorus = 1/5\ ). First, plan to flip 3 coins. Divide the 8 outcomes into 1 success encase, 4 failure cases, and 3 rehear cases. These can be distributed as

• accept: HHH
• failing: THH, THT, TTH, TTT
• retry: HHT, HTH, HTT

If the beginning coin is T, we can stop immediately. If the first two coins are HT, we can retry immediately. The merely font left is HH, for which we need to see the third flip before deciding what to do .
( Some of you may be getting flashbacks to prefix-free codes. If you haven ’ thymine seen those, don ’ thymine worry, they won ’ thyroxine prove up in this post. )
With clever rearrangement, we can bundle outcomes of a given type under as few alone prefixes as potential. This gives some improvement for rational \ ( p\ ), but still does not let us simulate irrational number \ ( p\ ). For this, we need to switch to a new model .

I did not come up with this method – I discovered it from here. I wish I had come up with it, for reasons that will become clear .
Let \ ( phosphorus = 0.b_1b_2b_3b_4b_5\ldots\ ) be the binary expansion of \ ( p\ ). Proceed as follows .

• Flip a coin until it lands on heads.
• Let $$n$$ be the number of coins flipped. Report success if $$b_n = 1$$ and report failure
otherwise

The probability the process stops after \ ( n\ ) flips is \ ( 1/2^n\ ), so the probability of success is

$P[success] = \sum_{n: b_n = 1}^\infty \frac{1}{2^n} = p$

Regardless of \ ( p\ ), it takes \ ( 2\ ) expected flips for the coin to land heads. frankincense, any bias coin can be simulated in \ ( 2\ ) expected flip. This beats out the other system, works for all \ ( p\ ) alternatively of only rational \ ( p\ ), and best of all you can compute bits \ ( b_i\ ) lazily, making this implementable in real life and not just in hypothesis .
Slick, right ? This idea may have been obvious to you, but it surely wasn ’ thyroxine to me. After thinking about the trouble more, I finally recreated a potential chain of reasoning to reach the same solution .

## Bits and Pieces

( Starting now, I will use \ ( 1\ ) interchangeably with heads and \ ( 0\ ) interchangeably with tails. )

• Construct a real number in $$[0,1]$$ by flipping an infinite number of coins,
generating a random number $$0.b_1b_2b_3\ldots$$, where $$b_i$$ is the outcome
of coin flip $$i$$. Let this number be $$x$$.
• Report success if $$x \le p$$ and failure otherwise.

This algorithm is right angstrom long as the decimals generated follow a consistent distribution over \ ( [ 0, 1 ] \ ). I won ’ t test this, but for an entreaty to intuition : any two bit strings of length \ ( k\ ) are generated with the same probability \ ( 1/2^k\ ), and the numbers these defend are evenly distributed over the time interval \ ( [ 0, 1 ] \ ). As \ ( k \to\infty\ ) this approaches a uniform distribution .
Assuming this all sounds reasonable, the algorithm works ! merely, there is the small problem of flipping \ ( \infty\ ) -ly many coins. however, similarly to the \ ( phosphorus = 1/4\ ) sheath, we can stop mint flipping angstrom soon as as it is guaranteed the number will fall in or out of \ ( [ 0, p ] \ ) .
When does this occur ? For now, limit to the case where \ ( p\ ) has an infinite binary star expansion. For the sake of an example, suppose \ ( phosphorus = 0.1010101\ldots\ ). There are 2 cases for the first flip .

1. The coin lands \ ( 1\ ). Starting from \ ( 0.1\ ), it is possible to fall inwardly or away \ ( [ 0, p ] \ ), depending on how the following flips go. The algorithm must continue .
2. The mint lands \ ( 0\ ). Starting from \ ( 0.0\ ), it is impossible to fall outside \ ( [ 0, p ] \ ). even if every coin lands \ ( 1\ ), \ ( 0.0111\ldots_2 = 0.1_2 < p\ ). ( This is why \ ( p\ ) having an space binary expansion is important - it ensures there will be another \ ( 1\ ) down the line. ) The algorithm can halt and report success .

so, the algorithm halts unless the coin flip matches the \ ( 1\ ) in \ ( p\ ) ’ south expansion. If it does not match, it succeeds. Consider what happens next. Starting from \ ( 0.1\ ), there are 2 cases.

1. Next coin lands $$1$$. Since $$0.11 > p$$, we can immediately report
failure.
2. Next coin lands $$0$$. Similarly to before, the number may or may not
end in $$[0,p]$$, so the algorithm must continue.

so, the algorithm halts unless the mint flip matches the \ ( 0\ ) in \ ( p\ ) ’ s expansion. If it does not match, it fails. This gives the following algorithm .

• Flip a coin until the $$n$$th coin fails to match $$b_n$$
• Report success if $$b_n = 1$$ and failure otherwise.

note this is basically the lapp algorithm as mentioned above ! The only difference is the ending condition. rather of halting on heads, the algorithm halts if the random piece does not match the “ true ” bit. Both find \ ( 1/2\ ) the time, so the two algorithms are equivalent .
( If you wish, you can extend this reason to \ ( p\ ) with finite binary expansions. merely make the expansion infinite by expanding the trail \ ( 1\ ) into \ ( 0.1_2 = 0.0111\ldots_2\ ). )
here ’ s a sample distribution run of the algorithm told in pictures. The fleeceable area represents the potential values of \ ( x\ ) as bits are generated. initially, any \ ( x\ ) is potential .

The foremost render act is \ ( 0\ ), reducing the valid region to

This placid overlaps \ ( [ 0, phosphorus ] \ ), thus stay. The second spot is \ ( 1\ ), giving

This hush overlaps, so retain. The third sting is \ ( 1\ ), giving

The feasible region for \ ( x\ ) no longer intersects \ ( [ 0, phosphorus ] \ ), so the algorithm reports failure .
CS-minded people may see similarities to binary star search. Each bite chooses which half of the feasible area we move to, and the halve continues until the feasible region is a subset of \ ( [ 0, phosphorus ] \ ) or disjoint from \ ( [ 0, phosphorus ] \ ) .

## Proving Optimality

This schema is very, very effective. But, is it the best we can do ? Is there an algorithm that does better than \ ( 2\ ) expected flips for general \ ( p\ ) ?
It turns out that no, \ ( 2\ ) expected flips is optimum. More amazingly, the proof is not excessively bad .
For a given \ ( p\ ), any algorithm can be represented by a calculation tree. That tree encodes whether the algorithm succeeds, fails, or continues, based on the adjacent spot and all previously generated bits .

Two sample distribution calculation trees for \ ( phosphorus = 3/4\ ) .
With the convention that the etymon is level \ ( 0\ ), children of the ancestor are tied \ ( 1\ ), and indeed on, let the slant of a node be \ ( 1/2^ { \text { level } } \ ). Equivalently, the weight is the probability the algorithm reaches that lymph node .
For a given algorithm and \ ( p\ ), the expect number of flips is the have a bun in the oven number of edges traversed in the algorithm ’ s calculation tree. On a given ply, the number of edges traversed is the number of vertices visited, ignoring the root. By one-dimensionality of arithmetic mean ,

$E[flips] = \sum_{v \in T, v \neq root} \text{weight}(v)$

To be decline, an algorithm must end at a achiever node with probability \ ( p\ ). thus, the sum of weights for success nodes must be \ ( p\ ). For \ ( p\ ) with boundlessly long binary expansions, we must have an infinitely deep calculation tree. If the tree had finite depth \ ( d\ ), any flick node weight would be a multiple of \ ( 2^ { -d } \ ), and the sum of success node weights would have at most \ ( d\ ) decimal places .
therefore, the calculation tree must be infinitely deep. To be boundlessly deep, every level of the corner ( except the root ) must have at least 2 nodes. therefore, a lower jump on the expect number of flips is

$E[flips] \ge \sum_{k=1}^\infty 2 \cdot \frac{1}{2^k} = 2$

and as we have shown earlier, this lower bound is accomplishable. \ ( \blacksquare\ )
( For \ ( p\ ) with finite binary expansions, you can do better than \ ( 2\ ) expected flips. Proving the optimum tie down for that is left as an use. )

## In Conclusion…

Every mint in your wallet is immediately an randomly biased bit generating machine that runs at proven-optimal efficiency. If you run into a bridge troll who demands you simulate several flips of a coin that lands heads \ ( 1/\pi\ ) of the meter, you ’ ll be ready. If you don ’ thyroxine, at least you know one more interesting thing .

source : https://leowiki.com
Category : Economy