Entropy of a Coin
Lesson notes
Overview
Uncertainty peaks at p = ½ and vanishes for certain outcomes. This topic is rated advanced and is number 241 of 245 in Mathematics. Deeper models, formal results and harder reasoning.
What you are seeing
The panel above shows a graph of the function. Each slider changes one parameter, so you can see exactly how the curve bends, shifts or stretches. The animation is drawn live from the model, so everything you see responds to the sliders. Press Space to pause, use Restart to replay from the start, and change the speed to slow down quick events.
In depth
Shannon entropy measures uncertainty in bits: H = −p log₂ p − (1 − p) log₂(1 − p). It is largest, exactly 1 bit, for a fair coin, p = ½, because the outcome is as unpredictable as possible. It falls to zero when the outcome is certain. Entropy gives the minimum average number of bits needed to encode the results, which is the foundation of data compression and information theory.
Reading the relationship
The expression "H = −p log₂ p − (1−p) log₂(1−p)" is a compact summary of the idea. It uses the symbols H, p; name what each one stands for before you start. Notice that it contains a logarithm, which turns multiplication into addition and compresses large values. Then use the sliders to see how changing one quantity changes the others.
A guided exploration
1. Set every slider to its middle value and note the shape of the curve. 2. Move one slider at a time and describe whether the curve shifts, stretches or turns. 3. Find the value where the behaviour changes sharply, then connect it to the formula.
Try this
Try to explain why the pattern holds, not just what it is. Look for limiting cases and what happens at the edges.
Common pitfalls
A frequent slip is memorising the rule without the picture. If a result surprises you, sketch the graph or test a simple number by hand before trusting the algebra.
How it connects
It builds on “Variance of a Sum”, the topic before it in Mathematics. Once you are comfortable, continue with “Gambler’s Ruin”.
Why it matters
Mathematical ideas like this one are the language that physics, engineering, computing and statistics are written in.
Where you might use it
- Revising “Entropy of a Coin” before a class, exam or certification
- Checking working on homework and exam questions by watching the graph respond
- Modelling growth, change and uncertainty in finance, data science and statistics
- Building the reasoning behind the algorithms and engineering tools people use every day
Frequently asked questions
What is Entropy of a Coin?
Uncertainty peaks at p = ½ and vanishes for certain outcomes. It is an advanced topic in Mathematics.
How do I use the Entropy of a Coin simulation?
The panel at the top shows a graph of the function. Each slider changes one parameter, so you can see exactly how the curve bends, shifts or stretches. Move the sliders one at a time, then press Space to pause the animation and look closer.
What does the formula mean?
The expression H = −p log₂ p − (1−p) log₂(1−p) packs the idea into symbols. Name each symbol first, then change one quantity with the sliders and see which others respond.
Who is this topic for?
Deeper models, formal results and harder reasoning. If the quiz feels hard, reread the lesson notes and try the previous topic first.
What should I learn before this?
Start with “Variance of a Sum”, the previous topic in Mathematics. Each topic in a subject builds on the ones before it.
How do I track my progress?
Press “Mark as learned” on this page. Progress is stored only in this browser and is never sent anywhere. The Browse page can show topics you have learned or still need to learn.
Quick quiz
More in Mathematics
See all →Gambler’s Ruin
The chance of ruin depends on starting wealth relative to the target.
Monty Hall Switching
Switching doors wins two thirds of the time — a counter-intuitive Bayesian result.
Regression to the Mean
Extreme first measurements are followed by less extreme ones when correlation is imperfect.