Chapter 10: Multinomial Distribution
1. What is the Multinomial distribution really?
The multinomial distribution is the generalization of the binomial distribution to more than two categories.
- Binomial = 2 outcomes (success/failure, heads/tails, yes/no)
- Multinomial = k outcomes (k ≥ 2 categories)
You have:
- n independent trials / experiments / draws
- k possible categories / outcomes / classes
- p₁, p₂, …, pₖ probabilities for each category (they sum to 1)
- Each trial must produce exactly one of the k categories
The multinomial tells you the probability of getting a particular combination of counts across all categories.
2. Classic real-world examples (these appear very often)
| Situation | n (trials) | Categories (k) | Probabilities p₁, p₂, … | What we count |
|---|---|---|---|---|
| Rolling a 6-sided die 100 times | 100 | {1,2,3,4,5,6} | each 1/6 | How many times each face appeared |
| Customers choosing product categories | 5000 purchases | {electronics, clothing, books, …} | different shares | How many purchases in each category |
| Words in a document | 800 words | vocabulary size (e.g. 5000) | word probabilities | Word counts (bag-of-words) |
| Image classification predictions | 10000 images | {cat, dog, bird, car, …} | predicted probabilities | Predicted class counts |
| A/B/C test results | 30000 visitors | {A, B, C} | conversion rates | Number of conversions per variant |
| Election votes | millions | {party A, B, C, …} | vote shares | Votes per party |
3. The core idea with a small example
Imagine you have a biased 3-sided die with probabilities:
- Face A: 0.5
- Face B: 0.3
- Face C: 0.2
You roll it n = 10 times.
Possible outcomes are all combinations where the counts add up to 10, e.g.:
- (A=6, B=3, C=1)
- (A=4, B=4, C=2)
- (A=10, B=0, C=0)
- etc.
The multinomial gives the probability of each possible count vector.
4. Generating multinomial data in NumPy
Python
|
0 1 2 3 4 5 6 7 8 9 10 11 12 |
# Classic 6-sided die rolled 1000 times # probabilities: all faces equally likely p = [1/6] * 6 rolls = np.random.multinomial(n=1000, pvals=p) print("Counts for faces 1 to 6:", rolls) print("Sum (should be n):", rolls.sum()) |
Multiple independent experiments at once (very common)
Python
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 |
# 20,000 simulated documents, each with 300 words # Simplified 5-word vocabulary p_vocab = [0.35, 0.25, 0.20, 0.12, 0.08] word_counts = np.random.multinomial(n=300, pvals=p_vocab, size=20000) print("Shape:", word_counts.shape) # (20000, 5) print("First document word counts:", word_counts[0]) print("Average count per word:", word_counts.mean(axis=0).round(1)) |
5. Visualizing multinomial counts (very important)
Python
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
# Simulate many rolls of a biased 4-sided die p = [0.4, 0.3, 0.2, 0.1] n_trials = 500 n_simulations = 30000 counts = np.random.multinomial(n_trials, p, size=n_simulations) fig, axes = plt.subplots(2, 2, figsize=(14, 9)) for i, ax in enumerate(axes.flat): sns.histplot(counts[:, i], bins=range(0, n_trials+5, 5), stat="probability", color=sns.color_palette()[i], discrete=True, alpha=0.8, ax=ax) ax.set_title(f"Category {i+1} (p = {p[i]})", fontsize=13) ax.set_xlabel("Number of occurrences") ax.set_ylabel("Probability") # Theoretical mean & std mean = n_trials * p[i] std = np.sqrt(n_trials * p[i] * (1 - p[i])) ax.axvline(mean, color='darkred', linestyle='--', lw=1.8, label=f"mean = {mean:.1f}") ax.legend() plt.suptitle(f"Multinomial counts after {n_trials} trials (30,000 simulations)", fontsize=15, y=1.02) plt.tight_layout() plt.show() |
What you should observe:
- Higher p → distribution centered farther to the right
- Lower p → distribution squeezed near zero, more skewed
- Variance = n × p × (1-p) → maximum when p=0.5
6. Realistic code patterns you will actually use
Pattern 1 – Simulating A/B/C test results
Python
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 |
n_visitors = 25000 p = [0.032, 0.038, 0.035] # conversion rates A, B, C conversions = np.random.multinomial(n_visitors, p) print("Visitors:", n_visitors) print("Conversions:", conversions) print("Conversion rates:", (conversions / n_visitors * 100).round(2), "%") |
Pattern 2 – Simulating bag-of-words / topic proportions
Python
|
0 1 2 3 4 5 6 7 8 9 10 11 12 13 |
# 4 topics with different probabilities topic_probs = [0.45, 0.25, 0.18, 0.12] words_per_doc = 400 # 1000 documents doc_topic_counts = np.random.multinomial(words_per_doc, topic_probs, size=1000) print("Average topic proportions:", doc_topic_counts.mean(axis=0) / words_per_doc) |
Pattern 3 – Simulating class distribution in imbalanced classification
Python
|
0 1 2 3 4 5 6 7 8 9 10 11 12 |
n_samples = 15000 class_probs = [0.65, 0.25, 0.08, 0.02] # very imbalanced labels = np.random.multinomial(1, class_probs, size=n_samples).argmax(axis=1) print("Class distribution:", np.bincount(labels)) print("Percentages:", np.bincount(labels) / n_samples * 100) |
Summary – Multinomial Distribution Quick Reference
| Property | Value / Formula |
|---|---|
| Number of trials | n (fixed) |
| Number of categories | k (≥ 2) |
| Probabilities | p₁ + p₂ + … + pₖ = 1 |
| Counts vector | (c₁, c₂, …, cₖ) where c₁ + … + cₖ = n |
| Expected count for category i | n × pᵢ |
| Variance for category i | n × pᵢ × (1 − pᵢ) |
| Covariance between i and j | −n × pᵢ × pⱼ (negative!) |
| NumPy function | np.random.multinomial(n, pvals, size=…) |
| When n large & pᵢ not extreme | Approximates normal (multivariate) |
Final teacher messages
- Whenever you are counting “how many times each category appeared after n trials” → think multinomial.
- Multinomial is the multivariate version of binomial.
- When you only care about one category vs everything else → you can collapse to binomial.
- Multinomial + Dirichlet is the foundation of topic modeling (LDA).
Would you like to continue with any of these next?
- Multinomial vs multinomial logistic regression
- Dirichlet-multinomial (topic modeling intuition)
- How multinomial becomes multivariate normal (large n)
- Realistic mini-project: simulate customer segments or A/B/C test
- Comparing multinomial to binomial & Poisson
Just tell me what you want to explore next! 😊
