{"id":2516,"date":"2026-02-02T09:45:34","date_gmt":"2026-02-02T09:45:34","guid":{"rendered":"https:\/\/demo.materiamedica.net\/demo6\/?p=2516"},"modified":"2026-02-02T09:45:34","modified_gmt":"2026-02-02T09:45:34","slug":"chapter-10-multinomial-distribution","status":"publish","type":"post","link":"https:\/\/demo.materiamedica.net\/demo6\/chapter-10-multinomial-distribution\/","title":{"rendered":"Chapter 10: Multinomial Distribution"},"content":{"rendered":"1. What is the Multinomial distribution really?<\/h3>\n
The multinomial distribution<\/strong> is the generalization of the binomial distribution<\/strong> to more than two categories<\/strong>.<\/p>\n\n- Binomial = 2 outcomes<\/strong> (success\/failure, heads\/tails, yes\/no)<\/li>\n
- Multinomial = k outcomes<\/strong> (k \u2265 2 categories)<\/li>\n<\/ul>\n
You have:<\/p>\n
\n- n<\/strong> independent trials \/ experiments \/ draws<\/li>\n
- k<\/strong> possible categories \/ outcomes \/ classes<\/li>\n
- p\u2081, p\u2082, …, p\u2096<\/strong> probabilities for each category (they sum to 1)<\/li>\n
- Each trial must produce exactly one<\/strong> of the k categories<\/li>\n<\/ul>\n
The multinomial tells you the probability of getting a particular combination of counts<\/strong> across all categories.<\/p>\n2. Classic real-world examples (these appear very often)<\/h3>\n\n
\n
\n\n\n| Situation<\/th>\n | n (trials)<\/th>\n | Categories (k)<\/th>\n | Probabilities p\u2081, p\u2082, \u2026<\/th>\n | What we count<\/th>\n<\/tr>\n<\/thead>\n |
\n\n| Rolling a 6-sided die 100 times<\/td>\n | 100<\/td>\n | {1,2,3,4,5,6}<\/td>\n | each 1\/6<\/td>\n | How many times each face appeared<\/td>\n<\/tr>\n |
\n| Customers choosing product categories<\/td>\n | 5000 purchases<\/td>\n | {electronics, clothing, books, \u2026}<\/td>\n | different shares<\/td>\n | How many purchases in each category<\/td>\n<\/tr>\n |
\n| Words in a document<\/td>\n | 800 words<\/td>\n | vocabulary size (e.g. 5000)<\/td>\n | word probabilities<\/td>\n | Word counts (bag-of-words)<\/td>\n<\/tr>\n |
\n| Image classification predictions<\/td>\n | 10000 images<\/td>\n | {cat, dog, bird, car, \u2026}<\/td>\n | predicted probabilities<\/td>\n | Predicted class counts<\/td>\n<\/tr>\n |
\n| A\/B\/C test results<\/td>\n | 30000 visitors<\/td>\n | {A, B, C}<\/td>\n | conversion rates<\/td>\n | Number of conversions per variant<\/td>\n<\/tr>\n |
\n| Election votes<\/td>\n | millions<\/td>\n | {party A, B, C, \u2026}<\/td>\n | vote shares<\/td>\n | Votes per party<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n<\/div>\n 3. The core idea with a small example<\/h3>\nImagine you have a biased 3-sided die with probabilities:<\/p>\n \n- Face A: 0.5<\/li>\n
- Face B: 0.3<\/li>\n
- Face C: 0.2<\/li>\n<\/ul>\n
You roll it n = 10<\/strong> times.<\/p>\nPossible outcomes are all combinations where the counts add up to 10, e.g.:<\/p>\n \n- (A=6, B=3, C=1)<\/li>\n
- (A=4, B=4, C=2)<\/li>\n
- (A=10, B=0, C=0)<\/li>\n
- etc.<\/li>\n<\/ul>\n
The multinomial gives the probability of each possible count vector<\/strong>.<\/p>\n4. Generating multinomial data in NumPy<\/h3>\n\n \n \n Python<\/div>\n \n # Classic 6-sided die rolled 1000 times\r\n# probabilities: all faces equally likely\r\np = [1\/6] * 6\r\nrolls = np.random.multinomial(n=1000, pvals=p)\r\n\r\nprint(\"Counts for faces 1 to 6:\", rolls)\r\nprint(\"Sum (should be n):\", rolls.sum())<\/code><\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\nMultiple independent experiments at once<\/strong> (very common)<\/p>\n\n \n \n Python<\/div>\n \n # 20,000 simulated documents, each with 300 words\r\n# Simplified 5-word vocabulary\r\np_vocab = [0.35, 0.25, 0.20, 0.12, 0.08]\r\nword_counts = np.random.multinomial(n=300, pvals=p_vocab, size=20000)\r\n\r\nprint(\"Shape:\", word_counts.shape) # (20000, 5)\r\nprint(\"First document word counts:\", word_counts[0])\r\nprint(\"Average count per word:\", word_counts.mean(axis=0).round(1))<\/code><\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n5. Visualizing multinomial counts (very important)<\/h3>\n\n \n \n Python<\/div>\n \n # Simulate many rolls of a biased 4-sided die\r\np = [0.4, 0.3, 0.2, 0.1]\r\nn_trials = 500\r\nn_simulations = 30000\r\n\r\ncounts = np.random.multinomial(n_trials, p, size=n_simulations)\r\n\r\nfig, axes = plt.subplots(2, 2, figsize=(14, 9))\r\n\r\nfor i, ax in enumerate(axes.flat):\r\n sns.histplot(counts[:, i], bins=range(0, n_trials+5, 5),\r\n stat=\"probability\", color=sns.color_palette()[i],\r\n discrete=True, alpha=0.8, ax=ax)\r\n \r\n ax.set_title(f\"Category {i+1} (p = {p[i]})\", fontsize=13)\r\n ax.set_xlabel(\"Number of occurrences\")\r\n ax.set_ylabel(\"Probability\")\r\n\r\n # Theoretical mean & std\r\n mean = n_trials * p[i]\r\n std = np.sqrt(n_trials * p[i] * (1 - p[i]))\r\n ax.axvline(mean, color='darkred', linestyle='--', lw=1.8, label=f\"mean = {mean:.1f}\")\r\n ax.legend()\r\n\r\nplt.suptitle(f\"Multinomial counts after {n_trials} trials (30,000 simulations)\", fontsize=15, y=1.02)\r\nplt.tight_layout()\r\nplt.show()<\/code><\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\nWhat you should observe<\/strong>:<\/p>\n\n- Higher p \u2192 distribution centered farther to the right<\/li>\n
- Lower p \u2192 distribution squeezed near zero, more skewed<\/li>\n
- Variance = n \u00d7 p \u00d7 (1-p) \u2192 maximum when p=0.5<\/li>\n<\/ul>\n
6. Realistic code patterns you will actually use<\/h3>\nPattern 1 \u2013 Simulating A\/B\/C test results<\/strong><\/p>\n\n \n \n Python<\/div>\n \n n_visitors = 25000\r\np = [0.032, 0.038, 0.035] # conversion rates A, B, C\r\n\r\nconversions = np.random.multinomial(n_visitors, p)\r\n\r\nprint(\"Visitors:\", n_visitors)\r\nprint(\"Conversions:\", conversions)\r\nprint(\"Conversion rates:\", (conversions \/ n_visitors * 100).round(2), \"%\")<\/code><\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\nPattern 2 \u2013 Simulating bag-of-words \/ topic proportions<\/strong><\/p>\n\n \n \n Python<\/div>\n \n # 4 topics with different probabilities\r\ntopic_probs = [0.45, 0.25, 0.18, 0.12]\r\nwords_per_doc = 400\r\n\r\n# 1000 documents\r\ndoc_topic_counts = np.random.multinomial(words_per_doc, topic_probs, size=1000)\r\n\r\nprint(\"Average topic proportions:\", doc_topic_counts.mean(axis=0) \/ words_per_doc)<\/code><\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\nPattern 3 \u2013 Simulating class distribution in imbalanced classification<\/strong><\/p>\n\n \n \n Python<\/div>\n \n n_samples = 15000\r\nclass_probs = [0.65, 0.25, 0.08, 0.02] # very imbalanced\r\n\r\nlabels = np.random.multinomial(1, class_probs, size=n_samples).argmax(axis=1)\r\n\r\nprint(\"Class distribution:\", np.bincount(labels))\r\nprint(\"Percentages:\", np.bincount(labels) \/ n_samples * 100)<\/code><\/pre>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\nSummary \u2013 Multinomial Distribution Quick Reference<\/h3>\n\n \n \n\n\n| Property<\/th>\n | Value \/ Formula<\/th>\n<\/tr>\n<\/thead>\n | \n\n| Number of trials<\/td>\n | n<\/strong> (fixed)<\/td>\n<\/tr>\n\n| Number of categories<\/td>\n | k<\/strong> (\u2265 2)<\/td>\n<\/tr>\n\n| Probabilities<\/td>\n | p\u2081 + p\u2082 + \u2026 + p\u2096 = 1<\/strong><\/td>\n<\/tr>\n\n| Counts vector<\/td>\n | (c\u2081, c\u2082, \u2026, c\u2096) where c\u2081 + \u2026 + c\u2096 = n<\/td>\n<\/tr>\n | \n| Expected count for category i<\/td>\n | n \u00d7 p\u1d62<\/strong><\/td>\n<\/tr>\n\n| Variance for category i<\/td>\n | n \u00d7 p\u1d62 \u00d7 (1 \u2212 p\u1d62)<\/strong><\/td>\n<\/tr>\n\n| Covariance between i and j<\/td>\n | \u2212n \u00d7 p\u1d62 \u00d7 p\u2c7c<\/strong> (negative!)<\/td>\n<\/tr>\n\n| NumPy function<\/td>\n | np.random.multinomial(n, pvals, size=…)<\/td>\n<\/tr>\n | \n| When n large & p\u1d62 not extreme<\/td>\n | Approximates normal (multivariate)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n<\/div>\n Final teacher messages<\/h3>\n\n- Whenever you are counting \u201chow many times each category appeared after n trials\u201d<\/strong> \u2192 think multinomial.<\/li>\n
- Multinomial is the multivariate version of binomial<\/strong>.<\/li>\n
- When you only care about one category vs everything else<\/strong> \u2192 you can collapse to binomial.<\/li>\n
- Multinomial + Dirichlet<\/strong> is the foundation of topic modeling (LDA).<\/li>\n<\/ol>\n
Would you like to continue with any of these next?<\/p>\n \n- Multinomial vs multinomial logistic regression<\/li>\n
- Dirichlet-multinomial (topic modeling intuition)<\/li>\n
- How multinomial becomes multivariate normal (large n)<\/li>\n
- Realistic mini-project: simulate customer segments or A\/B\/C test<\/li>\n
- Comparing multinomial to binomial & Poisson<\/li>\n<\/ul>\n
Just tell me what you want to explore next! \ud83d\ude0a<\/p>\n","protected":false},"excerpt":{"rendered":" 1. What is the Multinomial distribution really? The multinomial distribution is the generalization of the binomial distribution to more than two categories. Binomial = 2 outcomes (success\/failure, heads\/tails, yes\/no) Multinomial = k outcomes (k...<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[75],"tags":[],"class_list":["post-2516","post","type-post","status-publish","format-standard","hentry","category-numpy"],"_links":{"self":[{"href":"https:\/\/demo.materiamedica.net\/demo6\/wp-json\/wp\/v2\/posts\/2516","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/demo.materiamedica.net\/demo6\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/demo.materiamedica.net\/demo6\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/demo.materiamedica.net\/demo6\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/demo.materiamedica.net\/demo6\/wp-json\/wp\/v2\/comments?post=2516"}],"version-history":[{"count":1,"href":"https:\/\/demo.materiamedica.net\/demo6\/wp-json\/wp\/v2\/posts\/2516\/revisions"}],"predecessor-version":[{"id":2517,"href":"https:\/\/demo.materiamedica.net\/demo6\/wp-json\/wp\/v2\/posts\/2516\/revisions\/2517"}],"wp:attachment":[{"href":"https:\/\/demo.materiamedica.net\/demo6\/wp-json\/wp\/v2\/media?parent=2516"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/demo.materiamedica.net\/demo6\/wp-json\/wp\/v2\/categories?post=2516"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/demo.materiamedica.net\/demo6\/wp-json\/wp\/v2\/tags?post=2516"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}} | | | | | | |
|