# Definition of Expected Value ## Prerequisites If the formula below feels arbitrary — *"why this expression and not some other?"* — the issue isn't the formula itself. It's that **four upstream concepts** need to be in place. With them, $\mathbb{E}(X) = \sum x \cdot \Pr(X=x)$ becomes inevitable rather than invented. | # | Prerequisite | What you need from it | |---|--------------|------------------------| | 1 | [[Random Variable]] | $X$ is a function from outcomes to real numbers — not the outcome itself | | 2 | [[Discrete Probability Distribution]] | Where $\Pr(X=x)$ comes from (the PMF assigns probability mass to each value in img(X)) | | 3 | [[Weighted Average]] | The arithmetic move: $\sum w_i x_i$ when items don't count equally — the formula's *shape* | | 4 | [[Law of Large Numbers]] | The promise that $\mathbb{E}(X)$ is what the sample average **actually converges to** — the formula's *meaning* | **Reading order if starting from scratch:** 1. [[Random Variable]] — what is $X$? (a function, not an outcome) 2. [[Discrete Probability Distribution]] — what is $\Pr(X=x)$? (a PMF — probability mass assigned to each value) 3. [[Weighted Average]] — the formula's *shape* falls out of this — expected value is just a weighted average where the weights happen to be probabilities 4. [[Law of Large Numbers]] — the formula's *meaning* — $\mathbb{E}(X)$ is the number the running sample mean converges to 5. **This note** — clicks once you see: *expected value = weighted average + probabilities as weights, justified by LLN* **Deep-dive companions** (worth their own session, not required to grok this note): - [[Probability Density Function]] — the continuous-case analog (replaces $\Pr(X=x)$ with $f_X(x)\,dx$) - [[Variance]] — the same weighted-average machinery applied to *squared deviations from the mean* - [[Baseball Batting Average Statistical Convergence]] — concrete LLN instance you can feel ### Prerequisite Tree ```mermaid flowchart BT Target["Definition of Expected Value<br>(this note)"] P1["Random Variable"] P2["Discrete Probability Distribution"] P3["Weighted Average"] P4["Law of Large Numbers"] P1 --> Target P2 --> Target P3 --> Target P4 --> Target P1 --> P2 P3 --> P4 classDef shape fill:#fff4e1,stroke:#d97706 classDef meaning fill:#e1ffe1,stroke:#059669 class P3 shape class P4 meaning ``` Read bottom-up: [[Random Variable]] gives you $X$, [[Discrete Probability Distribution]] gives you the weights, [[Weighted Average]] gives you the **formula shape**, [[Law of Large Numbers]] gives you the **formula meaning**. The four merge into the definition below. ## Definition Expected Value (or Expectation) of the the **discrete** [[Random Variable]] $X$ is defined as $ \mathbb{E}(X) \coloneqq \sum_{x\in \text{img}(X)} x Pr(X=x) $ If we were to write this in Python, it will be ```python x = [1, 2, 3] p_x = [0.1, 0.8, 0.1] expected_value = 0 for _x, _p in zip(x, p_x): expected_value += _x * _p print(expected_value) # 2 ``` # Expected Value of Contiuous Random Variable For continuous Random Variable, it's defined as $ \mathbb{E}(X)\coloneqq \int_{-\infty}^{\infty} x f_X(x) dx $ See https://proofwiki.org/wiki/Definition:Expectation for detail.