Random Variables

• A random variable is a function $X:S\to \mathbb{R}$ that assigns a real number to each outcome in the sample space $S$.
• The cumulative distribution function (CDF) of a random variable $X$ is the function defined by $F(x)=P(X\le x)$
  • $F$ is nondecreasing.
  • $\underset{x\to -\infty }{\lim }F(x)=0$ and $\underset{x\to \infty }{\lim }F(x)=1$
  • $\forall x\in \mathbb{R},0\le F(x)\le 1$
  • $F$ is right continuous.
• A mode of a random variable $X$ is a value $x_0$ such that $p(x_0)\ge p(x)$ for all $x$, where $p$ is the pmf or pdf of $X$.
• The median of a random variable $X$ is a value $m$ such that $P(X\le m)\ge \frac{1}{2}$ and $P(X\ge m)\ge \frac{1}{2}$.
• (Chebyshev’s inequality) For any random variable $X$ with nonzero finite variance, and for any $k>0$, we have $P(|X-\mu |\ge k\sigma )\le \frac{1}{k^2}$. (or equivalently, $P(|X-\mu |\ge k)\le \frac{{\sigma }^2}{k^2}$)
• (Markov’s inequality) For any non-negative random variable $X$ and for any $a>0$, we have $P(X\ge a)\le \frac{E[X]}{a}$.
• (Memoryless Property) A random variable $X$ is said to have the memoryless property if $\forall s,t\ge 0,P(X>s+t\mid X>s)=P(X>t)$.
  • The exponential and geometric distribution have the memoryless property.

Moments

• The moment generating function (MGF) $M(t)$ of a random variable $X$ is defined for all $t\in \mathbb{R}$ by $M(t)=E[e^{tX}]$

  • If $X$ is discrete with PMF $p(x)$, then $M(t)=\sum _x{e}^{tx}p(x)$

  • If $X$ is continuous with PDF $f(x)$, then $M(t)={\int }_{-\infty }^{\infty }{e}^{tx}f(x)dx$

  • ${\mu }_n(a)=E[(a-X{)}^n]$ is called the $n$th moment of $X$ about $a$.

    • ${\mu }_n^{\prime }={\mu }_n(0)=M^{(n)}(0)=E[X^n]$ is called the $n$th raw moment (or moment about origin $a=0$) of $X$.
      • The 0th raw moment is ${\mu }_0^{\prime }=E[X^0]=1$.
      • The 1st raw moment, denoted by $\mu$ is ${\mu }_1^{\prime }=E[X]$, the mean of $X$.
      • The 2nd raw moment is ${\mu }_2^{\prime }=E[X^2]$.
    • ${\mu }_n={\mu }_n(\mu )=E[(X-\mu {)}^n]$ is called the $n$th central moment (or moment about the mean $a=\mu$) of $X$.
      • The 0th central moment is ${\mu }_0=E[(X-\mu {)}^0]=1$.
      • The 1st central moment is ${\mu }_1=E[X-\mu ]=0$.
      • The 2nd central moment is ${\mu }_2=E[(X-\mu {)}^2]={\mu }_2^{\prime }-{\mu }^2=\text{Var}(X)$.

  • $M^{\prime }(0)=E[X]$

  • $M^{\prime \prime }(0)=E[X^2]$ and $M^{\prime \prime }(t)=E[X^2{e}^{tX}]$

  • $M^{(n)}(0)=E[X^n]$ and $M^{(n)}(t)=E[X^n{e}^{tX}]$

  • $M_{X+Y}(t)=M_X(t)M_Y(t)$ where $X$ and $Y$ are independent

  • $M_{X_1+\cdots +X_n}(t)=M_{X_1}(t)\cdots M_{X_n}(t)$

  • If $X_1,\dots ,X_n$ are independent and $X_i\sim \mathcal{N}({\mu }_i,{\sigma }_i^2)$, then $\sum _{i=1}^n{X}_i\sim \mathcal{N}\left(\sum _{i=1}^n{\mu }_i,\sum _{i=1}^n{\sigma }_i^2\right)$

Expectation

• Equivalence definition of expected value (or expectation or mean) of a random variable $X$:
  • (via MGF) $E[X]=M^{\prime }(0)$
  • (via CDF) $E[X]={\int }_{-\infty }^{\infty }xdF(x)$ (Riemann–Stieltjes integral)
• $E[aX+b]=aE[X]+b$

Variance

• The variance of a random variable $X$ is defined as $\text{Var}(X)=E[(X-E[X]{)}^2]$ (the 2nd central moment)
  • $\text{Var}(X)=E[X^2]-E[X{]}^2$
  • (c2.1) $\text{Var}(aX+b)=a^2\text{Var}(X)$
  • $\text{Var}(X)\ge 0$
  • If $X$ and $Y$ are independent, then $\text{Var}(X+Y)=\text{Var}(X)+\text{Var}(Y)$
• The standard deviation (denoted by $\sigma$) of a random variable $X$ is defined as $\text{SD}(X)=\sqrt{\text{Var}(X)}$
• (8.2.3) $\text{Var}(X)=0⟹P(X=E[X])=1$

Covariance

• The covariance between $X$ and $Y$, denoted by $\text{Cov}(X,Y)$, is defined by $\text{Cov}(X,Y)=E[(X-E[X])(Y-E[Y])]=E[XY]-E[X]E[Y]$.
  • $\text{Cov}(X,Y)=\text{Cov}(Y,X)$
  • $\text{Cov}(X,X)=\text{Var}(X)$
  • $\text{Cov}(aX,Y)=a\text{Cov}(X,Y)$
  • $\text{Cov}\left(\sum _{i=1}^n{X}_i,\sum _{j=1}^m{Y}_j\right)=\sum _{i=1}^n\sum _{j=1}^m\text{Cov}(X_i,Y_j)$
• $\text{Var}\left(\sum _{i=1}^n{X}_i\right)=\sum _{i=1}^n\text{Var}(X_i)+2\sum _{i<j}\text{Cov}(X_i,X_j)$
• $\text{Cov}(X,Y)=0$ iff $\rho (X,Y)=0$.
• If $X\perp Y$, then $\text{Cov}(X,Y)=0$.
• If $X_1,\dots ,X_n$ are pairwise independent, then $\text{Var}\left(\sum _{i=1}^n{X}_i\right)=\sum _{i=1}^n\text{Var}(X_i)$

Independent Identically (i.i.d.)

• Random variables $X_1,\dots ,X_n$ are said to be independent and identically distributed (i.i.d.) if:

  • $X_i$ and $X_j$ are independent for all $i\ne j$
  • $X_i$ and $X_j$ have the same distribution for all $i,j$

• Let $X_1,\dots ,X_n$ be i.i.d. random variables with CDF $F$.

  • $X_1,\dots ,X_n$ are said to be a random sample (of size $n$) from the distribution $F$.
    • $\overline{X}=\frac{1}{n}\sum _{i=1}^n{X}_i$ is the sample mean of $X_1,\dots ,X_n$.
    • $S^2=\frac{1}{n-1}\sum _{i=1}^n(X_i-\overline{X}{)}^2$ is the sample variance of $X_1,\dots ,X_n$.
  • (Weak Law of Large Numbers) If $E[X_i]=\mu$ is finite, then $\forall \epsilon >0,\underset{n\to \infty }{\lim }P\left(\left|\frac{X_1+\cdots +X_n}{n}-\mu \right|>\epsilon \right)=0$.
  • (Central Limit Theorem) If $E[X_i]=\mu$ is finite and $\mathrm{Var}(X_i)={\sigma }^2$, then $\forall a\in \mathbb{R},\underset{n\to \infty }{\lim }P\left(\frac{\overline{X}-\mu }{\sigma /\sqrt{n}}\le a\right)=\Phi (a)$.

Correlation

• The Pearson correlation coefficient between $X$ and $Y$, denoted by $\rho (X,Y)$, is defined, as long as $\text{Var}(X)\text{Var}(Y)>0$, by $\rho (X,Y)=\frac{\text{Cov}(X,Y)}{\sqrt{\text{Var}(X)\text{Var}(Y)}}$
  • $-1\le \rho (X,Y)\le 1$
  • If $\rho (X,Y)=1$, then $X$ and $Y$ are said to be perfectly positively correlated.
  • If $\rho (X,Y)=-1$, then $X$ and $Y$ are said to be perfectly negatively correlated.
  • If $\rho (X,Y)=0$, then $X$ and $Y$ are said to be uncorrelated.

Discrete RV

• A discrete random variable is a random variable $X$, whose image, $X(S)=\{X(s):s\in S\}$, is a countable set.

  • The support of a discrete random variable $X$ is the set $\{x\in \mathbb{R}:P(X=x)>0\}$

• Let $X$ be a discrete random variable with CDF $F$.

  • $F(x)=P(X\le x)=\sum _{t\le x}P(X=t)$
  • $F(x)$ is a step function with jumps at the values of $X$.
  • The probability mass function (PMF) of $X$ is the function $p:\mathbb{R}\to [0,1]$ defined by $p(a)=P\{X=a\}$
    • $\sum _{x}p(x)=1$
    • $\forall x,0\le p(x)\le 1$
    • $\forall x\notin X(S),p(x)=0$
    • $p(x)=F(x)-\underset{t\to x^{-}}{\lim }F(t)$
    • $P(a<X\le b)=F(b)-F(a)$
    • $P(X<b)=\underset{n\to \infty }{\lim }F\left(b-\frac{1}{n}\right)$

• Let $X$ be a discrete random variable with PMF $p(x)$.

  • $E[X]=\sum _{x}xp(x)$. (weighted arithmetic mean)
  • (LOTUS) $E[g(X)]=\sum _{i}g(x_i)p(x_i)$
  • $E[X^n]=\sum _x{x}^{n}p(x)$

• If $X$ and $Y$ have a joint PMF $p(x,y)$, then:

  • (7.2.1) $E[g(X,Y)]=\sum _x\sum _{y}g(x,y)p(x,y)$
  • If $E[X]$ and $E[Y]$ are finite, then $E[X+Y]=E[X]+E[Y]$.

• For every infinite collection of random variables $X_i,i\ge 1$:

  • (7.2.6) If $\sum _{i=1}^{\infty }E[|X_i|]$ is finite, then $E\left[\sum _{i=1}^{\infty }{X}_i\right]=\sum _{i=1}^{\infty }E[X_i]$.
  • If $\forall i,P(X_i\ge 0)=1$, then $E\left[\sum _{i=1}^{\infty }{X}_i\right]=\sum _{i=1}^{\infty }E[X_i]$

• todo conditional variance and expectation

Distributions

• $p$ is probability of success

Distribution	$\mathrm{supp}(X)$	PMF	$E(X)$	$\text{Var}(X)$	CDF
$\text{Uniform}(a,b)$, $n=b-a+1$	$k\in \{a,a+1,\dots ,b-1,b\}$	$P(X=k)=\frac{1}{n}$	$\frac{a+b}{2}$	$\frac{n^2-1}{12}$	$F(k)=\frac{\lfloor k\rfloor -a+1}{n}$
$\text{Uniform}(\{x_1,\dots ,x_n\})$	$k\in \{x_1,\dots ,x_n\}$	$P(X=k)=\frac{1}{n}$	$\frac{1}{n}\sum _{i=1}^n{x}_i$
$\text{Bernoulli}(p)$	$x\in \{\underset{\text{fail.}}{0},\underset{\text{succ.}}{1}\}$	$P(X=x)=p^x(1-p{)}^{1-x}$	$p$	$p(1-p)$	$F(x)=\{\begin{array}{ll}0,& x<0\\ 1-(1-p{)}^x,& x\ge 0\end{array}$
$\text{Binomial}(\underset{\text{trials}}{n},p)$ (Run $n$ i.i.d. Ber(p) trials)	$\underset{\text{successes}}{k}\in \{0,1,\dots ,n\}$	$P(X=k)=\left(\genfrac{}{}{0ex}{}{n}{k}\right)p^k(1-p{)}^{n-k}$	$np$	$np(1-p)$	$F(x)=\sum _{k=0}^x\left(\genfrac{}{}{0ex}{}{n}{k}\right)p^k(1-p{)}^{n-k}$
$\text{NB}(\underset{\text{successes}}{r},p)$ (Run i.i.d. Ber($p$) trials until $r$th suc.)	$\underset{\text{trials}}{n}\in \{r,r+1,\dots \}$	$P(X=n)=\left(\genfrac{}{}{0ex}{}{n-1}{r-1}\right)(1-p{)}^{n-r}{p}^r$
$\text{Geo}(p)$ (Run i.i.d. Ber($p$) trials until $1$st suc.)	$\underset{\text{trials}}{n}\in \{1,2,\dots \}$	$P(X=n)=(1-p{)}^{n-1}p$	$\frac{1}{p}$	$\frac{1-p}{p^2}$
$\text{Pois}(\underset{\text{rate}}{\lambda })$	$\underset{\text{events}}{k}\in \{0,1,\dots \}$	$P(X=k)=\frac{{\lambda }^k{e}^{-\lambda }}{k!}$	$\lambda$	$\lambda$	$F(x)=\sum _{k=0}^x\frac{{\lambda }^k{e}^{-\lambda }}{k!}$
$\text{Hypergeo}(\underset{\text{pop.}}{N},\underset{\text{succ.}}{K},\underset{\text{draws}}{n})$	$\underset{\text{observed}}{k}\in \{\max (0,n+K-N),\dots ,\min (n,K)\}$	$P(X=k)=\frac{\left(\genfrac{}{}{0ex}{}{K}{k}\right)\left(\genfrac{}{}{0ex}{}{N-K}{n-k}\right)}{\left(\genfrac{}{}{0ex}{}{N}{n}\right)}$	$n\frac{K}{N}$	$n\cdot \frac{K}{N}\cdot \left(1-\frac{K}{N}\right)\cdot \frac{N-n}{N-1}$

Poisson

• (Average Rate of Occurrence) $\lambda =\frac{\text{Total number of occurrences}}{\text{Total observed time or space}}$
• $\lambda =\sum _{i=1}^n{p}_i$
• $\lambda \approx \bar{X}=\frac{1}{n}\sum _{i=1}^n{X}_i$
• (Binomial Approximation) If $n$ is large, $p$ is small, and $\lambda =np$, then $\text{Binom}(n,p)\approx \text{Pois}(\lambda )$
• $\frac{P(X=k+1)}{P(X=k)}=\frac{\lambda }{k+1}$
• If $X_i\sim \text{Pois}({\lambda }_i)$ then $X=\sum _{i=1}^n{X}_i\sim \text{Pois}({\lambda }_1+\cdots +{\lambda }_n)$

Poisson Process

• $X=\sum _{i=1}^k{X}_i\sim \text{Pois}(\lambda t)$

  • $X_1\perp X_2\perp \dots \perp X_k$
  • $\forall i\in \{1,\dots ,k\}$
    • $p_i$ is the probability of an event occurring in category $i$.
      • ${\sum }_{i=1}^k{p}_i=1$
      • $p_i\ge 0$
    • $X_i\sim \text{Pois}({\lambda }_i=\lambda t\cdot p_i)$ (number of events in category $i$ in time $t$)
      • $\mathbb{E}[X_i]={\lambda }_i$
      • $\text{Var}(X_i)={\lambda }_i$
      • $\mathbb{P}(X_i=m)=e^{-{\lambda }_i}\frac{{\lambda }_i^m}{m!}$
    • $X_i\mid X=n\sim \text{Bin}(n,p_i)$
      • $\mathbb{P}(X_i=m\mid X=n)=\left(\genfrac{}{}{0ex}{}{n}{m}\right)\cdot p_i^m(1-p_i{)}^{n-m}$
  • $(X_1,\dots ,X_k)\mid X=n\sim \text{Multinomial}(n,p_1,\dots ,p_k)$
  • If $\mu =\lambda t$ is large, then $X\approx \mathcal{N}(\mu ,\mu )$

• A set of random variables $\{N(t),t\ge 0\}$ (where $N(t)\in {\mathbb{N}}_0$) is the number of events in the interval $[0,t]$) is said to be a Poisson process having rate $\lambda$ (where $\lambda >0$) if:

  • (Initial Condition) $N(0)=0$
  • (Independent Increments) For $0\le t_1<t_2<\cdots <t_n$, the random variables $N(t_2)-N(t_1),\dots ,N(t_n)-N(t_{n-1})$ are independent.
  • (Stationary Increments) For all $s,t\ge 0$, and $k\in {\mathbb{N}}_0$, we have $P(N(s+t)-N(s)=k)=P(N(t)=k)$
  • $P(N(h)=1)=\lambda h+o(h)$
  • $P(N(h)\ge 2)=o(h)$

• Let $\{N(t),t\ge 0\}$ be a Poisson process with rate $\lambda$.

  • $P(N(t)=0)=e^{-\lambda t}$
  • $P(N(t)=k)=\frac{(\lambda t{)}^k}{k!}{e}^{-\lambda t}$
  • $\forall s,t\ge 0,N(s+t)-N(s)\sim \text{Pois}(\lambda t)$

theorems

• Geometric:
  • $P(X\ge k)=(1-p{)}^{k-1}$
• Speical Cases:
  • $\text{Bernoulli}(p)=\text{Binomial}(1,p)$
  • $\text{Geometric}(p)=\text{NB}(1,p)$
• Hypergeometric:
  • If $p=\frac{K}{N}$, then $E[X]=n\frac{K}{N}=np$
    • If $N$ is large in relation to $n$, then $\text{Var}(X)\approx np(1-p)$
• If $X\perp Y$ and $X\sim \text{Pois}({\lambda }_1)$ and $Y\sim \text{Pois}({\lambda }_2)$, then $X\mid X+Y=n\sim \text{Binomial}\left(n,\frac{{\lambda }_1}{{\lambda }_1+{\lambda }_2}\right)$
• If $X\perp Y$ and $X\sim \text{Bin}(n_1,p)$ and $Y\sim \text{Bin}(n_2,p)$, then $X\mid X+Y=n\sim \text{Hypergeo}\left(n_1+n_2,n_1,n\right)$

Continuous RV

• A continuous random variable is a random variable $X$, whose image, $X(S)=\{X(s):s\in S\}$, is an uncountable set.

• Let $X$ be a continuous random variable:

  • The probability density function (PDF) of $X$ is a function $f:\mathbb{R}\to [0,\infty )$ such that:
    • ${\int }_{-\infty }^{\infty }f(x)dx=1$
    • $f(x)=\frac{d}{dx}F(x)$ where $F$ is the CDF of $X$
  • Let $F$ be the CDF of $X$:
    • $F(x)$ is a continuous function.
    • The PDF of $X$ is given by $f(x)=\frac{d}{dx}F(x)$
  • Let $f$ be the PDF of $X$:
    • ${\int }_{-\infty }^{\infty }f(x)dx=1$
    • $\forall x,f(x)\ge 0$
    • $P(X\in B)={\int }_{B}f(x)dx$
    • $P(a\le X\le b)={\int }_a^{b}f(x)dx$
    • The CDF of $X$ is given by $F(x)={\int }_{-\infty }^{x}f(t)dt=P(X\le x)$
    • $E[X]={\int }_{-\infty }^{\infty }xf(x)dx$.
      • (2.1, LOTUS) $E[g(X)]={\int }_{-\infty }^{\infty }g(x)f(x)dx$
    • $\text{Var}(X)={\int }_{-\infty }^{\infty }(x-\mu {)}^{2}f(x)dx={\int }_{-\infty }^{\infty }{x}^{2}f(x)dx-{\mu }^2$

• (5.7.1) Let $X$ be continuous with PDF $f(x)$, and the support of $X$ is an interval $I$, and $g$ is differentiable on $I$, and $g^{\prime }(x)\ne 0$ for all $x\in I$, then, the PDF of $Y=g(X)$ is given by: $f_Y(y)=\{\begin{array}{ll}{f}_X(g^{-1}(y))\cdot \left|\frac{d}{dy}{g}^{-1}(y)\right|& \text{if }y\in g(I)\\ 0& \text{otherwise}\end{array}$

Distributions

Distribution	PDF	$E(X)$	Var(X)	CDF	MGF
$\text{Uniform}(a,b)$	$f(x)=\frac{1}{b-a}$	$\frac{a+b}{2}$	$\frac{(b-a{)}^2}{12}$	$F(x)=\frac{x-a}{b-a}$
$\mathcal{N}(\mu ,{\sigma }^2)$ (Normal)	$f(x)=\frac{1}{\sigma \sqrt{2\pi }}\cdot \exp \left(-\frac{(x-\mu {)}^2}{2{\sigma }^2}\right)$	$\mu$	${\sigma }^2$
$\mathcal{N}(0,1)$ (Standard Normal)	$\phi (x)=\frac{1}{\sqrt{2\pi }}\cdot \exp \left(-\frac{x^2}{2}\right)$	0	1	$\Phi (x)=\frac{1}{\sqrt{2\pi }}{\int }_{-\infty }^x\exp \left(-\frac{t^2}{2}\right)dt$
$\text{Exp}(\lambda )$	$f(x)=\lambda e^{-\lambda x}$ (for $x>0$)	$\frac{1}{\lambda }$	$\frac{1}{{\lambda }^2}$	$1-e^{-\lambda x}$

• (4.3, Symmetry) $\Phi (-x)=1-\Phi (x)$

• If $Z=\frac{X-\mu }{\sigma }$ then $Z\sim \mathcal{N}(0,1)⟺X\sim \mathcal{N}(\mu ,{\sigma }^2)$

• $P(a<Z<b)=\Phi (b)-\Phi (a)$

• (The 68–95–99.7, empirical, or $3\sigma$ rule) If $X\sim \mathcal{N}(\mu ,{\sigma }^2)$, then:

  • $P(\mu -\sigma <X<\mu +\sigma )\approx 68\%$
  • $P(\mu -2\sigma <X<\mu +2\sigma )\approx 95\%$
  • $P(\mu -3\sigma <X<\mu +3\sigma )\approx 99.7\%$

• (De Moivre–Laplace theorem) If $X_n\sim \mathrm{Bin}(n,p)$ and $Z_n=\frac{X_n-np}{\sqrt{np(1-p)}}$, then $\underset{n\to \infty }{\lim }P\left(a\le Z_n\le b\right)=P(a\le Z\le b)=\Phi (b)-\Phi (a)$

• Let $X\sim \text{Bin}(n,p)$ and $k\in \{0,1,\dots ,n\}$, then $P(X=k)\approx \Phi \left(\frac{k+0.5-\mu }{\sigma }\right)-\Phi \left(\frac{k-0.5-\mu }{\sigma }\right)$ where:

  • $\mu =np$
  • $\sigma =\sqrt{np(1-p)}$

Jointly Distributed RV

• $X$ and $Y$
  • $p(i,j)=P(X=i,Y=j)$ is the joint PMF of $X$ and $Y$
  • $P(X=i)=\sum _{j}p(i,j)$ is the marginal PMF of $X$
  • $P(Y=j)=\sum _{i}p(i,j)$ is the marginal PMF of $Y$

Multinomial

• If $P(X_1=n_1,\dots ,X_r=n_r)=\frac{n!}{n_1!\cdots n_r!}{p}_1^{n_1}\cdots p_r^{n_r}$, then $X_1,\dots ,X_r$ are said to be jointly multinomial, where:
  • $n=n_1+\cdots +n_r$
  • $p_1+\cdots +p_r=1$
• If $X_1,\dots ,X_r$ are jointly multinomial, then: -
  • $X_i\sim \text{Bin}(n,p_i)$
  • $X_1,\dots ,X_r$ are dependent
  • For all $i\ne j$, $X_i+X_j\sim \text{Bin}(n,p_i+p_j)$
  • If $r=2$, then $X\sim \text{Bin}(n,p_1)$

Independence

• $X$ and $Y$ are said to be independent, denoted by $X\perp Y$, if $P(X=x,Y=y)=P(X=x)P(Y=y)$ for all $x,y$, otherwise they are dependent.

  • If $X$ and $Y$ are independent, iff, their joint PMF can be written as $p_{X,Y}(x,y)=h(x)g(y)$ for some functions $h$ and $g$.
  • $X_1,\dots ,X_n$ are said to be (mutually) independent if $P(X_1=x_1,\dots ,X_n=x_n)=P(X_1=x_1)\cdots P(X_n=x_n)$ for all $x_1,\dots ,x_n$
  • $X_1,\dots ,X_n$ are said to be pairwise independent if $X_i$ and $X_j$ are independent for all $i\ne j$

• If $X_i\sim \text{Pois}({\lambda }_i)$, and $X_i$ are independent, then $X=\sum _{i=1}^n{X}_i\sim \text{Pois}({\lambda }_1+\cdots +{\lambda }_n)$

• If $X_i\sim \text{Bin}(n_i,p)$, and $X_i$ are independent, then $X=\sum _{i=1}^n{X}_i\sim \text{Bin}(n_1+\cdots +n_n,p)$

• The conditional PMF of $X$ given $Y=y$ is defined as $p_{X|Y}(x|y)=P(X=x|Y=y)=\frac{P(X=x,Y=y)}{P(Y=y)}=\frac{p(x,y)}{p_Y(y)}$, where $p$ is the joint PMF of $X$ and $Y$, and $p_Y$ is the marginal PMF of $Y$, and $p_Y(y)>0$.

• (7.3.1) If $X$ and $Y$ are independent, then for any functions $g$ and $h$, $E[g(X)h(Y)]=E[g(X)]E[h(Y)]$.

Continuous

• $X$ and $Y$ are said to be jointly continuous if there exists a function $f(x,y)$ such that $P((X,Y)\in C)={\iint }_{C}f(x,y)dxdy$ for all sets $C\subseteq {\mathbb{R}}^2$.
• The function $f(x,y)$ is called the joint probability density function of $X$ and $Y$.
• $P(X\in A,Y\in B)={\int }_A{\int }_{B}f(x,y)dxdy$ (where $A,B\subseteq \mathbb{R}$)
  • $P(X\in A)={\int }_A{\int }_{-\infty }^{\infty }f(x,y)dydx={\int }_A{f}_X(x)dx$
  • $P(Y\in B)={\int }_B^{}{\int }_{-\infty }^{\infty }f(x,y)dxdy={\int }_B{f}_Y(y)dy$
• $F(a,b)=P(X\le a,Y\le b)={\int }_{-\infty }^a{\int }_{-\infty }^{b}f(x,y)dxdy$
• $f(a,b)=\frac{{\partial }^2}{\partial a\partial b}F(a,b)$