From the cominatorics page, we know that the sample space of ordered samples is:

# ¶ Sampling with Replacement

## ¶ Binomial distribution

Sample space: .

Events: .

Probability: $\mathrm{P}\left({A}_{k}\right)=\left(\genfrac{}{}{0px}{}{n}{k}\right){p}^{k}{q}^{n-k}\Rho\left(A_k\right)=\displaystyle\binom\left\{n\right\}\left\{k\right\}p^kq^\left\{n-k\right\}$.

The set of probabilities $\left(\mathrm{P}\left({A}_{0}\right),\dots ,\mathrm{P}\left({A}_{n}\right)\right)\left\left(\Rho\left(A_0\right),\dots,\Rho\left(A_n\right)\right\right)$ is called the binomial distribution.

## ¶ Multinomial distribution

Sample space: .

Events: , where ${\nu }_{i}\left(\omega \right)\nu_i\left(\omega\right)$ is the number of elements of $\omega =\left({a}_{1},\dots ,{a}_{n}\right)\omega=\left(a_1,\dots,a_n\right)$ that are equal to ${n}_{i},i=1,\dots ,rn_i, i=1,\dots, r$.

Probability: $\mathrm{P}\left({A}_{{n}_{1},\dots ,{n}_{r}}\right)=\left(\genfrac{}{}{0px}{}{n}{{n}_{1},\dots ,{n}_{r}}\right){p}_{1}^{{n}_{1}}\cdots {p}_{r}^{{n}_{r}}\Rho\left(A_\left\{n_1,\dots,n_r\right\}\right)=\displaystyle\binom\left\{n\right\}\left\{n_1,\dots,n_r\right\}p_1^\left\{n_1\right\}\cdots p_r^\left\{n_r\right\}$.

The set of probabilities $\left\{\mathrm{P}\left({A}_{{n}_{1},\dots ,{n}_{r}}\right)\right\}\\left\{\Rho\left(A_\left\{n_1,\dots,n_r\right\}\right)\\right\}$ is called the multinomial distribution, where the number $\left(\genfrac{}{}{0px}{}{n}{{n}_{1},\dots ,{n}_{r}}\right)\displaystyle\binom\left\{n\right\}\left\{n_1,\dots,n_r\right\}$ is the multinomial coefficient.

# ¶ Sampling without Replacement

## ¶ Multidimensional hypergeometric distribution

Experiment: There is an urn containing $MM$ balls, where ${M}_{1}M_1$ balls have the color ${b}_{1},\dots ,{M}_{r}b_1, \dots, M_r$ balls have the color ${b}_{r}b_r$, and ${M}_{1}+\cdots +{M}_{r}=MM_1+\dots + M_r=M$. Draw a sample size of $n balls without replacement.

Sample space: .

Event: ${B}_{{n}_{1},\dots ,{n}_{r}}B_\left\{n_1,\dots,n_r\right\}$ in which ${n}_{1}n_1$ balls have the color ${b}_{1},\dots ,{n}_{r}b_1,\dots,n_r$ balls have color ${b}_{r}b_r$, where ${n}_{1}+\cdots +{n}_{r}=nn_1+\dots+n_r=n$.

Probability: $\mathrm{P}\left({B}_{{n}_{1},\dots ,{n}_{r}}\right)=\frac{N\left({B}_{{n}_{1},\dots ,{n}_{r}}\right)}{N\left(\mathrm{\Omega }\right)}=\frac{\left(\genfrac{}{}{0px}{}{n}{{n}_{1},\dots ,{n}_{r}}\right){A}_{{M}_{1}}^{{n}_{1}}\dots {A}_{{M}_{r}}^{{n}_{r}}}{{A}_{M}^{n}}=\frac{\left(\genfrac{}{}{0px}{}{n}{{n}_{1},\dots ,{n}_{r}}\right)\left(\genfrac{}{}{0px}{}{{M}_{1}}{{n}_{1}}\right)\dots \left(\genfrac{}{}{0px}{}{{M}_{r}}{{n}_{r}}\right)}{n!\cdot \left(\genfrac{}{}{0px}{}{M}{n}\right)}=\frac{\left(\genfrac{}{}{0px}{}{{M}_{1}}{{n}_{1}}\right)\dots \left(\genfrac{}{}{0px}{}{{M}_{r}}{{n}_{r}}\right)}{\left(\genfrac{}{}{0px}{}{M}{n}\right)}\displaystyle\Rho\left(B_\left\{n_1,\dots,n_r\right\}\right)=\frac\left\{N\left(B_\left\{n_1,\dots,n_r\right\}\right)\right\}\left\{N\left(\Omega\right)\right\}=\frac\left\{\binom\left\{n\right\}\left\{n_1,\dots,n_r\right\}A_\left\{M_1\right\}^\left\{n_1\right\}\dots A_\left\{M_r\right\}^\left\{n_r\right\}\right\}\left\{A^n_M\right\}=\frac\left\{\binom\left\{n\right\}\left\{n_1,\dots,n_r\right\}\binom\left\{M_1\right\}\left\{n_1\right\}\dots\binom\left\{M_r\right\}\left\{n_r\right\}\right\}\left\{n!\cdot \binom\left\{M\right\}\left\{n\right\}\right\}=\frac\left\{\binom\left\{M_1\right\}\left\{n_1\right\}\dots\binom\left\{M_r\right\}\left\{n_r\right\}\right\}\left\{\binom\left\{M\right\}\left\{n\right\}\right\}$.

The set of probabilities $\left\{\mathrm{P}\left({B}_{{n}_{1},\dots ,{n}_{r}}\right)\right\}\\left\{\Rho\left(B_\left\{n_1,\dots,n_r\right\}\right)\\right\}$ is called the multidimensional hypergeometric distribution. When $r=2r=2$, it is called the hypergeometric distribution.

### ¶ Hypergeometric distribution vs. Binomial distribution

When $r=2r=2$, $\mathrm{P}\left({B}_{{n}_{1},\dots ,{n}_{r}}\right)=\frac{\left(\genfrac{}{}{0px}{}{{M}_{1}}{{n}_{1}}\right)\left(\genfrac{}{}{0px}{}{{M}_{2}}{{n}_{2}}\right)}{\left(\genfrac{}{}{0px}{}{M}{n}\right)},{n}_{1}+{n}_{2}=n,{M}_{1}+{M}_{2}=M\displaystyle\Rho\left(B_\left\{n_1,\dots,n_r\right\}\right)=\frac\left\{\binom\left\{M_1\right\}\left\{n_1\right\}\binom\left\{M_2\right\}\left\{n_2\right\}\right\}\left\{\binom\left\{M\right\}\left\{n\right\}\right\}, n_1+n_2=n,M_1+M_2=M$.

If , and therefore ${M}_{2}\mathrm{/}M\to 1-pM_2/M\rightarrow 1-p$, then

$\mathrm{P}\left({B}_{{n}_{1},{n}_{2}}\right)\to \left(\genfrac{}{}{0px}{}{{n}_{1}+{n}_{2}}{{n}_{1}}\right){p}_{1}^{{n}_{1}}\left(1-{p}_{1}{\right)}^{{n}_{2}}\Rho\left(B_\left\{n_1,n_2\right\}\right)\rightarrow \binom\left\{n_1+n_2\right\}\left\{n_1\right\}p_1 ^\left\{n_1\right\}\left(1-p_1\right)^\left\{n_2\right\}$

Under the hypotheses above, the hypergeometric distribution is approximated by the binomial distribution. This is intuitive because when $MM$ and ${M}_{1}M_1$ are large (but finite), sampling without replacement should give almost the same result as sampling with replacement.