Calculus Review

Here’s a quick refresher on a few useful ideas from calculus.

$u$-substitution

Steps:

1. Choose $u=g(x)$ where $g(x)$ is a function inside the integral.
2. Compute $du/dx$ and solve for $dx$.
3. Substitute $u$ and $dx$ into the integral.
4. Integrate with respect to $u$.
5. Substitute the definition of $u$ back in to get the answer in terms of $x$.

Example:

$$\int x{e}^{x^2}dx$$

Let $u=x^2$, so that $du/dx=2x$ or $dx=\frac{du}{2x}$.

$$\int x{e}^{x^2}dx=\int x{e}^u\cdot \frac{du}{dx}=\frac{1}{2}\int e^{u}du=\frac{1}{2}{e}^u+C=\frac{1}{2}{e}^{x^2}+C.$$

Integration by Parts

Basically, integration by parts is the ‘product rule’ for derivatives in reverse (i.e., integration instead of differentiation).

The formula is always $$\int udv=uv-\int vdu.$$

Steps:

1. Identify the parts of the integral $u$ and $dv$.
2. Differentiate $u$ to get $du$.
3. Integrate $dv$ to get $v$.
4. Substitute into the formula.

Example:

$$\int x{e}^{x}dx.$$

Let $u=x$ and $dv=e^{x}dx$. Then $du=dx$ and $v=e^x$. $$\int x{e}^{x}dx=x{e}^x-\int e^{x}dx=x{e}^x-e^x+C=e^x(x-1)+C.$$

Tip

A Gamma-Poisson Relationship

Sometimes repeated application of integration by parts until reaching some kind of a base-case is the most elegant way to solve a problem.

Problem 3.19 from Casella & Berger asks one to show the relation $${\int }_x^{\mathrm{\infty }}\frac{1}{\mathrm{\Gamma }(\alpha )}{z}^{\alpha -1}{e}^{-z}dz=\sum _{y=0}^{\alpha -1}\frac{x^y{e}^{-x}}{y!},\alpha =1,2,3,...$$

First, notice what the left- and right-hand-sides are: $$\begin{array}{c}\text{RHS: }P(Y<\alpha ),\text{ where }Y\sim \text{Poisson}(x)\\ \text{LHS: }P(Z>x),\text{ where }Z\sim \text{Gamma}(\alpha ,1).\end{array}$$

Recall also that if $\alpha \in {\mathbb{N}}^{+}$, then $\mathrm{\Gamma }(\alpha )=(\alpha -1)!$.

Notice that repeated application of the differential $\frac{d}{dz}$ to $z^{\alpha -1}$ will yield a factorial term $(\alpha -1)!$.

Let $u=\frac{1}{\mathrm{\Gamma }(\alpha )}{z}^{\alpha -1}$ and $dv=e^{-z}dz$. Therefore, $$\frac{du}{dz}=\frac{d}{dz}\frac{1}{\mathrm{\Gamma }(\alpha )}{z}^{\alpha -1}=(\alpha -1)\frac{1}{\mathrm{\Gamma }(\alpha )}{z}^{\alpha -2}=\frac{1}{\mathrm{\Gamma }(\alpha -1)}{z}^{\alpha -2}.$$ Also, $$\int dv=v=\int e^{-z}dz=-e^{-z}.$$ $$\therefore {\int }_x^{\mathrm{\infty }}\frac{1}{\mathrm{\Gamma }(\alpha )}{z}^{\alpha -1}{e}^{-z}dz=\frac{1}{\mathrm{\Gamma }(\alpha )}{z}^{\alpha -1}\cdot (-e^{-z}){|}_{z=x}^{z=\mathrm{\infty }}+{\int }_x^{\mathrm{\infty }}\frac{1}{\mathrm{\Gamma }(\alpha -1)}{z}^{\alpha -2}{e}^{-z}dz.$$ Observe that as $z\to \mathrm{\infty }$, $e^{-z}\to 0$, so what remains is $$\frac{1}{\mathrm{\Gamma }(\alpha )}{x}^{\alpha -1}{e}^{-x}+{\int }_x^{\mathrm{\infty }}\frac{1}{\mathrm{\Gamma }(\alpha -1)}{z}^{\alpha -2}{e}^{-z}dz.$$ Repeat integration by parts until one is left with the term, ${\int }_x^{\mathrm{\infty }}\frac{1}{\mathrm{\Gamma }(1)}{z}^1{e}^{-z}dz$. Performing another $u$-substitution, similar to the first example above, let $u=z$ and $dv=e^{-z}dz$. Thus $du/dz=1$ and $v=-e^{-z}$. As a result, we have that $$\underset{=1}{\underbrace{\frac{1}{\mathrm{\Gamma }(1)}}}{\int }_x^{\mathrm{\infty }}z{e}^{-z}dz=e^{-x}+e^{-x}.$$ These correspond to the $y=1$ and $y=0$ terms of the series in the problem statement’s RHS series summation.

As such, we have shown that repeated integration by parts yields the following relation $$\begin{array}{rl}{\int }_x^{\mathrm{\infty }}\frac{1}{\mathrm{\Gamma }(\alpha )}{z}^{\alpha -1}{e}^{-z}dz=\frac{1}{(\alpha -1)!}{x}^{\alpha -1}{e}^{-x}+& \frac{1}{(\alpha -2)!}{x}^{\alpha -2}{e}^{-x}+\cdots +e^{-x}+e^{-x}\\ & =\sum _{y=0}^{\alpha -1}\frac{x^y{e}^{-x}}{y!}.\end{array}$$