Calculus Review

Here’s a quick refresher on a few useful ideas from calculus.

\(u\)-substitution

Steps:

  1. Choose \(u = g(x)\) where \(g(x)\) is a function inside the integral.
  2. Compute \(du/dx\) and solve for \(dx\).
  3. Substitute \(u\) and \(dx\) into the integral.
  4. Integrate with respect to \(u\).
  5. Substitute the definition of \(u\) back in to get the answer in terms of \(x\).

Example:

\[\int x e^{x^2} dx\]

Let \(u = x^2\), so that \(du/dx = 2x\) or \(dx = \frac{du}{2x}\).

\[\int x e^{x^2} dx = \int x e^u \cdot \frac{du}{dx} = \frac{1}{2} \int e^u du = \frac{1}{2} e^u + C = \frac{1}{2} e^{x^2} + C.\]

Integration by Parts

Basically, integration by parts is the ‘product rule’ for derivatives in reverse (i.e., integration instead of differentiation).

The formula is always \[ \int u dv = uv - \int v du.\]

Steps:

  1. Identify the parts of the integral \(u\) and \(dv\).
  2. Differentiate \(u\) to get \(du\).
  3. Integrate \(dv\) to get \(v\).
  4. Substitute into the formula.

Example:

\[\int x e^x dx.\]

Let \(u = x\) and \(dv = e^x dx\). Then \(du = dx\) and \(v = e^x\). \[\int x e^x dx = xe^x - \int e^x dx = xe^x - e^x + C = e^x(x-1) + C.\]

Tip
A Gamma-Poisson Relationship

Sometimes repeated application of integration by parts until reaching some kind of a base-case is the most elegant way to solve a problem.

Problem 3.19 from Casella & Berger asks one to show the relation \[\int_x^\infty \frac{1}{\Gamma(\alpha)} z^{\alpha - 1} e^{-z} dz = \sum_{y=0}^{\alpha - 1} \frac{x^y e^{-x}}{y!}, \quad \alpha = 1, 2, 3, ...\]

First, notice what the left- and right-hand-sides are: \[ \begin{array}{c} \text{RHS: } P(Y < \alpha), \text{ where } Y \sim \text{Poisson}(x) \\ \text{LHS: } P(Z > x), \text{ where } Z \sim \text{Gamma}(\alpha, 1). \end{array} \]

Recall also that if \(\alpha \in \mathbb N^{+}\), then \(\Gamma(\alpha) = (\alpha-1)!\).

Notice that repeated application of the differential \(\frac{d}{dz}\) to \(z^{\alpha-1}\) will yield a factorial term \((\alpha-1)!\).

Let \(u = \frac{1}{\Gamma(\alpha)} z^{\alpha - 1}\) and \(dv = e^{-z} dz\). Therefore, \[\frac{du}{dz} = \frac{d}{dz} \frac{1}{\Gamma(\alpha)} z^{\alpha-1} = (\alpha - 1)\frac{1}{\Gamma(\alpha)} z^{\alpha - 2} = \frac{1}{\Gamma(\alpha-1)} z^{\alpha-2}.\] Also, \[\int dv = v = \int e^{-z} dz = - e^{-z}.\] \[\therefore \int_x^{\infty} \frac{1}{\Gamma(\alpha)} z^{\alpha-1} e^{-z} dz = \frac{1}{\Gamma(\alpha)} z^{\alpha - 1} \cdot (-e^{-z}) \Bigg \vert_{z=x}^{z=\infty} + \int_{x}^\infty \frac{1}{\Gamma(\alpha - 1)} z^{\alpha-2} e^{-z}dz.\] Observe that as \(z \to \infty\), \(e^{-z} \to 0\), so what remains is \[\frac{1}{\Gamma(\alpha)} x^{\alpha - 1} e^{-x} + \int_x^{\infty} \frac{1}{\Gamma(\alpha - 1)} z^{\alpha - 2} e^{-z}dz.\] Repeat integration by parts until one is left with the term, \(\int_x^\infty \frac{1}{\Gamma(1)} z^1 e^{-z} dz\). Performing another \(u\)-substitution, similar to the first example above, let \(u = z\) and \(dv = e^{-z} dz\). Thus \(du/dz = 1\) and \(v = - e^{-z}\). As a result, we have that \[\underbrace{\frac{1}{\Gamma(1)}}_{=1} \int_x^\infty z e^{-z} dz = e^{-x} + e^{-x}.\] These correspond to the \(y = 1\) and \(y = 0\) terms of the series in the problem statement’s RHS series summation.

As such, we have shown that repeated integration by parts yields the following relation \[ \begin{aligned} \int_x^\infty \frac{1}{\Gamma(\alpha)} z^{\alpha - 1} e^{-z} dz = \frac{1}{(\alpha-1)!} x^{\alpha - 1} e^{-x} + & \frac{1}{(\alpha - 2)!} x^{\alpha - 2} e^{-x} + \; \cdots \; + e^{-x} + e^{-x} \\ & = \sum_{y=0}^{\alpha-1} \frac{x^y e^{-x}}{y!}. \end{aligned} \]