Calculus Review
Here’s a quick refresher on a few useful ideas from calculus.
\(u\)-substitution
Steps:
- Choose \(u = g(x)\) where \(g(x)\) is a function inside the integral.
- Compute \(du/dx\) and solve for \(dx\).
- Substitute \(u\) and \(dx\) into the integral.
- Integrate with respect to \(u\).
- Substitute the definition of \(u\) back in to get the answer in terms of \(x\).
Example:
\[\int x e^{x^2} dx\]
Let \(u = x^2\), so that \(du/dx = 2x\) or \(dx = \frac{du}{2x}\).
\[\int x e^{x^2} dx = \int x e^u \cdot \frac{du}{dx} = \frac{1}{2} \int e^u du = \frac{1}{2} e^u + C = \frac{1}{2} e^{x^2} + C.\]
Integration by Parts
Basically, integration by parts is the ‘product rule’ for derivatives in reverse (i.e., integration instead of differentiation).
The formula is always \[ \int u dv = uv - \int v du.\]
Steps:
- Identify the parts of the integral \(u\) and \(dv\).
- Differentiate \(u\) to get \(du\).
- Integrate \(dv\) to get \(v\).
- Substitute into the formula.
Example:
\[\int x e^x dx.\]
Let \(u = x\) and \(dv = e^x dx\). Then \(du = dx\) and \(v = e^x\). \[\int x e^x dx = xe^x - \int e^x dx = xe^x - e^x + C = e^x(x-1) + C.\]
A Gamma-Poisson Relationship
Sometimes repeated application of integration by parts until reaching some kind of a base-case is the most elegant way to solve a problem.
Problem 3.19 from Casella & Berger asks one to show the relation \[\int_x^\infty \frac{1}{\Gamma(\alpha)} z^{\alpha - 1} e^{-z} dz = \sum_{y=0}^{\alpha - 1} \frac{x^y e^{-x}}{y!}, \quad \alpha = 1, 2, 3, ...\]
First, notice what the left- and right-hand-sides are: \[ \begin{array}{c} \text{RHS: } P(Y < \alpha), \text{ where } Y \sim \text{Poisson}(x) \\ \text{LHS: } P(Z > x), \text{ where } Z \sim \text{Gamma}(\alpha, 1). \end{array} \]
Recall also that if \(\alpha \in \mathbb N^{+}\), then \(\Gamma(\alpha) = (\alpha-1)!\).
Notice that repeated application of the differential \(\frac{d}{dz}\) to \(z^{\alpha-1}\) will yield a factorial term \((\alpha-1)!\).
Let \(u = \frac{1}{\Gamma(\alpha)} z^{\alpha - 1}\) and \(dv = e^{-z} dz\). Therefore, \[\frac{du}{dz} = \frac{d}{dz} \frac{1}{\Gamma(\alpha)} z^{\alpha-1} = (\alpha - 1)\frac{1}{\Gamma(\alpha)} z^{\alpha - 2} = \frac{1}{\Gamma(\alpha-1)} z^{\alpha-2}.\] Also, \[\int dv = v = \int e^{-z} dz = - e^{-z}.\] \[\therefore \int_x^{\infty} \frac{1}{\Gamma(\alpha)} z^{\alpha-1} e^{-z} dz = \frac{1}{\Gamma(\alpha)} z^{\alpha - 1} \cdot (-e^{-z}) \Bigg \vert_{z=x}^{z=\infty} + \int_{x}^\infty \frac{1}{\Gamma(\alpha - 1)} z^{\alpha-2} e^{-z}dz.\] Observe that as \(z \to \infty\), \(e^{-z} \to 0\), so what remains is \[\frac{1}{\Gamma(\alpha)} x^{\alpha - 1} e^{-x} + \int_x^{\infty} \frac{1}{\Gamma(\alpha - 1)} z^{\alpha - 2} e^{-z}dz.\] Repeat integration by parts until one is left with the term, \(\int_x^\infty \frac{1}{\Gamma(1)} z^1 e^{-z} dz\). Performing another \(u\)-substitution, similar to the first example above, let \(u = z\) and \(dv = e^{-z} dz\). Thus \(du/dz = 1\) and \(v = - e^{-z}\). As a result, we have that \[\underbrace{\frac{1}{\Gamma(1)}}_{=1} \int_x^\infty z e^{-z} dz = e^{-x} + e^{-x}.\] These correspond to the \(y = 1\) and \(y = 0\) terms of the series in the problem statement’s RHS series summation.
As such, we have shown that repeated integration by parts yields the following relation \[ \begin{aligned} \int_x^\infty \frac{1}{\Gamma(\alpha)} z^{\alpha - 1} e^{-z} dz = \frac{1}{(\alpha-1)!} x^{\alpha - 1} e^{-x} + & \frac{1}{(\alpha - 2)!} x^{\alpha - 2} e^{-x} + \; \cdots \; + e^{-x} + e^{-x} \\ & = \sum_{y=0}^{\alpha-1} \frac{x^y e^{-x}}{y!}. \end{aligned} \]