Calculus

Definition of a Derivative

The derivative of a function measures how the function’s output changes as its input changes. Formally, the derivative of a function \(f(x)\) at a point \(x = a\) is defined as:

\[ \frac{df}{dx}\bigg|_{x=a} = \lim_{h \to 0} \frac{f(a + h) - f(a)}{h} \]

If this limit exists, \(f(x)\) is said to be differentiable at \(x = a\). The derivative represents the slope of the tangent line to the function at that point.

Derivatives of Common Functions

Here are some common derivative rules that you’ll frequently encounter:

Function	Derivative
\(c\) (constant)	\(0\)
\(x^n\)	\(n x^{n-1}\)
\(e^x\)	\(e^x\)
\(\ln(x)\)	\(\frac{1}{x}\)
\(\sin(x)\)	\(\cos(x)\)
\(\cos(x)\)	\(-\sin(x)\)
\(\tan(x)\)	\(\sec^2(x)\)
\(a^x\)	\(a^x \ln(a)\)
\(\log_a(x)\)	\(\frac{1}{x \ln(a)}\)

Differentiation Rules

Sum Rule

The sum rule states that the derivative of a sum of functions is the sum of their derivatives. If \(u(x)\) and \(v(x)\) are differentiable, then: $\( \frac{d}{dx}[u(x) + v(x)] = u'(x) + v'(x) \)$

Product Rule

The product rule allows us to differentiate the product of two functions. If \(u(x)\) and \(v(x)\) are differentiable, then:

\[ \frac{d}{dx}[u(x)v(x)] = u'(x)v(x) + u(x)v'(x) \]

Quotient Rule

The quotient rule is used to differentiate the ratio of two functions. If \(u(x)\) and \(v(x)\) are differentiable, then:

\[ \frac{d}{dx}\left[\frac{u(x)}{v(x)}\right] = \frac{u'(x)v(x) - u(x)v'(x)}{[v(x)]^2} \]

Chain Rule

The chain rule helps differentiate composite functions. If \(y = f(g(x))\), then:

\[ \frac{dy}{dx} = f'(g(x)) \cdot g'(x) \]

Partial Derivatives

When dealing with functions of multiple variables, we use partial derivatives to describe how the function changes with respect to one variable while keeping the others constant.

For a function \(f(x, y)\), the partial derivative with respect to \(x\) is denoted as:

\[ \frac{\partial f}{\partial x} \]

Similarly, the partial derivative with respect to \(y\) is:

\[ \frac{\partial f}{\partial y} \]

Example:

For \(f(x, y) = x^2y + 3xy^2\), the partial derivatives are:

\[ \frac{\partial f}{\partial x} = 2xy + 3y^2 \]

\[ \frac{\partial f}{\partial y} = x^2 + 6xy \]

Gradients

The gradient of a scalar-valued function \(f(x_1, x_2, \ldots, x_n)\) is a vector of its partial derivatives. It points in the direction of the greatest rate of increase of the function.

The gradient is denoted by \(\nabla f\) or \(\text{grad}(f)\):

\[ \nabla f = \left[ \frac{\partial f}{\partial x_1}, \frac{\partial f}{\partial x_2}, \ldots, \frac{\partial f}{\partial x_n} \right]^T \]

Since we treat vectors as column vectors, we want the gradient of a function taking in a vector to be a column vector as well. That is why we use the transpose \(T\) in the definition of the gradient.

Example:

For \(f(x, y) = x^2 + y^2\), the gradient is:

\[\begin{split} \nabla f = \begin{bmatrix} 2x \\ 2y \end{bmatrix} \end{split}\]

The gradient points away from the origin, indicating the direction in which \(f(x, y)\) increases most rapidly.