Univariate differentiation

Partial differentiation

The partial differential operator

Differential

When we change the value of an input to a function, we also change the output. We can examine these changes.

Consider the value of a function \(f(x)\) at points \(x_1\) and \(x_2\).

\(y_1=f(x_1)\)

\(y_2=f(x_2)\)

\(y_2-y_1=f(x_2)-f(x_1)\)

\(\dfrac{y_2-y_1}{x_2-x_1}=\dfrac{f(x_2)-f(x_1)}{x_2-x_1}\)

Let’s define \(x_2\) in terms of its distance from \(x_1\):

\(x_2=x_1+\epsilon\)

\(\dfrac{y_2-y_1}{\epsilon }=\dfrac{f(x_1+\epsilon )-f(x_1)}{\epsilon }\)

We define the differential of a function as:

\(\dfrac{\delta y}{\delta x}=\lim_{\epsilon \rightarrow 0^+}\dfrac{f(x+\epsilon )-f(x)}{\epsilon }\)

If this is defined, then we say the function is differentiable at that point.

Differential operator

Graph test

Differentiating constants, the identity function, and linear functions

Constants

\(f(x)=c\)

\(\dfrac{\delta y}{\delta x}=\lim_{\epsilon \rightarrow 0^+}\dfrac{f(x+\epsilon )-f(x)}{\epsilon }\)

\(\dfrac{\delta y}{\delta x}=\lim_{\epsilon \rightarrow 0^+}\dfrac{c-c}{\epsilon }=0\)

\(x\)

\(f(x)=x\)

\(\dfrac{\delta y}{\delta x}=\lim_{\epsilon \rightarrow 0^+}\dfrac{f(x+\epsilon )-f(x)}{\epsilon }\)

\(\dfrac{\delta y}{\delta x}=\lim_{\epsilon \rightarrow 0^+}\dfrac{x+\epsilon -x}{\epsilon }=1\)

Addition

\(f(x)=g(x)+h(g)\)

\(\dfrac{\delta y}{\delta x}=\lim_{\epsilon \rightarrow 0^+}\dfrac{g(x+\epsilon )+h(x+\epsilon )-g(x)-h(x)}{\epsilon }\)

\(\dfrac{\delta y}{\delta x}=\lim_{\epsilon \rightarrow 0^+}\dfrac{g(x+\epsilon )-g(x)}{\epsilon }+\lim_{\epsilon \rightarrow 0^+}\dfrac{h(x+\epsilon )-h(x)}{\epsilon }\)

\(\dfrac{\delta y}{\delta x}=\dfrac{\delta g}{\delta x}+\dfrac{\delta h}{\delta x}\)

Partial differentiation is a linear operator

Intro

The chain rule, the product rule and the quotient rule

Chain rule

\(f(x)=f(g(x))\)

\(\dfrac{\delta f}{\delta x}=\lim_{\epsilon \rightarrow 0^+}\dfrac{f(g(x+\epsilon) )-f(g(x))}{\epsilon }\)

\(\dfrac{\delta f}{\delta x}=\lim_{\epsilon \rightarrow 0^+}\dfrac{g(x+\epsilon )-g(x)}{g(x+\epsilon )-g(x)}\dfrac{f(g(x+\epsilon) )-f(g(x))}{\epsilon }\)

\(\dfrac{\delta f}{\delta x}=\lim_{\epsilon \rightarrow 0^+}\dfrac{g(x+\epsilon )-g(x)}{\epsilon }\dfrac{f(g(x+\epsilon) )-f(g(x))}{g(x+\epsilon )-g(x)}\)

\(\dfrac{\delta f}{\delta x}=\lim_{\epsilon \rightarrow 0^+}[\dfrac{g(x+\epsilon )-g(x)}{\epsilon }]\lim_{\epsilon \rightarrow 0^+}[\dfrac{f(g(x+\epsilon) )-f(g(x))}{g(x+\epsilon )-g(x)}]\)

\(\dfrac{\delta f}{\delta x}=\dfrac{\delta g}{\delta x}\dfrac{\delta f}{\delta g}\)

Product rule

\(y=f(x)g(x)\)

\(\dfrac{\delta y}{\delta x}=\lim_{\epsilon \rightarrow 0^+}\dfrac{f(x+\epsilon )g(x+\epsilon )-f(x)g(x)}{\epsilon }\)

\(\dfrac{\delta y}{\delta x}=\lim_{\epsilon \rightarrow 0^+}\dfrac{f(x+\epsilon )g(x+\epsilon )-f(x)g(x+\epsilon )+f(x)g(x+\epsilon )-f(x)g(x)}{\epsilon }\)

\(\dfrac{\delta y}{\delta x}=\lim_{\epsilon \rightarrow 0^+}\dfrac{f(x+\epsilon )g(x+\epsilon )-f(x)g(x+\epsilon )}{\epsilon }+\lim_{\epsilon \rightarrow 0^+}\dfrac{f(x)g(x+\epsilon )-f(x)g(x)}{\epsilon }\)

\(\dfrac{\delta y}{\delta x}=\lim_{\epsilon \rightarrow 0^+}g(x+\epsilon )\dfrac{f(x+\epsilon )-f(x)}{\epsilon }+\lim_{\epsilon \rightarrow 0^+}f(x)\dfrac{g(x+\epsilon )-g(x)}{\epsilon }\)

\(\dfrac{\delta y}{\delta x}=g(x)\dfrac{\delta f}{\delta x }+f(x)\dfrac{\delta g}{\delta x}\)

Quotient rule

\(y=\dfrac{f(x)}{g(x)}\)

\(\dfrac{\delta }{\delta x}y=\dfrac{\delta }{\delta x}\dfrac{f(x)}{g(x)}\)

\(\dfrac{\delta }{\delta x}y=\dfrac{\delta }{\delta x}f(x)\dfrac{1}{g(x)}\)

\(\dfrac{\delta }{\delta x}y=\dfrac{\delta f}{\delta x}\dfrac{1}{g(x)}-\dfrac{\delta g}{\delta x}\dfrac{f(x)}{g(x)^2}\)

\(\dfrac{\delta }{\delta x}y=\dfrac{\dfrac{\delta f}{\delta x}g(x)-\dfrac{\delta g}{\delta x}f(x)}{g(x)^2}\)

Differentiating natural number power functions

Other

\(\dfrac{\delta }{\delta x}x^n=\lim_{\delta \rightarrow 0}\dfrac{(x+\delta)^n-x^n}{\delta}\)

\(\dfrac{\delta }{\delta x}x^n=\lim_{\delta \rightarrow 0}\dfrac{(\sum_{i=0}^n x^i\delta ^{n-i}\dfrac{n!}{i!(n-i)!})-x^n}{\delta}\)

\(\dfrac{\delta }{\delta x}x^n=\lim_{\delta \rightarrow 0}\sum_{i=0}^{n-1} x^i\delta ^{n-i-1}\dfrac{n!}{i!(n-i)!}\)

\(\dfrac{\delta }{\delta x}x^n=\lim_{\delta \rightarrow 0}x^{n-1}\dfrac{n!}{(n-1)!(n-n+1)!}+\sum_{i=0}^{n-2} x^i\delta ^{n-i-1}\dfrac{n!}{i!(n-i)!}\)

\(\dfrac{\delta }{\delta x}x^n=nx^{n-1}\)

L’Hôpital’s rule

L’Hôpital’s rule

If there are two functions which are both tend to \(0\) at a limit, calculating the limit of their divisor is hard. We can use L’Hopital’s rule.

We want to calculate:

\(\lim_{x\rightarrow c}\dfrac{f(x)}{g(x)}\)

This is:

\(\lim_{x\rightarrow c}\dfrac{f(x)}{g(x)}=\lim_{x\rightarrow c}\dfrac{\dfrac{f(x)-0}{\delta}}{\dfrac{g(x)-0}{\delta}}\)

If:

\(\lim_{x\rightarrow c}f(x)=\lim_{x\rightarrow c}g(x)=0\)

Then

\(\lim_{x\rightarrow c}\dfrac{f(x)}{g(x)}=\lim_{x\rightarrow c}\dfrac{\dfrac{f(x)-f(c)}{\delta}}{\dfrac{g(x)-f(c)}{\delta}}\)

\(\lim_{x\rightarrow c}\dfrac{f(x)}{g(x)}=\dfrac{f'(x)}{g'(x)}\)

Rolle’s theorem

Rolle’s theorem

Take a real function \(f(x)\) on closed interval \([a,b]\), differentiable on \((a,b,)\), and \(f(a)=f(b)\).

Rolle’s theorem states that:

\(\exists c\in(a,b) (f’(c)=0)\)

Generalised Rolle’s theorem states that:

Generalised Rolle’s theorem implies Rolle’s theorem, so we only need to prove the generalised theorem.

Mean value theorem

Mean value theorem

Take a real function \(f(x)\) on closed interval \([a,b]\), differentiable on \((a,b,)\).

The mean value theorem states that:

\(\exists c\in(a,b) (f’(c)=\dfrac{f(b)-f(a)}{b-a})\)

Elasticity

Introduction

We have \(f(x)\)

\(Ef(x)=\dfrac{x}{f(x)}\dfrac{\delta f(x)}{\delta x}\)

This is the same as:

\(Ef(x)=\dfrac{\delta \ln f(x)}{\delta \ln x}\)

Smooth functions

Analytic function

Introduction

Higher-order differentials

Differentiable functions

Introduction

A differentiable function is one where the differential is defined at all points on the real line.

All differentiable functions are continuous. Not all continuous functions are differentiable.

Differentiability class

We can describe a function with its differentiability class. If a function can be differentiated \(n\) times and these differentials are all continous, then the function is class \(C^n\).

Smooth functions

If a function can be differentiated infinitely many times to produce continous functions, it is \(C^{\infty }\), or smooth.

Critial points

Critical points

Where partial derivative are \(0\).