Theorems and proofs of Weierstrass, Bolzano, Riemann, Cantor, Dedekind, Cauchy, L'Hospital, Heine, Leibniz
This article is a conclusive summary of the most important theorems and their proofs from my Analysis 1 course. I am going to talk about: real number sequences, functions and their convergence, among other properties. Derivation and Riemann integration.
1.0: The Cantor Axiom
Infinitely many bounded intervals nested into each other have a nonempty intersection and it is a single point. That means you keep placing a smaller bounded interval into a larger one, so that its size approaches infinity. This property is only true for real numbers and comes without proof, as it's an axiom.
1.1: Triangle inequality:
$$\begin{aligned} -\left| x \right| & \le x \le \left| x \right|\\ -\left| y \right| & \le y \le \left| y \right|\\ \implies -(\left| y \right| + \left| x \right|) & \le y + x \le \left| y \right| + \left| x \right|\\ \implies -(\left| y \right| + \left| x \right|) & \le y + x \le \left| y \right| + \left| x \right|\\ \implies \left| y + x \right| & \le \left| \left| y \right| + \left| x \right| \right|\\ \implies \left| y + x \right| & \le \left| y \right| + \left| x \right|\\ \end{aligned}$$
1.1.1: Reverse triangle inequality.
Using the triangle inequality (1.1):
$$\begin{aligned} \left| x \right| + \left| y - x \right| & \ge \left| x + y - x \right| = \left| y \right|\\ \left| y \right| + \left| x - y \right| & \ge \left| y + x - y \right| = \left| x \right| \end{aligned}$$
$$\begin{aligned} \left| y - x \right| & \ge \left| y \right| - \left| x \right|\\ \left| x - y \right| & \ge \left| x \right| - \left| y \right| \end{aligned}$$
Since $\left| y - x \right| = \left| x - y \right|$ and since $\left| y \right| - \left| x \right| = -1 \cdot (\left| x \right| - \left| y \right|):$
$$\left| x - y \right| \ge \left| \left| x \right| - \left| y \right| \right|$$
1.2: Definition of a Cauchy sequence:
For $\forall\ \epsilon > 0 $ there $ \exists\ \N(\epsilon):\quad \left| a_k - a_l \right| < \epsilon$ for $\forall\ k,\ l \ge \N $.
1.2.1: A function can be, oddly said, 'Cauchy' at $x_0$ if:
$\forall\ \epsilon > 0$ there $\exists\ \delta(\epsilon):\quad \left| \f(x_0) - \f(x_1) \right| < \epsilon$ for $\forall\ x_0,\ x_1 \in \left] x_0 - \delta,\ x_0 + \delta \right[$
1.3: A Cauchy sequence is bounded
Proof:
If a sequence is Cauchy then for $\forall\ \epsilon > 0$ there $\exists\ \N(\epsilon):\quad \left| a_n - a_m \right| < \epsilon$ for $\forall\ n,\ m > \N$
We wish to show $\exists\ C \in \mathbb{R}:\quad a_n < C$ for $\forall\ n \in \mathbb{N}$
Let $\epsilon = 1$, so there $\exists\ \N_0:\quad \left| a_n - a_m \right| < 1$ for $\forall\ m, n > \N_0$
For $\forall\ n$ it holds because of the triangle inequality (1.1):
$\left| a_n \right| = \left| a_n - a_m + a_m \right| = \left| a_n - a_m \right| + \left| a_m \right|$
Now let $m = \N_0 + 1$ so that $ \left| a_n \right| < 1 + \left| a_{\N_0 + 1}\right|$ for $\forall\ n > \N_0$
Therefore beyond $\N_ 0$ every term is bounded. There are finitely many terms below $\N_0$, so there must be a minimum and maximum. Therefore a Cauchy sequence is bounded.
1.3.1: If a function satisfies the Cauchy criterium at $x_0$, then there must exist a neighbourhood of $x_0$ in which $\f$ is bounded. Proof: if at every neighbourhood of $x_0$ $\f$ was not bounded, then it would be impossible for all elements at any neighbourhood to be at most a fix $\epsilon_0$ distance from each other, because we could always select two elements whose difference is larger than $\epsilon_0$. But if we could, then the function must be bounded in the neighbourhood.
1.4: A sequence converges to $A \in \mathbb{R}$ if:
$\forall\ \epsilon > 0 $ there $ \exists\ \N(\epsilon):\quad \left| a_n - A \right| < \epsilon$ for $\forall\ n \ge \N$.
1.4.1: A function converges to $A \in \mathbb{R}$ at $x_0$, or: $\lim_{x \to x_0} \f(x) = A$ if:
$\forall\ \epsilon > 0 $ there $ \exists\ \delta(\epsilon):\quad \left| \f(x) - A \right| < \epsilon$ for $\forall\ x \in A$.
1.5: The limit of a function at $x_0$ is unique
The proof is similar to the one of the uniqueness of a sequence. Let's indirectly assume a limit $A$ and $B$ at $x_0$, and $A \neq B$.
So then for $\forall\ \epsilon > 0$ there $\exists\ \delta_0(\epsilon):\quad \left| \f(x) - A \right| < \epsilon$ for $\forall\ x:\quad \left| x - x_0 \right|$
And for $\forall\ \epsilon > 0$ there $\exists\ \delta_1(\epsilon):\quad \left| \f(x) - B \right| < \epsilon$ for $\forall\ x:\quad \left| x - x_0 \right|$
Let's assume $A < B$ since order is irrelevant.
Then if we choose $d = \frac{A + B}{2}$ there $\exists\ \delta_2(\epsilon_0) = \min(\delta_0(\epsilon_0),\ \delta_1(\epsilon_0)):$
$$ \left] B - \epsilon_0,\ B + \epsilon_0 \right[ \cup \left] A - \epsilon_0,\ A + \epsilon_0 \right[ = \emptyset$$
For $\forall\ \epsilon_0 < d$. And that is a contradiction. It is not possible for any function value to be at $\epsilon_0$ neighbourhood of both $A$ and $B$, though for $\delta_2$ it should be the case.
1.5.1: The limit of a sequence is unique
Indirect proof:
Let's assume $a_n \to A$ and $a_n \to B$ and $A \neq B$.
Therefore for $\forall\ \epsilon > 0$ there $\exists\ \N_0(\epsilon):\quad \left| a_n - A \right| < \epsilon$ for $\forall\ n \ge \N_0$
And also for $\forall\ \epsilon > 0$ there $\exists\ \N_1(\epsilon):\quad \left| a_n - B \right| < \epsilon$ for $\forall\ n \ge \N_0$
So then if $\N_2(\epsilon) = \max(\N_0(\epsilon),\ \N_1(\epsilon))$ then it must hold that:
$$ \begin{aligned} A - \epsilon & < a_n < A + \epsilon\\ B - \epsilon & < a_n < B + \epsilon \end{aligned} $$
There must $\exists\ \epsilon_0$ for which $a_n$ is in so close neighbourhood of both $A$ and $B$ that the two neighbourhoods (intervals) are disjunct. That is, it is not possible for the $a_n$ to be in both of the intervals at the same time. That as I said, requires $\epsilon_0$ to be sufficiently small, beause if it's sufficiently large the two neighbourhoods might still intersect. For example, let $A < B$ (and note that the order is meaningless) then let $d = \frac{A + B}{2}$. If we wanted to set $\epsilon_0 \le d$, then the set of numbers in the open neighbourhood of $A$ and $B$ would be already disjunct, and $a_n$ is forced to be exactly one of them. In other words $\left] A - \epsilon_0, A + \epsilon_0 \right[ \cup \left] B - \epsilon_0, B + \epsilon_0 \right[ = \emptyset$
1.6: A function's left and right hand side limits
We get the left hand side limit at $x_0$ if set $D_{\f}' = D_{\f} \cup \left] -\infty,\ x_0 \right[$ and see if it now converges by original definition (1.4.1).
Its notation is: $\lim_{x \to x_0+0} \f(x) = \lim_{x \to x_0+} \f(x) = f(x_0 + 0)$
Similarly we get the right hand side limit at $x_0$ if set $D_{\f}' = D_{\f} \cup \left] x_0,\ \infty \right[$ and see if it now converges by original definition (1.4.1).
Its notation is: $\lim_{x \to x_0-0} \f(x) = \lim_{x \to x_0-} \f(x) = f(x_0 - 0)$
1.7: $\exists\ \lim_{x \to x_0} \f(x) \iff \exists\ \f(x_0+0),\ \f(x_0-0)$ and $\f(x_0+0) = \f(x_0-0)$
Proof $\implies$: Let's assume $\exists\ \lim_{x \to x_0} \f(x) = A$
This one is trivial. If we cut the domain into two at $x_0$ (and neither contains $x_0$) then both the right and left hand side of the function can still be brought as close to $A$ as we wish, therefore both are ought to converge to $A$ and so $\delta(\epsilon)$ is sufficient for $\forall\ \epsilon > 0$
Proof $\impliedby$: Let's assume $\exists\ \f(x_0+0),\ \f(x_0-0)$ and $\f(x_0+0) = \f(x_0-0)$
Then let $\delta_l(\epsilon),\ \delta_r(\epsilon)$ be the $\delta$ functions of the reduced-domain left hand and right hand functions, respectively. But then it inherently means that with $\delta(\epsilon) = \min(\delta_r(\frac{\epsilon}{2}),\ \delta_l(\frac{\epsilon}{2}))$ will bring the function with the united domain in the $\epsilon$ neighbourhood of A. And since a limit is always unique, $\exists\ \lim_{x \to x_0} \f(x) = A$.
1.8: Infinite sequence $a_n$ has a cluster point at $A$ if:
- $\exists\ a_{n_k}$ infinite subsequence whose limit is $A$
- Every neighbourhood of $A$ contains infinitely many sequence terms of $a_n$
And the two definitions are equivalent. Proof:
$\implies$
We assume that $\exists\ a_{n_k}$ subsequence whose limit is $A$, and wish to show every neighbourhood of A contains infinitely many sequence terms of $a_n$
By the definiton of the limit for sequences (1.4), for $\forall \epsilon > 0$ there $\exists\ \N(\epsilon):\quad \left| a_n - A \right| < \epsilon$ for $\forall\ n > \N$ This means there are infinitely many sequence terms in any chosen neighbourhood of $A$
$\impliedby$
We assume infinitely many sequence terms in any neighbourhood of $A$, and wish to show that for $\forall\ \epsilon > 0$ neighbourhood $\exists\ a_{n_k},\ \N(\epsilon):\quad \left| a_{n_k} - A \right| < \epsilon$ for $\forall\ n > \N$
Let's consider an $\epsilon_0$ neighbourhood. Then also consider an $\epsilon_1 < \epsilon 0$ neighbourhood. Both contain infinitely many sequence terms, but outside them there are
Consequence: $a_n$ is only convergent, or in other words, $\exists\ \lim_{n \to \infty} a_n$ $\iff$ $a_n$ has only one cluster point.
1.8.1: Function $\f$ has a cluster point at $x_0$ if:
- $\exists\ \lim_{x \to x_0} \f(x)$
- Every neighbourhood of $x_0$ contains infinitely many points of $D_f$
1.9: A function is continuous at $x_0$ if $\exists\ \lim_{x \to x_0} \f(x)$ and $\exists\ \f(x_0)$
2.0: A function is said to have a removable discontinuity at $x_0$ given $\exists\ \f(x_0 + 0),\ \f(x-0)$, $\f(x+0) = \f(x-0) \in \mathbb{R}$ and $x_0 \notin D_f\$ ($\nexists \f(x_0)$).
2.1: A function is said to have a jump discontinuity at $x_0$ given $\exists\ \f(x+0),\ \f(x-0)$, but $\f(x+0) \in \mathbb{R} \neq \f(x-0) \mathbb{R}$.
2.2: A function is said to have an infinite discontinuity at $x_0$ given either $\f(x+0) = \pm \infty$ and or $\f(x-0) = \pm \infty$
2.3: $a_n \to \infty$ when for $\forall\ P > 0$ there $\exists\ \N(P):\quad a_n > P$ for $\forall\ n > \N(P)$
2.3.1: $\lim_{x \to x_0} \f(x) = \infty$ when for $\forall\ P > 0$ there $\exists\ \delta(P):\quad \f(x) > P$ for $\forall\ x \in \left] x_0 - \delta,\ x_0 + \delta \right[$
2.4: Squeeze theorem for sequences
$a_n \to A$ and $c_n \to A$ and if $\exists\ N_0:\quad a_n \le b_n \le c_n$ for $\forall\ n > N_0$ then $c_n \to A$
Proof:
Since $a_n \to A$ for $\forall\ \epsilon > 0$ there $\exists\ \N_a(\epsilon):\quad \left| a_n - A \right| < \epsilon$ for $\forall\ n > \N_a$
Also $c_n \to A$ for $\forall\ \epsilon > 0$ there $\exists\ \N_c(\epsilon):\quad \left| c_n - A \right| < \epsilon$ for $\forall\ n > \N_c$
So then for $\forall\ n > \N(\epsilon) = \max(\N_a(\epsilon),\ N_c(\epsilon), N_0):$
$$ \begin{aligned} A - \epsilon & < a_n < A + \epsilon\\ A - \epsilon & < c_n < A + \epsilon\\ a_n & \le b_n \le c_n\\ \implies A - \epsilon < a_n & < b_n < c_n < A + \epsilon\\ \implies A - \epsilon & < b_n < A + \epsilon \end{aligned} $$
And the last inequality holds for $\N(\epsilon)$ and is equivalent to:
$$\left| b_n - A \right| < \epsilon$$
2.4.1: Squeeze theorem for functions
$\lim_{x \to x_0} \g(x) = A = \lim_{x \to x_0} \h(x)$ and there $\exists\ \delta_0(\epsilon):\quad \g(x) \le \f(x) \le \h(x)$ for $\forall\ x \in \left] x_0 - \delta_0,\ x_0 + \delta_0 \right[$
Since $\lim_{x \to x_0} \g(x) = A$ for $\forall\ \epsilon > 0$ there $\exists\ \delta_g(\epsilon):\quad \left| \g(x) - A \right| < \epsilon$ for $\forall\ x \in \left] x_0 - \delta_g,\ x_0 + \delta_g \right[$
Also since $\lim_{x \to x_0} \h(x) = A$ for $\forall\ \epsilon > 0$ there $\exists\ \delta_h(\epsilon):\quad \left| \h(x) - A \right| < \epsilon$ for $\forall\ x \in \left] x_0 - \delta_h,\ x_0 + \delta_h \right[$
So then for $\forall\ x \in \left] x_0 - \delta,\ x_0 + \delta \right[,\ \delta(\epsilon) = \min(\delta_g(\epsilon),\ \delta_h(\epsilon),\ \delta_0)$ it holds:
$$ \begin{aligned} A - \epsilon & < \g(x) < A + \epsilon\\ A - \epsilon & < \h(x) < A + \epsilon\\ \g(x) & \le \f(x) \le \h(x)\\ \implies A - \epsilon < \g(x) & \le \f(x) \le \h(x) < A + \epsilon\\ \implies A - \epsilon & < \f(x) < A + \epsilon \end{aligned} $$
And the last inequality holds for $\delta(\epsilon)$ and is equivalent to:
$$\left| \f(x) - A \right| < \epsilon$$
2.5: Dedekind Completeness Theorem
If a nonempty set is bounded from above or below (by $K$) then it has a least upper bound or greatest lower bound.
This implies that a bounded sequence should also have an infumum or suprenum. I do not prove this one.
2.6: $\forall$ sequence has a monotonic subsequence (Where by subsequence I mean finite in case of finite and infinite in case of infinite sequence.)
Let $a_{n_0}$ be a peak element in the sequence. Then for $\forall\ n > n_0\quad a_n \le a_{n_0}$
Let's assume the sequence is infinite, then:
- If we assume our sequence to have infinite peak elements then these elements construct a monotonically decreasing sequence and the theorem is proven
- If we assume finite number of peak elements (or none) then let $a_{s_1}$ be the element just after the last peak element.
In this case, for $\forall\ s_2 > s_1$ it must hold that $a_{s_2} \le a_{s_1}$ otherwise $a_{s_1}$ would be a peak point.
But then also, for $\forall x_3 > x_2$ it must hold that $a_{s_3} \le a_{s_2}$ otherwise $a_{s_2}$ would be a peak point.
But then these $a_{s_1},\ a_{s_2},\ ....$ elements constitute a monotonically increasing subsequence, and hence the theorem is proven.
2.6.1: $\forall$ function has a monotonic subsequence
Let $\f(x_0)$ be a peak element in the sequence. Then for $\forall\ x > x_0\quad \f(x) \le \f(x_0)$
- If we assume our sequence to have infinite peak elements then these elements construct a monotonically decreasing sequence and the theorem is proven
- If we assume finite number of peak elements (or none) then let $\f(x_1),\ x \in D_f \cap \left] x_0,\ \infty \right[$ be an element after the last peak element.
In this case, for $\forall\ x_2 > x_1$ it must hold that $\f(x_2) \le \f(x_1)$ otherwise $\f(x_1)$ would be a peak point.
But then also, for $\forall s_3 > s_2$ it must hold that $\f(x_3) \le \f(x_2)$ otherwise $\f(x_2)$ would be a peak point.
But then these $\f(x_1),\ \f(x_2),\ ....$ elements constitute a monotonically increasing subsequence, and hence the theorem is proven.
2.7: A monotonically increasing/decreasing sequence which is bounded above/below is convergent
Proof: let sequence $a_n$ be monotonically increasing and bounded from above by a constant $K_0$, the opposite statement is analogous to this one. Now since our sequence is bounded from above, it must have a least upper bound $K$ as well.
Then by the definition of the least upper bound, for $\forall\ \epsilon > 0$ there $\exists\ n_0:\quad K - \epsilon < a_{n_0} \le K$
(otherwise, by common sense, the upper bound is too large and is NOT least upper bound).
But then you see, since our sequence is monotonically increasing:
For $\forall\ n \ge n_0\quad K - \epsilon < a_n <= K$ holds and therefore $\left| a_n - K \right| < \epsilon$
2.7.1: A monotonically increasing/decreasing function which is bounded above/below is convergent
Proof: let sequence $\f(x)$ be monotonically increasing and bounded from above by a constant $K_0$, the opposite statement is analogous to this one. Now since our function is bounded from above, it must have a least upper bound $K$ as well.
Then by the definition of the least upper bound, $\forall\ \epsilon > 0$ there $\exists\ x_0 \in \mathbb{N}$ so that:
$K - \epsilon < \f(x_0) \le K$
(otherwise, by common sense, the upper bound is too large and is NOT least upper bound).
But then you see, since our function is monotonically increasing:
$$K - \epsilon < \f(x) <= K$$
For $\forall\ x \ge x_0$. Which is equivalent to:
$$\left| \f(x) - K \right| < \epsilon$$
2.8: Bolzano-Weierstrass Theorem: Every bounded sequence has a convergent subsequence
Now by theorem 2.6 the sequence must have a monotonic subsequence.
And then by theorem 2.7 this subsequence must be convergent since it is also bounded.
2.8.1: Every bounded function has convergent subsequence (Bolzano-Weierstrass Theorem for functions)
Now by theorem 2.6.1 the sequence must have a monotonic subsequence.
And then by theorem 2.7.1 this subsequence must be convergent since it is also bounded.
2.9: A convergent sequence is bounded
Since $a_n$ is convergent for $\forall\ \epsilon > 0$ there $\exists\ \N(\epsilon):\quad \left| a_n - A \right| < \epsilon$ for $\forall\ n \ge \N$
From that it follows every term beyond this index is bounded such that:
$$A - \epsilon < a_n < A + \epsilon$$
Since we have finitely many terms below this index, there must be a maximum and minimum element among them which bounds the sequence.
2.9.1: If a function is convergent at $x_0$, then there exists a neighbourhood of $x_0$ in which $\f$ is bounded.
Indirect proof:
Let's assume whatever neighbourhood of $x_0$ we choose the function is not bounded, while we still assume convergence at $x_0$.
Due to convergence at $x_0$, for $\forall\ \epsilon > 0$ there $\exists\ \delta(\epsilon):\quad | \f(x) - A | < \epsilon$ for $\forall\ x:\quad | x - x_0 | < \delta$
We can look at any $\delta_0(\epsilon_0)$ neighbourhood of $x_0$. Then it means that here $\f$ is not bounded but still $| \f(x) - A | < \epsilon_0$. That is of course contradiction. Whichever $\f(x_1)$ we look we should always find an $\f(x_2):\quad \f(x_2) > \f(x_1)$ where $x_1,\ x_2 \in ] x_0 - \delta_0,\ x_0 + \delta_0 [$. We can therefore choose a function value arbitrarily large, so must also be able to find $\f(x_3):\quad | \f(x_3) - A | \ge \epsilon_0$
Therefore if the functino is convergent at x_0, there must $\exists$ a neighbourhood of $x_0$ in which teh function is bounded.
3.0: Definitions 1.2 and 1.4 are equivalent:
Proof for $1.2 \impliedby 1.4$:
Assumed: $\forall\ \epsilon > 0 $ there $\exists\ \N(\epsilon):\quad \left| a_n - A \right| < \epsilon$ for $\forall\ n \ge \N $
So that for $\forall\ k,\ l \ge \N:$
$$ \begin{aligned} & \left| a_l - a_k \right| = \left| a_l - A - (a_k - A) \right| =\\ & = \left| (A - a_k) + (a_l - A) \right| \end{aligned} $$
Due to 1.1 (Tringle Equality) it follows:
$ \left| (A - a_k) + (a_l - A) \right| \le \left| A - a_k \right| + \left| a_l - A \right| < 2\epsilon$
Which then means for $\N_c(\epsilon) = \N(\frac{\epsilon}{2}):$
$$ \left| a_l - a_k \right| < \epsilon $$
for $\forall\ n \ge \N_c(\epsilon)$
Proof for $1.2 \implies 1.4$:
Convergence of a Cauchy sequence can be proven using either the Bolzano-Weierstrass Theorem (2.8.1), or the Cantor Axiom
Proof using 2.8 Bolzano-Weierstrass Theorem:
Since by 1.3 a Cauchy sequence is always bounded, every Cauchy sequence has a convergent subsequence according to 2.0. We wish to show that the Cauchy sequence converges to the limit of its convergent subsequence.
Since the subsequence converges to some $A \in \mathbb{R}$, for $\forall\ \epsilon > 0$ there must $\exists\ a_{\N_0(\epsilon)}:\quad \left| a_{\N_0} - A \right| < \epsilon$
But then there $\exists\ \N_1:\quad \left| a_{\N_0} - a_m \right| < \epsilon$ for $\forall\ m \ge \N_1$
But then if $\N_2 = \max(\N_0, \N_1),\ \left| a_m - A \right| < 2\epsilon$ for $\forall\ m \ge \N_2$
So that for $\N_3(\epsilon) = \max(\N_0(\frac{\epsilon}{2}),\ \N_1(\frac{\epsilon}{2})):$
$$\left| a_n - A \right| < \epsilon$$
For $\forall\ n \ge \N_3$
3.0.1: Definitions 1.2.1 and 1.4.1 are equivalent
Proof for $1.2.1 \impliedby 1.4.1$:
Assumed: $\forall\ \epsilon > 0 $ there $\exists\ \delta(\epsilon):\quad | \f(x) - A | < \epsilon$ for $\forall\ x:\quad |x - x_0| < \delta $
So that for $\forall\ x_2,\ x_1:\quad | x_1 - x_0 | < \delta,\ | x_2 - x_0 | < \delta :$
$$ \begin{aligned} & | \f(x_2) - \f(x_1) | = | \f(x_2) - A - (\f(x_1) - A) | =\\ & = | (A - \f(x_1)) + (\f(x_2) - A) | \end{aligned} $$
Due to 1.1 (Tringle Equality) it follows:
$$ | (A - \f(x_1)) + (\f(x_2) - A) | \le | A - \f(x_1) | + | \f(x_2) - A | < 2\epsilon$$
Which then means for $\delta_0(\epsilon) = \delta(\frac{\epsilon}{2}):$
$$ | \f(x_2) - \f(x_1) | < \epsilon $$
for $\forall\ x:\quad | x - x_0 | < \delta_0$
Proof for $1.2 \implies 1.4$:
I am using the Bolzano-Weierstrass theorem (2.1) for functions to prove the reverse. We assume Cauchy criterion for $x_0$. Let's consider only the the left or right neighbourhood of $x_0$. We know that $\forall$ function has a monotonic subsequence. Since the left and right hand sides are bounded functions, they also have convergent subsequences. And there is ought to be a bounded portion of the function in a neighbourhood of $x_0$ othewrise if at every neighbourhood of $x_0$ the function would be unbounded then the Cauchy criterion would not hold. Therefore there must exist a neghibourhood of $x_0$ so that the function on the left and right is bounded. And also, by common sense, the function should be bounded on the left and right for any possible neighbourhood in which we assume the Cauchy criterion.
Let $a_{n_k}$ be the convergent subsequence of the left hand side function. So for $\forall\ \epsilon > 0$ there $\exists\ \N_0(\epsilon):\quad | a_{n_k - A} | < \epsilon$ for $\forall\ k > \N_0$. And also there $\exists\ \delta_0(\N_0)$ neighbourhood of $x_0$ in which every such $a_{n_k}$ sequence terms reside.
Then because of the Cauchy criterion, we have that for $\forall\ \epsilon > 0$ there $\exists\ \delta_1(\epsilon):\quad | \f(x_1) - \f(x_2) | < \epsilon$ for $\forall\ x:\quad | x - x_0 | < \delta_1$
So for $\delta_2(\epsilon) = \min(\delta_0(\epsilon), \delta_1(\epsilon)):$
$$| a_{n_k} - A | + | \f(x) - a_{n_k} | < 2\epsilon$$
Due to the triangle inequality (1.1):
$$| \f(x) - A | = | a_{n_k} - A + \f(x) - a_{n_k} | \le | a_{n_k} - A | + | \f(x) - a_{n_k} | < 2\epsilon$$
Therefore for $\delta_3(\epsilon) = \delta_2(\frac{\epsilon}{2}):$
$$| \f(x) - A | < \epsilon$$
What I have essentially shown is the left hand side converges to $A$. We can also show that the right hand side also converges somewhere. And by common sense, the right and left had sides of the function shall converge to the value otherwise it would not satisfy the Cauchy criterion. But if the left and right hand side limits of teh function tend to the same value then the function is convergent at that point and is either continuous there, or have at most a removable discontinuity there.
There's also an easier way to show this result. We can also use the Cantor Axiom. Let's consider a $\delta_0(\epsilon_0)$ neighbourhood of $x_0$. We presume any $| \f(x_1) - \f(x_2) | < \epsilon_0$ for $\forall\ x_0,\ x_1: \in ] x_0 - \delta_0,\ x_0 + \delta_0 [$
Now let $M = \max \{\f(x) | x \in ] x_0 - \delta_0,\ x_0 + \delta_0 [ \}$
And let $m = \min \{\f(x) | x \in ] x_0 - \delta_0,\ x_0 + \delta_0 [ \}$
So then $\f(x) \in [m,\ M]$ for $\forall\ x:\quad | x - x_0 | < \delta_0$ because if any function value would fall otuside this range, then the Cauchy criterion would not hold. (This element would not be in the $\epsilon_0$ neighbourhood of the minimum or maximum element).
That means the minimum and maximum function values inside a smaller $\delta_1 < \delta_0$ neighbourhood of $x_0$ would also fall in this larger interval. This new interval must also be smaller, otherwise the Cauchy criterion would not hold. Because between these extreme values the distance is limited by a smaller $\epsilon_1 < \epsilon_0$. Therefore we are nesting ever strictly decreasing bounded and closed function value intervals into each other. By the Cantor axiom (1.0) the union of these intervals is nonempty and is a single point. Let this single point be $A$.
I wish to show the function converges to $A$. Then looking back, for any given $\delta$ neighbourhood of $x_0$, the absolute difference between the minimum and maximum function value should not reach $\epsilon$. That means the difference of any function value in this interval should not reach $2\epsilon$.
$$| \f(x) - A | < 2\epsilon$$
For $\forall\ x:\quad | x - x_0 | < \delta$
Therefore $\delta_f(\epsilon) = \delta(\frac{\epsilon}{2})$ is sufficient for:
$$| \f(x) - A | < \epsilon$$
3.1: First Theorem of Weierstrass
A function which is continuous over a closed and bounded interval is bounded over this interval.
Proof (indirect): We assume that there $\nexists\ K > a_n$ for $\forall\ n$. Then:
There $\exists\ x_1 \in \left[ a,\ b \right]$ so that $\f(x_1) > 1$
And there $\exists\ x_2 \in \left[ a,\ b \right]$ so that $\f(x_2) > 2$
....
And there $\exists\ x_n \in \left[ a,\ b \right]$ so that $\f(x_n) > n$
....
Sequence $x_n$ is bounded since $\forall\ x_n \in \left[ a,\ b \right]$
By 2.8 (Bolzano-Weierstrass theorem) $x_n$ must have a convergent subsequence $x_{n_i} \to x_0$
Since $a \le x_{n_i} \le b$ it must hold that $a \le x_0 \le b$ so $x_{n_i} \to x_0 \in \left[ a,\ b \right]$
We assumed that $\f(x_{n_i}) \to \infty$ but it can not be true since $\f$ is continuous at $x_0$ $\f(x_{n_i}) \to \f(x_0)$
3.2: Second theorem of Weierstrass (Extreme value theorem)
if $\f$ is continuous over the closed and bounded interval $\left[ a,\ b\right]$ then there must $\exists\ \alpha$ such that $\f(\alpha) = \sup \{\f(x) : x \in \left[a,\ b\right]\}$ and there must $\exists\ \beta$ such that $\f(\beta) = \inf \{\f(x) : x \in \left[a,\ b\right]\}$
Proof:
Let set $A$ denote the range of function $\f$.
Then due to 3.1 $A$ is bounded.
And since it is bounded, due to Dedekind completeness 2.5
there $\exists\ \sup A = M$ and there $\exists\ \inf A = m$
We wish to show that there $\exists\ \alpha \in \left[a,\ b\right]$ so that $\f(\alpha) = M$ and
there $\exists\ \beta \in \left[a,\ b\right]$ so that $\f(\beta) = m$
Indirect proof:
Let's assume there $\nexists\ \alpha \in \left[a,\ b\right]$ for which $\f(\alpha) = M$
From this it follows that $M - \f(x) > 0$ for $x \in \left[a,\ b\right]$
$\implies \g(x) = \frac{1}{M - \f(x)}$ is a continuous function over \left[a,\ b\right].
So then by 3.1 $\g$ is bounded over $\left[a,\ b\right]$ so there $\exists\ K$ for which:
$\frac{1}{M - \f(x)} < K$ for $\forall\ x \in \left[a,\ b\right]$
$$ \begin{aligned} \implies M - \f(x) > \frac{1}{K}\\ \implies \f(x) < M - \frac{1}{K} < M \end{aligned} $$
But then the last inequlity would mean that $M - \frac{1}{K}$ is an upper bound which is trivially smaller than $M$. But since we assumed $M$ to be least upper bound, it must be a contradiction.
3.3: Heine definition of limit, and its equivalence with convergence
$\lim{x \to x_0} \f(x) = A \iff$ for $\forall\ x_n \to x_0\quad \f(x_n) \to A$ where $x_n \in D_f$ and $x_n \neq x_0$
Proof for $\implies$: We know that $\lim_{x \to x_0} \f(x) = A$, and we wish to show that $\f(x_n) \to A$ for $\forall\ x_n:\quad x_n \to x_0,\ x_n \in D_f,\ x_n \neq x_0$
Since $\lim_{x \to x_0} \f(x) = A$, for $\forall\ \epsilon > 0$ there $\exists\ \delta(\epsilon):\quad \left| \f(x) - A \right| < \epsilon$ for $\forall\ x:\quad \left| x - x_0 \right|$
In case of any $x_n \to x_0$ for $\forall\ \xi > 0$ there $\exists\ \N(\xi):\quad \left| x_n - x_0 \right| < \xi$ for $\forall\ n > \N$
Therefore in case of $\N(\delta(\epsilon)) < n$:
$$ \begin{aligned} \left| x_n - x_0 \right| < \delta(\epsilon)\\ \implies \left| \f(x_n) - A \right| < \epsilon \end{aligned} $$
Proof for $\impliedby$: We know that $\f(x_n) \to A$ for $\forall\ x_n:\quad x_n \to x_0,\ x_n \in D_f,\ x_n \neq x_0$, and wish to show that $\lim_{x \to x_0} \f(x) = A$
This is an indirect proof. We wish to show that despite the conditions there $\exists\ \epsilon_0 > 0$ for which $\nexists\ \delta(\epsilon)$. Let $\delta_0$ fixed and we wish to show that like any other value, this one is also insufficient. This means $\exists\ x_n:\quad \left| x_n - x_0 \right| < \delta_0$ but $\left| \f(x_n) - A \right| \ge \epsilon_0$, implying that this way we can always be bale to find a sequence for which our function does not converge, but it is contradictory to our initial assumption.
3.4: Bolzano theorem
If $\f(x)$ is continuous over $\left[ a,\ b \right]$ then for $\forall\ c \in \left[ a\ b \right]$ there $\exists\ x_0:\quad \f(x_0) = c$
Proof:
Let $c \in \mathbb(R):\quad \f(a) < c < \f(b)$ so wish to show that there $\exists\ \xi:\quad \f(\xi) = c$
Let $a_1 = a$ and $b_1 = b$. Then let $m = \frac{a_1 + b_1}{2}$
If $\f(m) = c$ then we finished the proof
If $\f(m) > c$ then let $a_2 = m$ and $b_2 = b_1$
if $\f(m) < c$ then let $a_2 = a_1$ and $b_2 = m$
We want to show that the process somewhere ends with the result. Please note that we are nesting bounded and closed intervals into each other, and the size of a subsequent interval is always smaller, tends to zero. By the Cantor Axiom (1.0), the intersection of infinitely many such nested intervals is a nonempty set and is a single point, let this point be denoted by $\xi$
Since $a_n \le \xi \le b_n:$
$$ \begin{aligned} 0 \le \left| a_n - \xi \right| \le \left| a_n - b_n \right|$\\ 0 \le \left| b_n - \xi \right| \le \left| a_n - b_n \right|$ \end{aligned} $$
We know that $\left| a_n - b_n \right| \to 0$, and so by the squeeze theorem for sequences (2.4) $\left| a_n - \xi \right| \to 0$ and therefore $a_n \to \xi$, and similarly $b_n \to \xi$
Because $\f(x)$ is continuous at $x_0$ and because of the Heine definition of limit:
$$\lim_{n \to \infty} \f(a_n) = \lim_{n \to \infty} \f(b_n) = \f(\xi)$$
But most importantly:
$$ \begin{aligned} \f(a_n) < c\\ \f(b_n) > c\\ \implies \lim_{n \to \infty} \f(a_n) \le c\\ \implies \lim_{n \to \infty} \f(b_n) \ge c\\ \implies \f(\xi) \le c\\ \implies \f(\xi) \ge c\\ \implies \f(\xi) = c \end{aligned} $$
3.5:
$a_n \to \infty$ then $b_n = \frac{1}{a_n} \to 0$
Since $a_n \to \infty$, for $\forall\ P > 0$ there $\exists\ \N_a(P):\quad a_n > P$ for $\forall\ n > \N_a$
From this it follows that $0 < \frac{1}{a_n} < \frac{1}{P}$. So for this to hold $\N_b(\epsilon) = \N_a(\frac{1}{\epsilon})$ is sufficient.
3.5.1:
$\lim_{x \to x_0} \f(x) = \infty$ then $\g(x) = \frac{1}{\f(x)} \to 0$
Since $\f(x) \to \infty$, for $\forall\ P > 0$ there $\exists\ \delta_f(P):\quad \f(x) > P$ for $\forall\ x:\quad \left| x - x_0 \right| < \delta_f$
From this it follows that $0 < \frac{1}{\f(x)} < \frac{1}{P}$. So for this to hold $\delta_g(\epsilon) = \delta_f(\frac{1}{\epsilon})$ is sufficient.
3.6:
$a_n \to A \in \mathbb{R}$ and $b_n \to B \in \mathbb{R}$, then $c_n = a_n + b_n \to (A + B)$
Proof 1:
$$\left| a_n + b_n - (A + B) \right| = \left| (a_n - A) + (b_n - B) \right|$$
Now since $a_n \to A$, for $\forall\ \epsilon > 0$ there $\exists\ \N_a(\epsilon):\quad \left| a_n - A \right| < \epsilon$ for $\forall\ n \ge \N_a(\epsilon)$
And since $b_n \to B$, for $\forall\ \epsilon > 0$ there $\exists\ \N_b(\epsilon):\quad \left| b_n - B \right| < \epsilon$ for $\forall\ n \ge \N_b(\epsilon)$
Now let $\N_d(\epsilon) = \max(\N_a(\epsilon),\ \N_b(\epsilon))$ and let $n \ge \N_d(\epsilon)$
Using the triangle inequality (1.1) we get:
$$\left| (a_n - A) + (b_n - B) \right| \le \left| a_n - A \right| + \left| b_n - B \right| < 2 \epsilon$$
So $\N_c(\epsilon) = \N_d(\frac{\epsilon}{2})$ is sufficient.
Proof 2:
Since $a_n \to A$, for $\forall\ \epsilon > 0$ there $\exists\ \N_a(\epsilon):\quad \left| a_n - A \right| < \epsilon$ for $\forall\ n \ge \N_a(\epsilon)$
And since $b_n \to B$, for $\forall\ \epsilon > 0$ there $\exists\ \N_b(\epsilon):\quad \left| b_n - B \right| < \epsilon$ for $\forall\ n \ge \N_b(\epsilon)$
Now let $\N_d(\epsilon) = \max(\N_a(\epsilon),\ \N_b(\epsilon))$ and let $n \ge \N_d(\epsilon)$
Then:
$$ \begin{aligned} A - \epsilon & < a_n < A + \epsilon\\ B - \epsilon & < b_n < B + \epsilon\\ \implies (A + B) - 2\epsilon & < a_n + b_n < (A + B) + 2\epsilon\\ \implies \left| a_n + b_n \right| & < 2\epsilon\\ \implies \N_c(\epsilon) & = \N_d(\frac{\epsilon}{2}) \end{aligned}$$
3.6.1:
$\lim_{x \to x_0} \g(x) = A \in \mathbb{R}$ and $\lim_{x \to x_0} \h(x) = B \in \mathbb{R}$, then $\lim_{x \to x_0} g(x) + h(x) = \lim_{x \to x_0} \f(x) = (A + B)$
Proof 1:
$$ \begin{aligned} & \left| \g(x) + \h(x) - (A + B) \right|\\ & = \left| (\g(x) - A) + (\h(x) - B) \right| \end{aligned} $$
For $\forall\ \epsilon > 0$ there $\exists\ \delta_g(\epsilon):\quad \left| \g(x) - A \right| < \epsilon$ for $\forall\ x:\quad \left| x_0 - x \right| <= \delta_g$
For $\forall\ \epsilon > 0$ there $\exists\ \delta_h(\epsilon):\quad \left| \h(x) - B \right| < \epsilon$ for $\forall\ x:\quad \left| x_0 - x \right| <= \delta_h$
Now let $\delta_d(\epsilon) = \min(\delta_g(\epsilon),\ \delta_h(\epsilon))$
Using the triangle inequality (1.1) we get:
$\left| (a_n - A) + (b_n - B) \right| \le \left| a_n - A \right| + \left| b_n - B \right| < 2 \epsilon$
For $\forall\ x:\quad \left| x_0 - x \right| <= \delta_d$
So $\delta_f(\epsilon) = \delta_d(\frac{\epsilon}{2})$ is sufficient.
Proof 2:
For $\forall\ \epsilon > 0$ there $\exists\ \delta_g(\epsilon):\quad \left| \g(x) - A \right| < \epsilon$ for $\forall\ x:\quad \left| x_0 - x \right| <= \delta_g$
For $\forall\ \epsilon > 0$ there $\exists\ \delta_h(\epsilon):\quad \left| \h(x) - B \right| < \epsilon$ for $\forall\ x:\quad \left| x_0 - x \right| <= \delta_h$
For $\forall\ x:\quad \left| x - x_0 \right| < \delta_d(\epsilon) = \min(\delta_g(\epsilon),\ \delta_h(\epsilon)):$
$$ \begin{aligned} A - \epsilon & < \g(x) < A + \epsilon\\ B - \epsilon & < \h(x) < B + \epsilon\\ \implies (A + B) - 2\epsilon & < \f(x) < (A + B) + 2\epsilon\\ \implies \left| \f(x) \right| & < 2\epsilon\\ \implies \delta_f(\epsilon) & = \delta_f(\frac{\epsilon}{2}) \end{aligned}$$
3.7:
If $a_n \to A \in \mathbb{R}$ then $\left| a_n \right| \to \left| A \right|$
Proof:
Due to the reverse triangle inequality (1.1.1):
$\left| \left| a_n \right| - \left| A \right| \right| \le \left| a_n - A \right| < \epsilon$ for $\forall\ n \le \N(\epsilon)$
3.7.1:
If $\lim_{x \to x_0} \f(x) \to A \in \mathbb{R}$ then $\lim_{x \to x_0} \left| \f(x) \right| = \left| A \right|$
Proof:
Due to the reverse triangle inequality (1.1.1):
$$\left| \left| \f(x) \right| - \left| A \right| \right| \le \left| \f(x) - A \right| < \epsilon$ for $\forall\ x:\quad \left| x - x_0 \right| < \delta(\epsilon)$$
3.8:
$a_n \to A \in \mathbb{R} \setminus \{ 0 \}$, then $b_n = \frac{1}{a_n} \to \frac{1}{A}$
Proof 1:
Since $a_n \to A$, for $\forall\ \epsilon > 0$ there $\exists\ \N(\epsilon):\quad \left| a_n - A \right| < \epsilon$ for $\forall\ n \le \N(\epsilon)$
$$ \begin{aligned} A - \epsilon < a_n < A + \epsilon\\ \implies \frac{1}{A - \epsilon} > \frac{1}{a_n} > \frac{1}{A + \epsilon}\\ \end{aligned} $$
We can clearly see that if $A \neq 0$ then $\lim_{\epsilon \to 0} \frac{1}{A \pm \epsilon} = \frac{1}{A}$
Let $\g(\epsilon) = \frac{1}{A - \epsilon}$ and $\h(\epsilon) = \frac{1}{A + \epsilon}$
Then by definition of convergence of smooth functions (1.4.1):
For $\forall\ \xi > 0$ there $\exists\ \delta_0(\xi) > 0:\quad \left| \g(\epsilon) - \frac{1}{A} \right| < \xi$ for $\forall\ \epsilon:\quad \left| \epsilon - 0 \right| = \left| \epsilon \right| < \delta_0$
And for $\forall\ \epsilon > 0$ there $\exists\ \delta_1(\xi):\quad \left| \h(\epsilon) - \frac{1}{A} \right| < \xi$ for $\forall\ \epsilon:\quad \left| \epsilon \right| < \delta_1$
So then if $\delta_2(\xi) = \min(\delta_0(\xi),\ \delta_1(\xi))$, then:
$$ \begin{aligned} \frac{1}{A} - \xi & < \g(\epsilon) < \frac{1}{A} + \xi\\ \frac{1}{A} - \xi & < \h(\epsilon) < \frac{1}{A} + \xi \end{aligned} $$
Now let $\eps_0(\delta) = \frac{\delta}{2}$ so that $\eps_0 < \delta(\xi)$ holds and so that it will be adequate for input $\epsilon$ value for $\N(\epsilon)$ so that for $\forall\ n \ge \N(\eps_0(\delta(\xi))):$
$$ \begin{aligned} \g(\epsilon) > \frac{1}{a_n} > \h(\epsilon) \end{aligned} $$
Combining it all into one:
$$ \begin{aligned} \frac{1}{A} + \xi > \g(\epsilon) > \frac{1}{a_n} > \h(\epsilon) > \frac{1}{A} - \xi \end{aligned} $$
And so the important takeaway is:
$$ \begin{aligned} \frac{1}{A} + \xi > \frac{1}{a_n} > \frac{1}{A} - \xi \end{aligned} $$
For $\forall\ n \ge \N(\eps_0(\delta(\xi)))$
Proof 2:
Due to 3.7 $\left| a_n \right| \to \left| A \right|$. Therefore there $\exists\ \N_1(\frac{\left| A \right|}{2}):$
$$ \left| \left| a_n \right| - \left| A \right| \right| < \frac{\left| A \right|}{2}$$
For $\forall\ n \ge \N_1$
Which is equivalent to:
$$ \begin{aligned} \left| A \right| - \frac{\left| A \right|}{2} & < \left| \f(x) \right| < \left| A \right| + \frac{\left| A \right|}{2}\\ \implies - \frac{\left| A \right|}{2} & < \left| \f(x) \right| - \left| A \right| < \frac{\left| A \right|}{2}\\ \implies \left| \f(x) \right| & > \frac{\left| A \right|}{2} \end{aligned} $$
Also, for $\forall\ \epsilon > 0$ there $\exists\ \N_2(\frac{\epsilon}{2}{\left| A \right|}^2) = \N_2(\epsilon^*):$
$$\left| a_n - A \right| < \frac{\epsilon}{2}{\left| A \right|}^2 = \epsilon^*$$
For $\forall\ n \ge \N_2$. Now let's write up:
$$\left| \frac{1}{a_n} - \frac{1}{A} \right| = \frac{\left| A - a_n \right|}{\left| A\cdot a_n \right|} = \frac{\left| A - a_n \right|}{\left| A \right| \cdot \left| a_n \right|}$$
Now let $N(\epsilon) = \max(N_1(\frac{\left| A \right|}{2}),\ N_2(\frac{\epsilon}{2}{\left| A \right|}^2)):$
$$\frac{\left| A - a_n \right|}{\left| A \right| \cdot \left| a_n \right|} < \frac{\left| A - a_n \right|}{\left| A \right|\cdot \frac{\left| A \right|}{2}} < \frac{\frac{\epsilon}{2}{\left| A \right|}^2}{\left| A \right| \cdot \frac{\left| A \right|}{2}} = \epsilon$$
For $\forall\ n \ge N(\epsilon)$. So we can make the difference of $\frac{1}{A}$ with respect to $\frac{1}{A}$ arbitrarily small beyond a certain index thresold.
3.8.1:
$\lim_{x \to x_0} \f = A \in \mathbb{R} \setminus \{ 0 \}$, then $\k(x) = \frac{1}{\f(x)} = \frac{1}{A}$
Proof 1:
Since $\lim_{x \to x_0} \f = A$ for $\forall\ \epsilon > 0$ there $\exists\ \delta_f(\epsilon):$
$$ \begin{aligned} A + \epsilon > \f(x) > A - \epsilon\\ \implies \frac{1}{A + \epsilon} < \frac{1}{\f(x)} < \frac{1}{A - \epsilon} \end{aligned} $$
For $\forall\ x:\quad \left| x - x_0 \right| < \delta_f(\epsilon)$. Now let $\eps_0(x) = \left| x - x_0 \right|$ so that:
$$ \frac{1}{A + \left| x - x_0 \right|} < \frac{1}{\f(x)} < \frac{1}{A - \left| x - x_0 \right|} $$
for $\forall\ x:\quad \left| x - x_0 \right| < \delta_f(\eps_0(x))$.
Let $\g(x) = \frac{1}{A - \left| x - x_0 \right|}$ and $\h(x) = \frac{1}{A + \left| x - x_0 \right|}$
We can clearly see that if $A \neq 0$ then $\lim_{x\to 0} \frac{1}{A \pm x} = \frac{1}{A}$
By the squeeze theorem for functions 2.4.1, $\lim_{x \to x_0} \frac{1}{\f(x)} = \frac{1}{A}$
Proof 2:
Due to 3.7.1 $\left| \f(x) \right| \to \left| A \right|$. Therefore there $\exists\ \delta_1(\frac{\left| A \right|}{2}):$
$$ \left| \left| a_n \right| - \left| A \right| \right| < \frac{\left| A \right|}{2}$$
$\forall\ x:\quad \left| x - x_0 \right| < \delta_1$. This is equivalent to:
$$ \begin{aligned} \left| A \right| - \frac{\left| A \right|}{2} & < \left| a_n \right| < \left| A \right| + \frac{\left| A \right|}{2}\\ \implies - \frac{\left| A \right|}{2} & < \left| \f(x) \right| - \left| A \right| < \frac{\left| A \right|}{2}\\ \implies \left| \f(x) \right| & > \frac{\left| A \right|}{2} \end{aligned} $$
Also, for $\forall\ \epsilon > 0$ there $\exists\ \delta_2(\frac{\epsilon}{2}{\left| A \right|}^2) = \delta_2(\epsilon^*):$
$$\left| a_n - A \right| < \frac{\epsilon}{2}{\left| A \right|}^2 = \epsilon^*$$
For $\forall\ x:\quad \left| x - x_0 \right| < \delta_2$. Now let's write up:
$$\left| \frac{1}{\f(x)} - \frac{1}{A} \right| = \frac{\left| A - \f(x) \right|}{\left| A\cdot \f(x) \right|} = \frac{\left| A - \f(x) \right|}{\left| A \right| \cdot \left| \f(x) \right|}$$
Now let $\delta(\epsilon) = \min(\delta_1(\frac{\left| A \right|}{2}),\ \delta_2(\frac{\epsilon}{2}{\left| A \right|}^2)):$
$$\frac{\left| A - a_n \right|}{\left| A \right| \cdot \left| a_n \right|} < \frac{\left| A - a_n \right|}{\left| A \right|\cdot \frac{\left| A \right|}{2}} < \frac{\frac{\epsilon}{2}{\left| A \right|}^2}{\left| A \right| \cdot \frac{\left| A \right|}{2}} = \epsilon$$
for $\forall\ x:\quad \left| x - x_0 \right| < \delta$. So we can make the difference of $\frac{1}{A}$ with respect to $\frac{1}{A}$ arbitrarily small beyond a certain index thresold.
3.9: $a_n \to A$ then $c_n = c \cdot a_n \to A \cdot c$ and $c \in \mathbb{R}$
Proof 1:
Since $a_n \to A$ for $\forall\ \epsilon > 0$ there $\exists\ \N_a(\epsilon)$ so that for $\forall\ n \ge \N_a:$
$$ \begin{aligned} A - \epsilon & < a_n < A + \epsilon\\ \implies c(A - \epsilon) & < ca_n < c(A + \epsilon)\\ \implies Ac - \epsilon c & < a_n < Ac + \epsilon c \end{aligned} $$
Which is equivalent to:
$$\left| ca_n - Ac \right| < \epsilon c$$
So then $\N_c(\epsilon) = \N(\frac{\epsilon}{c})$ will be sufficient for the above to hold true for given $\epsilon$
Proof 2:
$$\left| ca_n - cA \right| = \left| c(a_n - A) \right| = \left| c \right| \left| a_n - A \right|$$
Now given that for $\forall\ \epsilon > 0$ there $\exists\ N_a(\epsilon):$
$\left| c \right| \left| a_n - A \right| < \left| c \right|\epsilon$
For $\forall\ n \ge \N_a$
Therefore $\N_c(\epsilon) = \N_a(\frac{\epsilon}{\left| c \right|})$ is sufficient for this to hold.
3.9.1: $\lim_{x \to x_0} \f(x) = A$ then $\lim_{x \to x_0} \g(x) = c \cdot \f(x) = A \cdot c$ and $c \in \mathbb{R}$
Proof 1:
Since $\lim_{x \to x_0} \f(x) = A$ for $\forall\ \epsilon > 0$ there $\exists\ \delta_f(\epsilon):$
$$ \begin{aligned} A - \epsilon & < a_n < A + \epsilon\\ \implies c(A - \epsilon) & < ca_n < c(A + \epsilon)\\ \implies Ac - \epsilon c & < a_n < Ac + \epsilon c \end{aligned} $$
For $\forall\ x:\quad \left| x - x_0 \right| < \delta_f$. This is equivalent to:
$$\left| c\f(x) - Ac \right| < \epsilon c$$
So then $\delta_g(\epsilon) = \delta_f(\frac{\epsilon}{c})$ will be sufficient for the above to hold true for given $\epsilon$
Proof 2:
$$\left| c\f(x) - cA \right| = \left| c(\f(x) - A) \right| = \left| c \right| \left| \f(x) - A \right|$$
Now given that for $\forall\ \epsilon > 0$ there $\exists\ \delta_f(\epsilon):$
$$\left| c \right| \left| a_n - A \right| < \left| c \right|\epsilon$$
For $\forall\ x:\quad \left| x - x_0 \right| < \delta_f$
Therefore $\delta_g(\epsilon) = \delta_f(\frac{\epsilon}{\left| c \right|})$ is sufficient for this to hold true.
4.0: If $a_n \to 0$ and $b_n \to 0$ then $a_n \cdot b_n \to 0$
Then there $\exists\ \N_a(\frac{\epsilon}{2})$ and there $\exists\ \N_b(2)$ so that for $\N_c(\epsilon) = \max(\N_a(\epsilon),\ \N_b(\epsilon)):$
$$ \begin{aligned} \left| a_n - 0 \right| = \left| a_n \right| & < \frac{\epsilon}{2}\\ \left| b_n - 0 \right| = \left| b_n \right| & < 2 \end{aligned} $$
Holds for $\forall\ n \ge \N_c$. But then it immediately follows that:
$$\left| a_nb_n - 0 \right| = \left| a_nb_n \right| = \left| a_n \right| \left| b_n \right| < 2 \frac{\epsilon}{2} = \epsilon$$
4.0.1: If $\lim_{x \to x_0} \f(x) = 0$ and $\lim_{x \to x_0} \g(x) = 0$ then $\lim_{x \to x_0} \f(x) \cdot \g(x) = 0$
Then there $\exists\ \delta_f(\frac{\epsilon}{2})$ and there $\exists\ \delta_g(2)$ so that for $\delta(\epsilon) = \min(\delta_f(\epsilon),\ \delta_g(\epsilon)):$
$$ \begin{aligned} \left| \f(x) - 0 \right| = \left| \f(x) \right| & < \frac{\epsilon}{2}\\ \left| \g(x) - 0 \right| = \left| \g(x) \right| & < 2 \end{aligned} $$
Holds for $\forall\ x:\quad \left| x - x_0 \right| < \delta$. But then it immediately follows that:
$$\left| \f(x)\g(x) - 0 \right| = \left| \f(x)\g(x) \right| = \left| \f(x) \right| \left| \g(x) \right| < 2 \frac{\epsilon}{2} = \epsilon$$
4.1: if $a_n \to 0$ and $\left| b_n \right|$ is bounded by a constant $K$ then $a_nb_n \to 0$
Proof:
For $\forall\ \epsilon > 0$ there $\exists\ \N_a(\epsilon):\quad \left| a_n - A \right| = \left| a_n \right| < \epsilon$ for $\forall\ n \ge \N_a$ and also $\left| b_n \right| < K$
So then:
$$\left| a_nb_n - 0\right| = \left| a_n \right| \left| b_n \right| < \left| a_n \right| K < \epsilon K$$
Therefore $\N(\epsilon) = \N_a(\frac{\epsilon}{K})$ is sufficient for the above to hold true.
4.1.1: if $\lim_{x \to x_0} \f(x) = 0$ and $\left| \g(x) \right|$ is bounded by a constant $K$ then $\f(x)\g(x) \to 0$
Proof:
For $\forall\ \epsilon > 0$ there $\exists\ \delta_f(\epsilon):\quad \left| a_n - A \right| = \left| a_n \right| < \epsilon$ for $\forall\ x:\quad \left| x - x_0 \right| < \delta_f$ and also $\left| \g(x) \right| < K$
So then:
$$\left| \f(x)\g(x) - 0\right| = \left| \f(x) \right| \left| \g(x) \right| < \left| \f(x) \right| K < \epsilon K$$
Therefore $\delta(\epsilon) = \delta_f(\frac{\epsilon}{K})$ is sufficient for the above to hold true.
4.2: if $a_n \to A$ and $b_n \to B$ then $a_n \cdot b_n \to A \cdot B$
$$a_nb_n = (a_n - A)(b_n - B) + Ab_n + Ba_n - AB$$
Now we learned that the limit of a sum of sequences is the sum of their limits.
So by 4.0 $(a_n - A)(b_n - B) \to 0$
By 3.9 $Ab_n \to AB$ and $Ba_n \to BA$
And so then by 3.7:
$$a_nb_n \to AB$$
4.2.1: if $\lim_{x \to x_0} = A$ and $\lim_{x \to x_0} \g(x) = B$ then $\lim_{x \to x_0} \f(x) \cdot \g(x) = A \cdot B$
$$ \begin{aligned} & \f(x)\g(x) = \\ & (\f(x) - A)(\g(x) - B) + A\g(x) + B\f(x) - AB \end{aligned} $$
Now we learned that the limit of a sum of sequences is the sum of their limits.
So by 4.0.1 $(\f(x) - A)(\g(x) - B) \to 0$
By 3.9.1 $A\g(x) \to AB$ and $B\f(x) \to BA$
And so then by 3.7.1:
$$\f(x)\g(x) \to AB$$
4.3: if $a_n \to A$ and $b_n \to B$ then $\frac{a_n}{b_n} \to \frac{A}{B}$ if $B \neq 0$
Proof:
$$\frac{a_n}{b_n} = a_n \cdot \frac{1}{b_n}$$
Therefore by 3.8 $\frac{1}{b_n} \to \frac{1}{B}$ and so then by 4.2:
$$\frac{a_n}{b_n} \to \frac{A}{B}$$
4.3.1: if $\lim_{x \to x_0} \f(x) = A$ and $\lim_{x \to x} \g(x) = B$ then $\frac{\f(x)}{\g(x)} \to \frac{A}{B}$ if $B \neq 0$
Proof:
$$\frac{\f(x)}{\g(x)} = \f(x) \cdot \frac{1}{\g(x)}$$
Therefore by 3.9.1 $\lim_{x \to x_0} \frac{1}{\g(x)} \to \frac{1}{B}$ and so then by 4.2.1:
$$\lim_{x \to x_0} \frac{\f(x)}{\g(x)} = \frac{A}{B}$$
4.4: Difference quotient
The difference quotient is the average rate of change of a function over an interval. Or, change in function value over change in 'time' (interval)
$$\frac{\triangle y}{\triangle x} = \frac{\f(x + h)- \f(x)}{h}$$
4.4.1: Derivative
By geometric intuition the derivative at a given point is the slope of the tangent to the function. We are looking at what is at the limit when we are looking at the change in function value over smaller and smaller intervals. An interval always takes 2 points from the function, but in the limit, it 'touches just one point'. It is used to precisely measure instantenious speed at a given moment. A function is differentiable at $x_0$ if it is continuous there. But then it means the function converges here, meaning the smaller the neighbourhood you choose for $x_0$ the closer the function values are in the neighbourhood to $\f(x_0)$. That means your measurement for average rate of change gets more and more precise by choosing points closer and closer to $x_0$
By mathematical definition the derivative at $x_0$ is:
$$\f'(x_0) = \lim_{h \to 0}\frac{\f(x_0 + h) - \f(x_0)}{h}$$
But only when $\f'(x_0) \in \mathbb{R}$. $x_0$ may not be a fix value. Then the result will a variable and it is called the derivative function which will have discontinuities where the original function is not continuous.
4.5: If $\f(x)$ has a local minimum or maximum at $x_0$ then $\f'(x_0) = 0$
Let's prove for a local maximum. Then $\lim_{h \to 0+} \frac{\f(x + h) - \f(x)}{h} \ge 0$ because both the numerator and denominator is positive. The numerator is poisitive because in the close neighbourhood of $x$ we have values smaller han $\f(x)$, and the denominator is positive because we are looking at the right hand side limit of the derivative. Therefore, it may only converge to positive value or zero.
Now in case of $\lim_{h \to 0-} \frac{\f(x + h)}{h} \le 0$ because the numerator is still positive for the same reason, but because we look at the left hand side limit now, it is negative, and therefore may only converge to a negative value or zero
Since we assumed the existence of the derivative, the right and left hand side limits must be equal, therefore it must be zero.
4.6: Rolle's theorem
If $\f$ is continuous over $\left[ a,\ b \right]$ and differentiable over $\left] a,\ b \right[$ and $\f(a) = \f(b)$ then $\exists\ \xi \in \left] a,\ b \right[:\quad \f'(\xi) = 0$
Proof: Due to the second theorem of Weierstrass (3.2), a function that is continuous over a closed and bounded interval obtains obtains its infimum ad supremum at some points on this interval.
If we assumed both extreme values at the end of the interval at $a,\ b$ then it would mean $\f$ is a constant function due ot $\f(a) = \f(b)$ and therefore any inner point of the interval satisfies the theorem since the derivative at any point of a constant function is 0.
If we assume only one or none of the extreme values at the end of the interval, then at least one of the extreme values should be obtained at an inner point, proving the theorem.
4.6.1: Lagrange mean value theorem
If $\f$ is continuous over $\left[ a,\ b \right]$ and differentiable over $\left] a,\ b \right[$ then there $\exists\ \xi \in \left] a,\ b \right[:\quad$
$$\f'(\xi) = \frac{\f(b) - \f(a)}{b - a}$$
Let $\h(x) = \f(a) + \frac{\f(b) - \f(a)}{b - a}(x - a)$
Let $\g(x) = \f(x) - \h(x)$
So that $\g(a) = \g(b) = 0$, and $\g$ is continuous over $\left[ a,\ b \right[$ and differentiable over $\left] a,\ b \right[$ (the latter is due to $\f$).
Therefore, Rolle's theorem (4.6) is applicable, and there $\exists\ \xi \in \left] a,\ b \right[:\quad \g'(\xi) = 0 = \f'(\xi) - \frac{\f(b) - \f(a)}{b - a}$
$$\implies \frac{\f(b) - \f(a)}{b - a} = \f'(\xi)$$
It may worth notign that Lagrange mean value theorem is a generalisation of Rolle's. Or conversely, Rolle's is a special case of this theorem.
4.6.2: Cauchy mean value theorem
If $\f$ and $\g$ are continuous over $\left[ a,\ b \right]$ and differentiable in $\left] a,\ b \right[$ and $\g'(x) \neq 0$ for $\forall\ x \in \left] a,\ b \right[$ then there $\exists\ \xi \in \left] a,\ b \right[:$
$$\frac{\f'(\xi)}{\g'(\xi)} = \frac{\f(b) - \f(a)}{\g(b) - \g(a)}$$
Proof:
Let $\h(x) = (\f(b) - \f(a))\g(x) - (\g(b) - \g(a))\f(x)$ so that it is continuous over $\left[ a,\ b \right]$ and is differentiable in $\left] a,\ b \right[$. So then:
$$ \begin{aligned} \h(x) & = (\f(b) - \f(a))\g(x) - (\g(b) - \g(a))\f(x)\\ \implies \h(a) & = \f(b)\g(a) - \f(a)\g(b)\\ \implies \h(b) & = -\f(a)\g(b) + \f(b)\g(a)\\ \implies \h(a) & = \h(b) \end{aligned} $$
Therefore, using Rolle's theorem (4.6), there $\exists\ \xi \in \left] a,\ b \right[:\quad \h'(\xi) = 0$ Therefore:
$$ \begin{aligned} & (\f(b) - \f(a))\g'(\xi) - (\g(b) - \g(a))\f'(\xi) = 0\\ & \implies \frac{\f'(\xi)}{\g'(\xi)} = \frac{\f(b) -\f(a)}{\g(b) - \g(a)} \end{aligned} $$
Please note that Cauchy mean value theorem is a generalisation of the Lagrange mean value theorem, or conversely, the Cauchy mean value theorem is a specialisation of the Cauchy mean value theorem. Note that if we let $\g(x) = x$ then:
$$ \begin{aligned} \g(b) & = b\\ \g(a) & = a\\ \g'(\xi) & = 1\\ \implies \frac{\f(b) - \f(a)}{b - a} & = \f'(\xi) \end{aligned} $$
4.7: L'Hospital rule
If $\f,\ \g$ are differentiable in a neighbourhood of $x_0$ and here $\g,\ \g' \neq 0$ and $\lim_{x \to x_0} \f(x) = \lim_{x \to x_0} \g(x) = 0$ then:
$\lim_{x \to x_0} \frac{\f'(x)}{\g'(x)} = \frac{\f(x)}{\g(x)}$
Proof:
Please note that $\f$ and $\g$ were not required to be continuous at $x_0$, only to have a limit of $0$ at $x_0$. If these functions had a discontinuity at $x_0$, then it would be a removable one for both the left and right limits are equal. Now let $\f_0(x)$ and $\g_0(x)$ be altered functions, so that if $x = x_0$, then $\f(x_0) = \g(x_0) = 0$, but otherwise they are the exact same functions. Please note that the altered functions are continuous at $x_0$. This means the following shall be true:
$$ \frac{\f(x)}{\g(x)} = \frac{\f_0(x) - \f_0(x_0)}{\g_0(x) - \g_0(x_0)} $$
Due to the right hand side formula, we are able to apply the Cauchy mean value theorem. So for $\forall\ x$ there $\exists\ \xi \in \left] x,\ x_0 \right[:$
$$ \frac{\f_0'(\xi)}{\g_0'(\xi)} = \frac{\f_0(x) - \f_0(x_0)}{\g_0(x) - \g_0(x_0)} $$
Now please note that if $x \to x_0$ then the above equality remains true only if $\xi \to x_0$. BUT! Also note that as $x \to x_0$ the fraction of the original functions tend to the same limit of $0$ as the fraction of the altered functions. Therefore:
$$\lim_{x \to x_0} \frac{\f(x)}{\g(x)} = \lim_{\xi \to x_0} \frac{\f_0'(\xi)}{\g_0'(\xi)} = \lim_{\xi \to x_0} \frac{\f'(\xi)}{\g'(\xi)}$$
Also note that $\f$ and $\g$ may not be differentiable at $x_0$ but the limit of the their derivatives at $x_0$ still match the derivatives of the altered functions everywhere else. Therefore:
$$\lim_{\xi \to x_0} \frac{\f'(\xi)}{\g'(\xi)} = \lim_{x \to x_0} \frac{\f'(x)}{\g'(x)}$$
$$\lim_{x \to x_0} \frac{\f(x)}{\g(x)} = \lim_{x \to x_0}$$
4.8: If $\f$ is continuous over $\left[ a,\ b \right]$ and differentiable in $\left] a,\ b \right[$ and there $\f'(x) = 0$, then $\f$ is constant function.
Proof:
Due to the Lagrange mean value theorem (4.6.1), for $\forall\ x_1,\ x_2 \in \left[ a,\ b \right]$ there $\exists\ \xi \in \left] x_1,\ x_2 \right[:$
$$ \begin{aligned} \f'(\xi) & = \frac{\f(x_1) - \f(x_2)}{x_1 - x_2}\\ \f'(\xi) & = 0\\ \implies \f(x_1) & = \f(x_2) \end{aligned} $$
For $\forall x_1 \neq x_2$, therefore $\f$ must be constant function.
4.9: Let $\f$ be differentiable over $I$, then:
$\f$ monotonically increasing/decreasing $\iff$ $\f'(x) \ge 0$ or $\f'(x) \le 0$
Proof for $\implies$:
Since $\f$ monotonically increases:
$$ \begin{aligned} \implies \frac{\f(x + h) - \f(x)}{h} & \ge 0\\ \implies \lim_{h \ to 0} \frac{\f(x + h) - \f(x)}{h} & = \f'(x) \ge 0 \end{aligned} $$
Proof for $\impliedby$
Let $x_1,\ x_2 \in I:\quad x_1 < x_2$. Then we can apply the Lagrange mean value theorem (4.6.1) and combine with our assumption:
$$ \begin{aligned} \frac{\f(x_2) - \f(x_1)}{x_2 - x_1} & = \f'(\xi) \ge 0\\ x_2 - x_1 & > 0\\ \implies \f(x_2) - \f(x_1) & \ge 0\\ \implies \f(x_1) & \ge \f(x_1) \end{aligned} $$
5.0: Let $\f$ be differentiable in $I$, then:
If $\f'(x) > 0$ then $\f$ strictly monotonically increases
Proof
Let $x_1,\ x_2 \in I:\quad x_1 < x_2$. Then we can apply the Lagrange mean value theorem (4.6.1) and combine with our assumption:
$$ \begin{aligned} x_2 - x_1 & > 0\\ \frac{\f(x_2) - \f(x_1)}{x_2 - x_1} & = \f'(\xi) > 0\\ \implies \f(x_2) & > \f(x_1) \end{aligned} $$
5.1: Let $\f$ be differentiable in $I$, then:
$\f'$ monotonically increases/decreases $\iff$ $\f$ is convex/concave
Proof for $\impliedby$
Let $x_1,\ x_2 \in I$, then due to convexity for $\forall\ x \in \left] x_1,\ \right[:$
$$ \begin{aligned} \lim_{x \to x_1} \frac{\f(x) - \f(x_1)}{x - x_1} & \le \f'(x) \le \lim_{x \to x_2} \frac{\f(x) - \f(x_2)}{x - x_2}\\ \implies \f'(x_1) & \le \f'(x) \le \f'(x_2)\\ \implies \f'(x_1) & \le \f'(x_2) \end{aligned} $$
Proof for $\implies$
Let's give a precise axiomatic definition for convexity. $\f$ is convex in $I$ when:
$$\frac{\f(x_2) - \f(x_1)}{ x_2 - x_1} < \frac{\f(x_3) - \f(x_2)}{ x_3 - x_2}$$
For $\forall\ x_1,\ x_2,\ x_3 \in I$
This condition of convexity is easily proven using Lagrange mean value theorem (4.8.1). For $\forall\ x_1 < x_2 < x_3,\ x_1,\ x_2,\ x_3 \in I$ there $\exists\ \xi_0,\ \xi_2:$
$$ \begin{aligned} \f'(\xi_0) & = \frac{\f(x_2) - \f(x_1)}{x_2 - x_1}\\ \f'(\xi_1) & = \frac{\f(x_3) - \f(x_2)}{x_3 - x_2}\\ \xi_0 & < \xi_1\\ \implies \f'(\xi_0) & \le \f'(\xi_1)\\ \implies \frac{\f(x_2) - \f(x_1)}{x_2 - x_1} & \le \frac{\f(x_3) - \f(x_2)}{x_3 - x_2} \end{aligned} $$
5.2: $\f''(x) \ge 0 \iff \f(x)$ convex
Proof: (using 4.9, 5.1)
$$\f''(x) \le 0 \iff \f'(x) \text{monotonically increases} \iff \f(x) \text{convex}$$
5.3: We learned that if a function has a local extreme value at $x_0$ then $\f'(x_0) = 0$. Now here is a sufficient condition for the reverse: if $\f'(x_0) = 0$ and $\f'$ chanegs sign at $x_0$ then it has a local extreme value there
5.4: $\f$ has a point of inflexion at $x_0$ if it goes from concave to convex or vice versa. This happens if and only if $\f''$ changes sign at $x_0$. (And by common sense $\f''(x_0) = 0$)
Proof:
Since $\f'$ changes sign at $x_0$ there is a nonzero neighbourhood of $x_0$ to the left and right in which $\f'$ is nonzero and has opposite sign. But then by 5.0, $\f$ is strictly monotonically increasing/decreasing on one side, and is strictly monotonically decreasing/increasing on the other. Therefore, $x_0$ must be a point at which the function goes from decreasing to increasing or from increasing to decreasing and so it must be a minimum or maximum.
5.5: $\f'(x) + \g'(x) = (\f(x) + \g(x))'$
Proof:
$$ \begin{aligned} & (\f(x) + \g(x))' = \lim_{h \to 0} \frac{\f(x + h) + \g(x + h) - (\f(x) + \g(x))}{h} =\\ & = \lim_{h \to 0} \frac{\f(x + h) - \f(x)}{h} + \frac{\g(x + h) - \g(x)}{h} = \f'(x) + \g'(x) \end{aligned} $$
By rules of operations with limes:
$$(\f(x) + \g(x))' = \f'(x) + \g'(x)$$
5.6: $(c\cdot\f(x))' = c \cdot \f'(x)$
Proof:
$$ \begin{aligned} (c\f(x))' & = \lim_{h \to 0} \frac{c\f(x + h) - c\f(x)}{h} =\\ & = c \cdot \lim_{h \to 0} \frac{\f(x + h) - \f(x)}{h} = c \cdot \f'(x) \end{aligned} $$
5.7: $(\f(x)\g(x))' = \f'(x)\g(x) + \f(x)\g'(x)$
Proof:
$$ \begin{aligned} & (\f(x)\g(x))' = \lim_{h \to 0} \frac{\f(x + h)\g(x + h) - \f(x)\g(x)}{h} = \\ & = \lim_{h \to 0} \frac{\f(x + h)\g(x + h) \pm \f(x + h)\g(x) - \f(x)\g(x)}{h}\\ & = \lim_{h \to 0} \frac{\f(x + h)(\g(x + h) - \g(x)) + \g(x)(\f(x + h) - \f(x))}{h}\\ & = \lim_{h \to 0} \f(x + h)\frac{\g(x + h) - \g(x)}{h} + \g(x) \frac{\f(x + h) - \f(x)}{h} =\\ & \f(x)\g'(x) + \f'(x)\g(x) \end{aligned} $$
5.8: $(\f(x)\g(x)\h(x))' = \f'(x)\g(x)\h(x) + \f(x)\g'(x)\h(x) + \f(x)\g(x)\h'(x)$
Proof: (in the last step, 5.7 is used!)
$$ \begin{aligned} (\f(x)\g(x)\h(x))' & = \lim_{d \to 0} \frac{\f(x + d)\g(x + d)\h(x + d) - \f(x + d)\g(x)\h(x) + \f(x + d)\g(x)\h(x) - \f(x)\g(x)\h(x)}{d} = \\ & = \lim_{d \to 0} \frac{\f(x + d)(\g(x + d)\h(x + d) - \g(x)\h(x)) + \g(x)\h(x)(\f(x + d) - \f(x))}{d}\\ & = \lim_{x \to 0} \f(x + d)\frac{\g(x + d)\h(x + d) - \g(x)\h(x)}{d} + \g(x)\h(x)\frac{\f(x + d) - \f(x)}{d} =\\ & = \f(x)\g'(x)\h(x) + \f(x)\g(x)\h'(x) + \f'(x)\g(x)\h(x) \end{aligned} $$
We can use the same scheme to obtin a formula for a product of any number of functions
5.9: $(x^k)' = k \cdot x^{k - 1},\ k \in \mathbb{N} $
Proof: (second step uses binomial theorem)
$$ \begin{aligned} & (x^k)' = \lim_{h \to 0} \frac{(x + h)^k - x^k}{h} =\\ & = \lim_{h \to 0} \frac{{k \choose 0}x^k + {k \choose 1}x^{k - 1} h + {k \choose 2}x^{k - 2} h^2 + ... + {k \choose k} h^k - x^k}{h} =\\ & = \lim_{h \to 0} \frac{{k \choose 1}x^{k - 1} h + {k \choose 2}x^{k - 2} h^2 + ... + {k \choose k} h^k}{h} =\\ & = \lim_{h \to 0} {k \choose 1}x^{k - 1} + {k \choose 2}x^{k - 2} h + ... + {k \choose k} h^{k - 1} =\\ & = k \cdot x^{k - 1} \end{aligned} $$
6.0: a function can be derived at $x_0$ $\iff$ the growth $\triangle \f$ of the function can be written as $A(\triangle x) + \epsilon(\triangle x)(\triangle x),\ A \in \mathbb{R}$
Proof for $\implies$
$$ \begin{aligned} \lim_{h \to 0} \frac{\f(x + h) - \f(x)}{h} & = A\\ \frac{\f(x + h) - \f(x)}{h} & = A + \epsilon(h)\\ \f(x + h) - \f(x) & = Ah + \epsilon(h)h \end{aligned} $$
Proof for $\impliedby$
$$ \begin{aligned} Ah + \epsilon(h)h & = \f(x + h) - \f(x)\\ A + \epsilon(h) & = \frac{\f(x + h) - \f(x)}{h}\\ \lim_{h \to 0} A + \epsilon(h) & = A = \lim_{h \to 0} \frac{\f(x + h) - \f(x)}{h} \end{aligned} $$
6.1: Chain rule
$$(\f(\g(x)))' = \f'(\g(x))\g'(x)$$
$$ \triangle \f(\g(x)) = \f(\g(x + h)) - \f(\g(x)) = \triangle \f\\ $$
We assume differentiability of $\f$ in $\g$ and differentiability of $\g$ in $x$, then by 6.0:
$$ \begin{aligned} & \triangle \f = \f'(\g(x))(\triangle \g) + \epsilon_f(\triangle \g)(\triangle \g)\\ & \triangle \g = \g'(x)h + \epsilon_g(h)h\\ & \implies \f'(\g(x))(\g'(x)h + \epsilon_g(h)h) + \epsilon_f(\triangle \g)(\g'(x)h + \epsilon_g(h)h) =\\ & = \f'(\g(x))\g'(x)h + \f'(\g(x))\epsilon_g(h)h + \epsilon_f(\triangle \g)\g'(x)h + \epsilon_f(\triangle \g)\epsilon_g(h)h\\ & \implies \frac{\f(\g(x + h)) - \f(\g(x))}{h} = \\ & = \f'(\g(x))\g'(x) + \f'(\g(x))\epsilon_g(h) + \epsilon_f(\triangle \g)\g'(x) + \epsilon_f(\triangle \g)\epsilon_g(h)\\ & h \to 0\\ & \implies (\f(\g(x)))' = \f'(\g(x))\g'(x) \end{aligned} $$
6.2: $(e^x)' = e^x$
Proof:
By definition $e = \lim_{n \to \infty} \left( 1 + \frac{1}{n} \right)^n = \lim_{h \to 0} \left( 1 + h \right)^{\frac{1}{h}}$
$$ \begin{aligned} & (e^x)' = \lim_{h \to 0} \frac{e^{x + h} - e^x}{h} = \lim_{h \to 0} e^x \frac{e^h - 1}{h}\\ & e = \left(1 + h \right)^{\frac{1}{h}}\\ & \implies e^h = 1 + h\\ & \implies \lim_{h \to 0} e^x \frac{e^h - 1}{h} = \lim_{h \to 0} e^x \frac{1 + h - 1}{h} = e^x \end{aligned} $$
6.3: $(\ln{x})' = \frac{1}{x}$
Proof:
$$ \begin{aligned} & (\ln{x})' = \lim_{h \to 0} \frac{\ln(x + h) - \ln(x)}{h} = \lim_{h \to 0} \frac{\ln\left(1 + \frac{1}{h}\right)}{h}\\ & = \lim_{h \to 0} \frac{1}{h} \ln\left(1 + \frac{h}{x}\right) = \\ & = \lim_{h \to 0} \frac{1}{h} \ln\left( \left( 1 + \frac{h}{x} \right) \right)\\ & = \lim_{h \to 0} \ln\left( \left( 1 + \frac{h}{x} \right)^{\frac{1}{h}} \right) = \\ & h = xy\\ & \implies = \lim_{y \to 0} \ln\left( \left( 1 + y \right)^{\frac{1}{xy}} \right) = \\ & = \lim_{y \to 0} \frac{1}{x} \ln\left( \left( 1 + y \right)^{\frac{1}{y}} \right) = \\ & e = \lim_{m \to 0} \left(1 + m \right)^\frac{1}{m}\\ & \implies = \frac{1}{x} \ln\left( e \right) = \frac{1}{x} \end{aligned} $$
6.4: $(x^n)' = n \cdot x^{n - 1},\ n \in \mathbb{R}$
$$ \begin{aligned} & x^n = y\\ & \ln {y} = \ln {x^n} \end{aligned} $$
Now we apply the chain rule (6.1):
$$ \begin{aligned} (\ln {y})' & = (\ln {x^n})'\\ \frac{y'}{y} & = \frac{n}{x}\\ \implies y' & = \frac{ny}{x}\\ \implies y' & = \frac{nx^n}{x}\\ \implies y' & = n \cdot x^{n - 1} \end{aligned} $$
6.5: $\left(\frac{\f(x)}{\g(x)}\right)' = \frac{\f'(x)\g(x) - \f(x)\g'(x)}{\g(x)^2}$
Proof: (using 5.7, 6.1 and and 6.4):
$$ \begin{aligned} & \frac{\f(x)}{\g(x)} = \f(x) \cdot \frac{1}{\g(x)} = \f(x) \cdot \g(x)^{-1}\\ & \implies \left(\frac{\f(x)}{\g(x)}\right)' = \f'(x)\g(x)^{-1} + \f(x) \cdot (-1) \cdot \g(x)^{-2}\g'(x) =\\ & = \f'(x)\g(x)\g(x)^{-2} + \f(x) \cdot (-1) \cdot \g(x)^{-2}\g'(x) = \frac{\f'(x)\g(x) - \f(x)\g'(x)}{\g(x)^2} \end{aligned} $$
When we think about integration one (of the two) important meaning(s) of it is its geometric meaning. The 'integral' function of a function can be used to calculate the area under the graph of the function on a given interval. You usually hear 'area under the curve' as there is no challenge in calculating the area under, for example, a constant or first grade function because it has a constant slope everywhere.
6.6: First mean value theorem for definite integrals
If $\f$ is continuous over the closed and bounded interval $\left[ a,\ b \right]$ then there $\exists\ c \in \left[ a,\ b \right]:\quad \f(c)(b - a) = \int_{a}^{b} \f(x) \ dx$
Let this theorem be a special case of $\int_{a}^{b} \f(x)\g(x)\ dx = \f(c) \int_{a}^{b} \g(x)\ dx$ when $\g(x) = 1$
Proof:
Let $\f$ be continuous over $[a,\ b]$ and $\g$ integrable function. By the second theorem of Weierstrass (3.2), there $\exists\ n,\ M:\quad n \le \f(x) \le M$ for $\forall\ x \in [a,\ b]$. Therefore if $\g$ is a nonnegative function:
$$ \int_{a}^{b} \g(x) \cdot m\ dx \le \int_{a}^{b} \f(x)\g(x)\ dx \le \int_{a}^{b} \g(x) \cdot M\ dx $$
Now if $\g$ is a negative function, the reverse inequality would be true, but then we could still get the same results.
Let $I = \int_{a}^{b} \g(x)\ dx$. If $I = 0$ then we are done, because then:
$$ \begin{aligned} & 0 \le \int_{a}^{b} \f(x)\g(x)\ dx \le 0\\ & \implies \int_{a}^{b} \f(x)\g(x)\ dx = 0 = \f(c) \cdot I,\ c \in \mathbb{R} \end{aligned} $$
If $I \neq 0$, we can divide by it:
$$ m \le \frac{1}{I} \int_{a}^{b} \f(x)\g(x)\ dx \le M $$
By the Bolzano theorem (intermediate value theorem) (3.4) $\f$ must attain every value in the interval $[m,\ M]$ so that for some $c \in [a,\ b]:$
$$ \begin{aligned} & \f(c) = \frac{1}{I} \int_{a}^{b} \f(x)\g(x)\ dx\\ & \implies \f(c)\int_{a}^{b} \g(x)\ dx = \int_{a}^{b} \f(x)\g(x)\ dx\\ & \text{if}\ \g(x) = 1\\ & \f(c)(b - a) = \int_{a}^{b} \f(x)\ dx\\ & \implies \f(c) = \frac{\int_{a}^{b} \f(x)\ dx}{b - a} \end{aligned} $$
6.7: Newton-Leibniz Theorem
If there $\exists\ \F(x):\quad \F'(x) = \f(x)$, then the sign-aware area under the function over a given bounded and closed interval is $\F(b) - \F(a),\ a \ge b$. That is why we call $\F$ the antiderivative of $\f$.
Proof 1, for a function continuous over $\left[ a,\ b \right]$:
Let $\F(x),\ x \in \left[ a,\ b \right]$ be the area function giving the area under the function over $\left[ c,\ x \right],\ c \in \left[ a,\ b \right]$.
Then the area under the function over $\left[ x,\ x_0 \right],\ x,\ x_0 \in \left[ a,\ b \right]$ is: $\F(x) - \F(x_0)$
Due to the second theorem of Weierstrass (3.2), a continuous function over a closed and bounded interval obtains its infimum and supremum. Therefore it must hold that:
$$ \begin{aligned} \f(m) \cdot h \le \F(x) - \F(x_0) \le \f(M) \cdot h \end{aligned} $$
Where $h = x - x_0$ and $m,\ M \in \left[ x,\ x_0 \right]$ is where the minimum/maximum is of the function over $\left[ x,\ x_0 \right]$ respectively. Now the only thing you need note is $\lim_{x \to x_0} \f(m) = \f(x_0) = \lim_{x \to x_0} \f(M)$. Then we can use the squeeze theorem to obtain the result:
$$ \begin{aligned} & \lim_{h \to 0} \f(m) \cdot h \le \lim_{h \to 0} \F(x) - \F(x_0) \le \lim_{h \to 0} \f(M) \cdot h\\ & \implies \lim_{h \to 0} \f(m) \le \lim_{h \to 0} \frac{\F(x) - \F(x_0)}{h} \le \lim_{h \to 0} \f(M)\\ & \implies \f(x_0) = \F'(x_0) = \f(x_0) \end{aligned} $$
So it turns out that the derivative of our area function is our original function, therefore obtaining the area function from our function is the reverse process of derivation. What does it mean? It means that function with a finite number of discontinuities can still be integrated, since the theorem can be applied to its continuous pieces.
Proof 2, using the first mean value theorem for definite integrals (6.6)
Still let $\F(x) - \F(x_0)$ be the area under our function over $[x,\ x_0]$. So then by the first mean value theorem for definite integrals (6.6):
$$ \frac{\F(x) - \F(x_0)}{x - x_0} = \f(c),\ c \in [x,\ x_0]\\ $$
Now note that $\lim_{x \to x_0} \f(c) = \f(x_0)$ so:
$$ \lim_{x \to x_0} \frac{\F(x) - \F(x_0)}{x - x_0} = \lim_{x \to x_0} \f(c) = \F'(x_0) = \f(x_0) $$