The path to Continuous Fourier Transform

Maclaurin Series, Euler's Formula, Fourier Series, Continuous Fourier Transform


Maclaurin Series

The Maclaurin Series is a special case of Taylor Series, the latter being more useful when the function and its derivatives are better known at some point other than $x = 0$. The Taylor Series will be discussed in a separate article, as it is not necessary to know in order to fully understand Fourier Series.

This article is intended to be as minimalistic as possible, very focused on introducing only the most important concepts and basically only shows a series of proofs.

So the concept of the Maclaurin Series (and also of Taylor and Fourier Series) is the expression of our function at a given point $x$ as an infinite power series. Like other series and Riemann sums, the series gets ever more precise as the upper limit grows. We need to find those coefficients (multipliers) of the first $n$ powers of $x$ so that adding them up approaches the function value, and increasing $n$ makes the estimate more precise. This leaves us with:

$$f(x) = C_0x^0 + C_1x^1 + C_2x^2 + C_3x^3 + ... + C_nx^n$$

How do we find the value of any of those coefficients? Let's write up how the power series change as you are taking the derivatives of the function from the first down to the $n$th:

$$\begin{aligned} f(x) & = C_0x^0 + C_1x^1 + C_2x^2 + C_3x^3 + ... + C_nx^n\\ f'(x) & = C_1 + 2C_2x^1 + 3C_3x^2 + ... + nC_nx^{n - 1}\\ f''(x) & = 2C_2 + 3\times2C_3x^1 + ... + n(n-1)C_nx^{n - 2}\\ f'''(x) & = 3\times2C_3 + ... + n(n-1)(n-2)C_nx^{n - 3}\\ ....\\ f^{(n)}(x) &= n!C_n\\ \end{aligned}$$

Now if all those equations are evaluated at $x = 0$, then all the terms after the first multiplied by a power of $x$ cancel out so that we obtain:

$$\begin{aligned} f(0) & = C_0\\ f'(0) & = C_1\\ f''(0) & = 2!C_2\\ f'''(0) & = 3!C_3\\ & ....\\ f^{(n)}(0) &= n!C_n\\ \end{aligned}$$

Now rearranging them:

$$\begin{aligned} C_0 & = \frac{f(0)}{0!}\\ C_1 & = \frac{f'(0)}{1!}\\ C_2 & = \frac{f''(0)}{2!}\\ C_3 & = \frac{f'''(0)}{3!}\\ & ....\\ C_n &= \frac{f^{(n)}}{n!}\\ \end{aligned}$$

And placing them back to the original series:

$$f(x) = f(0) + \frac{f'(0)}{1!}x^1 + \frac{f''(0)}{2!}x^2 + \frac{f'''(0)}{3!}x^3 + ... + \frac{f^{(n)}(0)}{n!}x^n$$

As a single expression:

$$f(x) = \sum_{n = 0}^{\infty}{\frac{f^{(n)}(0)}{n!}x^n}$$

So that is the Maclaurin Series of a function. It goes without saying that you will need to know the function value of any derivative of the original function at $x = 0$ down to the $n$th as you can see, and Taylor Series is the way around that.

Euler's Formula

So we start by writing up the Maclaurin Series for sine and cosine first, simply because it is so easy to take their derivatives indefinitely, and because their values at $x = 0$ are commonsense. I encourage you to write your own trigonometric functions in some programming language using a Maclaurin or a Taylor, and it all just makes a lot more sense immediately :)

So the cycle of the derivation of $sin(x)$ is as follows:

$$\begin{aligned} & \sin(x) = \sin(x)\\ & \sin'(x) = \cos(x) \\ & \sin''(x) = -\sin(x) \\ & \sin'''(x) = -\cos(x) \\ & .... \\ & \sin''''(x) = \sin(x) \\ \end{aligned}$$

Now evaluating each at zero:

$$\begin{aligned} \sin(0) & = 0\\ \sin'(0) & = 1\\ \sin''(0) & = 0\\ \sin'''(0) & = -1\\ & ....\\ \sin''''(0) & = 0\\ \end{aligned}$$

And so writing up the Maclaurin Series of sine:

$$\begin{aligned} & \sin(x) = 0 + \frac{1}{1!}x + \frac{0}{2!}x^2 + \frac{-1}{3!}x^3 +\\ & + \frac{0}{4!}x^4 + \frac{1}{5!}x^5 + \frac{0}{6!}x^6 + \frac{-1}{7!}x^7 + .... \end{aligned}$$

Canceling out all that's zero, rewriting a little...

$$\sin(x) = x - \frac{1}{3!}x^3 + \frac{1}{5!}x^5 - \frac{1}{7!}x^7 + \frac{1}{9!}x^9 - ....$$

If you wanted to express it as a single sum

$$\sin(x) = \sum_{n=0}^{\infty}{(-1)^n\frac{1}{(2n + 1)!}x^{2n+1}}$$

You get to see those terms with odd factorials and powers of $x$, and with alternating signs. That's a very nice looking pattern, and cosine follows something similar. Here is the derivation cycle:

$$\begin{aligned} \cos(x) & = \cos(x)\\ \cos'(x) & = -\sin(x)\\ \cos''(x) & = -\cos(x)\\ \cos'''(x) & = \sin(x)\\ ....\\ \cos''''(x) & = \cos(x)\\ \end{aligned}$$

Evaluating at $x = 0$...

$$\begin{aligned} \cos(0) & = 1\\ \cos'(0) & = 0\\ \cos''(0) & = -1\\ \cos'''(0) & = 0\\ ....\\ \cos''''(0) & = 1\\ \end{aligned}$$

Now putting these into the Maclaurin Series of cosine:

$$\cos(x) = 1 + \frac{0}{1!} + \frac{-1}{2!}x^2 + \frac{0}{3!}x^3 + \frac{1}{4!}x^4 + \frac{0}{5!}x^5 + \frac{-1}{6!}x^6 + ....$$

Canceling out, rewriting...

$$\cos(x) = 1 - \frac{1}{2!}x^2 + \frac{1}{4!}x^4 - \frac{1}{6!}x^6 + \frac{1}{8!}x^8 - ....$$

If you wanted to express it as a single sum:

$$\cos(x) = \sum_{n=0}^{\infty}{(-1)^n\frac{1}{(2n)!}x^{2n}}$$

It is exactly the same story, but now we get the terms with the even factorials and powers of $x$, and with alternating signs. Now we are going to look into one more interesting Maclaurin, which is $e^x$. We know very well that the derivative of $e^x$ is always itself, and that anything to the power of zero is always $1$. So the Maclaurin Series of $e^x$ is:

$$e^x = 1 + \frac{1}{1!}x + \frac{1}{2!}x^2 + \frac{1}{3!}x^3 + \frac{1}{4!}x^4 + ... + \frac{1}{n!}x^n$$

If you wanted to express it as a single sum:

$$e^x = \sum_{n=0}^{\infty}{\frac{1}{n!}x^{n}}$$

That's easy as $\pi$, right? Only if it had alternating signs could we express it in terms of the Maclaurin Series of sine and cosine, which is the key concept! If we want to do that, we will need to break the rules a little, some very little very subjective rule. What if we invented a number whose square is $-1$? That must be something really imaginary, but mind you, the fact that the square of something is always positive is as subjective as politics. So let's say we have a rule such that $i^2=-1$ but only for $i$, which is an imaginary number. So if we proceed with the Maclaurin Series of $e^{ix}$ we obtain:

$$\begin{aligned} & e^{ix} = 1 + \frac{i}{1!}x + \frac{i^2}{2!}x^2 + \frac{i^3}{3!}x^3 + \frac{i^4}{4!}x^4 +\\ & + \frac{i^5}{5!}x^5 \frac{i^6}{6!}x^6 + \frac{i^7}{7!}x^7 + .... + \frac{i^n}{n!}x^n \end{aligned}$$

Little of rewriting...

$$\begin{aligned} e^{ix} & = 1 + \frac{i}{1!}x + \frac{i^2}{2!}x^2 + \frac{i^2i}{3!}x^3 + \\ & + \frac{(i^2)^2}{4!}x^4 + \frac{(i^2)^2i}{5!}x^5 \frac{(i^2)^3}{6!}x^6 + \frac{(i^2)^3i}{7!}x^7 + .... + \frac{i^n}{n!}x^n \end{aligned}$$

So there will be a pattern of sign alternation:

$$\begin{aligned} e^{ix} & = 1 + ix + \frac{-1}{2!}x^2 + \frac{-i}{3!}x^3 + \\ & + \frac{1}{4!}x^4 + \frac{i}{5!}x^5 + \frac{-1}{6!}x^6 + \frac{-i}{7!}x^7 + .... + \frac{i^n}{n!}x^n \end{aligned}$$

The pattern is $1$, $i$, $-1$, $-i$. One more rewrite:

$$\begin{aligned} e^{ix} & = 1 + ix - \frac{1}{2!}x^2 - \frac{i}{3!}x^3 +\\ & + \frac{1}{4!}x^4 + \frac{i}{5!}x^5 - \frac{1}{6!}x^6 - \frac{i}{7!}x^7 + .... + \frac{i^n}{n!}x^n \end{aligned}$$

So what you eventually realise is that factoring out the imaginary number from all terms that have it will yield the Maclaurin Series of sine, and the terms which does not have the imaginary number constitute the Maclaurin Series of cosine. This is the final rearranged form in which it is evident:

$$\begin{aligned} e^{ix} & = (1 - \frac{1}{2!}x^2 + \frac{1}{4!}x^4 - \frac{1}{6!}x^6 + ...) +\\ & + i(x - \frac{i}{3!}x^3 + \frac{i}{5!}x^5 - \frac{i}{7!}x^7 + ...) \end{aligned}$$

So taking the limit of the Maclaurin Series of $e^{ix}$ we get:

$$e^{ix} = \cos(x) + i\sin(x)$$

That is sometimes also called cisx, meaning "Cosine x plus I times Sine x". Now, this is Euler's Formula. We need to put it away for some time now, as I need to introduce you to the Fourier Series. It has 2 forms, and I am gonna start out with the trigonometric variant. Euler's Formula is going to be used to prove the equivalence of the trigonometric form and the rather compact complex exponential form.

Trigonometric Fourier Series

The Fourier Series can also be used to estimate a function. We can select an interval on which we wish to approach the function, and we can get the coefficients required for that. That means, if you wanted to estimate teh function on another interval, you would need to recalculate teh coefficients. And since Fourier series uses trigonometric functions to approach the function, it is going to be periodic, and the period will be the length of the interval. So if our function was also periodic, then it would converge to teh function everywhere. Most commonly, the Fourier series is shown to approach the input function on the interval $]-\pi,\ \pi[$ since by nature sines and cosines are $2\pi$ periodic. But with modifications, we can make the series estimate on arbitrary interval.

Sidenote. In the following, I am not going to prove the convergence of the Foueries series. Instead, I will use it as already proven and start out from it, proving the coefficients if it is true. If you are interested in knowing the proof of the rigorous proof of pointwise convergence then please check out my other article written about it. So the Fourier series says that if $f$ is $2\pi$ periodic then it holds that:

$$\begin{aligned} & f(x) = \\ & + b_0\sin(0x) + b_1\sin(x) + b_2\sin(2x) + ... + b_n\sin(nx) +\\ & + a_0\cos(0x) + a_1\cos(x) + a_2\cos(2x) + ... + a_n\cos(nx) \end{aligned}$$

As a single expression:

$$f(x) = \sum_{n=0}^{n=\infty}{(a_n\sin(nx) + b_n\cos(nx))}$$

If we want to find out the values of $a_0$ and the coefficients of the trigonometric functions of different frequencies, we only need to prove the following integrals:

$$\begin{aligned} \int_{t}^{t + P}{\sin(\frac{2\pi}{P}kx)dx} & = 0 \quad \text{if} \quad k \in \mathbb{Z}\\ \int_{t}^{t + P}{\cos(\frac{2\pi}{P}kx)dx} & = \begin{cases} 0, \text{if}\ k \neq 0\\ 2\pi, \text{if}\ k = 0 \end{cases} \quad k \in \mathbb{Z}\\\\ \int_{t}^{t + P}{\sin(\frac{2\pi}{P}qx)\cos(\frac{2\pi}{P}kx)dx} & = 0\\ \int_{t}^{t + P}{\sin(\frac{2\pi}{P}qx)\sin(\frac{2\pi}{P}kx)dx} & = \int_{t}^{t + P}{\cos(\frac{2\pi}{P}qx)\cos(\frac{2\pi}{P}kx)dx} = \begin{cases} 0, & \text{if}\ q \neq k\\ \pi & \text{if}\ q = k \end{cases} \end{aligned} $$

For proving those integrals, we need to use the Addition Formulae, which will now come without proof. Maybe later it will also come in some other batch of proofs.

$$\begin{cases} \sin(\alpha + \beta) = \sin(\alpha)\cos(\beta) + \sin(\beta)\cos(\alpha)\\ \sin(\alpha - \beta) = \sin(\alpha)\cos(\beta) - \sin(\beta)\cos(\alpha) \end{cases}$$

From the 2 it follows:

$$\begin{aligned} \sin(\alpha + \beta) + \sin(\alpha - \beta) & = 2\sin(\alpha)\cos(\beta)\\ \sin(\alpha + \beta) - \sin(\alpha - \beta) & = 2\sin(\beta)\cos(\alpha) \end{aligned}$$

So you basically rewrite the product of trigonometric equations to their sums, which comes in pretty handy in case of integrals. So let's substitute!

$$\begin{aligned} & \int_{t}^{t + P}{\sin(\frac{2\pi}{P}kx)\cos(\frac{2\pi}{P}qx)dx} = \int_{t}^{t + P}{\frac{1}{2}(\sin(\frac{2\pi}{P}kx + \frac{2\pi}{P}qx) + \sin(\frac{2\pi}{P}kx - \frac{2\pi}{P}qx))dx} = \\ & = \frac{1}{2}(\int_{t}^{t + P}{\sin((k + q)\frac{2\pi}{P}x)dx} + \int_{t}^{t + P}{\sin((k - q)\frac{2\pi}{P}x)dx}) \end{aligned}$$

Now, if we let $\int_{t}^{t + P}{\sin(\frac{2\pi}{P}kx)} = \int_{t}^{t + P}{\cos(\frac{2\pi}{P}kx)} = 0$ when $k \neq 0$ by intuition (since $k$ and $q$ are integers sine and cosine will complete a whole number of periods wich cancel out to zero), then the above integral equates to:

$$\frac{1}{2}(0 + 0) = 0$$

Now let's continue with the product of 2 sines or 2 cosines. If we want to express them as sums, we will want to use cosine addition formula instead of the sine:

$$\begin{cases} \cos(\alpha - \beta) & = \cos(\alpha)\cos(\beta) + \sin(\alpha)\sin(\beta)\\ \cos(\alpha + \beta) & = \cos(\alpha)\cos(\beta) - \sin(\alpha)\sin(\beta) \end{cases}$$

From these it follows:

$$\begin{aligned} \cos(\alpha - \beta) + \sin(\alpha + \beta) & = 2\cos(\alpha)\cos(\beta)\\ \cos(\alpha - \beta) - \sin(\alpha + \beta) & = 2\sin(\beta)\sin(\alpha) \end{aligned}$$

So with these we can rewrite the 2 integrals:

$$\begin{aligned} \int_{t}^{t + P}{\cos(\frac{2\pi}{P}kx)\cos(\frac{2\pi}{P}qx)dx} & = \int_{t}^{t + P}{\frac{1}{2}(\cos(\frac{2\pi}{P}kx - \frac{2\pi}{P}qx) + \sin(\frac{2\pi}{P}kx + \frac{2\pi}{P}qx))dx}\\ & = \frac{1}{2}(\int_{t}^{t + P}{\cos((k - q)\frac{2\pi}{P}x)dx} + \int_{t}^{t + P}{\sin((k + q)\frac{2\pi}{P}x)dx})\\ \int_{t}^{t + P}{\sin(\frac{2\pi}{P}kx)\sin(\frac{2\pi}{P}qx)dx} & = \int_{t}^{t + P}{\frac{1}{2}(\cos(\frac{2\pi}{P}kx - \frac{2\pi}{P}qx) - \sin(\frac{2\pi}{P}kx + \frac{2\pi}{P}qx))dx}\\ & = \frac{1}{2}(\int_{t}^{t + P}{\cos((k - q)\frac{2\pi}{P}x)dx} - \int_{t}^{t + P}{\sin((k + q)\frac{2\pi}{P}x)dx}) \end{aligned}$$

Now, in these 2 cases we need to be careful, because of the $\cos(kx - qx)$ terms. Given that $k = q$, we are integrating $f(x) = 1$ over one perod $P$, so we are ought to get $P$ (we may also get this result when the frequencies are opposites but it won't happen in the context of the Fourier Series) as the sine term cancels out:

$$\begin{aligned} \frac{1}{2}(\int_{t}^{t + P}{\cos((k - q)\frac{2\pi}{P}x)dx} + \int_{t}^{t + P}{\sin((k + q)\frac{2\pi}{P}x))dx} & = \begin{cases} \frac{1}{2}(0 + 0), \text{if}\ k \neq q\\ \frac{1}{2}(2\pi + 0), \text{if}\ k = q \end{cases}\\ \frac{1}{2}(\int_{t}^{t + P}{\cos((k - q)\frac{2\pi}{P}x)dx} - \int_{t}^{t + P}{\sin((k + q)\frac{2\pi}{P}x))dx} & = \begin{cases} \frac{1}{2}(0 - 0), \text{if}\ k \neq q\\ \frac{1}{2}(2\pi - 0), \text{if}\ k = q \end{cases} \end{aligned}$$

Which is:

$$\begin{aligned} \frac{1}{2}(\int_{t}^{t + P}{\cos((k - q)\frac{2\pi}{P}x)dx} + \int_{t}^{t + P}{\sin((k + q)\frac{2\pi}{P}x))dx} & = \begin{cases} 0, \text{if}\ k \neq q\\ \pi, \text{if}\ k = q \end{cases}\\ \frac{1}{2}(\int_{t}^{t + P}{\cos((k - q)\frac{2\pi}{P}x)dx} - \int_{t}^{t + P}{\sin((k + q)\frac{2\pi}{P}x))dx} & = \begin{cases} 0, \text{if}\ k \neq q\\ \pi, \text{if}\ k = q \end{cases} \end{aligned}$$

So summing up these integrals, the integral of the product of a sine and cosine is always zero regardless of coefficients, and the product of 2 equivalent trigonometric functions is $\pi$ given the coefficients are equal, or zero given they are not, in Fourier Series. Now having these in mind, it is child's play to get the coefficients in our Fourier Series. Let's first go just for the first term denoted by $a_0$. This is the coefficient for the cosine term with frequence $0$. This is considered to be first term as the correspnding $b_0$ coefficient of the first sine term with frequency zero is nonexistent as this term is always zero. We basically just integrate the series on both sides, so that every other term inherently cancels out since the integral of sine or cosine on any interval of length $2\pi$ is always zero in case the frequency is never zero (in which case cosine would yield $2\pi$):

$$\begin{aligned} & \int_{t}^{t + P}{f(x)dx} = \int_{t}^{t + P}{a_0dx} = \\ & = Pa_0 = \\ & = \int_{t}^{t + P}{a_0dx} +\\ & + b_1\int_{t}^{t + P}{\sin(\frac{2\pi}{P}x)dx} + b_2\int_{t}^{t + P}{\sin(\frac{2\pi}{P}2x)dx} + ... + b_n\int_{t}^{t + P}{\sin(\frac{2\pi}{P}nx)dx} +\\ & + a_1\int_{t}^{t + P}{\cos(\frac{2\pi}{P}x)dx} + a_2\int_{t}^{t + P}{\cos(\frac{2\pi}{P}2x)dx} + ... + a_n\int_{t}^{t + P}{\cos(\frac{2\pi}{P}nx)dx} \end{aligned}$$

So rearranging the equation to obtain the value of $a_0$:

$a_0 = \frac{\int_{t}^{t + P}{f(x)dx}}{P}$

Which is the average function value over one period. Now, the technic for finding the coefficient for a trigonometric term of given frequency, is multiplying both sides by the same trigonometric function with the same frequency, so that every other term will cancel out since, either because it is the other trigonometric function, or because it has a different frequency. So let's find the coefficient for $\sin(\frac{2\pi}{P}nx)$. We multiply both sides by $\sin(\frac{2\pi}{P}nx)$ and integrate both sides:

$$\begin{aligned} & \int_{t}^{t + P}{\sin(\frac{2\pi}{P}nx)f(x)dx} = \int_{t}^{t + P}{a_0\sin(\frac{2\pi}{P}nx)dx} +\\ & + \int_{t}^{t + P}{b_1\sin(\frac{2\pi}{P}x)\sin(\frac{2\pi}{P}nx)dx} + \int_{t}^{t + P}{b_2\sin(\frac{2\pi}{P}2x)\sin(\frac{2\pi}{P}nx)dx} + ... +\\ & + \int_{t}^{t + P}{b_n\sin(\frac{2\pi}{P}nx)^2dx} +\\ & + \int_{t}^{t + P}{a_1}{\cos(\frac{2\pi}{P}x)\sin(\frac{2\pi}{P}nx)dx} + \int_{t}^{t + P}{a_2\cos(\frac{2\pi}{P}2x)\sin(\frac{2\pi}{P}nx)dx} + ... +\\ & + \int_{t}^{t + P}{a_n\cos(\frac{2\pi}{P}nx)\sin(\frac{2\pi}{P}nx)dx} \end{aligned}$$

Which finally equates to:

$$\int_{t}^{t + P}{\sin(\frac{2\pi}{P}nx)f(x)dx} = \int_{t}^{t + P}{b_n\sin(\frac{2\pi}{P}nx)^2dx} = \frac{P}{2} b_n$$

Rearranging the equation for $b_n$:

$$b_n = \frac{2}{P}\int_{t}^{t + P}{\sin(\frac{2\pi}{P}nx)f(x)dx}$$

And you are going to get the exact same for a cosine term of given frequency:

$$\begin{aligned} & \int_{t}^{t + P}{\cos(\frac{2\pi}{P}nx)f(x)dx} = \int_{t}^{t + P}{a_0\cos(\frac{2\pi}{P}nx)dx} +\\ & +\int_{t}^{t + P}{b_1\sin(\frac{2\pi}{P}x)\cos(\frac{2\pi}{P}nx)dx} + \int_{t}^{t + P}{b_2\sin(\frac{2\pi}{P}2x)\cos(\frac{2\pi}{P}nx)dx} + ... +\\ & + \int_{t}^{t + P}{b_n\sin(\frac{2\pi}{P}n)\cos(\frac{2\pi}{P}nx)dx} +\\ & + \int_{t}^{t + P}{a_1}{\cos(\frac{2\pi}{P}x)\cos(\frac{2\pi}{P}nx)dx} + \int_{t}^{t + P}{a_2\cos(\frac{2\pi}{P}2x)\cos(\frac{2\pi}{P}nx)dx} + ... +\\ & + \int_{t}^{t + P}{a_n\cos(\frac{2\pi}{P}nx)^2dx} \end{aligned}$$

$$\int_{t}^{t + P}{\cos(\frac{2\pi}{P}nx)f(x)dx} = \int_{t}^{t + P}{b_n\cos(\frac{2\pi}{P}nx)^2dx} = \frac{P}{2}a_n$$

$$a_n = \frac{2}{P}\int_{t}^{t + P}{\cos(\frac{2\pi}{P}nx)f(x)dx}$$

So the Trigonometric Fourier Series in full resolution:

$$f(x) = \frac{\int_{t}^{t + P}{f(u)}du}{P} + \sum_{n=1}^{\infty}{(\frac{2\int_{t}^{t + P}{\cos(\frac{2\pi}{P}nx)f(u)du}}{P}\cos(\frac{2\pi}{P}nx) + \frac{2\int_{t}^{t + P}{\sin\frac{2\pi}{P}nx)f(u)du}}{P}\sin(\frac{2\pi}{P}nx))}$$

We need to note that we have a sum in which the upper bound if the index $n$ is approaching $\infty$. If a concrete value is given to the upper bound, we obtain a so called partial sum. The larger the partial sum is, taht is, the larger teh upper bound of $n$ is, the finer is our approximation on the selected period. By intuition, the value of this upper bound should be a lot larger than the size of the period. That means you should choose a larger upper bound given a larger period for same precision, and of course oyu still increment teh upper bound for an even finer approximation. Please remember this, it is going to be an important information during our transition to Continuous Fourier Transform!

Complex Exponential Fourier Series

The Fourier Series is easier to understand when shown in its trigonometric form first. But then it also has a much more compact form which is easier to use when taken further, for example with the different types of the Fourier Transformation, and that is its complex exponential form. You see, in the trigonometric version, you can observe the 2 (or 3) distinct parts of the formula: (the first term since $b_0=0$), the series of cosines, and the series of sines. That is too much terms! There must be an easier representation! And here comes Euler's Formula which expresses $e^{ix}$ in terms of sine and cosine. And there's no furhter magic here. If you do some algebra, you can express sine and cosine in terms of complex exponentials, and then you just simply do a substitution in the trigonometric form and there you are. The beauty of complex numbers lies in the fact that they can do thrilling algebraic simplifications while producing a real number result as the complex conjugates of complex numbers cancel out each other. That being said, the Fourier Series and Transform (or anything else) would lack any kind of practical sense given in its equation there would be left at least a single imaginary part. But as you will see us summing up the product of a complex exponential with a complex coefficient from $-\infty$ to $\infty$ this way each of the complex numbers' complex conjugates are in fact present - the complex conjugate of $a + ib$ is $a - ib$ - and therefore the imaginary parts cancel out each other and make no practical difference, if not that we have a supersimplified formula that is very easy to work with. But in fact since any equation down the line are equaivalent to the former should mean that imaginary parts actually cancel out each other as they must produce sine and cosine.

Let's first do some algebraic manipulations with Euler's Formula:

$$\begin{cases} e^{ix} = \cos(x) + i\sin(x)\\ -\sin(x) = \sin(-x)\\ \cos(x) = \cos(-x) \end{cases}$$

From these identities we gain:

$$e^{-ix} = \cos(x) - i\sin(x)$$

$$\begin{cases} e^{ix} = \cos(x) + i\sin(x)\\ e^{-ix} = \cos(x) - i\sin(x) \end{cases}$$

Adding and subtracting the 2 equations, and rearranging for the expression of the trigonometric functions we obtain:

$$\begin{aligned} \frac{1}{2}e^{ix} + \frac{1}{2}e^{-ix} = \cos(x)\\ \frac{1}{2}ie^{ix} - \frac{1}{2}ie^{-ix} = \sin(x) \end{aligned}$$

In case it is not directly clear why after the rearrangement of the second equation results in multiplying $\frac{1}{2}$ by $i$ and not dividing by it, it is because of the following identity:

$$\frac{a}{bi} = \frac{ai}{bi^{2}}=-\frac{a}{b}i$$

Now observe that we will get a new supernatural equation by substitution, but all you need to keep in mind is that the above formulas break down to sine and cosine, so after performing the math our equation still produces real number result. So we substitute these in the Trigonometric Fourer Series, we get the following:

$$f(x) = \sum_{n=0}^{\infty}{((\frac{1}{2}e^{i\frac{2\pi}{P}nx} + \frac{1}{2}e^{-i\frac{2\pi}{P}nx})a_n + (\frac{1}{2}e^{i\frac{2\pi}{P}nx} - \frac{1}{2}e^{-i\frac{2\pi}{P}nx})ib_n)}$$

Now you rewrite the equation so that we get coefficients expressed in terms of $a_n$ and $b_n$, which multiply $e^{inx}$ and $e^{-inx}$

$$\begin{aligned} f(x) & = \sum_{n=0}^{\infty}{((\frac{1}{2}a_n + \frac{1}{2}ib_n)e^{i\frac{2\pi}{P}nx} + (\frac{1}{2}a_n - \frac{1}{2}ib_n)e^{-i\frac{2\pi}{P}nx})}\\ & = \sum_{n=0}^{\infty}{(\frac{1}{2}a_n + \frac{1}{2}ib_n)e^{i\frac{2\pi}{P}nx}} + \sum_{n=1}^{\infty}{(\frac{1}{2}a_n - \frac{1}{2}ib_n)e^{-i\frac{2\pi}{P}nx}} \end{aligned}$$

Now what we are going to do is define $f(x)$ as a single sum, running from $-\infty$ to $\infty$. We can do that by generalising the coefficients of the 2 sums as $C_n$, which can be different for every value of $n$. But of course, we can tell the relationship between $C_n$, $a_n$ and $b_n$ when $n$ is either positive or negative. The better question is what should $C_n$ be when $n = 0$, if we know that the term should result in $a_0$

$$C_0e^0 = C_n = a_0$$

So at the end we get:

$$\begin{aligned} f(x) & = \sum_{n=-\infty}^{\infty}{C_ne^{i\frac{2\pi}{P}nx}}\\ C_n & = \begin{cases} \frac{1}{2}(a_n - ib_n), \text{if}\ n > 0\\ a_0, \text{if}\ n = 0 (since b_0 = 0)\\ \frac{1}{2}(a_{\mid n \mid } + ib_{\mid n\mid }), \text{if}\ n < 0 \end{cases} \end{aligned}$$

But we are not stopping there. We know now how to derive the complex exponential coefficient from the trigonometric ones. We also need to rewrite the trigonometric derived complex coefficient to be purely exponential and more compact. Yes we can do that. So reqriting the coefficient by substituting $a_n$ and $b_n$ by their trigonometric definitions:

$$ C_{\pm n} = \frac{1}{2}\frac{2\int_{t}^{t + P}{f(x)\cos(\frac{2\pi}{P}\mid n \mid x)dx}}{P} \mp \frac{1}{2}\frac{2\int_{t}^{t + P}{f(x)\sin(\frac{2\pi}{P}\mid n \mid x)dx}}{P}i $$

$$ C_{\pm n} = \frac{\int_{t}^{t + P}{f(x)\cos(\frac{2\pi}{P}\mid n \mid x)dx}}{P} \mp \frac{\int_{t}^{t + P}{f(x)\sin(\frac{2\pi}{P}\mid n \mid x)dx}}{P}i $$

Now what we do is adding up the 2 fractions:

$$ C_{\pm n} = \frac{\int_{t}^{t + P}{f(x)\cos(\frac{2\pi}{P}\mid n \mid x)dx} \mp \int_{t}^{t + P}{f(x)i\sin(\frac{2\pi}{P}\mid n \mid x)dx}}{P} $$

Now we turn the 2 integrals into 1:

$$ C_{\pm n} = \frac{\int_{0}^{2\pi}{f(x)\cos(\mid n \mid x) \mp f(x)i\sin(\mid n \mid x)dx}}{P} $$

Now raise out $f(x)$, and observe the 3 identities under the equation:

$$ \begin{aligned} \begin{cases} C_{\pm n} = \frac{\int_{t}^{t + P}{f(x)(\cos(\frac{2\pi}{P}\mid n \mid x) \mp i\sin(\frac{2\pi}{P}\mid n \mid x))dx}}{P}\\ \cos(x)=\cos(+x)\\ -\sin(x)=\sin(-x)\\ e^{ix}=\cos(x)+i\sin(x) \end{cases} \end{aligned} $$

And now observe cisx in the top equation and see that it can be expressed as follows using Euler's Equation and the trigonometric identities:

$$ C_{n} = \frac{\int_{t}^{t + P}{f(x)e^{-i\frac{2\pi}{P}nx}dx}}{P} $$

We know that this will work as $i\sin(nx)$ will obtain the sign opposite to the sign of $n$ and the sign change does not affect cosine.

Now having the most compact form of the complex coefficient we can write up the most compact form of the complex exponential Fourier series:

$$ f(x) = \sum_{n=-\infty}^{\infty}{\frac{\int_{t}^{t + P}{f(u)e^{-i\frac{2\pi}{P}nx}du}}{P}e^{i\frac{2\pi}{P}nx}} $$

Continuous Fourier Transform

And so again I need to note that the Fourier Series can not be used to estimate an aperiodic function on its entire infinite domain. It is important information that the Fourier Series does estimate any function on a chosen period of $P$ even if the function is aperiodic. The idea behind the Fourier Transform is that if this chosen period is infinitely large (period tends to infinity) then "taking the limit" of what we obtain this way should give the original function even if it is aperiodic. I wrote taking the limit in quotes because I do not think it is the best way to describe how the Fourier Series turns into the Fourier Transform, but you will get to it soon. In brief words, the Fourier Series makes a discrete approximation of a continuous function with finite number of cosines and sines, paired with finite number of frequencies. The Continuous Fourier Transform is its generalisation, making a "perfect" translation from our function based on an infinitely large continuous time domain to a function based on an infinitely large and continuous frequency domain. So we are expressing the input function with infinitely many sines and cosines paired with infinitely many frequencies. So let's write up again our Fourier Series in its Complex Exponential form and take its limit as $P$ tends to infinity. As the period will go to infinity and stretch along the entire domain the variable $t$ customising the starting point of the period is fixed in the following:

$$\lim_{P \to \infty}f(x) = \sum_{n=-\infty}^{\infty}{\frac{\int_{-\frac{P}{2}}^{\frac{P}{2}}{f(u)e^{-i\frac{2\pi}{P}nx}du}}{P}e^{i\frac{2\pi}{P}nx}}$$

Again, I note taking the limit of this expression as $P \nearrow \infty$ has a special meaning here, and I will explain it very soon. I rewrite it a little:

$$\lim_{P \to \infty}f(x) = \sum_{n=-\infty}^{\infty}{\frac{1}{P}(\int_{-\frac{P}{2}}^{\frac{P}{2}}{f(u)e^{-i2\pi\frac{n}{P}x}du})e^{i\frac{2\pi}{P}nx}}$$

Now we introduce a new function on the domain defiend by $\frac{n}{P}$. The new function is:

$$g(\frac{n}{P}) = \int_{-\frac{P}{2}}^{\frac{P}{2}}{f(x)e^{-i2\pi\frac{n}{P} x}dx}$$

So if we substitute this function into the original equation:

$$\lim_{P \to \infty}f(x) = \sum_{n=-\infty}^{\infty}{\frac{1}{P}g(\frac{n}{P})e^{i\frac{2\pi}{P}nx}}$$

And here comes the real deal. We need to define what we actually mean by taking the limit as $P \nearrow \infty$. You need to understand that $n$ has a range of $[-\infty,\infty]$, meaning the absolute value of its upper and lower bound, besides being the same, tends to $\infty$. So let it be $N$, and so $N \nearrow \infty$ while at the same time $P \nearrow \infty$. You need to realise that incrementing $N$ will cause the number of used frequencies to get larger and larger as the width of the spectrum also increases, while incrementing $P$ will cause the frequencies to get closer and closer to each other and shrinking the spectrum. We know 2 things: the larger $N$ is the finer our approximation is on a given period. But we also know that the period should be infinitely large. So if we want to approximate our aperiodic signal on a larger and larger domain, and at the same time we aim at getting more and more precise, we will need to increment our upper bound $N$ at a greater pace than our period $P$. If $N$ and $P$ tend to $\infty$ in such a way, you will get a spectrum of discrete frequencies which gets larger and wider, while at the same time also gets denser and denser as they get closer to each other due to $P \nearrow \infty$. For example, if you increase your period by mulitplying it by a whole multiplier $k$, then you should increase the upper bound $N$ by multiplying it by a whole multiple of $k$ so that it increases at a greater pace: this will keep the spectrum widening, since if you just multiplied both quantities by the same mulitplier your spectrum would maintain a constant width since $\frac{n}{P}$ would be constant and so the approximation would not get more precise as $P \nearrow \infty$. So if a Fourier Series approximates a given period with a given precision, such icrement would cause it to approximate a larger period with a greter precision. If you are incrementing these 2 in exactly this way you will get an ever increasing, widening and densifying range of frequencies used. If this tendency is pushed to infinity we shall get an infinitely large and continuous frequency domain, and we can clearly state what limit this sum is converging to. Please realise that our approximation of the input function gets more and more precise as our frequency range gets larger, wider and denser. You need to look at the multiplier of $\frac{1}{P}$ and look at the quantity $\frac{n}{P}$ inside our new function. We also note that $n \in \mathbb{Z}$, so it is always a whole multiple of $\frac{1}{P}$. Finally, we note that as $P \nearrow \infty$ $\frac{1}{P} \nearrow 0$, and we also note that in case $P$ and $N$ tend to $\infty$ in the abovementioned manner, then what we have on table is a Riemann sum approximation, not on a fixed interval, but from $-\infty$ to $\infty$. If you take literally any kind of function, take its integral between $-\infty$ and $\infty$, and write it back to its Riemann form, you get something similar to our rearranged Fourier Series. This is how a Riemann works right? It sort of adds up rectangles, whose width is the "step size", here $\frac{1}{P}$. The height of the rectangle is given by a function - here $g$ - evaluated always at a point which is a single step size larger than the point at which it was evaluated the last time: here the point which is always step size larger is $\frac{n}{P}$. Tha fact that in our context $g$ is also multiplied by sines and cosines does nto mke a difference, it could also be made part of our function if we wanted. If the step size gets smaller and smaller, our approximation gets more and more precise. But if you are approximating on the entire domain, you not just have a step size tending to zero, but you also have a sum index running from $-\infty$ to $\infty$. So if we take $P \nearrow \infty$ in exactly this sense, we can happily rewrite it in integral form, making the right changes.

One more rewrite before turning it into an integral:

$$\lim_{P \to \infty}f(x) = \sum_{n=-\infty}^{\infty}{\Delta g(n\Delta)e^{i2\pi n\Delta x}}$$

Let $\xi = \frac{n}{P}$ (this will be the dummy variable of our integral), and note that $\frac{1}{P}$ will become $d\xi$ as our Riemann turns into an integral by taking its limit, since that is the step size of $\frac{n}{P}$:

$$\int_{-\infty}^{\infty}{g(\xi)e^{i2\pi\xi x}d\xi}$$

And taking the limit affects $g(\xi)$ as well (since $P \nearrow \infty$):

$$g(\xi) = \int_{-\infty}^{\infty}{f(x)e^{-i2\pi\xi x}dx}$$

So we eventually obtain:

$$f(x) = \int_{-\infty}^{\infty}{(\int_{-\infty}^{\infty}{f(u)e^{-i2\pi\xi x}du})e^{i2\pi\xi x}d\xi}$$

That whole integral is what produces our input function $f(x)$, and is called the Synthesis/Inverse Fourier Transform and formally speaking $g(\xi)$ is called our (Forward or Analysis) Fourier Transform. It is called a transform because we have sort of transformed our function of time to a function of frequency.

There is also another form of the Fourier Transform, which is just an algebraic rearrangement of this one. We know that $\omega = \frac{2\pi}{P}$ and $\omega_0$ is our fundamental/base frequency. So we can rewrite our Fourier Transform - and literally any of the above equations substituting $\omega$ - as:

$$ \lim_{P \to \infty}f(x) = \sum_{n=-\infty}^{\infty}{\frac{1}{P}(\int_{-\frac{P}{2}}^{\frac{P}{2}}{f(u)e^{-in\omega_0x}du})e^{i\omega_0nx}} $$

Now, our Forward Fourier Transform or analysis equation will be a function of $n\omega_0$:

$$ g(n\omega_0) = \int_{-\frac{P}{2}}^{\frac{P}{2}}{f(x)e^{-in\omega_0x}dx} $$

And here it is the quantity $n\omega_0$ which gets continuous since as $P \nearrow \infty$ $\omega_0 \nearrow 0$ because $\frac{2\pi}{P}=\omega$ and $n \in [-\infty,\infty]$, and so this quantity becomes $\omega$:

$$ g(\omega) = \int_{-\infty}^{\infty}{f(x)e^{-i\omega x}dx} $$

We also know that $\frac{1}{P} = \frac{\omega_0}{2\pi}$, so that will change in the inverse Riemann, and since the step size of $n\omega_0$ is $\omega_0$, that will become $d\omega$, and we will be left with an additional multiplier of $\frac{1}{2\pi}$:

$$f(x) = \frac{1}{2\pi}\int_{n=-\infty}^{\infty}{g(\omega)e^{iwx}d\omega}$$

And so back to its full resolution:

$$f(x) = \frac{1}{2\pi}\int_{n=-\infty}^{\infty}{(\int_{-\infty}^{\infty}{f(u)e^{-i\omega x}du})e^{iwx}d\omega}$$