Introduction to APMA
Complex Variables
- Introduction
- Real numbers
- Complex numbers
- Polar representation
- Riemann's sphere
- Topology
- Integration
- Differentiation
- Conformal maps
- Reduced Row-Echelon Form
- Equation A x = b
- Sensitivity of solutions
- Linear independence
- Plane transformations
- Space transformations
- Linear transformations
- Affine maps
- Exercises
- Answers
Series and Limits
- Introduction
- Definition of derivative
- Geometric meaning
- Properties Matrix operators
- Determinants
- Cofactors
- Cramer's rule
- Equivalent matrices
- Elimination: A = L U
- PLU factorization
- Reflection
- Givens rotation
- Special matrices
- Exercises
- Answers
Differentiation
- Introduction
- Definition of derivative
- Geometric meaning
- Properties Compositions
- Derivative of inverse
- Power rule
- Right and Left-hand derivatives
- Higher order derivatives
- Infinitesimals
- Differentials
- Extrema of a real-valued functions
- Fermat's and Rolle's theorems
- Lagrange theorem
- de l'Hospital's rule
- Dual transformations
- Direct sums
- Quotient spaces
- Rank
- Solving A x = b
- Exercises
- Answers
Integration
- Introduction
- Riemann integral
- Area and integral
- Properties of integration
- Primitive functions
- Fundamental theorem of calculus
- Integration by parts
- Indefinite integrals
- Improper integrals
- Improper integrals II
- Mean value theorems
- Double integral
- Polar coordinates
- Self-adjoint operators
Vector Calculus
- Introduction
- Vector fields
- Paths
- Line integrals
- Answers
- Inner product
- Norm and distance
- Matrix norms
- Dual norms
- Dual transformations
- Examples of transformations
Tensors
- Introduction
- LU-decomposition
- QR-decomposition
- Cholesky decomposition
- Schur decomposition
- Positive matrices
- Roots
- Polar factorization
- Spectral decomposition
- Singular values
- SVD <
- Pseudoinverse
- Exercises
- Answers
Calculus of Variations
- Introduction
- GPS problem
- Poisson equation
- Graph theory
- Error correcting codes
- Electric circuits
- Markov chains
- Cryptography
- Wave-length transfer matrix
- Computer graphics
- Linear Programming
- Hill's determinant
- Fibonacci matrices
- Discrete dynamic systems
- Discrete Fourier transform
- Fast Fourier transform
- Curve fitting
Classical mechanics
- Introduction
- Similar matrices
- Diagonalization
- Sylvester formula
- The Resolvent method
- Polynomial interpolation
- Positive matrices
- Roots <
- Pseudoinverse
- Exercises
- Answers
Quantum mechanics
- Introduction
- Circles along curves
- TNB frames
- Tensors
- Tensors in ℝ³
- Tensors & Mechanics
- Differential forms
- Calculus
- Vector Representations
- Matrix Representations
- Change of Basis
- Orthonormal Diagonalization
- Generalized Inverse
- Differential forms
Preliminaries
- Complex Number Operations
- Sets
- Polynomials
- Polynomials and Matrices
- Computer solves Systems of Linear Equations
- Location of Eigenvalues
- Power Method
- Iterative Method
- Similarity and Diagonalization
Glossary
Reference
This Book is licensed under Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License
Rational Numbers
There are two main types of rational numbers: proper and improper fractions. In addition to proper and improper fractions, there are also mixed numbers, which are a combination of a whole number and a proper fraction (such as 1 ½). A proper fraction is a rational number where the numerator is less than the denominator. Some examples of proper fractions include ½, ¾. An improper fraction, on the other hand, is a rational number where the numerator is greater than or equal to the denominator.
The system of rational numbers ℚ is ordered, i.e. if we have two different numbers 𝑎 and b of this system, one of them is greater than the other. Also, if 𝑎 > b and b > c, then 𝑎 > c, when 𝑎, b, and c are numbers of the system.
Further, if two different rational numbers 𝑎 and b are given, we can always find another rational number greater than the one and less than another. It follows from this that between any two not equal rationals there are an infinite number of rational numbers.
Consider the rational numbers \[ x = \frac{1}{3} , \qquad y = \frac{2}{6} . \] Mathematically \[ x = y = \frac{1}{3} = \frac{2}{6} . \] Their decimal representations are infinite: \[ x = y = 0.3333333\ldots . \] Although the decimal representations contain infinitely many digits, they represent exactly the same rational number. Thus, equality is determined by the value represented by the entire infinite expansion, not by comparing a finite number of displayed digits.
A more interesting example is \[ x = 0.9999999\ldots , \qquad y = 1.0000000\ldots . \] These appear different, but they are exactly equal: 0.999999… = 1.0. Indeed, \[ 10\,x = 9.9999999999\ldots \] and therefore \[ 10\,x - x = 9.00000\ldots , \] so 9x = 9.0, which gives \[ x = 1. \] Consequently, two distinct-looking infinite decimal representations can denote the same rational number.
Comparing infinite representations
Suppose two rational numbers are represented by infinite decimal expansions \[ x = 0.x_1 x_2 x_3 \ldots \qquad\mbox{and} \qquad y = 0.y_1 y_2 y_3 \ldots . \] If they are not equal, their ordering can be determined by the first position at which their digits differ. For example, \[ x = 0.31415926\ldots \qquad y=0.31416000\ldots . \] The first five digits are identical \[ 0.3141\underbrace{5}_{x} \qquad\quad 0.3141\underbrace{6}_{y} . \] Since 5 < 6, we conclude that x < y. This works because the contribution of the first differing digit is larger than the maximum possible contribution of all subsequent digits.However, the equality case is more subtle. If two infinite representations agree at every position, they are obviously equal. But they can also be equal when they differ syntactically, because of the familiar phenomenon \[ 0.2500000\ldots = 0.2499999\ldots . \] Thus, lexical comparison of infinite digit strings is not by itself a reliable definition of equality.
Exact rational representation
For rational numbers, the cleanest approach is to represent a number as a pair of integers \[ \frac{p}{q} , \qquad q \ne 0 , \] rather than as a floating-point number. For example, \[ \frac{2}{6} \qquad \mbox{and} \qquad \frac{1}{3} \] can be compared using cross multiplication: \[ \frac{2}{6} = \frac{1}{3} \quad \iff \quad 2 \cdot 3 = 1 \cdot 6 . \] More generally, \[ \frac{a}{b} = \frac{c}{d} \quad \iff \quad a\,d = b\,c , \] provided b, d ≠ 0. Similarly, \[ \frac{a}{b} < \frac{c}{d} \] can be determined by comparing ad and bc, after taking the signs of the denominators into account.Why this matters for floating-point numbers
A machine floating-point number is not an infinite float in the mathematical sense. It normally has a finite number of bits. Consequently, a value such as \[ \frac{1}{3} \] is generally stored as an approximation: \[ fl\left( \frac{1}{3} \right) = 0.3333333333333333\ldots \] with only finitely many significant binary digits actually stored. This creates an important distinction: \[ \fbox{\text exact rational equality} \] versus \[ \fbox{\mbox{equality of floating-point approximations}} . \] For example, in an exact rational system, \[ \frac{1}{3} + \frac{1}{3} + \frac{1}{3} = 1. \] But if 1/3 is represented by a finite-precision floating-point value f, then computing \[ f +f +f \] may produce a value that is extremely close to 1, but not necessarily exactly equal to the floating-point representation of 1.
Therefore, for exact rational arithmetic, equality should be defined mathematically—for example by reducing fractions or cross multiplication. For approximate floating-point arithmetic, equality often needs to be defined using a tolerance, such as \[ | x-y | < \varepsilon , \] although the appropriate comparison depends on the numerical problem.
In summary: infinite decimal representations provide a conceptual way of representing rationals, but equality should be based on the mathematical value they denote, not merely on their textual representations. For exact computation, a canonical representation such as a reduced fraction qp is preferable; for finite floating-point representations, one must explicitly account for rounding and approximation. ■
The Greeks had already encountered the irrational. The discovery that the diagonal of a unit square has length (\sqrt{2}) revealed something deeply unsettling: not every magnitude could be expressed as a ratio of integers. The Pythagorean dream of a world governed entirely by rational proportions had been broken. Yet the irrational numbers were not immediately admitted as full algebraic citizens. They were treated primarily as geometrical magnitudes—lengths, ratios, and points on a line.
For centuries, mathematics lived comfortably with this ambiguity. One could draw the diagonal of a square and say that its length was (\sqrt{2}), even if one could not construct (√{2}) as a ratio of integers. Geometry provided an intuitive home for these mysterious quantities. The number line seemed to contain them, even when arithmetic did not know quite what they were.
Then came calculus
In the seventeenth century, Newton and Leibniz developed a mathematical language capable of dealing with continuously varying quantities, limits, derivatives, and integrals. Calculus became extraordinarily powerful. It could describe motion, curves, areas, planetary trajectories, and physical laws. Yet beneath this magnificent machinery lay a remarkable conceptual gap: what exactly were the numbers over which calculus operated?
The calculus of the eighteenth and early nineteenth centuries routinely appealed to continuity and limits without possessing a completely satisfactory arithmetic construction of the continuum. Mathematicians could calculate with infinitesimal quantities, infinite series, and limits long before they had settled precisely what a real number was.
This is perhaps the most fascinating paradox: mathematics learned how to use the real line before it had rigorously explained what the real line was.
In the nineteenth century, German mathematicians such as Richard Dedekind and Georg Cantor helped bring these ancient ghosts into the light. Dedekind's construction through cuts gave a way of defining irrational numbers arithmetically: a real number could be understood as a precise division of the rational numbers into two classes. Cantor approached the continuum through sequences and the theory of sets. In different ways, both transformed the intuitive continuum into a mathematical object that could be constructed and studied.
The philosophical significance is greater than the technical achievement. The irrational number had existed for millennia as a geometrical presence without a fully satisfactory algebraic identity. It was visible on the line but elusive in arithmetic. The nineteenth century did not so much invent these numbers as change their ontological status: the ghost acquired a body.
The history therefore contains a beautiful reversal: first came the intuition, then the calculation, and only later the foundation. The real numbers were not absent from mathematics before Dedekind and Cantor. They were already haunting it. What the nineteenth century accomplished was to catch the ghosts.
Real numbers
A real number that is not rational is called irrational. Irrational numbers include the square, cubic, and fourth root of 3 (√3, ∛3, and ∜3), π ≈ 3.141592653…, e ≈ 2.71828…, and the golden ratio (φ = (1 + √5)/2 ≈ 1.61803…). Since the set of rational numbers is countable, and the set of real numbers is uncountable, almost all real numbers are irrational. The field of rational numbers is the unique field that contains the integers, and is contained in any field containing the integers.
There are two familiar ways to represent real numbers. Geometrically, they may be pictured as the points (abstract objects of zero size) on a line, once the two reference points corresponding to 0 and 1 have been picked. For computation, however, we represent a real number as an infinite decimal, consisting of an integer part and the sign (plus or minus), followed by infinitely many decimal places:
On a straight line L, we take any segment of this line as unit of length, a definite point of the line as origin or zero point, and the directions of right and left for the positive and negative senses. To every rational number corresponds a definite point on the line. If the number is an integer, the point is obtained by taking the required number of unit segments one after the other in the proper direction. If it is a fraction ±p/q, it is obtained by dividing the unit of length into q equal parts and taking p of these to the right or left according as the sign is positive or negative. These numbers are called the measures of the corresponding segments, and the segments are said to be commensurable with the unit of length. The points corresponding to rational numbers may be called rational points.
There are, however, an infinite number of points on the line L that are not rational points. Although we may approach them as nearly as we please by choosing more and more rational points on the line, we can never quite reach them in this way. The simplest example is the case of the points coinciding with one end of the diagonal of a square, the sides of which are the unit of length, when the diagonal lies along the line L and its other end coincides with any rational point.
Thus, without considering any other case of incommensurability, we see that the line L is infinitely richer in points than the system of rational numbers.
Hence, it is clear that if we desire to follow arithmetically all the properties of the straight line, the rational numbers are insufficient, and it will be necessary to extend this system by creation of other numbers.
Recall that a rational number is any number that can be expressed as a fraction of integers and the denominator is not zero. Examples include all integers (which can be written as a fraction with a denominator of 1), terminating decimals (like 0.3), and repeating decimals (like 0.3(21) = 0.3212121…). The set of all rational numbers is denoted by ℚ. But if the decimal does not terminate or recur, the number is not the ratio of two whole numbers and is said to be irrational.
Dedekind's cut
Richard Dedekind (1831--1916) introduced his theory of real numbers using what are now called Dedekind cuts in 1872. He was a German mathematician who made important contributions to number theory, abstract algebra (particularly ring theory), and the axiomatic foundations of arithmetic.
We are going to describe Dedekind's method (1872) of introducing the irrational number, in its most general form, into analysis. Dedekind cuts, named after the German mathematician Richard Dedekind (1831--1916), but previously considered by Joseph Bertrand, are а method of constructing real numbers from rational numbers.
Let us suppose that by some method or other we have divided all the rational numbers into two nonempty classes, a lower class A and an upper class B, such that every number α of the lower class is less than every number β of the upper class. Such a division of the rational numbers into two classes is called a section.
When this division has been made, if number α belongs to the class A, every number less than α does so also; and if a number β belongs to the class B, every number greater than β does so also.
These two classes may be separates by choosing a number N, so that all rational numbers less than N belong to A, and the others belong to B. Therefore, we say that, corresponding to any given rational number there exists a section that divides the aggregate of rational numbers into two classes, such that all the numbers of the first class are less than all those of the second class. Three different cases can arise.
-
The lower class can have a greatest number and the upper class no smallest number.
This would occur, if, for example, we put the number 7 and every number less than 7 in the lower class, and if we put in the upper class all the numbers greater than 7.
-
The upper class can have a smallest number and the lower class no greatest number.
This would occur, if, for instance, we put the number 7 and all the numbers greater than 7 in the upper class, while in the lower class we put all the numbers less than 7.
It is impossible that the lower class can have a greatest number m, and the upper class a smallest number n, in the same division of the rational numbers; for between the rational numbers m and n there are rational numbers, so that our hypothesis that the two classes contain all the rational numbers is contradicted.
- The lower class can have no greatest number and the upper class no smallest rational number.
Let us consider a section generated by a root. If 𝑚 is a positive integer which is not a square number, we may conceive the rational numbers to be divided into two classes, the first of which contains all the negative numbers and also those positive numbers that the square is less than 𝑚, including zero; the second class contains all the positive numbers of which the square is greater than 𝑚. The first class contains no greatest number, and the second class contains no least number; this section is said to be related to an irrational number √𝑚, in the same way as a section is related to a rational number. This example shows that sections of the rational numbers ℚ exist, such that ℚ is divided into two classes A, B, where every number of A is less than every number of B and such that A contains no number greater than all the others, and also B contains no number less than all the others.
For example, let us arrange the positive integers and their squares in two rows, so that the squares are underneath the number to which they correspond. Since the square of a fraction in its lowest terms is a fraction whose numerator and denominator are perfect squares, we see that there are not rational numbers whose squares are 2, 3, 5, 6, 7, …,
We assume that 𝑎/b, the quotient of positive integers 𝑎 and b. For x ∈ [0,1]; define the polynomial \( \displaystyle \quad f(x) = \frac{x^n \left( 1-x \right)^n}{n!} \ \) and \( \displaystyle \quad F(x) = \pi^{2n} f(x) - \pi^{2n-2} f^{(2)} (x) + \pi^{2n-4} f^{(4)} (x) + \cdots + (-1)^n f^{(2n)} , \quad \) with the positive integer n to be specified later.
Since the polynomial n! f(x) has integer coefficients and all its monomials in x are of degree at least n (this positive integer will be specified later), every single derivative f(2k)(0) with 0 ≤ k ≤ n is an integer. Indeed, differentiating an expression of the form \( \displaystyle \quad \frac{a_j}{n!}\, x^{n+ j} , \quad \) with 𝑎j and j ≥ 0 integers, 2k-times with respect to x, the evaluation at x = 0 is nonzero only if 2k = n + j. For 2k = n + j we obtain the integer \( \displaystyle \quad \frac{a_j (2k)!}{n!} . \ \) Since f(x) = f(1 − x), we also have that every single derivative f(2k)(1) with 0 ≤ k ≤ n is an integer. A straightforward calculation yields
To every section (A, B) of the set ℚ of rational numbers, such that every number of ℚ belongs to one or other of the two classes A , B and every number in A is ordinally less than every number in B there corresponds a real number.
In case neither A contains a number which is ordinally greater than all the others in A, nor B contains a number which is ordinally less than all the others in B, the real number corresponding to the section is said to be an irrational number.
In case either A has a greatest number 𝑥, or B has a least number 𝑥, the section is said to define a real number corresponding to a rational number 𝑥. The real number that corresponds to a rational number 𝑥, though conceptually distinct from 𝑥, has no properties distinct from those of 𝑥, and is usually denoted by the same symbol.
Properties of the cut representing √7:
- No greatest element in A
- No least element in B
- 3) A ∩ B= ∅.
Cantor's Theory
We will use expressions such as "as small as we please," or "arbitrarily small," to be applied to positive numbers. These expressions denote only that the number can be so chosen as to be of lower rank than any other arbitrarily chosen positive number. Now in mathematics, it is common to use instead ε-δ approach.
The symbol |𝑥| is used to denote one of the two numbers 𝑥, −𝑥 which is positive; |𝑥| is said to be the absolute value of 𝑥.
Georg Cantor (1845–1918) was a German mathematician who revolutionized mathematics by founding set theory and proving that there are multiple, distinct sizes of infinity.
Georg Cantor developed his theory of the real numbers gradually during the early 1870s.
- 1872: Cantor published "On the Extension of a Theorem from the Theory of Trigonometric Series", where he gave one of the first rigorous constructions of the real numbers using Cauchy sequences of rational numbers. This work was published in the same year as independent constructions by Richard Dedekind (using cuts, now called Dedekind cuts) and Eduard Heine (1821–1881).
- 1874: Cantor published his groundbreaking paper proving that the set of real numbers is uncountable, while the rational numbers are countable. This established that there are different "sizes" (cardinalities) of infinity and laid the foundation for modern set theory.
Georg Cantor (1845–1918) was a German mathematician who revolutionized mathematics by founding set theory and proving that there are multiple, distinct sizes of infinity.
A sequence is usually denoted by {𝑎ₙ}. For the convergent sequence {𝑎ₙ}, if we choose n such that |𝑎n − 𝑎n+m| < ε, then for m = 1, 2, 3, …, the value of 𝑎n+m for all values of m, lies between 𝑎ₙ + ε, and 𝑎ₙ − ε, that is to say, from and after some value of n, all the elements lie between two rational numbers whose difference is arbitrary small.
A truncation (also called a convergent) is obtained by stopping after finitely many terms. For example: \begin{align*} \left[ 2 \right] &= 2, \\ \left[ 2;1 \right] &= 2 + \frac{1}{1} = 3 , \\ \left[ 2;1,1 \right] &= 2 + \dfrac{1}{1 + \frac{1}{1}} = 2 + \frac{1}{2} = \frac{5}{2} = 2.5 , \\ \left[ 2;1,1,1 \right] &= 2 + \dfrac{1}{1 + \dfrac{1}{1 + \frac{1}{1}}} = 2 + \frac{2}{3} = \frac{8}{3} \approx 2.66667 , \\ \left[ 2;1,1,1,4 \right] &= 2 + \dfrac{1}{1 + \dfrac{1}{1 + \dfrac{1}{1 + \frac{1}{4}}}} = \frac{37}{14} \approx 2.642857. \end{align*} The next few convergents are \begin{align*} \left[ 2;1,1,1,4,1 \right] &= \frac{45}{17} \approx 2.647059 , \\ \left[ 2;1,1,1,4,1,1 \right] &= \frac{82}{31} \approx 2.645161 . \end{align*} These fractions alternate around \[ \sqrt{7} \approx 2.64575131, \] and rapidly become more accurate.
A useful shortcut is that once you've found the periodic continued fraction for √7, you can generate all convergents recursively using \begin{align*} p_n &= a_n p_{n-1} + p_{n-2} , \\ q_n &= a_n q_{n-1} + q_{n-2} , \end{align*} with initial values \[ p_{-2} = 0, \quad p_{-1} = 1, \qquad q_{-2} = 1 , \quad q_{-1} = 0, \] where 𝑎ₙ = 2,1,1,1,4,1,1,1,4,… . The n-th convergent is then pₙ/qₙ.
We can also can find a rational approximation to a square root by solving numerically equation x² - n = 0.
And there is an even deeper point: there is no single privileged sequence representing √7. You can use another sequence; for instance, you can use Newton's iterative process \[ x_{n+1} = \frac{x_n^2 + 7}{2\,x_n}, \qquad x_0 = 2 \mbox{ or } 3. \] Since 2 < √7 < 3, the initial point could be either of these two integers, but not necessarily. Consequently, \[ x_n^2 \,\to\, 7. \] The sequence therefore determines the positive real number whose square is 7. That real number is what we call √7. ■
- π = 3.1415926535… e = 2.71828182845904523536029…,
- √7 = 2.64575131106459059050162…,
- √11 = 3.31662479035539984911493…
Cantor proved several remarkable facts:
- There are infinitely many real numbers.
- They are uncountable.
- Almost every real number is irrational.
- The interval [0,1] already contains uncountably many numbers (diagonal argument).
For a convergent sequence, corresponding to any arbitrary chosen positive number ε, an integer n can be found, such that from and after that value of n, the absolute difference of any two elements is less than ε. It can be shown that the sequences
Representations of rational numbers
Dedekind cuts---rigorous, but not practical. Cantor's method defines real numbers as equivalence classes of Cauchy sequences of rational numbers, treats rational and irrational numbers uniformly within this framework, and finds rational approximations by truncating or evaluating terms of these convergent sequences. Two different rational sequences belong to the same real number if the distance between their corresponding terms shrinks to zero as you go further down the sequences.
Because every real number in Cantor's theory is a sequence of rational numbers, any specific term 𝑎ₙ in that sequence serves directly as a rational approximation. By taking the explicit rule of the sequence (such as computing successive decimal expansions like \( \displaystyle \quad 3,\ \frac{31}{10}, \ \frac{157}{50} , \mbox{etc.}\quad \) for π, each stopping point gives a precise rational number that approaches the real taget.
Since Cantor did not specify how to determine rational numbers in each sequence, it is natural to use electronic resourses. To compute rational approximations to real numbers like π or functions like √x and ex, computer processors do not use infinite precision or guess. They use fast, deterministic numerical algorithms hardcoded into software libraries and executed directly by hardware.
Modern hardware handles the first guess using a dedicated CPU instruction, such as rsqrtss on x86 processors. The CPU runs 3 to 4 quick Newton-Raphson iterations using that seed guess. Because the algorithm features quadratic convergence, the number of correct decimal digits doubles with every single iteration.
For instance, you want to approximate the square root √7. Modern processors use a hybrid hardware-software approach to find square roots using Newton-Raphson Iteration combined with a fast initial lookup table. To solve y² = x, the computer sets up the root-finding function:
There is another method to determine rational approximations (convergents) with fractions that have the least possible number of digits in corresponding denominators. A continued fraction is an expression obtained by representing a number as an integer plus a fraction, whose denominator is an integer plus a fraction, and so on. They are classified into finite (for rational numbers) and infinite (for irrational numbers).
The most fundamental estimate states that the distance between the true value and the approximation is strictly less than 1 divided by the product of consecutive denominators:
While representing numbers as exact fractions (using integer pairs (p/q) provides perfect arithmetic precision, it is rarely used for general computing. Standard long division or the Babylonian method is often faster and easier to code or compute by hand. Floating-point numbers are preferred for several practical reasons:
- Fixed and Predictable Memory Size: Floating-point numbers always occupy a fixed number of bits (e.g., 32 or 64 bits). In contrast, exact fractional arithmetic causes the numerator (p) and denominator (q) to grow rapidly with each addition or multiplication (a problem known as "coefficient explosion"). This requires dynamic memory allocation, which degrades performance.
- Massive Dynamic Range: Floating-point notation can represent incredibly tiny numbers and massive scales using the exact same amount of memory. For example, a 64-bit float can scale from 10−308 up to 10308. A 64-bit fraction cannot match this breadth without losing resolution.
- Hardware Acceleration: Modern CPUs and GPUs contain dedicated physical hardware called Floating-Point Units (FPUs). These circuits process floating-point operations in a single clock cycle, making them orders of magnitude faster than software-driven fractional arithmetic.
The IEEE 754 Floating-Point Structure
ANSI/IEEE 754 is a technical standard for floating-point arithmetic, established in 1985 by the Institute of Electrical and Electronics Engineers (IEEE) and approved as an American National Standard (ANSI). It defines how computers store and handle real numbers (fractional and large numbers) using binary and decimal formats.
IEEE 754 specifies three types or Formats of floating-point numbers:
| Floating-Point Precision | Requirement |
|---|---|
Single (Fortran's REAL*4, C's float) |
Obligatory |
Double (Fortran's REAL*8, C's double) |
Ubiquitous |
Double-Extended (Fortran's REAL*10+, C's long double) |
Optional |
The IEEE 754 standard organizes a floating-point number into three structural components packed into a contiguous bit-string
In standard floating-point representation (such as the IEEE 754 standard), the significand (also called the mantissa) is 1 + fraction, meaning it is greater than or equal to 1 and strictly less than 2 (1 ≤ significand < 2). Why it is 1 + fraction (The Hidden Bit). In binary, every non-zero normalized number begins with a leading 1. Because this leading 1 is always there, computer scientists realized they do not need to waste a bit storing it in memory. It is omitted from the hardware representation
The bits you actually see stored in the "fraction" or "mantissa" field of a float represent only the fractional part after the binary point. When the CPU performs math, it automatically re-attaches that implicit, hidden to the front. Thus, the full significand becomes 1 + fraction,
The technical specifications for the two most common formats defined by the standard are outlined below:
| Technical Feature | Single Precision (binary32) |
Double Precision (binary64) |
|---|---|---|
| Total Bit Width | 32 bits | 64 bits |
| Sign Bit Allocation | 1 bit (Bit 31) | 1 bit (Bit 63) |
| Exponent Allocation | 8 bits (Bits 30–23) | 11 bits (Bits 62–52) |
| Exponent Bias | 127 | 1023 |
| Fraction/Mantissa Allocation | 23 bits (Bits 22–0) | 52 bits (Bits 51–0) |
| Effective Significant Width | 24 bits (with implicit leading 1) | 53 bits (with implicit leading 1) |
| Decimal Precision | ~7 decimal digits | ~15 to 17 decimal digits |
| Minimum Normal Value | 2-126 ≈ 1.18 × 10-38 | 2-1022 ≈ 2.22 × 10-308 |
| Maximum Normal Value | (2 - 2-23) × 2127 ≈ 3.40 × 1038 | (2 - 2-52) × 21023 ≈ 1.80 × 10308 |
- Sign (S): A single bit determining if the number is positive (0) or negative (1).
- Exponent (E): Determines the magnitude. It uses a Bias system so that negative exponents can be represented as positive integers without needing a separate sign bit.
- Significant / Mantissa (M): Represents the precision bits of the number. It assumes a hidden, implicit leading bit of 1. (e.g., 1.fraction), maximizing the available storage efficiency.
- Zero (0.0): Represented by setting all exponent bits and all fraction bits to 0. It can be positive or negative depending on the sign bit.
- Infinity (∞): Triggered by overflow or dividing by zero. Represented by setting all exponent bits to 1 and all fraction bits to 0.
- Not a Number (NaN): Triggered by invalid operations (like \( \ \sqrt{-1}\ \) or 0/0). Represented by setting all exponent bits to 1 and leaving a non-zero fraction.
Double-precision real number bit-level representation
(1 bit) Sign | (11 bits) exponent | (52 bits) fraction
| Now we sequentially multiply mantissa by 2 till the result is equal to or exceeds 1. Once this happens, we deduct 1 from the value and keep performing multiplication by 2 (binary number).
- Step 1: Since sign of √77 is positive, sign bit is 0.
-
Step 2: Let's put the irrational number √77 ≈8.774964387392122… in the format (-1)sign bit × (1+fraction) × 2power. The mantissa is supposed to be a number as 1 ≤ b < 2. As approximation of √77, we use a number slightly greater than 8.77. In order to bring it to the value within the required base interval, we need to divide it by 2³ which is the last component 2power of the format. In other words,
Significand = 8.774964 / 2³ ≈ 1.0968705
and √77 is approximated by 8.774964 = (-1)⁰ * mantissa * 2exp = 1.0968705 * 2³'.
- Step 3: exponent = 3 + 1023 = 1026 = 10000000010
-
Step 4:
Let's now represent the mantissa (1 + fractional part or significand) with fraction (0968705) in binary format. We multiply it by 2 till the number is equal to 1 or exceeds it.
We write fraction in binary format:
Calculations 0.0968705 × 2 = 0.193741 0.193741 × 2 = 0.387482 0.387482 × 2 = 0.774964 0.774964 × 2 = 1.549928 0.549928 × 2 = 1.099856 0.099856 × 2 = 0.199712 0.199712 × 2 = 0.399424 0.399424 × 2 = 0.798848 0.798848 × 2 = 1.597696 0.597696 × 2 = 1.195392 0.195392 × 2 = 0.390784 0.390784 × 2 = 0.781568 0.781568 × 2 = 1.563136 0.563136 × 2 = 1.126272 0.126272 × 2 = 0.252544 0.252544 × 2 = 0.505088 0.505088 × 2 = 1.010176 0.010176 × 2 = 0.020352 0.020352 × 2 = 0.040704 0.040704 × 2 = 0.081408 0.081408 × 2 = 0.162816 0.162816 × 2 = 0.325632 0.325632 × 2 = 0.651264 0.651264 × 2 = 1.302528 0.302528 × 2 = 0.605056 0.605056 × 2 = 1.210112 0.210112 × 2 = 0.420224 0.420224 × 2 = 0.840448 0.840448 × 2 = 1.680896 0.680896 × 2 = 1.361792 0.361792 × 2 = 0.723584 0.723584 × 2 = 1.447168 0.447168 × 2 = 0.894336 0.894336 × 2 = 1.788672 0.788672 × 2 = 1.577344 0.577344 × 2 = 1.154688 0.154688 × 2 = 0.309376 0.309376 × 2 = 0.618752 0.618752 × 2 = 1.237504 0.237504 × 2 = 0.475008 0.475008 × 2 = 0.950016 0.950016 × 2 = 1.900032 0.900032 × 2 = 1.800064 0.800064 × 2 = 1.600128 0.600128 × 2 = 1.200256 0.200256 × 2 = 0.400512 0.400512 × 2 = 0.801024 0.801024 × 2 = 1.602048 0.602048 × 2 = 1.204096 0.204096 × 2 = 0.408192 0.408192 × 2 = 0.816384 0.816384 × 2 = 1.632768 Total of Steps: 52
Properties of real numbers
We are going to show that the set ℝ of real numbers, defined in Dedekind's or Cantor's manner, can be ordered so that every real number has a definite rank in the set ℝ, i.e., of any two real numbers it is determinate which has the higher and which the lower rank. Let us denote by n, m any two real numbers, and let the sections by which they are defined be denoted by (A, B), (A₁, B₁), respectively. The following cases may arise:
- If (A, B) and (A₁, B₁) are identical, that is, if every number in A is also in A₁, and every number in B is also in B₁, the two numbers n, m are identical; thus, n ≡ m.
- Suppose that there is one rational number q ≡ q₁, which is contained in A, but not in A₁: it is consequently contained in B₁. All the numbers in A₁ are less than q₁, and hence all the numbers in A₁ are in A. Since q is the only number in A that is contained in B₁, it follows that q is greater than all the other numbers in A; and thus the number n defined by section (A, B) is a number corresponding to the rational number q or q₁. All the elements of A₁ are contained in A, and are less than q₁; all the numbers in B₁ except q₁, are greater than q₁, for if not they would be contained in A: hence, the section (A₁, B₁) defined the real number m ≡ n, corresponding to the rational number m ≡ n. The two sections are essentially identical, the only difference being that the rational number q ≡ q₁ is regarded as belonging to the first class in one section and to the second class in the other section.
- If there are two different numbers belonging to A that also belong to B₁, there are an infinite number of other numbers which have the same property, since unlimited number of rational numbers can be found that lie between two given rational numbers. In this case, we define the real number n or (A, B) to be greater in the ordinary sense of the term, than q₁ or (A₁, B₁) agreeably with the definition already given for the rational numbers.
To derive this polynomial equation, we observe \[ x^2 = m + n + 2\,\sqrt{m\,n} \qquad \Longrightarrow \qquad \left( x^2 - m - n \right)^2 = 4\,m\,n . \] The last equation is equivalent to the polynomial equation \[ P(x) = x^4 - 2\left( m+n \right) x^2 + \left( m-n \right)^2 = 0 . \] Thus, 𝑥 is the unique positive root of the polynomial equation P(𝑥) = 0. In our particular case, we have \[ P(x) = x^4 -36\, x^2 + 16 = 0 . \] In order to identify a positive root of polynomial equation P(𝑥) = 0, we choose rational numbers c < d such that
- P(c) < 0,
- P(d) > 0,
- P(𝑥) is strictly increasing on [c, d].
For example, 𝑚 = 7, 𝑛 = 11. There roots are √7 ≈ 2.64575, √11 ≈ 3.31662, so their sum is √7 + √11 ≈ 5.96238. The polynomial is \[ P(x) = x^4 - 36\, x^2 + 16 \] has only one positive root on interval [5. 6].
The operations between two real numbers may, in accordance with the above definition of real numbers by means of sections or Cauchy sequences, be so defined that the result of each operation corresponds to the section or Cauchy sequences of the rational numbers; thus, the arithmetical operations are reduced to operations with rational numbers.
This definition is known as the arithmetical definition of a limit, and was first given in a form substantially identical with the above, by John Wallis. The necessary and sufficient condition that a sequence 𝑥₁, 𝑥₂, 𝑥₃, … 𝑥ₙ, … of real numbers may have a limit is that, corresponding to every arbitrarily chosen positive number ε, a value of n can be found such that 𝑥ₙ − 𝑥n+1, 𝑥ₙ − 𝑥n+2, 𝑥ₙ − 𝑥n+3, … shall be all numerically less than ε.
Elementary algebra is concerned with the application of the arithmetic operations (+, − *, and ÷) to symbols representing real numbers. However, there are difficulties with decimal representation that we need to think about. The first is that two different infinite decimals can represent the same real number, for according to well-known rules, a decimal having only 9's after some place represents the same real nuber as a different decimal ending with all 0's (such decimals are called finite or terminating):
Another difficulty with infinite decimals is that it is not immediately obvious how to calculate with them. For finite decimals or ratios of integers (that could be represented by infinite decimals with periodic fractional parts) there is no problem; in this case we just follow the usual rules---add/subtract or multiply/divide starting at the right-hand end:
To get around this, instead of calculating with the infinite decimal, we use its truncations to finite decimals, viewing these as approximations to the infinite decimals. For instance, the increasing sequence of finite decimals
The decimal representation of this sequence is not as simple as it was for the sequence representing e or π where each new decimal digit is added on. The sequence representing e + π may contain changes in two decimal laces. For instance, in the fifth step of the last row, the first decimal place changes from 7 to 8. Nevertheless, as we compute more and more places, the earlier part of the decimals in this sequence ultimately does not change any more, and in this way we get the decimal expansion of a new number; we then define the sum e + π to be this number.
Computers do not actively calculate during runtime operations. Because is a universal physical constant, its value is calculated in advance to maximum precision and hard coded directly into computer libraries as a fixed, hexadecimal floating-point bit pattern.
In 64-bit double precision, π is hard coded as:
- Decimal: 3.141592653589793
- Hexadecimal IEEE 754 representation: 0x400921FB54442D18
To calculate exponential functions like ex, computers do not use Taylor series directly. Taylor series converge slowly at high numbers, requiring too many operations. Instead, systems use Remez Polynomial Approximations or CORDIC algorithms
■As this example shows, even the simplest arithmetic operations with real numbers require an understanding of sequences and their limits. So you get an answer not at once, but rather by making closer and closer approximations to it.
ℝ as a field
We discuss properties of arithmetic operations (addition, subtraction, multiplication, and division) from formal algebraic prospective.
- Closure: For any two elements 𝑎, b in semigroup S, the result of the operation, 𝑎 • b, is also an element in S.
- Associativity: For all 𝑎, b, and c in group S, the equation (𝑎 • b) • c = 𝑎 • (b • c) holds.
- Identity element: There exists an element e in S, such that for all elements 𝑎 in S, the equation e • 𝑎 = 𝑎 • e = 𝑎 holds. This identity element is usually denoted by 1.
- Commutativity: For all 𝑎, b in S, 𝑎 • b = b • 𝑎.
- Closure: For any two elements 𝑎, b in A, the result of the operation, 𝑎 + b, is also an element in A.
- Associativity: For all 𝑎, b, and c in group A, the equation (𝑎 + b) + c = 𝑎 + (b + c) holds.
- Identity element: There exists an element e in A, such that for all elements 𝑎 in A, the equation e + 𝑎 = 𝑎 + e = 𝑎 holds. This element is usually denoted by zero.
- Inverse element: For each 𝑎 in A there exists an element b in A such that 𝑎 + b = b + 𝑎 = e (= 0), where e is the identity element (zero).
- Commutativity: For all 𝑎, b in A, 𝑎 + b = b + 𝑎.
The set ℝ, equipped with the usual addition "+" and multiplication "·" (also denoted by "•"), is a field. This means it satisfies the field axioms: commutative groups under addition, commutative monoid under multiplication with multiplicative inverses for nonzero elements, and distributively of multiplication over addition.
| name | addition | multiplication | ||
|---|---|---|---|---|
| associativity: | (𝑎 + b) + c = 𝑎 + (b + c) | (𝑎 • b) • c = 𝑎 • (b • c) | ||
| commutativity: | 𝑎 + b = b + 𝑎 | 𝑎 • b = b • 𝑎 | ||
| distributivity: | 𝑎 • (b + c) = 𝑎•b + 𝑎•c | (𝑎 + b) • c = 𝑎•c + b•c | ||
| identity: | 𝑎 + 0 = 𝑎 = 0 + 𝑎 | 𝑎•1 = 1•𝑎 = 𝑎 | ||
| inverses: | 𝑎 + (−𝑎) = 0 = (−a) + 𝑎 | 𝑎•𝑎−1 = 1 = 𝑎−1•𝑎 if 𝑎 ≠ 0 |
- Apostol, T.M., Calculus, Vol. 2: Multi-Variable Calculus and Linear Algebra with Applications to Differential Equations and Probability, Wiley; 2nd edition, 1991; ISBN-13: 978-0471000075.