Real numbers, and the lies programming languages tell about them

LLM-authored, human-reviewed

Foundations

A real number, to a mathematician, is any value on the number line - every integer, every fraction, every irrational like π or sqrt2\\sqrt{2}, all the way out to the uncountable infinity of points between any two of them. A real number, to a computer, is different. A computer has a fixed number of bits, and those bits must stand in for something that is, by construction, larger than any finite representation can hold.

This is the gap every programming language must bridge. JavaScript and Elixir bridge it in genuinely different ways, and each choice has costs. This article explains what a real number is, why memory cannot represent it perfectly, and how the two languages’ choices affect code you write every day. It draws on Ronald Kneusel’s Math for Programming (whose opening chapter, “Computers and Numbers,” is the clearest short treatment of this material for working programmers) and on Thomas Nield’s perspective from Essential Math for Data Science on why number fluency matters more than library fluency for anyone doing real computation.

What a real number is, and why it’s a problem

Kneusel draws the important line at the start: “when it comes to implementation, computers deal primarily with two types: integers and real numbers. Integers are all the positive and negative numbers on the number line, including zero, that have no fractional or decimal part. Real numbers are all the numbers on the number line, including those between the integers.”

The difference between those two is the whole story. “Computers generally represent integers exactly,” Kneusel writes. “When the computer stores a representation of 11, that representation is 11, no more and no less.” An integer is a finite object - there are only so many of them between any two bounds - so a finite machine can hold one precisely. Real numbers are different: “because they include everything (numerically speaking). Because computers have a finite amount of memory, they can’t possibly store all real numbers and must approximate instead.” Between 0 and 1 alone there are uncountably many real numbers, more than any finite memory could enumerate. A computer does not store real numbers. It stores approximations in a format called floating point, almost everywhere according to one standard.

Keep Kneusel’s distinction as vocabulary: he reserves “real number” for the mathematical object and “floating-point number” for the in-memory representation. “For example, π is a valid real number in math. When it comes to computers, I’ll instead refer to floating-point numbers to mean real numbers as represented in the computer’s memory. A computer cannot represent π but only approximate it.” That gap - between the real number and its floating-point stand-in - explains every surprise in this article.

IEEE 754: the standard almost everyone picked

The approximation scheme nearly every language uses is IEEE 754, the floating-point standard first introduced in 1985. Kneusel calls it “the floating-point standard in use almost everywhere.” It defines several sizes of floating-point number; the two you’ll meet are binary32 (32 bits, called float in C, f32 in Rust) and binary64 (64 bits, called double in C, f64 in Rust, and just “a number” in JavaScript). Here’s the bit layout, straight from Kneusel’s explanation of the 32-bit form:

“Bit 31 is the sign bit, where 0 is positive and 1 is negative. The next eight bits (E) are the exponent stored as an unsigned 8-bit number. The remaining 23 bits (M) are the mantissa, also called the significand. The actual number is stored in scientific notation but in binary: ±1.b22b21b0b_{22}b_{21}\ldots b_{0} × 2E1272^{E-127}. There’s an implied 1 on the mantissa.”

The 64-bit form (binary64) uses the same scheme with more bits: 1 sign bit, 11 exponent bits (subtract a bias of 1023 instead of 127), and 52 mantissa bits. That “implied 1” saves space. Every non-zero binary number in scientific notation starts with 1., so the standard leaves out that leading bit and uses the space for one more bit of precision.

Kneusel’s worked example shows where the approximation enters. The C program float v = 2.718; prints 2.71799994, not 2.718. Why? “We wanted v to be exactly 2.718, but we didn’t get that. We’re victims of finite memory. With only 23 bits in the mantissa, there’s no possible way to accurately store all real numbers; therefore, 2.718 rounds to the nearest representable value. We call rounding to the nearest representable value round-off error, and it’s your constant companion when using floating-point numbers.” Keep that phrase: constant companion.

The example everyone meets: 0.1

The most famous round-off error, and the one that catches every programmer the first time, is that 0.1 cannot be represented exactly in binary floating point. Kneusel explains why: “a terminating ‘decimal’ in one base isn’t necessarily a terminating ‘decimal’ in another. For example, 0.1 terminates in decimal but becomes 0.0001100110011… in binary” - the pattern 0011 repeats forever, just as 1/3 becomes 0.333… in decimal. A finite number of mantissa bits must cut that repeating expansion off, and the stored value is the closest available one. Add 0.1 to 0.1 and you don’t get 0.2; you get the nearest representable value to 0.2. This happens in both JavaScript and Elixir:

0.1 + 0.2 === 0.30000000000000004   // true, in JavaScript
iex> 0.1 + 0.2
0.30000000000000004                 # the same result, in Elixir

The answer matches because both languages use the same standard: binary64 doubles. The surprise is not a JavaScript bug or an Elixir bug. It is IEEE 754 returning the nearest representable value.

Kneusel’s pi demonstration shows how round-off error compounds. He runs a sequence of calculations that, mathematically, should leave π equal to π. Run it once and the floating-point result is “off in the 15th decimal.” Run it five times and it’s off in the ninth. Eight times, the fourth decimal. Twelve times, the result is 5.888... instead of 3.14159.... Fourteen times, 148565.99.... “Round-off error has proven fatal.” His grimmest footnote is worth quoting: “Lives have literally been lost to round-off error - search for ‘patriot missile round-off error’ to see one such instance.” (They have. The 1991 Patriot Missile failure, where an accumulated timing error of 0.000000095 seconds per second caused the system to miss a scud, is the canonical real-world case.) Kneusel points readers to David Goldberg’s classic 1991 paper, “What Every Computer Scientist Should Know About Floating-Point Arithmetic,” still the definitive reference.

That is the background for both languages’ number-type designs. Neither solved the real-number problem - nothing finite can. They made different trade-offs about which approximations to expose and how clearly to expose them.

JavaScript: one number type, and it’s a double

JavaScript made the simplest possible choice: there is one numeric type, and it’s IEEE 754 binary64. Every number in JavaScript - 1, 3.14, Number.MAX_SAFE_INTEGER, the result of 0.1 + 0.2, a loop counter, a monetary amount - is a 64-bit floating-point double. JavaScript has no separate integer and float types in its everyday semantics; even integer literals are doubles. The comparison file’s terse summary is exactly right: 1 === 1.0; // true (single Number type).

This is a defensible choice for a language designed in ten days in 1995 to script a browser. One type means no coercion rules between integer and float, no two sets of arithmetic operators, and no choice of numeric type for the programmer. For most browser work, it is fine - you’re counting DOM nodes and computing CSS pixel values, not landing spacecraft.

The costs appear at the edges, where IEEE 754 always charges. Because integers are doubles, JavaScript integers are exact only up to Number.MAX_SAFE_INTEGER (25312^{53} - 1, or 9007199254740991). Past that, even whole numbers lose precision: 9007199254740992 + 1 === 9007199254740992 is true, because the mantissa cannot hold the 54th bit of significance. Division returns a float whether you want one or not (7 / 2 is 3.5, and there’s no built-in integer-division operator - you reach for Math.floor(7 / 2) or Math.trunc). Every real-number computation carries round-off error, silently. The type system does not tell you whether a number must be exact (an array index, a count, a money amount in cents) or is approximate (a measurement, a probability).

The Math object

JavaScript puts its non-arithmetic numeric operations on one global object, Math, a static namespace of functions and constants. It is not a constructor (you don’t new Math(); the spec gives it an internal [[Math]] slot and forbids using it as one), and every property on it is a static member. The constants are Math.PI, Math.E, Math.SQRT2, Math.LN2, Math.LN10, Math.LOG2E, Math.LOG10E, and Math.SQRT1_2 - all stored as the nearest binary64 approximation of the true value. Math.PI is 3.141592653589793, not π.

The functions fall into a few groups. Trigonometric (Math.sin, Math.cos, Math.tan, and their hyperbolic and inverse variants), exponential and logarithmic (Math.exp, Math.log, Math.log2, Math.log10, Math.pow, Math.sqrt, Math.cbrt), rounding (Math.floor, Math.ceil, Math.round, Math.trunc), and miscellany (Math.abs, Math.max, Math.min, Math.sign, Math.hypot, Math.random). Every one takes doubles and returns doubles - even Math.max of two integers gives you a double, because that’s all there is. Math.min.length is 2 because that’s the declared parameter count, even though the function is variadic and accepts any number of arguments.

Three details about Math connect directly to the real-number problem. First, everything is approximate and the object doesn’t tell you so. Math.sin(Math.PI) returns 1.2246467991473532e-16, not zero - the same floating-point anomaly Kneusel’s Elixir counterpart demonstrates with :math.sin(:math.pi()). The sine of π is zero in mathematics; in IEEE 754 it is a tiny non-zero number because Math.PI is an approximation and the sine implementation operates on that approximation. The API does not surface this; you have to know it. Second, Math.random() is the canonical example of a function that’s hard to replicate in pure JavaScript - pseudo-random number generation needs state and bit-level tricks the language doesn’t expose, so the language ships it as a built-in. Third, the rounding functions are non-obvious: Math.round(2.5) is 3 but Math.round(-2.5) is -2 (it rounds half up, toward positive infinity, not the half-to-even “banker’s rounding” that’s safer for statistical work). Math.round(0.1 + 0.2) is 0, because 0.1 + 0.2 is 0.30000000000000004, which rounds down. The approximation and the rounding interact. Your defense is to understand the representation.

The Math object admits what the single-type design means: if every number is a double, a namespace of double operations is the toolbox you need. It is a well-stocked toolbox - the trigonometric, exponential, and rounding functions cover the scientific-calculator basics - but it serves one numeric world, the approximate one, and has no door into exact integer arithmetic bigger than 2532^{53}.

Elixir: two types, and the integer story is honest

Elixir made a traditional split: integers and floats are separate types, with different representation and different guarantees. That distinction lets each type be clear about what it is.

Elixir integers are arbitrary precision. Programming Elixir states it flatly: “There is no fixed limit on the size of integers - their internal representation grows to fit their magnitude.” Elixir in Action agrees: “There’s no upper limit on an integer’s size, and you can use arbitrarily large numbers.” Learning Elixir adds the memory detail: “Big integers will start at 3 words and will grow to n words to fit.” So factorial(10000) in Elixir returns the full 35,665-digit answer, exactly, with no overflow or silent precision loss - integers are not squeezed into a fixed-width field the way JavaScript’s doubles are. Big-integer arithmetic is slower than fixed-width arithmetic, and the representation grows with the number, but the value is, as Introducing Elixir puts it, “always precise. You don’t need to worry about their values being off by just a little.”

Elixir floats are IEEE 754 binary64 double precision - the same format JavaScript uses for everything. Programming Elixir: “Floats are IEEE 754 double precision, giving them about 16 digits of accuracy and a maximum exponent of around 1030810^{308}.” Introducing Elixir elaborates: “Floats, on the other hand, cover a wide range of numbers but with limited precision. Elixir uses the 64-bit IEEE 754-1985 ‘double precision’ representation. This means that it keeps track of about 15 decimal digits, plus an exponent… because it tracks only a limited number of digits, results will vary a little more than may seem convenient, especially when you want to do comparisons.” The same language that is precise about integers says plainly that floats approximate.

Two types make operations clear about which world they use. Division shows it: the / operator always returns a float, and integer division is explicit. Learning Elixir: “the / operator will always return a floating point type. If you want an integer type back, you can use the div and rem functions.” So 7 / 2 is 3.5 (float), div(7, 2) is 3 (integer), and rem(7, 2) is 1 (integer). The boundary between exact and approximate arithmetic is visible in the source. You cannot accidentally divide two integers and get a float without writing the operator that means “give me a float.” JavaScript forces you to know that / always floats and to reach for Math.trunc when you want an integer; Elixir makes the choice part of the call.

The same visibility appears in equality. Both languages distinguish strict from loose equality, but Elixir’s distinction is only about the integer/float boundary, where the representation difference lives: 1 == 1.0 is true (numeric equality, they’re the same number) while 1 === 1.0 is false (strict type check, they’re different types). Learning Elixir: “integers are not of the same type as floating point numbers. Therefore, they are not equivalent.” In JavaScript, 1 === 1.0 is true because there is only one type. The distinction Elixir makes cannot arise. The comparison file’s observation captures the inversion: in JS, === is “the safe default” for avoiding type coercion; in Elixir, == is the safe default, and === matters when you need to tell an integer from a float.

For the operations JavaScript puts on Math, Elixir uses a few homes. The transcendental functions live in Erlang’s :math module - :math.sin, :math.cos, :math.sqrt, :math.pow, :math.pi(), :math.log, and so on. Introducing Elixir calls it “Erlang’s math module, which offers pretty much the classic set of functions supported by a scientific calculator. These functions return floating-point values.” So :math.pow(2, 16) is 65536.0 (note the .0 - float result, as advertised). Rounding on floats lives in the Float module (Float.round(3.14159, 2)3.14), while the general-purpose round/1 and trunc/1 live in Kernel and return integers from any number. Because integers are separate and exact, the Integer module provides Integer.gcd/2, Integer.mod/2, Integer.is_even/1 - operations that in JavaScript would either not exist or would be awkward double-arithmetic.

The practical advice from Introducing Elixir is direct: “If you need to keep track of money, integers are going to be a better bet. Use the smallest available unit - cents for US dollars, for instance - and remember that those cents are 1/100 of a dollar.” That’s the integer-cents pattern, and it works cleanly in Elixir because arbitrary-precision integers do not hit MAX_SAFE_INTEGER on a ledger total or introduce floating-point error when you sum cents. For full arbitrary-precision decimal arithmetic - exact base-10 fractions, the right tool for money that needs sub-cent precision - the Elixir ecosystem has the Decimal library, which represents numbers as exact decimal digits rather than binary floats. Use it when integer cents are not precise enough.

The design trade-off, in one table

JavaScript Elixir
Integer type None - integers are doubles Separate type, arbitrary precision
Float type IEEE 754 binary64 (the only type) IEEE 754 binary64
1 === 1.0 true (one type) false (different types)
7 / 2 3.5 (always float) 3.5 (always float)
Integer division Math.trunc(7 / 2)3 div(7, 2)3
Max exact integer Number.MAX_SAFE_INTEGER (25312^{53}-1) No limit (grows to fit)
0.1 + 0.2 0.30000000000000004 0.30000000000000004 (same IEEE 754)
Transcendental functions Math.* static object :math.* Erlang module
Exact decimal for money None in language (use a lib) Decimal library; integer-cents idiomatic
What signals “this is approximate” Nothing - it’s all doubles The / operator; the float type itself

Read the table one way and JavaScript looks simpler: one type, no decisions. Read it another way and Elixir looks more honest: the syntax shows the boundary between exact and approximate arithmetic, and the integer world does not silently degrade past 2532^{53}. Neither is wrong. JavaScript optimized for not making the programmer think about numeric types, the right call for a browser scripting language in 1995. Elixir optimized for letting the programmer choose exactness where it matters, the right call for a language that runs banks and telephone switches.

When it matters, and what to do

Kneusel’s framing - round-off error is “your constant companion” - is the right mental model. Floating-point arithmetic is “almost always wrong, it’s close enough to be generally useful,” as he puts it. Do not ask whether arithmetic is exact; it is not. Ask whether the approximation is close enough for the job and where it stops being close enough.

For counting, indexing, loop bounds, and any quantity that must be a whole number, use integers. In Elixir this is automatic and unlimited. In JavaScript, stay under Number.MAX_SAFE_INTEGER, or use BigInt (the 9007199254740991n literal form) for larger values - though BigInt doesn’t interoperate with Number without explicit conversion, which is the language’s way of reminding you they’re different worlds.

For money, measurements that must compare equal across runs, and anything where 0.1 + 0.2 !== 0.3 would be a bug, don’t use binary floating point. Use integer cents (or integer micro-cents), or use a decimal library (Decimal in Elixir, or a JS equivalent) that stores numbers as exact base-10 digits. Floating point is the wrong tool when exact decimal fractions matter. That is the case Kneusel makes with 0.1 and the case Introducing Elixir makes for money.

For scientific computing, graphics, statistics, probabilities, and most physical quantities where a tiny error in the 15th digit is irrelevant, use binary64 floats freely. Remember Kneusel’s compounding demonstration when you chain operations. If the calculation is sensitive, scale it so the exponent stays near zero (where binary64 has its highest precision), or reach for a higher-precision format.

For comparisons, never test floating-point values for exact equality. Compare against an epsilon - Math.abs(a - b) < 1e-9 in JS, or Elixir’s assert_in_delta in tests. The sine-of-pi anomaly (1.22e-16, not zero) is the warning case: a value that should be exactly zero is not, so an exact equality check fails.

Thomas Nield’s broader point, from the data-science side of this material, is that the library matters less than understanding what its numbers are doing. A practitioner who uses a statistics library without knowing that its inputs are approximate doubles will trust outputs that carry the approximation forward, sometimes amplified. The same applies to any programmer using Math or :math without knowing that the inputs and outputs are doubles: close enough, usually, but never exact and occasionally catastrophic. Kneusel and Nield make the same case for number fluency as a prerequisite for using numeric tools well. The languages can give you exact integers or approximate floats. You must decide which one the problem needs.

The honest summary

A real number is any value on the number line - an object so abundant that no finite memory can hold more than a vanishing fraction of them. Computers represent integers exactly (because they’re finite and countable) and approximate real numbers as IEEE 754 floating point (because they’re not). The approximation is universal - binary64 doubles are the lingua franca of scientific and application computing, and 0.1 + 0.2 produces 0.30000000000000004 in JavaScript, Elixir, Python, C, and every other language that uses the standard. Round-off error, as Kneusel warns, is a constant companion, occasionally a fatal one.

The languages differ in how much of this they expose and let you control. JavaScript collapses everything to one type - the double - and puts the transigonometric and rounding operations on a static Math object; the design is simple, but integers silently degrade past 2532^{53} and the syntax does not distinguish exact work from approximate work. Elixir keeps integers and floats as separate types, makes integers arbitrary-precision and therefore always exact, makes the / operator’s float-returning nature explicit, and reserves the :math module for transcendental functions that belong to the floating-point world. The same IEEE 754 standard underlies both; the difference is how honestly the type system reflects it.

Know what a real number is, why no finite representation can hold it, and which of your language’s numeric types is exact versus approximate. Then 0.30000000000000004 is no surprise. You know where it came from, when to use integers, when to use decimals, and when floating-point approximation is, as Kneusel says, close enough to be useful.

Where to go next

Related exercises

← Back to articles