## Programming with Absolute Values

Posted on 2014-02-23 16:00 in Blog • Tagged with math, programming

For what values of x is the following equation true? x = |x| (absolute value)

Most people would say, the above equation is value for x >= 0.

For what values of x is the following equation true? |x| < 0

Most people would claim there are no such values of x, and they would be correct. Except when dealing with computers.

Numbers within a computer are confined to a limited range, dependent on the number of bits in memory that has been allocated. Integers typically get 32-bits and longs get 64-bits. This allows for 4294967296 (integer) and 1.84467440737096E19 (long) possible combination of values to store numbers in. The numbers are encoded with a format known as 2's complement for addition/ subtraction simplicity within the processor, but there is a subtle problem. 4294967296 is an even number. One pattern is used to represent zero, leaving 4294967295 patterns for storing both the positive and negative values. In other words there are 2147483648 patterns for one side and 2147483647 for the other. There is not an even pairing between positive and negative.

As a direct result of the 2's complement number encoding, it is clear to see there is one more possible negative value than positive value. A positive integer maxes at 2,147,483,647 and negative values bottom out at −2,147,483,648.

What happens when we want to take the absolute value of −2,147,483,648? The is not a positive pattern that can represent the number 2,147,483,648. In some programming languages, taking the absolute value of this smallest possible value, results in getting the smallest value back. This can cause problems if your application has logic that requires a positive value and assumes Math.abs() works as expected.

Good static analysis tools like Findbugs have rules to identify a class of errors that can result from Math.abs(), but they aren't fool proof.

I advocate abandoning sized numbers where ever possible. If an application isn't directly interfacing with a network, the file system, or some other hardware component, we should stick to number classes which do not impose an arbitrary limit (like BigInteger in Java).

If we must stick with what fits into a single RAM slot, then I would argue we should arbitrarily shrink the size of MIN_VALUE by one (disallowing 0x80000000). That way Math.abs(Int.MIN_VALUE) == Int.MAX_VALUE. Conveniently, we would then have a free pattern to natively represent the value, NOT_A_NUMBER.

Continue reading