AI’s rise has triggered a surge in experimentation with numeric formats—the digital schemes computers use to represent numbers. Reducing the number of bits in these representations can cut energy use and speed up computation. IEEE Spectrum tells that for AI workloads, this trade-off often works: neural networks tolerate lower precision because they rely on statistical averaging rather than exact values. Formats such as 16-bit or 8-bit representations, for example, deliver performance gains with tolerable accuracy loss in training and inference.
Scientific computing tells a different story. Fields such as computational physics, fluid dynamics, molecular modeling, and engineering simulation depend on solving systems of equations that model real-world behavior. Here, small numeric errors don’t fade into averages; they propagate and can distort results. Simulations need high dynamic range and precise representation of both very large and very small values to maintain fidelity. Standard 64-bit floating-point formats (such as IEEE 754 double precision) evolved to meet these needs, but recent low-bit formats in AI lack equivalent range and accuracy.
Laslo Hunhold, an AI engineer at Openchip, argues that the explosion of AI-driven number formats has exposed limitations in traditional designs. Many emerging formats prioritize density around values common in neural networks, which doesn’t align with the value distributions typical in scientific problems. Posit formats, for instance, cluster representable values tightly around one but thin out at extremes, a mismatch for many physical simulations. In response, Hunhold is developing the takum number format. Its design seeks to preserve dynamic range even when reducing bit counts, making it better suited for scientific applications than AI-centric alternatives.
The conversation underscores a broader point: innovations optimized for AI don’t automatically translate to other domains. If scientific computing is to benefit from advances in numeric representation, its unique accuracy and range requirements must inform new formats and hardware support.