John Gustafson developed the new UNUM and POSIT formats for floating-point numbers which mathematically and computationally appear to be very attractive. Background can be found on posithub.org and in his wonderful book entitled The End of Error, UNUM Computing.
John and I have been discussing applications of this to grand challenge compute problems like those encountered by the SKA radio telescope. In outline, the idea is to compress floating-point data to reduce the required memory bandwidth. For example, one could contemplate storing numbers with lower precision in memory, yet performing computations with higher precision before sending data back to memory.
I gave a keynote at the CONGA conference in March 2019 explaining the challenges and the ideas. The slides and a video of my presentation are online.
At the conference, I mentioned the importance of not looking at this problem one-float-at-a-time but instead considering the compression and decompression of arrays of floating-point numbers. I learned at the conference that there is very interesting work being done by Milan Klower in the Physics Department at Oxford (one feels somewhat embarrassed to discover such things 5000 miles away from our common department). Milan looks at the information content of arrays before using them in computations. Perhaps this is the way forward.
There are steep obstacles in creating good software for decompression of data in CPU caches because few instructions and even fewer API’s are available to manage data while controlling its write-back to RAM.
