fix get10Byte on platforms where long double is >80 bits.#1195
fix get10Byte on platforms where long double is >80 bits.#1195katrinafyi wants to merge 1 commit intoavast:masterfrom
Conversation
for platforms with a non 80-bit "long double" type, we change get10ByteImpl to reduce to double then use the language's conversion to long double. this handles both where long double is smaller and greater than 80 bits, though only 64 bits of precision are preserved. this fixes the tests on aarch64 platforms.
PeterMatula
left a comment
There was a problem hiding this comment.
The code looks fine I guess 👍 .
But for whatever reason, CI tests refuse to run for it, and I don't even see an option to trigger them manually. I will have to look at it, and get it to work, so that we see if it works on Windows, etc. (I was a bit worried about LDBL_MANT_DIG, but it looks like it is in standard).
Looking at this whole thing again, I'm not even sure there should be systemHasLongDouble(). It would be best to avoid using it altogether, but that would require deeper investigation into what is actually happening and why.
|
Thanks for the review! These changes are certainly an ad-hoc fix, if you can further refactor the code, that would certainly be better. I think, in general, is a difficult problem to place a fixed number of bytes into a numeric type. While investigating this issue, I found the standard does not like prescribing explicit widths to its number types. For integers, this is somewhat alleviated by the intN_t types, but the situation is murkier for floating types. Indeed, |
For platforms such as aarch64 with a non 80-bit "long double" type, the tests fail since they assume an 80-bit long double type.
Specifically, the tests construct a long double with the byte pattern:
00000000000081a0fd3f000000000000. When interpreted as:We fix this by restricting systemHasLongDouble to return true only when the platform has 80-bit long double (detected by 64-bit mantissa). Then, we change the fallback get10ByteImpl to construct a correct long double by using the language's builtin float handling, instead of naively copying
This handles both where long double is smaller and greater than 80 bits, though the trip through double does limit maximum precision. Given the fallback already assumes 64-bits is acceptable, I don't think this is a big problem.
If more precision is desired, it is not too hard to write a float10 to float16 method. I found the code in the linked perl blog didn't seem to be correct, but it is possible with not too much difficulty.