Pfthb

I stumpled into a test failure caused by floating point precision and am trying to understand it.

In short: Python3 round returns a different value depending on whether the type is a float or a numpy.float64 although I thought float==double==float64 and both Python3 and NumPy should round nearest to even.

Here the example:

npVal = np.float64(435)/100
pyVal = 435/100
print(round(npVal,1)) // 4.4
print(round(pyVal,1)) // 4.3
print(round(np.float64(pyVal),1)) // 4.4
print(round(float(npVal),1)) // 4.3

I understand that 4.35 and 4.4 might not be exactly representable in double but why is numpy round differently than Python although they both use the same datatypes and specify the function similar? I used the explicit division to avoid input rounding errors.

I don't know for sure, whether the double value for 4.35 is a bit more or less, so I can't say which of those implementations is (might be?) wrong.

There is a similar question: Strange behavior of numpy.round

There it was noted, that NumPy "rounds to the nearest even value" and "behaviour changed between Python 2 and Python 3; Python 3 behaves the same as NumPy here".

So both should do the same and round to nearest even value. So if 4.35 would be an exact float, 4.4 would the correct answer and needed to be returned by both.

Calculating 435/100 in IEEE-754 basic 64-bit binary floating-point yields 4.3499999999999996447286321199499070644378662109375.

When this is rounded to the nearest decimal numeral with one digit after the decimal point, the result ought to be “4.3”. The Python rounding for this case appears to be correct.

For numpy.round, the documentation refers to numpy.around. The documentation for that says “Results may also be surprising due to … errors introduced when scaling by powers of ten.” Thus, it may be that numpy.round does not calculate the correct conversion of 4.3499999999999996447286321199499070644378662109375 to decimal but rather performs a 64-bit binary floating-point multiplication of that by 10, which yields exactly 43.5 due to floating-point rounding, and then numpy.round rounds that to 44 and formats it as “4.4”.

In summary, numpy.round is not a correct rounding routine.