Re: [GP/PARI-2.1.0] arithmetic weirdness

[Bill Allombert <allomber@math.u-bordeaux.fr>]

On Mon, Apr 02, 2001 at 09:15:16AM +0200, Philippe Elbaz-Vincent wrote:
> (09:30) gp > .1
> time = 0 ms.
> %25 = 0.09999999999999999999999999999

I feel it is indeed a bug in PARI :

consider the following computation:

? (2^96\10)/2^96+0.
%10 = 0.09999999999999999999999999999
? (2^96\/10)/2^96+0.
%11 = 0.1000000000000000000000000000

The second result seem more accurate, since we
use the smallest residue.

We can check it:

? a=abs((2^96\10)/2^96-1/10)
%19 = 3/396140812571321687967719751680
? b=abs((2^96\/10)/2^96-1/10)
%20 = 1/198070406285660843983859875840
? a>b
%21 = 1

Internally :

? (2^96\10);
? \x
[&=080de15c] INT(lg=5,CLONE):03000005  (+,lgef=5):40000005  19999999  99999999  99999999
? (2^96\/10);
? \x
[&=080de1ec] INT(lg=5,CLONE):03000005  (+,lgef=5):40000005  19999999  99999999  9999999a
?

(the 0x9 has been rounded up to 0xa, which is better since next hexa is a 9)

So the best representation of .1 is the second one, that lead
to the expected result.

However this bug lie deep in the pari parser.
aaaa.bbbb is read as:

A=aaaa; B=bbbb; n=<number of digit of B>;
result: A+B/10.^n

It is easy to fix but the main problem is that

? 1/10.
%1 = 0.09999999999999999999999999999

Does other systems (outside those using base ten numbers) have this
problem ?

(fix:compute p=lgefint(B);
then result is A+(2^p*B\/10^n)/2^p

Bill.