Gerhard Niklasch on Wed, 5 Aug 1998 15:59:24 +0200 (MET DST)


[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

Fix for sign extension buglet


And another small patch for 2.0.11.beta.  Guillaume Hanrot (thanks!)
noted that under some conditions, notably on DEC alphas, one line
in the cgcd() function in arith1.c would produce a (signed) C int, of
half the size of a signed long, with the most significant bit turned on,
which would then get converted to a (negative) C long by sign extension,
although a positive value was intended.  The fix is to tell the compiler
that the thing is a C long right from the start.  Two more potential
problems of this kind are secured below;  if anybody can find any other
similar places in the code, please do let us know.

Enjoy, Gerhard

Apply with patch -p0 -l from the pari-2.0.11.beta directory  (-l because
TABs/spaces otherwise might not match).

bash$ diff -u src/basemath/arith1.c.orig src/basemath/arith1.c
--- src/basemath/arith1.c.orig  Tue Aug  4 20:02:23 1998
+++ src/basemath/arith1.c       Tue Aug  4 20:02:25 1998
@@ -921,7 +921,7 @@
   if (a>b) { a %= b; if (!a) return b; }
   else { b %= a; if (!b) return a; }
   v=vals(a|b); a>>=v; b>>=v;
-  if (a==1 || b==1) return 1<<v;
+  if (a==1 || b==1) return 1L<<v;
   if (b&1)
     return ((long)ugcd((ulong)a, (ulong)b)) << v;
   else

bash$ diff -u src/basemath/arith2.c.orig src/basemath/arith2.c
--- src/basemath/arith2.c.orig  Tue Aug  4 21:30:38 1998
+++ src/basemath/arith2.c       Tue Aug  4 21:30:42 1998
@@ -316,7 +316,7 @@
     return 16384;
   else if (size <= 512)
     return (size-16) << 10;
-  return 1<<19;		/* Rho will generally be faster above this */
+  return 1L<<19;		/* Rho will generally be faster above this */
 }
 
 /********** about to become obsolete --GN **********/

bash$ diff -u src/basemath/ifactor1.c.orig src/basemath/ifactor1.c
--- src/basemath/ifactor1.c.orig        Tue Aug  4 21:31:57 1998
+++ src/basemath/ifactor1.c     Tue Aug  4 21:31:59 1998
@@ -620,7 +620,7 @@
   else if (size < 65500)
     c0 = size<<6;		/* back to linear; ECM taking over */
   else
-    c0 = 1<<22;
+    c0 = 1L<<22;
 				/* improbable, but size _could_ be
 				   very large indeed... */
   c = c0 << 4;			/* 16 iterations per round */