Different behavior in library mode (PARI) vs. in GP (when using Z_issmooth)

[Daniel Berger <Daniel.Berger@hs-rm.de>]

Hello,

a simplified version of what I am doing is the following code using PARI:

```C
#include <pari/pari.h>
GEN
function(ulong B, GEN d, ulong m)
{
    GEN I, e, e_, ret = NULL;
    ulong i;
    pari_sp ltop = avma;
    I = const_vecsmall(m+1,1);
    e = quadunit0(d,-1);
    if (gsigne(gnorm(e)) == -1) e = gsqr(e);
    e_ = e
    for (i = 1; i < lg(I); i++)
    {
        if(!gel(I,i)) continue;
        if(Z_issmooth(gel(e,3),B))
        {
            if (ret == NULL) ret = 
mkvec(diviiexact(subii(gel(e,2),gen_1),gen_2));
            else ret = 
vec_append(ret,diviiexact(subii(gel(e,2),gen_1),gen_2));
        }
        else for (j = i; j < lg(I); j += i= gel(I,j) = 0;
        e = gmul(e,e_);
    }
    //copy ret to a clean place in memory
    return gerepileupto(ltop,ret);
}
```

Now when i run this code as part of compiled C code, it works fine and 
does what it's supposed to do. For example `function(10, 2, 3)` returns 
`[1, 8, 49]`

If I however compile this as part of a shared library and install it for 
use in gp, it doesn't. It appears to me that `Z_issmooth` always returns 
1, for example `function(10, 2, 3)` now returns `[1, 8, 49, 288]`. I 
tried testing `Z_issmooth` separately in gp by installing it with 
`install(Z_issmooth,"GU",smoothtest,"libpari.so")` and for example 
`smoothtest(408,10)` (should return 0) but it breaks with a segmentation 
fault.

Any insights as to what I might be doing wrong are appreciated.

Kind Regards,

Daniel