Mailing List Archive

5.14.0-RC1 test failures when CFLAGS is set
Hi,

we're seeing a few test failures with 5.14.0-RC1 on Debian unstable/i386.

cpan/Module-Build/t/xs.........................................Not a CODE reference at /home/niko/tmp/perl-5.14.0-RC1/cpan/Module-Build/../../lib/DynaLoader.pm line 213.
END failed--call queue aborted at /home/niko/tmp/perl-5.14.0-RC1/cpan/Module-Build/MB-OzogcClg/Simple/blib/lib/Simple.pm line 213.
Compilation failed in require.
BEGIN failed--compilation aborted.
# Failed test at t/xs.t line 49.
Not a CODE reference at /home/niko/tmp/perl-5.14.0-RC1/cpan/Module-Build/../../lib/DynaLoader.pm line 213.
END failed--call queue aborted at /home/niko/tmp/perl-5.14.0-RC1/cpan/Module-Build/MB-OzogcClg/Simple/blib/lib/Simple.pm line 213.
Compilation failed in require.
BEGIN failed--compilation aborted.
FAILED at test 3

dist/ExtUtils-ParseXS/t/basic..................................# Failed test at t/basic.t line 53.
# got: 'Not a CODE reference at /home/niko/tmp/perl-5.14.0-RC1/dist/ExtUtils-ParseXS/../../lib/DynaLoader.pm line 213.
# END failed--call queue aborted at XSTest.pm line 213.
# Compilation failed in require at t/basic.t line 52.
# '
# expected: ''
FAILED at test 8
dist/ExtUtils-ParseXS/t/more...................................# Failed test at t/more.t line 60.
# got: 'Not a CODE reference at /home/niko/tmp/perl-5.14.0-RC1/dist/ExtUtils-ParseXS/../../lib/DynaLoader.pm line 213.
# END failed--call queue aborted at t/more.t line 213.
# '
# expected: ''
FAILED at test 7
dist/ExtUtils-ParseXS/t/usage..................................# Failed test at t/usage.t line 57.
# got: 'Not a CODE reference at /home/niko/tmp/perl-5.14.0-RC1/dist/ExtUtils-ParseXS/../../lib/DynaLoader.pm line 213.
# END failed--call queue aborted at XSUsage.pm line 213.
# Compilation failed in require at t/usage.t line 56.
# '
# expected: ''

Failed 4 tests out of 2037, 99.80% okay.
../cpan/Module-Build/t/xs.t
../dist/ExtUtils-ParseXS/t/basic.t
../dist/ExtUtils-ParseXS/t/more.t
../dist/ExtUtils-ParseXS/t/usage.t

What's happening seems to be that when CFLAGS is set, it overrides
other flags in ExtUtils::CBuilder. One of these, -D_FILE_OFFSET_BITS=64,
is apparently crucial on this platform for some reason.

Without CFLAGS, all goes fine:

# unset CFLAGS; ./perl harness -v ../dist/ExtUtils-ParseXS/t/basic.t
../dist/ExtUtils-ParseXS/t/basic.t ..
1..10
ok 1 - require ExtUtils::ParseXS;
ok 2 - Test that output contains some text
ok 3 - Create an output file
cc -I/home/niko/tmp/perl-5.14.0-RC1 -fPIC -c -D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -o XSTest.o XSTest.c
ok 4
ok 5 - Make sure XSTest.o exists
cc -shared -O2 -L/usr/local/lib -fstack-protector -o XSTest.so XSTest.o
ok 6
ok 7 - Make sure XSTest.so exists
ok 8
ok 9
ok 10
ok
All tests successful.

With CFLAGS set:

# CFLAGS="-g -O2" ./perl harness -v ../dist/ExtUtils-ParseXS/t/basic.t
../dist/ExtUtils-ParseXS/t/basic.t ..
1..10
ok 1 - require ExtUtils::ParseXS;
ok 2 - Test that output contains some text
ok 3 - Create an output file
cc -I/home/niko/tmp/perl-5.14.0-RC1 -fPIC -c -g -O2 -O2 -o XSTest.o XSTest.c
ok 4
ok 5 - Make sure XSTest.o exists
cc -shared -O2 -L/usr/local/lib -fstack-protector -o XSTest.so XSTest.o
ok 6
ok 7 - Make sure XSTest.so exists

not ok 8
# Failed test at t/basic.t line 53.
# got: 'Not a CODE reference at /home/niko/tmp/perl-5.14.0-RC1/dist/ExtUtils-ParseXS/../../lib/DynaLoader.pm line 213.
# END failed--call queue aborted at XSTest.pm line 213.
# Compilation failed in require at t/basic.t line 52.
# '
# expected: ''
ok 9
ok 10
# Looks like you failed 1 test of 10.
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/10 subtests


Bisecting shows the crucial thing that breaks this is
-D_FILE_OFFSET_BITS=64 missing. Don't know why.

It looks like this is a change in ExtUtils::CBuilder, maybe
https://github.com/dagolden/extutils-cbuilder/commit/e653d24a

Somewhat surprisingly, while I see the same loss of gcc options
when building ExtUtils::ParseXS from CPAN with Perl 5.12.3 and
ExtUtils::CBuilder 0.280202, it doesn't cause the test failures there.

I doubt this is a blocker for 5.14.0, but that's not my call of course.
I'm able to reproduce it with perl-5.14.0-RC1.tar.bz2 and
export CFLAGS="-g -O2"
sh Configure -des -Dusethreads -Dprefix=/opt/p5.14 && make && make test

( A side track is that a tool used for building all Debian packages,
dpkg-buildpackage, sets CFLAGS to "-g -O2" by default for all builds,
which is why I noticed this while packaging 5.14.0-RC1.)

Summary of my perl5 (revision 5 version 14 subversion 0) configuration:

Platform:
osname=linux, osvers=2.6.38-2-amd64, archname=i686-linux-thread-multi
uname='linux madeleine 2.6.38-2-amd64 #1 smp sat apr 23 18:47:49 utc 2011 i686 gnulinux '
config_args='-des -Dusethreads -Dprefix=/opt/p5.14'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=undef, use64bitall=undef, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2',
cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
ccversion='', gccversion='4.5.2', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=4, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
libpth=/usr/local/lib /lib/../lib /usr/lib/../lib /lib /usr/lib /usr/lib64
libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc -lgdbm_compat
perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
libc=/lib/libc-2.11.2.so, so=so, useshrplib=false, libperl=libperl.a
gnulibc_version='2.11.2'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector'


Characteristics of this binary (from libperl):
Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV
PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
PERL_PRESERVE_IVUV USE_ITHREADS USE_LARGE_FILES
USE_PERLIO USE_PERL_ATOF USE_REENTRANT_API
Locally applied patches:
RC1
Built under linux
Compiled at Apr 27 2011 21:01:16
@INC:
lib
/opt/p5.14/lib/perl5/site_perl/5.14.0/i686-linux-thread-multi
/opt/p5.14/lib/perl5/site_perl/5.14.0
/opt/p5.14/lib/perl5/5.14.0/i686-linux-thread-multi
/opt/p5.14/lib/perl5/5.14.0


Thanks everybody for your work on Perl,
--
Niko Tyni ntyni@debian.org
Re: 5.14.0-RC1 test failures when CFLAGS is set [ In reply to ]
On Wed, Apr 27, 2011 at 4:39 PM, Niko Tyni <ntyni@debian.org> wrote:

> Failed 4 tests out of 2037, 99.80% okay.
>        ../cpan/Module-Build/t/xs.t
>        ../dist/ExtUtils-ParseXS/t/basic.t
>        ../dist/ExtUtils-ParseXS/t/more.t
>        ../dist/ExtUtils-ParseXS/t/usage.t
>
> What's happening seems to be that when CFLAGS is set, it overrides
> other flags in ExtUtils::CBuilder. One of these, -D_FILE_OFFSET_BITS=64,
> is apparently crucial on this platform for some reason.

> Bisecting shows the crucial thing that breaks this is
> -D_FILE_OFFSET_BITS=64 missing. Don't know why.

> It looks like this is a change in ExtUtils::CBuilder, maybe
>  https://github.com/dagolden/extutils-cbuilder/commit/e653d24a

I doubt there's anything crucial about the particular flag, but rather
it's the fact that you're building extensions using flags that give
you code that is binary incompatible with the perl binary it's being
built against. It looks like your expectation is that you can add an
optimization flag by specifying only that flag in CFLAGS. Sounds
reasonable, but clearly what EU::CB is doing is dropping everything it
got from $Config{ccflags} and replacing rather than supplementing it
with what you gave it in CFLAGS.

With options like -D_LARGEFILE_SOURCE and -D_FILE_OFFSET_BITS=64' used
to build Perl but dropped when testing extension building, you could
be getting a different and incompatible stat structure or other binary
incompatible differences between the extension and the Perl core.

It seems like the principle of least surprise might be to make EU::CB
take CFLAGS as a supplement to $Config{ccflags} rather than as a
replacement. It would still be possible to create binary
incompatibilities, but perhaps less likely.