Serious bug in atlas, yet no action

[Cyrus Hall]

Scott-

I wont have time to try and package 3.8.3 until this evening, but will
do so then.

In terms of the feature freeze, I have two comments.  First, it does
seem worth trying to get a non-broken version of the SSE2 version into
the repo.  The seriousness of this bug is very clear: a large number of
scientific computing tasks use atlas, and are at risk of producing
incorrect results.  This reflects poorly on Unbuntu and OSS in general.

However, if it's not desirable to ask for an exception, maybe it would
be possible to argue for the removal of the SSE2 optimized package in
Karmic.  This would not be an inconvenience for users, as the package
produces incorrect results anyway, and would in fact help them by making
sure they do not produce erroneous data.

Cheers,
Cyrus

On Wed, 2009-10-07 at 19:42 +0200, Scott Howard wrote:
> Hello all,
> 
> The reason why Ubuntu has an old version of atlas is that Debian has
> an old version of atlas. However, a week or two ago debian packaged
> the newest version of atlas into the experimental section:
> http://packages.qa.debian.org/a/atlas.html
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=382880
> 
> If it was earlier in the release cycle, it would be much simpler to
> get atlas in to Karmic. However, since we are in feature freeze with
> the release a few weeks away, we are almost bound to 3.6 since
> upstream (in this case debian) released 3.8 too recently.
> 
> Science motus - if an upgrade to 3.8 fixes this problem, do you think
> it is even worth trying to get a freeze exemption for an upgrade from
> 3.6 to 3.8? My guess is no, since several library names have changed
> and other libraries have been removed which won't play nice with other
> packages.
> 
> Upstream and Debian changelogs are below.
> 
> 
> I'll try to package 3.8 later. Cyrus, if you'd like to try packaging,
> you can get the source and package it by doing the following:
> 
> $ dget http://ftp.debian.org/debian/pool/main/a/atlas/atlas_3.8.3-2.dsc
> $ dpkg-source -x atlas_3.8.3-2.dsc
> 
> Then build it (using pbuilder, for example - see the wiki for detailed
> instructions).
> 
> 
> ATLAS 3.8.3 released 02/21/09, Changes from 3.8.2:
>    * Fixed bugs:
>      - Numerous improvements to configure's architecture recognition
>      - Fixed D/ZGEMM cleanup error on MIPS
>      - Fixed TRMV tuning Makefile error
>      - Fixed Makefile error preventing TRSM tuning
>      - Worked around gcc's Solaris division bug
>    * Backported Core2 and K10h GEMM kernels
>      - Makes a *huge* perf diff on Intel boxes, slight improvement for K10h
>    * Added arch defs for Corei7 (64 bit only)
> ATLAS 3.8.2 released 06/06/08, Changes from 3.8.1:
>    * Fixed bugs:
>      - Pervasive performance bug in GEMM, affecting all architectures
>      - Occasional access of C when BETA=0
>    * Configure improvements:
>      - Improved freebsd architecture probe
>      - Improved linux cpu throttling probe
>    * Added mu=4 SSE M cleanup for extra performance
> ATLAS 3.8.1 released 02/22/08, Changes from 3.8.0:
>    * Fixed bug in slvtst that counted complex flops same as real
>    * Fixed bug causing wrong answer for row-major gemm C=A*A' or A'A
>    * Fixed bug in configure causing Pentium-M to be IDed as CoreDuo
>    * Fixed bug in tfc.c causing memory overwrite when too many samples taken
>    * Improved L1 BLAS timers so they work like the rest of the package, and
>      thus don't die all the time on tolerance failures
>    * Improved ATLAS/tune/blas/gemm/mmsearch.c:
>      - for x86, tried more registers, since smart compiler can reduce A & B
>        regs to 2 (and possibly even 1)
>      - Made it so search tries both load-C-at-top and load-C-at-bottom of
>        M loop.  Bottom is superior for error, and ATLAS originally defaulted
>        to load-C-at-top.
>    * Added configure support for new K10h platform from AMD, as well as
>      basic architectural defaults (no new kernels, just good search)
> 
> 
> 
> 
> 
> 
> 
> The debian changelog is below:
>  atlas  (3.8.3-2) experimental; urgency=low
> 
>    * Improve CPU extension check
> 
>  -- Sylvestre Ledru <sylvestre at debian.org>  Sat, 26 Sep 2009 02:39:07 +0200
> atlas (3.8.3-1) experimental; urgency=low
> 
>    * New upstream version (Closes: #382880)
>    * debian/rules rewrited from scratch
>    * Provides more documentation in the libatlas-doc packages
>    * debian/copyright updated
>    * Add myself to the uploader
>    * Vcs-Browser & Vcs-Svn added
>    * packages libatlas-3dnow-dev & libatlas3gf-3dnow obsoletes
>    * packages libatlas-sse3-dev and libatlas3-sse3 added
>    * libatlas-headers renamed to libatlas-dev
>    * libatlas-dev is now in section libdevel
>    * fakeroot debian/rules custom will generated a package called
>      ../libatlas3gf-base_*.deb which will contain an optimized version of
>      Atlas just for the current arch.
>    * README.Debian updated to reflect the different changes
>    * README.source added to explain where sources of the PDF are
>    * Standards-Version updated to 3.8.3
>    * compat updated to version 7
>    * Camm & Ondrej removed as uploaders
>    * Use of David Cournapeau's patches (many thanks for this great work)
>    * Change of my email address since I am now DD
> 
> 
> On Wed, Oct 7, 2009 at 11:31 AM, Cyrus Hall <hallc at lu.unisi.ch> wrote:
> > On Tue, 2009-10-06 at 16:41 +0200, Cyrus Hall wrote:
> >> I've recently run into what appears to be a known and reported problem
> >> in Atlas 3.2.1
> >
> > Sorry, that should have read Atlas 3.6.0, not 3.2.1.  I was reading the
> > wrong version number in aptitude.
> >
> > Scott, if you want to test any new build you make, you can also run the
> > test posted at
> >  https://bugs.launchpad.net/ubuntu/+source/atlas/+bug/406520
> > or
> >  https://bugs.launchpad.net/ubuntu/+source/atlas/+bug/376739
> >
> > You can switch between the SSE2 optimized version and the non-optimized
> > version (which appears to function correctly) by using LD_PRELOAD:
> >
> > LD_PRELOAD=/usr/lib/libblas.so                 # Non-optimized version
> > LD_PRELOAD=/usr/lib/sse2/atlas/libblas.so.3gf  # SSE2 optimized version
> > (or whatever the appropriate directories are for a non-installed build.
> > You can confirm LD_PRELOAD is working by checking the result of runtime
> > linking with ldd.  This probably isn't news to anyone on this list...)
> >
> > I can tell you that the version you posted yesterday is still vulnerable
> > to the bug, at least as tested by:
> >  http://launchpadlibrarian.net/25714126/test.f
> >
> > Is there any particularly reason not to upgrade atlas to a version that
> > isn't six years old?  The newest stable version of atlas is 3.8.3 from
> > 18-02-09.  3.6.0 was released December of 2003.
> >
> > I can confirm that the test program works successfully with a build of
> > 3.8.3 I made.
> >
> > I would be happy to package up 3.8.3 for ubuntu, at least for
> > architectures I have access to (x86 CoreDuo and a x86-64 Opteron), if
> > someone could offer me some pointers.  I am not familiar with apt beyond
> > the level of a daily user, meaning I've never created packages for
> > distribution before, but I'd be more than willing to give it a try.
> >
> > No matter, take away message here is that the new build does not fix the
> > problem.  If I have more spare time later this week I'll try and see if
> > I can't hunt down the version in which the problem went away.
> >
> > Cheers,
> > Cyrus
> >
> >