Catégorie: "Debian"
Update of the linear algebra libraries in Debian
April 6th, 2010In the numerical computing world, the cornerstones libraries are BLAS and LAPACK. They have been used in most of the numerical software for decades (like Scilab, R, numpy, OpenOffice with calc, etc).
During that time, many implementations appeared to improve the performances taking advantages of clusters, multicore, SEE{1,2,3,4}, various levels of cache...
Between the reference BLAS (refblas) to an optimized one like ATLAS or MKL (Math Kernel Library by Intel - non-free), it is not rare to have a 15 factor.
In Debian, we use by default the reference implementation of BLAS (168 reverse dependencies) and LAPACK (178 reverse dependencies). If the results are usually bad, they are pretty easy to use. What is hard to use, is switch between highly optimized libraries.
For now, the main one in the archive is ATLAS. ATLAS build process will launch many computations to know what will work best on the architecture. Results are usually excellent.
1) Upload of a refactoring of the ATLAS package.
I have been working on this for a while and after 19 uploads into Debian Experimental and I am happy (and kind of relief) to upload into debian unstable the release 3.8.3 of ATLAS.
The new key elements in this release are:
- Package of the release 3.8.3 ... Long overdue
- Much more packages for recent architectures (sse3, core2sse3, etc)
- A simplified maintenance
- Easy to build a custom package: fakeroot debian/rules custom
- Easy upgrade to version 3.9.X when it is stable
- 12 bugs closed in Debian (including 4 RCs)
- 6 bugs closed in Launchpad.
- MMX optimized package removed
Note that, as before, all prebuilt binaries of ATLAS will be always slower than if you built them on the target architecture (but using Debian binary packages will save a few kilograms of Uranium).
And one of most important feature is the capability to switch to any ATLAS implementation.
2) Switch between the different implementation
The problem in Debian (and Ubuntu) was that it was hard to switch between the ref BLAS/LAPACK and the optimized libraries. The user has to play with the LD_LIBRARY_PATH to use the various optimized packages and since there is no convention between the various distribution, the upstream developer has to develop crappy tricks to handle such things.
It is why I implemented the following proposal: Handle different versions of BLAS and LAPACK.
The main idea is to use the update-alternatives system to allow a quick and easy switch. For example:
# update-alternatives --config libblas.so.3gf There are 3 choices for the alternative libblas.so.3gf (providing /usr/lib/libblas.so.3gf). Selection Path Priority Status ------------------------------------------------------------ * 0 /usr/lib/atlas-core2sse3/atlas/libblas.so.3gf 55 auto mode 1 /usr/lib/atlas-base/atlas/libblas.so.3gf 35 manual mode 2 /usr/lib/atlas-core2sse3/atlas/libblas.so.3gf 55 manual mode 3 /usr/lib/libblas/libblas.so.3gf 10 manual mode # update-alternatives --config liblapack.so.3gf There are 3 choices for the alternative liblapack.so.3gf (providing /usr/lib/liblapack.so.3gf). Selection Path Priority Status ------------------------------------------------------------ * 0 /usr/lib/atlas-core2sse3/atlas/liblapack.so.3gf 55 auto mode 1 /usr/lib/atlas-base/atlas/liblapack.so.3gf 35 manual mode 2 /usr/lib/atlas-core2sse3/atlas/liblapack.so.3gf 55 manual mode 3 /usr/lib/lapack/liblapack.so.3gf 10 manual mode
Thanks to this, it is just trivial to switch from one to the other...
Conclusion:
I just pushed the changes into Debian unstable for blas, lapack and atlas.
I have been testing a lot these deep modifications and I fixed all the problems that I found. However, in case I missed something, please report a bug...
Back from the dead
April 6th, 2010I know, my blog is pretty much dead. Not much time to blog on my various travels.
However, I am going to try to resurrect it for Debian and Scilab.
My next messages related to Debian will be also published on Debian Planet.
A few news around Scilab (packaging & other stuff)
Décembre 30th, 2008Here is a quick list of new things around Scilab (note that it is slightly modified message of the one I sent on the dev mailing list).
-
Sagemath - it is a software which combines the power of various opensource software.
A Experimental "Scilab/Sage package" is planed for Sage 3.4 and an experimental package by Jaap Spies is already available Debian/Ubuntu
Packages are available on my Scilab homepage
Debian packages are up-to-date (5.0.3-2). I will to upload the new Ubuntu's packages in 2009 (for now, it is 5.0.3-1 which is working too). I might backport them to Debian Lenny (future stable) & Ubuntu Hardy.Mandriva
Tomasz Pawel Gajc (a regular Mandriva contributor) created a package available on zarb.orgOpensuse
A Scilab package for Opensuse has been created by Andrea Florio.
It is available on packman
and should be included in the next version of opensuse.
Note that Mandriva & Opensuse packages have been created for Scilab 5.0.3 and I applied most of their patches (or update some part of the code) for Scilab 5.0.4.Redhat/Fedora
The work is still going on.
They are also doing a great work packaging the misc dependencies but they are a bit stuck about the JOGL packaging (jogl and glugen should produce two different packages ... which I should also do in Debian/Ubuntu too)Arch Linux
It is also available under Arch Linux by Simon Lipp (one of our former trainee).Gentoo
A bit stuck for now but some activities have been seen lately around on jrosetta (one of the dependency introduced by Scilab 5).Slackware
Scilab has been packaged by the Italian Slackware community. It is available on their website. I don't know if it is going to be included in Slackware by default or not;
Guake: New in Debian
August 27th, 2008Thanks to my great & usual sponsor, I have uploaded a new package into Debian (therefore in Ubuntu)
This package is a drop-down terminal for Gnome. Drop-down means basically the ² key in Quake 2 (where it takes his name). Just press the F12 key and quickly, your terminal will be displayed. Press it again, it disappears.
Debian Science policy
August 12th, 2008We are finally publishing the work on the Debian Science policy. This page is automatically generated from the git repository. It is a still a draft but the core is there.
In the meantime, we are still maintaining the Science CDD [1]. Inputs are welcome to improve this list.
For more information, please have a look on the Debian Wiki
[1] CDD: Custom Debian Distribution: a subset of Debian that is configured to support a particular target group out-of-the-box.