Catégorie: "Développement"

About optimization flags in software

Septembre 6th, 2013

Following a thread on the Clang mailing list, I did a quick analysis of the optimization flags used to build C or C++ code.

I used a rebuild result from last mid July of all Debian native packages (meaning that I don't have the log of full Java, Python or other packages without native code) and wrote a small script to count all their occurrences.
By default, deb packages uses the command "dpkg-buildflags --get CFLAGS" which will set the optimization level to -O2. However, many packages and upstream build systems do not respect such flags.

On 10320 binary-only packages (in total, Debian has around 19000), the results are the following:

Optimization levelNumber of occurrencesNumber of packages
-O35597269
-Os30849145
-O013993294
-O125494809
-O29958037685
-O3106048531
-O4140216
-O5
-O6693549

Since a graphic talks more than numbers (note that the Y axis is logarithmic):

Disclaimer:
Some packages are using several optimization flags during the same build.
Some packages are not showing the full command line (see jessie release goal: verbose build logs)
The results might be approximate since I am basically grepping "-OX " on the log files.

Clang 3.3 and Debian

Août 19th, 2013

The LLVM toolchain version 3.3 has been released a couple months ago.
Here are now the result of the rebuild of Debian archive using this version of the compiler.
Like the previous releases, we are at a bit less than 12% of packages failing (2188 packages on a total of 18854).
More warnings / errors detections have been added to the software (for example: like this defect of the C++ standard or the detection of unused linker option) causing more build failures, but, in the mean time, we fixed some issues in the Debian packages...

As usual, the following image shows clearly the evolution of the build failures over time.

As stated in my blog post for the release 3.2, this rebuilds prove that Clang is ready for production in term of support of the C, C++ and Objective C languages.

With the performance results showed by Chandler Carruth from Google at the last Euro LLVM summit (see this video from 5:40), I believe that it is now time to report and fix the bugs in the upstream packages.

I also presented this work (video) at the Debconf 13 last week in Vaumarcus (Switzerland) and I will be also presenting this work at the Linux Plumbers Conference, New Orleans.

With Léo Cavaille (as part of his GSOC) and Paul Tagliamonte, we are also working on providing a better automatic rebuild infrastructure for clang-built packages (and other static analyzers). More in the next few weeks.

Finally, I would like to thank folks at AWS for the Debian credit and David Suarez for helping on with the Ruby segfaults.

Rebuild of Debian using clang 3.2

Février 6th, 2013

After the studies of the rebuild of the Debian archive using clang 2.9 & 3.0 and 3.1, the results of the 3.2 rebuild have been published on http://clang.debian.net.

The percentage of failure is the same as clang 3.1: 12.1% of failure. That means that on 18264 packages, 2204 failed to be built with clang (it was 17710/2137 with the version 3.1).

The fact that the percentage is the same can be explained by at least two reasons:
First, some upstreams packages have been fixed to be built correctly with clang.
Second, the new warnings + -Werror (example: -Wsometimes-uninitialized) introduced in clang 3.2 triggered some new build failures.

To conclude, I think that the 3.2 release confirms the turning point of the 3.1. In term of support of the C and C++ standard, clang has now reached an equivalent quality to gcc (with better detection of errors and an extended set of warnings in -Wall).
Now, from the perspective of the Debian rebuilds, the decrease of number of failures will come from the upstream developers improving their codes (exemple of the error non-void function should return a value) or the Debian Developer/maintainer fixing the programming errors.

libc++: New C++ standard library in Debian

Août 15th, 2012

Thanks to the work of Andrej Belym, Debian has now a new C++ standard library. This work has been done in the context of the Google Summer of Code 2012.
Available in Debian Experimental, this new packages provides both the runtime libraries (libc++abi1) and the C++ headers (libc++-dev).

With this library, it is possible to build a C++-based program without any dependency on libstdc++.

For example, as detailed in README.Debian, with the (amazing) C++ code:

// foo.cpp
#include <iostream>

int main() {
std::cout < < "plop" << std ::endl;
return 0;
}

with clang++ (or g++) will give:

$ clang++ -o plop foo.cpp
$ ldd plop |grep c++
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f15791cb000)

Using libc++, it will drop the dependency on libstdc++ to use lib++

$ clang++ -stdlib=libc++ -o plop foo.cpp
$ ldd plop |grep c++
libc++.so.1 => /usr/lib/x86_64-linux-gnu/libc++.so.1 (0x00007f87464df000)

For the record, it is not that trivial to do with g++. The command being:

g++ -nostdlib -lc++ -lc++abi -std=c++11 -o plop /usr/lib/x86_64-linux-gnu/crt1.o /usr/lib/x86_64-linux-gnu/crti.o foo.cpp /usr/lib/x86_64-linux-gnu/crtn.o -isystem/usr/include/c++/v1 -lc -lgcc_s

Besides this change, libc++ provides a support of C++ 11, considered as "correct" against the C++11 standard by upstream.

// bar.cpp
#include <chrono>
int main() {
return 0;
}

clang++ -stdlib=libc++ -o bar bar.cpp

More information:

News on Debian & clang

Juillet 24th, 2012

A couple week ago, during the last debconf (Debian Conference) in Managua/Nicaragua, I presented the latest developments about the inclusion of clang in the Debian architecture.

To sum up (details are available in the slides and the video), the rebuild of the Debian archive with Clang 3.1 increased the number of failures from 8.8 to 12.1%. The main reason is that further checks have been added to clang.

With Paul Tagliamonte as co-mentor and Alexander Pashaliyski as a GSoC student, we made great progress in bringing Clang as A new compiler in the Debian infrastructure.

The various feedbacks that I had during Debconf were pretty good. It interests many people for reasons like Quality Assurance (more checks), performances, hackability or to decorrelate Debian and GCC.

Build Debian with another compiler - Slides
Build Debian with another compiler - Video

A nice resume of my talk has been made by Michael Larabel on Phoronix:
Decoupling GCC From Debian By Using LLVM/Clang