Catégories: "Informatique"

Some thoughts concerning LLVM & Clang and their evolutions: Release of LLVM & Clang 3.4

Janvier 15th, 2014

We just released LLVM & Clang 3.4 (already available in Debian Jessie and Ubuntu Trusty). The clang new release introduces some more warning and error detections. The point of the blog post is not to detail the changes (LLVM / Clang) but more to take a step back on the LLVM and Clang toolchains, usages and dissemination.

For the last couple years, I have been presenting our work on the Debian side to make Clang part of the Debian infrastructure. I add the chance to speak at many events (Debconf 12 & 13, FOSDEM 2013, Linux Plumbers 2013, Distro recipes, etc). I even had Linus Torvalds attending to one of these talks (and proposing new approach). I had many feedback from the Debian/Ubuntu communities but also from other communities.
More and more projects are using Clang as part of their workflows, in term of development, production and QA. For example, some of them are running continuous integration using clang instead of gcc, others are running scan-build (Wireshark or LLVM itself) or Address Sanitizer (asan) (Firefox for example), etc.
More and more actors are moving from gcc to clang for their developments (Sony with their PS4, Chrome under Mac OS X, Apple, the Linux Kernel with the LLVMLinux project, FreeBSD, etc).
I stopped counting the number of times when people answered me "Yeh, I am now using Clang" when I said I am involved upstream.

In parallel, more and more projects are using LLVM as a backend or computing engine and, in the meantime, getting more and more attraction (not because they are based on LLVM but just because they are great projects). For example, Mesa (llvmpipe), Julia, Rust, Emscripten or Native Client (NaCL) are excellent examples of this. The two last are excellent proofs that LLVM based technologies have an impact which goes further than geek-tools and will be used on a daily basis by millions of users.

I also saw an interesting and growing number of new projects using libclang and libTooling. My feeling is that the lack of plugin capabilities in gcc for a long time and, now, its current complexity limited the innovation in term of compilation capabilities, analysis and fun tools. The features and the quality of this library are enabling new developments and I am sure we will see more and more excellent tools based on this library during 2014.
For example, the LLVM toolchain ships Clang modernize and clang format and some cool projects like include-what-you-use, DXR, OCLint or creduce are emerging.

Thanks to this great competition, gcc is also improving (ASAN, JIT, better warnings, etc), remaining still relevant as before. No doubt that 2014 will be a great year for compilers.

Include What You Use - Yet a new cool tool in Debian.

Septembre 8th, 2013

iwyu (Include What You Use) has just been accepted in Debian.

This tool, based on LLVM/Clang, checks the headers of a C/C++ file. It will detect which include directives are useless, what could be improve and add comments to explain why #include are there.

Let's take this C file:

#include <stdio.h>
#include <malloc.h>
#include <limits.h>

int foo(void) {
fprintf(stderr, "an error\n");
return INT_MAX;
}

$ iwyu foo.c

will return:

foo.c should remove these lines:
- #include <malloc.h> // lines 2-2

The full include-list for foo.c:
#include <limits.h> // for INT_MAX
#include <stdio.h> // for fprintf, stderr
---

Package on Debian

About optimization flags in software

Septembre 6th, 2013

Following a thread on the Clang mailing list, I did a quick analysis of the optimization flags used to build C or C++ code.

I used a rebuild result from last mid July of all Debian native packages (meaning that I don't have the log of full Java, Python or other packages without native code) and wrote a small script to count all their occurrences.
By default, deb packages uses the command "dpkg-buildflags --get CFLAGS" which will set the optimization level to -O2. However, many packages and upstream build systems do not respect such flags.

On 10320 binary-only packages (in total, Debian has around 19000), the results are the following:

Optimization levelNumber of occurrencesNumber of packages
-O35597269
-Os30849145
-O013993294
-O125494809
-O29958037685
-O3106048531
-O4140216
-O5
-O6693549

Since a graphic talks more than numbers (note that the Y axis is logarithmic):

Disclaimer:
Some packages are using several optimization flags during the same build.
Some packages are not showing the full command line (see jessie release goal: verbose build logs)
The results might be approximate since I am basically grepping "-OX " on the log files.

Clang 3.3 and Debian

Août 19th, 2013

The LLVM toolchain version 3.3 has been released a couple months ago.
Here are now the result of the rebuild of Debian archive using this version of the compiler.
Like the previous releases, we are at a bit less than 12% of packages failing (2188 packages on a total of 18854).
More warnings / errors detections have been added to the software (for example: like this defect of the C++ standard or the detection of unused linker option) causing more build failures, but, in the mean time, we fixed some issues in the Debian packages...

As usual, the following image shows clearly the evolution of the build failures over time.

As stated in my blog post for the release 3.2, this rebuilds prove that Clang is ready for production in term of support of the C, C++ and Objective C languages.

With the performance results showed by Chandler Carruth from Google at the last Euro LLVM summit (see this video from 5:40), I believe that it is now time to report and fix the bugs in the upstream packages.

I also presented this work (video) at the Debconf 13 last week in Vaumarcus (Switzerland) and I will be also presenting this work at the Linux Plumbers Conference, New Orleans.

With Léo Cavaille (as part of his GSOC) and Paul Tagliamonte, we are also working on providing a better automatic rebuild infrastructure for clang-built packages (and other static analyzers). More in the next few weeks.

Finally, I would like to thank folks at AWS for the Debian credit and David Suarez for helping on with the Ruby segfaults.

Video of Debconf 13 available on the IRILL website

Août 19th, 2013

While raw videos of the previous Debconf are available on:
http://video.debian.net/

We, at IRILL, are glad to provide a nice HTML5 interface (and flash if HTML5 videos are not available) to browse all the videos of Debconf 13:
http://www.irill.org/videos/debconf13