|clang 3.4, 3.5 and 3.6 are now coinstallable in Debian »|
Rebuild of Debian using Clang 3.5.0
Clang 3.5.0 has just been released. A new rebuild has been done highlight the progress to get Debian built with clang.
tl;dr: Great progress. We decreased from 9.5% to 5.7% of failures. Full results are available on http://clang.debian.net
At time of the rebuild with 3.4.2, we had 2040 packages failing to build with clang. With 3.5.0, this dropped to 1261 packages.
With Arthur Marble and Alexander Ovchinnikov, both GSoC students, we worked on various ways to decrease the number of errors.
First, the most obvious way, we fixed programming bugs/mistakes in upstream sources. Basically, we took categories of failure and fixed issues one after the other. We started with simple bugs like 'Wrong main declaration', 'non-void function should return a value' or 'Void function should not return a value'.
They are trivial to fix. We continued with harder fixes like ' Undefined reference' or 'Variable length array for a non POD (plain old data) element'.
So, besides these one, we worked on:
- No support of nested C function
- Wrong C++ default declaration in a method
- Variable length array for a non POD (plain old data) element
- Tautological comparison
- Conflicting types
- Changes of default constructor
- Potential usage of an uninitialized variable
- Use of old GNU field designator
- Missing symbols at link time (inline in C99)
- and others
In total, we reported 295 bugs with patches. 85 of them have been fixed (meaning that the Debian maintainer uploaded a new version with the fix).
In parallel, I think that the switch by FreeBSD and Mac OS X to Clang also helped to fix various issues by upstreams.
Hacking in clang
As a parallel approach, we started to implement a suggestion from Linus Torvalds and a few others. Instead of trying to fix all upstream, where we can, we tried to update clang to improve the gcc compatibility.
gcc has many flags to disable or enable optimizations. Some of them are legacy, others have no sense in clang, etc. Instead of failing in clang with an error, we create a new category of warnings (showing optimization flag '%0' is not supported) and moved all relevant flags into it. Some examples, r212805, r213365, r214906 or r214907
We also updated clang to silent some useless arguments like -finput-charset=UTF-8 (r212110), clang being UTF-8 compliant.
Finally, we worked on the forwarding of linker flags. Clang and gcc have a very different behavior: when gcc does not know an argument, it is going to forward the argument to the linker. Clang, in this case, is going to reject the argument and fail with an error. In clang, we have to explicitly declare which arguments are going to be transfer to the linker. Of course, the correct way to pass arguments to the linker is to use -Xlinker or -Wl but the Debian rebuild proved that these shortcuts are used. Two of these arguments are now forwarded:
- -z keyword - r213198
- -u Force symbol to be entered in the output file as an undefined symbol - r211756. This one fixed most of the haskell build failures. It fixed the most common issue that we had (701 occurrences but this does not mean that all these packages build fine now, some haskell-based package are failing later in the process)
Just like in other releases, new warnings are added in clang. With (bad) usage of -Werror by upstream software, this causes new build failures:
I also took the opportunity to add some further categorizations in the list of errors. Some examples:
The Debile project being close to ready with Clément Schreiner's GSoC, we will now have an automatic and transparent way to rebuild packages using clang.
As stated, we can see a huge drop in term of number of failures over time:
Hopefully, Clang getting better and better, more and more projects adopting it as the default compiler or as a base for plugin/extension developments, this percentage will continue to decrease.
Having some kind of release goal with clang for Jessie+1 can now be considered as potentially reachable.
Want to help?
There are several things which can be done to help:
- Point me common error patterns in the Not categorized list of errors to create new categories
- Report and fix packages
- As an upstream, integrate clang as part of your continuous integration system
- Hack on cqa-scanlogs, the error detection tool to detect error patterns (example: Undetected error). This tool is used also for the regular rebuilds of the archive.
- Improve clang.debian.net website
Thanks to David Suarez for the rebuilds of the archive, Arthur Marble and Alexander Ovchinnikov for their GSoC works and Nicolas Sévelin-Radiguet for the few fixes.
I guess it will be possible to compile a full Linux distribution with clang in only a few years.
> what's the general reason why everything won't compile on Clang?
There are various reasons:
* bad code that gcc accepts
* non standard C code accepted by gcc (variable length array for example)
* -Wall (+ -Werror) does not enable the same set of warnings
* clang is C99 by default, not gcc.