Catégorie: "LLVM/Clang"
Follow up on Debian & Clang
Mai 6th, 2013A week after the Euro LLVM conference, I am glad to share the results I presented and discussed during the event.
1) Parallel building infrastructure using clang instead of gcc
Based on the work that Alexander Pashaliyski and I did during the GSoC 2012, we are now able to automatically rebuild each package uploaded in Unstable transparently. Results are published here:
http://buildd-clang.debian.net/
This provides a quick and easy way to any Debian contributors or upstream packaged in Debian to check how the package behaves with clang.
For Debian Developer and maintainer, a per maintainer view is also provided. This will list all packages maintained by a Debian packager. My page as example
Here is an example of the list of package:
And an example of package failing:
At time of writing, most of the packages in unstable have been rebuilt (16 % are remaining). However, due to resource constraints, only amd64 packages are built. Contact me if you have resources to share for other architectures.
Thanks to Paul Wise, the PTS (Debian package tracking system) proposes a link to the clang build logs (failing or not).
A repository will be automatically updated with the results of these builds (once I find the time to get into buildd).
2) clang-built repository
Thanks to the Amazon Cloud, with our new DPL (Debian Project Leader), Lucas Nussbaum, we rebuilt unstable last April and published it at the following address:
deb http://clang.debian.net/repository-2013-04-07/ unstable-clang main
This allows some cool stuff like:
$ echo "deb http://clang.debian.net/repository-2013-04-07/ unstable-clang main">>/etc/apt/sources.list $ apt-get update $ apt-get install coreutils/unstable-clang $ ls $ awk
Note that this repository was only one-shoot and does not get updated.
Quality of the binary produced (performance and size) can be tested thanks to this repository. Feedbacks are welcome!
LLVM Debian/Ubuntu nightly packages
Avril 3rd, 2013Lately, I have been working on providing nightly packages of the whole LLVM toolchain.
With the help of folks from Intel, Google and Apple, I am happy to announce the publication of these packages:
http://llvm.org/apt/
Built through a Jenkins instance (http://llvm-jenkins.debian.net/), packages for Debian wheezy and Unstable and Ubuntu quantal, precise and raring are created twice a day.
3.2 and 3.3 llvm-toolchain packages are currently waiting in the Debian NEW queue.
More information on the LLVM blog.
Rebuild of Debian using clang 3.2
Février 6th, 2013After the studies of the rebuild of the Debian archive using clang 2.9 & 3.0 and 3.1, the results of the 3.2 rebuild have been published on http://clang.debian.net.
The percentage of failure is the same as clang 3.1: 12.1% of failure. That means that on 18264 packages, 2204 failed to be built with clang (it was 17710/2137 with the version 3.1).
The fact that the percentage is the same can be explained by at least two reasons:
First, some upstreams packages have been fixed to be built correctly with clang.
Second, the new warnings + -Werror (example: -Wsometimes-uninitialized) introduced in clang 3.2 triggered some new build failures.
To conclude, I think that the 3.2 release confirms the turning point of the 3.1. In term of support of the C and C++ standard, clang has now reached an equivalent quality to gcc (with better detection of errors and an extended set of warnings in -Wall).
Now, from the perspective of the Debian rebuilds, the decrease of number of failures will come from the upstream developers improving their codes (exemple of the error non-void function should return a value) or the Debian Developer/maintainer fixing the programming errors.
Some more cool stuff with LLVM/Clang
Janvier 12th, 2013One of the drawback of C is the hard memory management.
It is because of this aspect that Clang provides a way to detect, at runtime, memory errors. Called Address Sanitizer, it allows, while the program is running, to keep track of the memory and detect typical errors (out-of-bound accesses, use of a variable after a free, etc).
# With the following packages (version 3.2-1~exp3)
$ sudo apt-get install clang clang-3.2 compiler-rt -t experimental
Taking the simple following example, even if the errors are obvious, compilers will accept this code.
#include <stdlib.h>
int main() {
char *x = (char*)malloc(10 * sizeof(char*));
free(x);
return x[5];
}
Built and run with:
$ clang -O1 -g -fsanitize=address -fno-omit-frame-pointer foo.c -o foo
$ ./foo &> memoryDebug.log
The previous command will generate a log file. Log which can be post processed with the asan_symbolize command.
$ asan_symbolize memoryDebug.log
which will give:
================================================================= ==21368== ERROR: AddressSanitizer: heap-use-after-free on address 0x7fb22e547f45 at pc 0x408c44 bp 0x7ffff60c10b0 sp 0x7ffff60c10a8 READ of size 1 at 0x7fb22e547f45 thread T0 #0 0x408c43 in main /tmp/foo.c:5 #1 0x7fb22d99f6ac in __libc_start_main /home/adconrad/eglibc-2.16/csu/libc-start.c:227 0x7fb22e547f45 is located 5 bytes inside of 80-byte region [0x7fb22e547f40,0x7fb22e547f90) freed by thread T0 here: #0 0x408c90 in __interceptor_free ??:0 #1 0x408c0a in main /tmp/foo.c:4 #2 0x7fb22d99f6ac in __libc_start_main /home/adconrad/eglibc-2.16/csu/libc-start.c:227 previously allocated by thread T0 here: #0 0x408d50 in __interceptor_malloc ??:0 #1 0x408bff in main /tmp/foo.c:3 #2 0x7fb22d99f6ac in __libc_start_main /home/adconrad/eglibc-2.16/csu/libc-start.c:227 Shadow byte and word: 0x1ff645ca8fe8: fd 0x1ff645ca8fe8: fd fd fd fd fd fd fd fd More shadow bytes: 0x1ff645ca8fc8: fa fa fa fa fa fa fa fa 0x1ff645ca8fd0: fa fa fa fa fa fa fa fa 0x1ff645ca8fd8: fa fa fa fa fa fa fa fa 0x1ff645ca8fe0: fa fa fa fa fa fa fa fa =>0x1ff645ca8fe8: fd fd fd fd fd fd fd fd 0x1ff645ca8ff0: fd fd fd fd fd fd fd fd 0x1ff645ca8ff8: fa fa fa fa fa fa fa fa 0x1ff645ca9000: fa fa fa fa fa fa fa fa 0x1ff645ca9008: fa fa fa fa fa fa fa fa Stats: 0M malloced (0M for red zones) by 1 calls Stats: 0M realloced by 0 calls Stats: 0M freed by 1 calls Stats: 0M really freed by 0 calls Stats: 0M (128 full pages) mmaped in 1 calls mmaps by size class: 8:2047; mallocs by size class: 8:1; frees by size class: 8:1; rfrees by size class: Stats: malloc large: 0 small slow: 1 ==21368== ABORTING
The main advantage compare to valgrind is that asan is supposed to be way more faster.
Threads can be also tricking to develop.
With the following example from stolen from upstream:
#include <pthread.h>
int Global;
void *Thread1(void *x) {
Global = 42;
return x;
}int main() {
pthread_t t;
pthread_create(&t, NULL, Thread1, NULL);
Global = 43;
pthread_join(t, NULL);
return Global;
}
$ clang -fsanitize=thread -g -O1 foo2.c -fPIE -pie -o foo $ ./foo ================== WARNING: ThreadSanitizer: data race (pid=21416) Write of size 4 at 0x7f2f1a214a50 by thread 1: #0 Thread1 /tmp/foo2.c:4 (exe+0x00000000f850) Previous write of size 4 at 0x7f2f1a214a50 by main thread: #0 main /tmp/foo2.c:10 (exe+0x00000000f8a4) Thread 1 (tid=21417, running) created at: #0 pthread_create ??:0 (exe+0x00000001267e) #1 main /tmp/foo2.c:9 (exe+0x00000000f894) ================== ThreadSanitizer: reported 1 warnings
Note that clang also provides scan-build, a static analyzer for memory issues. Not as powerful as the Address Sanitizer (it only works on a file), it provides some excellent reports. See the Wouter's blog post on this subject or the automatic report of scan-build on Scilab.
libc++: New C++ standard library in Debian
Août 15th, 2012Thanks to the work of Andrej Belym, Debian has now a new C++ standard library. This work has been done in the context of the Google Summer of Code 2012.
Available in Debian Experimental, this new packages provides both the runtime libraries (libc++abi1) and the C++ headers (libc++-dev).
With this library, it is possible to build a C++-based program without any dependency on libstdc++.
For example, as detailed in README.Debian, with the (amazing) C++ code:
with clang++ (or g++) will give:// foo.cpp
#include <iostream>int main() {
std::cout < < "plop" << std ::endl;
return 0;
} std>
$ clang++ -o plop foo.cpp
$ ldd plop |grep c++
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f15791cb000)
Using libc++, it will drop the dependency on libstdc++ to use lib++
$ clang++ -stdlib=libc++ -o plop foo.cpp
$ ldd plop |grep c++
libc++.so.1 => /usr/lib/x86_64-linux-gnu/libc++.so.1 (0x00007f87464df000)
For the record, it is not that trivial to do with g++. The command being:
g++ -nostdlib -lc++ -lc++abi -std=c++11 -o plop /usr/lib/x86_64-linux-gnu/crt1.o /usr/lib/x86_64-linux-gnu/crti.o foo.cpp /usr/lib/x86_64-linux-gnu/crtn.o -isystem/usr/include/c++/v1 -lc -lgcc_s
Besides this change, libc++ provides a support of C++ 11, considered as "correct" against the C++11 standard by upstream.
// bar.cpp
#include <chrono>
int main() {
return 0;
}
clang++ -stdlib=libc++ -o bar bar.cpp
More information: