Thursday, April 12, 2012

Building ARM toolchain - part 2: gcc and eglibc

Unfortunately after few tries of cross compiling eglibc using different source for instructions I alway end with hard to solve issues. Luckily, in the sources of eglibc I noticed instructions for cross-compiling written long time ago by Jim Blandy(I know i should start here). Lot of thanks to him for it. Below I describe my expierence which I gained during eglibc cross cpomliation for arm-unknown-linux-gnueabi and procedure that I used. Commands below contain some constants that I used in previous works. See this post. Eglibc library and the compiler itself is built with many various parameters this post is not the place to explain their meaning, please RTFM.
  1. Checkout eglibc from svn (as alwyas I try to use a latest sources possible). Version used r17815:
    svn co http://www.eglibc.org/svn/trunk eglibc
  2. Link working ports to GNU/Linux on some machine architectures. They are not maintained in the official glibc source tree so we need to add it in this way:
    ln -s ../ports eglibc/libc/ports/
  3. Create eglibc-headers directory:
    mkdir eglib-headers; cd eglib-headers
  4. Configure eglibc and preliminary objects:
    BUILD_CC=gcc CC=arm-unknown-linux-gnueabi-gcc CXX=arm-unknown-linux-gnueabi-cpp \
    AR=arm-unknown-linux-gnueabi-ar RANLIB=arm-unknown-linux-gnueabi-ranlib \
    ../eglibc/libc/configure --prefix=/usr --with-headers=$TARGET/usr/include \
    --build=x86_64-pc-linux-gnu --host=arm-unknown-linux-gnueabi --disable-profile \
    --without-gd --without-cvs --enable-add-ons
  5. Install eglibc headers:
    make install-headers install_root=$TARGET install-bootstrap-headers=yes
  6. We need few object file to link shared libraries, which will be built and installed by hand:
    mkdir -p $TARGET/usr/lib
    make csu/subdir_lib
    cp csu/crt1.o csu/crti.o csu/crtn.o $TARGET/usr/lib
  7. To produce libgcc_s.so we need libc.so, but only need its dummy version because we'll never use it. It doesn't matter what we will point as a libc.so we use /dev/null as C file.
    arm-unknown-linux-gnueabi-gcc -nostdlib -nostartfiles -shared -x c /dev/null -o \
    $TARGET/usr/lib/libc.so
  8. Get latest gcc sources using git repository mirror. Latest commit while writing this post was 5b9a8c3:
    cd ..
    git clone git://repo.or.cz/official-gcc.git
  9. Now, we can build gcc which can compile eglibc.
    mkdir eglibc-gcc; cd eglibc-gcc
    ../official-gcc/configure --target=arm-unknown-linux-gnueabi \
    --prefix=$TARGET/arm-x-tools --with-sysroot=$TARGET --disable-libssp \
    --disable-libgomp --disable-libmudflap --enable-languages=c \
    --with-gmp=$TARGET/arm-x-tools --with-mpfr=$TARGET/arm-x-tools \
    --with-mpc=$TARGET/arm-x-tools --disable-libquadmath --build=$MACHTYPE \
    --host=$MACHTYPE --with-local-prefix=$TARGET/arm-x-tools --disable-multilib \
    --with-float=soft --with-pkgversion="pietrushnic" --enable-threads=no \
    --enable-target-optspace --disable-nls --enable-c99 --enable-long-long
    make -j4
    make install
  10. Confugure and compile final version of eglibc.
    mkdir eglibc-final
    cd eglibc-final/
    BUILD_CC=gcc CC=arm-unknown-linux-gnueabi-gcc CXX=arm-unknown-linux-gnueabi-cpp \
    AR=arm-unknown-linux-gnueabi-ar RANLIB=arm-unknown-linux-gnueabi-ranlib \
    ../eglibc/libc/configure --prefix=/usr --with-headers=$TARGET/usr/include \
    --build=x86_64-pc-linux-gnu --host=arm-unknown-linux-gnueabi --disable-profile \
    --without-gd --without-cvs --enable-add-ons
    make
    make install install_root=$TARGET
  11. Install libelf library
    wget http://www.mr511.de/software/libelf-0.8.13.tar.gz
    tar zxvf libelf-0.8.13.tar.gz
    cd libelf-0.8.13/
    ./configure  --prefix=$TARGET/arm-x-tools --disable-shared --enable-static
    make;make install
  12. Prepare final version of gcc.
    cd ..
    mkdir final-gcc
    cd final-gcc
    ../official-gcc/configure --target=arm-unknown-linux-gnueabi \
    --prefix=$TARGET/arm-x-tools --with-sysroot=$TARGET --disable-libssp \
    --disable-libgomp --disable-libmudflap --enable-languages=c,c++ 
    --with-gmp=$TARGET/arm-x-tools --with-mpfr=$TARGET/arm-x-tools 
    --with-mpc=$TARGET/arm-x-tools --disable-libquadmath --build=$MACHTYPE \
    --host=$MACHTYPE --with-local-prefix=$TARGET/arm-x-tools --disable-multilib \
    --with-float=soft --with-pkgversion="pietrushnic" --enable-threads=posix \
    --enable-target-optspace --disable-nls --enable-c99 --enable-long-long \
    --enable-__cxa_atexit --enable-symvers=gnu --with-libelf=$TARGET/arm-x-tools \
    --enable-lto
    make
    make install
  13. Few libraries should be copied manualy
    cp -d $TARGET/arm-x-tools/arm-unknown-linux-gnueabi/lib/libgcc_s.so* $TARGET/lib
    cp -d $TARGET/arm-x-tools/arm-unknown-linux-gnueabi/lib/libstdc++.so* $TARGET/lib
  14. Compile and install chrpath - this is useful tool to remove the rpath or runpath setting from binary.
    cd ..
    sudo apt-get install libc6-i386 gcc-multilib
    apt-get source chrpath
    cd chrpath-0.13/
    CFLAGS=-m32 ./configure --prefix=$TARGET/arm-x-tools \
    --program-prefix=arm-unknown-linux-gnueabi-
    make
    make install
  15. Strip debug symbols
    strip --strip-debug $TARGET/arm-x-tools/lib/* \
    $TARGET/arm-x-tools/arm-unknown-linux-gnueabi/lib/* $TARGET/arm-x-tools/libexec/*
    
    strip --strip-unneeded $TARGET/arm-x-tools/bin/* \
    $TARGET/arm-x-tools/arm-unknown-linux-gnueabi/bin/*
    arm-unknown-linux-gnueabi-strip --strip-debug $TARGET/lib/* $TARGET/usr/lib/*
  16. At the end simple test to find out if basic functionality works:
    $ cat > hello.c <<EOF
    > #include <stdio.h>
    > int
    > main (int argc, char **argv)
    > {
    >   puts ("Hello, world!");
    >   return 0;
    > }
    > EOF
    
    Try to cross compile C file:
    $TARGET/arm-x-tools/bin/arm-unknown-linux-gnueabi-gcc -Wall hello.c -o hello
    
    $ cat > c++-hello.cc <<EOF
    > #include <iostream>
    > int
    > main (int argc, char **argv)
    > {
    >   std::cout << "Hello, C++ world!" << std::endl;
    >   return 0;
    > }
    > EOF
    
    Try to cross compile C++ file:
    $TARGET/arm-x-tools/bin/arm-unknown-linux-gnueabi-g++ -Wall c++-hello.cc -o \
    c++-hello
    
    Displays the information contained in the ELF header and in the file's segment headers:
    $TARGET/arm-x-tools/bin/arm-unknown-linux-gnueabi-readelf -hl hello
    $TARGET/arm-x-tools/bin/arm-unknown-linux-gnueabi-readelf -hl c++-hello
    
    Result should look like that:
    ELF Header:
      Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
      Class:                             ELF32
      Data:                              2's complement, little endian
      Version:                           1 (current)
      OS/ABI:                            UNIX - System V
      ABI Version:                       0
      Type:                              EXEC (Executable file)
      Machine:                           ARM
      (...)
      Flags:                             0x5000002, has entry point, Version5 EABI
      (...)
    Program Headers:
      (...)
      INTERP         0x000134 0x00008134 0x00008134 0x00013 0x00013 R   0x1
          [Requesting program interpreter: /lib/ld-linux.so.3]
      LOAD           0x000000 0x00008000 0x00008000 0x004b8 0x004b8 R E 0x8000
      (...)
    
    $TARGET/arm-x-tools/bin/arm-unknown-linux-gnueabi-readelf -d \
    $TARGET/lib/libgcc_s.so.1
    Result should look like that:
      (...)
      Tag        Type                         Name/Value
     0x00000001 (NEEDED)                     Shared library: [libc.so.6]
     0x0000000e (SONAME)                     Library soname: [libgcc_s.so.1]
     0x0000000c (INIT)                       0xcc2c
      (...)
    
I hope you find above manual useful. If you need more detailed descriptions it can be found here. Also don't bother to ask me by comment below.

Friday, April 6, 2012

How to download videos from videos.linux.com

Therefore, I'm leaving for the Easter holidays I wanted to download some lecture on embedded systems, which was presented at the 2012 Embedded Linux Conference. Although I regret I found that I could not find as good quality copy in the network as on the Linux Foundation page. It is unfortunate that linux.com site does not have the possibility of direct downloading video files. But there is a workaround. Follow below tutorial:

  1. Go to page with video. For example:  http://video.linux.com/videos/to-provide-a-long-term-stable-linux-for-industry
  2. Click play on the video and if you using Chrome browser right click on player window and inspect this element. Result should look like that: 


  3. Expand div tag marked in red on screenshot above. If video was start you should see video tag which contain two links to video files mp4 and webm. Screen shot below shows hot it should look like:


These links are only temporary, so if you want to use them do it as soon as possible. The sad part of all is that the organization intended to promote one of the most libertarian solutions in software history does not provide materials for download.

Tuesday, March 20, 2012

Building ARM toolchain - part 1: libs and binutils

Searching the internet for information on how to build arm toolchain from scratch I realize that it is very hard to find  information about this matter (and recent one even harder). I will try to fill this lack of information and try to build toolchain. My main goal is to use a component based on the GNU public license, and using them in as the newest version as it is possible.

What is toolchain ? (according to wikipedia):

In software, a toolchain is the set of programming tools that are used to create a product (typically another computer program or system of programs). The tools may be used in a chain, so that the output of each tool becomes the input for the next, but the term is used widely to refer to any set of linked development tools.

Requirements:
- Cross compiler, I create one using corsstool-ng and describe this process in previous post. I will use arm-unknown-linux-gnueabi as entry point compiler. - $TARGET is defined as my destination directory
export TARGET=/home/pietrushnic/sandbox/toolchain

Procedure:
  1. Kernel header files:
    1. clone linux git repository
      git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
    2. install headers $TARGET is home of our toolchain
      make ARCH=arm INSTALL\_HDR\_PATH=\$TARGET/usr headers\_install
  2. Libraries (gmp, mpfr, mpc)
    1. GMP (GNU Multiple Precision Arithmetic Library) - changeset used: 14765:0acae62fa162
      • As I said before I use "latest greatest" ;P version, and for gmp we can reach it using:
        hg clone http://gmplib.org:8000/gmp
      • create configuration files
        ./.bootstrap
      • configure
        ./configure --prefix=$TARGET/arm-x-tools --enable-shared=n \
        --enable-static
      • compile
        make
        make check
        make install
    2. MPFR (GNU Multiple Precision Floating-Point Reliably) - version: r8103
      • get latest version
        svn checkout svn://scm.gforge.inria.fr/svn/mpfr/trunk mpfr
      • create configuration file
        autoreconf -i
      • configure
        ./configure --prefix=$TARGET/arm-x-tools --enable-thread-safe \
        --with-gmp=$TARGET/arm-x-tools --disable-shared --enable-static
      • compile
        make
        make install
    3. MPC (Multiple Precision Complex)- version: r1146
      • checkout svn
        svn checkout svn://scm.gforge.inria.fr/svnroot/mpc/trunk mpc
      • create configuration file
        autoreconf -i
      • configure
        ./configure --prefix=$TARGET/arm-x-tools \
        --with-gmp=$TARGET/arm-x-tools --with-mpfr=$TARGET/arm-x-tools \
        --disable-shared --enable-static
      • compile
        make
        make install
  3. Binutils - collection of a GNU binary tools:
    • checkout version from anonymous cvs
      cvs -z 9 -d :pserver:anoncvs@sourceware.org:/cvs/src login
      create directory for checkout
      mkdir binutils
      checkout sources
      cvs -z 9 -d :pserver:anoncvs@sourceware.org:/cvs/src co binutils
    • configure
      LDFLAGS="-Wl,-rpath -Wl,$TARGET/arm-x-tools/lib" ./configure \
      --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu \
      --target=arm-unknown-linux-gnueabi --prefix=$TARGET/arm-x-tools \
      --disable-nls --disable-multilib --disable-werror --with-float=soft \
      --with-gmp=$TARGET/arm-x-tools --with-mpfr=$TARGET/arm-x-tools \
      --with-mpc=$TARGET/arm-x-tools --with-sysroot=$TARGET
      
    • compile
      make configure-host
      make
      make install
    • to check if everything was made correctly
      ldd $TARGET/arm-x-tools/bin/arm-unknown-linux-gnueabi-ldd
      it should show that it use library compiled previously by us:
      libz.so.1 => /home/pietrushnic/sandbox/toolchain/arm-x-tools/lib/libz.so.1 \
      (0x00007f0086cc5000)
This set gives us a solid base to build the compiler. However, it will be in the next section.

Wednesday, March 14, 2012

Quick build of arm-unknown-linux-gnueabi with crosstool-ng

You might be surprised at how much you have to make to correctly build arm-unknown-linux-gnueabi config based toolchain with crosstool-ng. As you can see examples of many open source projects, the man's work is a rare resource. The result of this economic fact is that the attempt to build configuration arm-unknown-linux-gnueabi is not a simple task and during an operation you can come across many problems. Although I am not afraid of problems and effectively try to fight them and of course sharing the results of my work. My build system parameters:
Debian GNU/Linux wheezy/sid 3.2.0-2-amd64

  1. Clone crosstools-ng (you need mercurial)
    hg clone http://crosstool-ng.org/hg/crosstool-ng
  2. Create temporary directory and run
    ct-ng arm-unknown-linux-eabi
    choose latest kernel (for me it was 3.2.9)
  3. We probably want to change directory to which all stuff will be build (default is $HOME/x-tools):
  4. ct-ng menuconfig
    And go to:
    Paths and misc options  ---> (${HOME}/x-tools/${CT_TARGET}) Prefix directory
    Change it according to your needs. Exit end save configuration.
  5. Build (number depend on how many command we want to run simultaneously):
    ct-ng build.4
    It can take a lot of time. On my machine with 5k BogoMips it takes over 1h.
Problems that you can encounter:
  1. gcj - latest changeset 2916:6f758ed4c0b9 have trouble finding gcj binary, which it show using following message:
    [ERROR]  Missing: 'x86_64-unknown-linux-gnu-gcj' or 
    'x86_64-unknown-linux-gnu-gcj' or 'gcj' : either needed!
    To workaround this install gcj and link binary like this:
    sudo ln -s /usr/bin/gcj-4.6 /usr/bin/gcj
  2. duma - mentioned changeset also has problem with url to D.U.M.A library, apply below changes to workaround problems:
    --- a/scripts/build/debug/200-duma.sh   Mon Mar 12 21:19:26 2012 +0100
    +++ b/scripts/build/debug/200-duma.sh   Wed Mar 14 20:02:22 2012 +0100
    @@ -4,7 +4,7 @@
         # Downloading an non-existing file from sourceforge will give you an
         # HTML file containing an error message, instead of returning a 404.
         # Sigh...
    -    CT_GetFile "duma_${CT_DUMA_VERSION}" .tar.gz http://kent.dl.sourceforge.net/sourceforge/duma/
    +    CT_GetFile "duma_${CT_DUMA_VERSION}" .tar.gz http://downloads.sourceforge.net/project/duma/duma/2.5.15
         # Downloading from sourceforge may leave garbage, cleanup
         CT_DoExecLog ALL rm -f "${CT_TARBALLS_DIR}/showfiles.php"*
     }
    
  3. mawk - if mawk return syntax error like this:
    mawk: scripts/gen-sorted.awk: line 19: regular expression compile failed (bad 
    class -- [], [^] or [)
    It could be fixed in two ways. First is to change line 19 in /path/to/tmp/dir/.build/src/glibc-2.9/scripts/gen-sorted.awk Is:
    sub(/\/[^/]+$/, "", subdir);
    Should be:
    sub(/\/[^\/]+$/, "", subdir);
    Or simply by installing gawk, reconfigure and recompile crosstools-ng.
This was my first post related to linux embedded enviroment. Hope it will be more. Enjoy!

Splitting my personal repository

After writing a few posts in which my workspace scripts could be useful, I realized that I should split blog works from the scripts that can be useful to others. From now materials related to the blog I will track in a separate repository and I'll add them as submodule into my environment. This implies that if we intend to deliver something to a broader audience this should include only those things which the recipient is expected and which we say that we provide. Any additional items should be removed. We get a lot of unwanted stuff today, at every opportunity. From an economic point of view, it is known that very often margin value law is used here i.e. by hypermarkets, I will touch on this subject on another occasion, related to economics.

Monday, March 12, 2012

Debugging coreboot in qemu environment - part 1

First of all I use testing version of Debian - wheezy. Clone coreboot repository:
git clone http://review.coreboot.org/p/coreboot
Conifigure:
cd coreboot; make menuconfig
Set:
Payload  ---> Add a payload ---> FILO
Payload  ---> FILO version ---> HEAD
Debugging  ---> Check PIRQ table consistency
Debugging  ---> Output verbose malloc debug messages
Debugging  ---> Output verbose ACPI debug messages
Debugging  ---> Enable debug messages for option ROM execution
Debugging  ---> Built-in low-level shell
Debugging  ---> Trace function calls
Try to build:
make
If everything builds correctly you can process. Sometimes there is need to use cross compiler. To build one:
cd util/crossgcc
./buildgcc
To explore coreboot code effectively I suggest to create tags and cscope database for coreboot. In my personal workspace I've got process that I go through before I start work (if you use my workspace configuration which is available @github you can follow below steps directly, if not adjust to your enviroment):
  1. run vim ;)
  2. :cd /path/to/code
  3. s<Tab> (fuzzyfinder -> bookmark dir)
  4. si (fuzzyfinder -> change dir)
  5. sr (run ctags to generate tags and cscope to build symbol database - ctags -R;cscope -R -q -b -v)
After steps above we can start work with code. Run vim in coreboot dirctory. Type:
:e src/cpu/x86/16bit/reset16.inc
Put cursor over protected_start and press Ctrl-]. If everything goes ok you should jump to build/mainboard/emulation/qemu-x86/bootblock.s line 537.


In second article we dive into first phase of coreboot execution in emulated environment.  

Wednesday, March 7, 2012

"Human Action" Ludwig von Mises


"Human Action" Ludwig von Mises's work by some of called the pope of free market. IMHO is one of the best books on economics I read in my life. It is not easy, however, giving it time with certainly pays off. Allows to understand the basic laws which ruled the modern world of finance. Mises in a sensational way talk about the myths explaining what is inflation, why it has impact on our lives, what is and how works the supply of credit, and who depend most on the production of money without coverage (fiat money). I can say that until I read this book, I was lost in the dark. Thanks to it I received a weapon to overthrow ridiculous  world order concepts, absurd messages communicated in the media. I recommend: mises.org store - Human Action