Catching Up

20 Aug 2013

Well... I've been bad at updating the blog on my process again...

Last time I missed updating, I made a branch from an earlier git commit, and threw away a bit of work and messy code rather than trying to catch up.

This time, however, I'm actually quite happy that I didn't write too much, because I've recently changed a lot of what I would have written about.

So instead, I'll describe one of my most recent changes here - a new cross compiler and a newlib port - and then in subsequent posts go over chosen parts of the other things I've done.

So let's jump into it.

Why a cross compiler

The be-or-not-to-be of cross compilers has been discussed at length and it seems every time someone posts a description on how to build one they are told that you don't need one if you know how to use your compiler or whatever. For examples, read the osdev wiki - Why do I need a Cross Compiler? talk page (I don't recommend it if you're in a bad mood).

Well, I'm developing on OSX but have chosen to build an X86 kernel in elf format due to the availability of documentation, support and expertize. That's reason enough for a cross compiler build of binutils at least.

Clang offers excellent support for cross compilation out of the box, so I didn't use to have a need for a gcc cross compiler as illustrated in this post. As it turns out, however, compiling newlib with that setup is rather annoying.

I'm also a fan of clean makefiles. Take a look at this:

VPATH := ../src

CC := i586-pc-myos-gcc

TARGETS := $(shell find ../src -name "*.c")
TARGETS := $(notdir $(TARGETS))
TARGETS := $(patsubst %.c, %, $(TARGETS))

all: $(TARGETS)

    -rm $(TARGETS) 2>/dev/null

That's the makefile for the entire /bin directory in my os.

So let's move on...

Building newlib and gcc

For my toolchain build, I followed the tutorial at the osdev wiki with a few changes. I chose to build everything inside the scope of Homebrew to keep track of all files and eventually make a formula for it. So after applying the patches described in the post (I even kept the name i586-pc-myos since I don't have a working name for my kernel besides an iteration number...) I did

$ export TARGET=i586-pc-myos
$ export PREFIX=/usr/local/Cellar/osdev/1.0
# Configure, build and install binutils
$ brew link osdev
# Configure, build and install gcc and libgcc
$ brew unlink osdev
$ brew link osdev

And that prepared me for building newlib.

Building newlib

Now this was an adventure...

Newlib uses a directory structure that hasn't been supported by automake and autoconf since a couple of versions. More specifically, you need automake version 1.12 or earlier and autoconf version 2.64. Unfortunately, those versions are not available through Homebrew, so ...

$ curl -O
$ tar -zxf automake-1.12.tar.gz
$ mkdir -p build-automake
$ pushd build-automake
$ ../automake-1.12/configure --prefix=/usr/local/Cellar/automake/1.12
$ make all -j
$ make install
$ popd
$ curl -O
$ tar -zxf autoconf-2.64.tar.gz
$ pushd build-autoconf
$ ../autoconf-2.64/configure --prefix=/usr/local/Cellar/autoconf/2.64
$ make all -j
$ make install
$ popd
$ brew switch automake 1.12
$ brew switch autoconf 2.64

Those last two lines tells Homebrew that you want to use those specific versions for now.

Now for the neat part. I followed the wiki post and used the syscall interface i've described earlier but I also wrapped crt0.S and syscalls.c in


Then I built it all through

$ pushd build-newlib
$ ../newlib/configure --target=$TARGET --prefix=$PREFIX
$ make -j
$ make install
$ mv $PREFIX/$TARGET/lib/libc.a $PREFIX/$TARGET/lib/libkernel.a
$ rm -rf *
$ ../newlib/configure --target=$TARGET --prefix=$PREFIX
$ make -j
$ make install
$ popd

This gives me two versions of the newlib c library. One with all syscalls defined and one without. The latter is suitable for linking into the kernel and allows me to use a standard C library with things like sprintf and even a ready made malloc.

"But wait," you say. "Doesn't malloc make an sbrk() syscall? You can't make a syscall from inside the kernel, can you?"

Well. Remember my last post where I said that keeping versions of every syscall function inside the kernel would turn out to have some really cool advantages? Here they are.

I need an sbrk anyway, so all I have to do differently is make it check whether the request came from a user process or from the kernel and handle them a slight bit differently (i.e. make sure the brk is above 0xC0000000 for kernel and below for user mode...).

Similar checks can be made for all syscalls and some doesn't actually need any changing at all.

Needless to say, this revelation changed my kernel structure quite a bit and I entirely threw away my own kernel heap manager.

Using this

Having the cross compiler toolchain and library setup allows me to compile the kernel using a very simple makefile.

Somewhat simplified:

OBJECTS := $(shell find . -name "*.S")
OBJECTS += $(shell find . -name "*.c")
OBJECTS := $(OBJECTS:%.c=%.o)

CC := i586-pc-myos-gcc
LDFLAGS := -nostdlib -T linkfile.ld
LDLIBS := -lkernel

kernel: $(OBJECTS)
    $(LINK.o) $^ $(LOADLIBES) $(LDLIBS) -o $@

and everything else is taken care of by the default rules of gnu make, including preprocessing and assembling .S files.

For executables running under my operating system it's even easier

CC := i586-pc-myos-gcc

That's all.


On a final note, the automatic rules for building object files out of .S files runs them through CC, which means they are assembled by i586-pc-myos-as which uses AT&T syntax rather than NASM which uses Intel syntax. I actually converted all my assembly code to AT&T syntax, but you might want to use the .intel_syntax directive instead.


The methods described in this post has been implemented in the directory toolchain of git commit f09bd57b8e.


comments powered by Disqus
© 2012 Thomas Lovén - @thomasloven - GitHub