20 Aug 2013
Well... I've been bad at updating the blog on my process again...
Last time I missed updating, I made a branch from an earlier git commit, and threw away a bit of work and messy code rather than trying to catch up.
This time, however, I'm actually quite happy that I didn't write too much, because I've recently changed a lot of what I would have written about.
So instead, I'll describe one of my most recent changes here - a new cross compiler and a newlib port - and then in subsequent posts go over chosen parts of the other things I've done.
So let's jump into it.
The be-or-not-to-be of cross compilers has been discussed at length and it seems every time someone posts a description on how to build one they are told that you don't need one if you know how to use your compiler or whatever. For examples, read the osdev wiki - Why do I need a Cross Compiler? talk page (I don't recommend it if you're in a bad mood).
Well, I'm developing on OSX but have chosen to build an X86 kernel in elf format due to the availability of documentation, support and expertize. That's reason enough for a cross compiler build of binutils at least.
Clang offers excellent support for cross compilation out of the box, so I didn't use to have a need for a gcc cross compiler as illustrated in this post. As it turns out, however, compiling newlib with that setup is rather annoying.
I'm also a fan of clean makefiles. Take a look at this:
VPATH := ../src CC := i586-pc-myos-gcc TARGETS := $(shell find ../src -name "*.c") TARGETS := $(notdir $(TARGETS)) TARGETS := $(patsubst %.c, %, $(TARGETS)) all: $(TARGETS) clean: -rm $(TARGETS) 2>/dev/null
That's the makefile for the entire /bin directory in my os.
So let's move on...
For my toolchain build, I followed the tutorial at
the osdev wiki with a few
changes. I chose to build everything inside the scope of
Homebrew to keep track of all files and eventually
make a formula for it. So after applying the patches described in the
post (I even kept the name i586-pc-myos since I don't have a working
name for my kernel besides an iteration number...) I did
$ export TARGET=i586-pc-myos $ export PREFIX=/usr/local/Cellar/osdev/1.0 # Configure, build and install binutils $ brew link osdev # Configure, build and install gcc and libgcc $ brew unlink osdev $ brew link osdev
And that prepared me for building newlib.
Now this was an adventure...
Newlib uses a directory structure that hasn't been supported by automake and autoconf since a couple of versions. More specifically, you need automake version 1.12 or earlier and autoconf version 2.64. Unfortunately, those versions are not available through Homebrew, so ...
$ curl -O http://ftp.gnu.org/gnu/automake/automake-1.12.tar.gz $ tar -zxf automake-1.12.tar.gz $ mkdir -p build-automake $ pushd build-automake $ ../automake-1.12/configure --prefix=/usr/local/Cellar/automake/1.12 $ make all -j $ make install $ popd $ curl -O http://ftp.gnu.org/gnu/autoconf/autoconf-2.64.tar.gz $ tar -zxf autoconf-2.64.tar.gz $ pushd build-autoconf $ ../autoconf-2.64/configure --prefix=/usr/local/Cellar/autoconf/2.64 $ make all -j $ make install $ popd $ brew switch automake 1.12 $ brew switch autoconf 2.64
Those last two lines tells Homebrew that you want to use those specific versions for now.
Now for the neat part. I followed the wiki post and used the
syscall interface i've described earlier
but I also wrapped crt0.S and syscalls.c in
#ifndef KERNEL_MODE ... #endif
Then I built it all through
$ pushd build-newlib $ ../newlib/configure --target=$TARGET --prefix=$PREFIX $ export CPPFLAGS_FOR_TARGET=-DKERNEL_MODE $ make -j $ make install $ mv $PREFIX/$TARGET/lib/libc.a $PREFIX/$TARGET/lib/libkernel.a $ rm -rf * $ ../newlib/configure --target=$TARGET --prefix=$PREFIX $ export CPPFLAGS_FOR_TARGET= $ make -j $ make install $ popd
This gives me two versions of the newlib c library. One with all syscalls
defined and one without. The latter is suitable for linking into the
kernel and allows me to use a standard C library with things like
sprintf and even a ready made malloc.
"But wait," you say. "Doesn't malloc make an sbrk() syscall? You can't make a syscall from inside the kernel, can you?"
Well. Remember my last post where I said that keeping versions of every syscall function inside the kernel would turn out to have some really cool advantages? Here they are.
I need an sbrk anyway, so all I have to do differently is make it
check whether the request came from a user process or from the kernel
and handle them a slight bit differently (i.e. make sure the brk is
above 0xC0000000 for kernel and below for user mode...).
Similar checks can be made for all syscalls and some doesn't actually need any changing at all.
Needless to say, this revelation changed my kernel structure quite a bit and I entirely threw away my own kernel heap manager.
Having the cross compiler toolchain and library setup allows me to compile the kernel using a very simple makefile.
Somewhat simplified:
OBJECTS := $(shell find . -name "*.S") OBJECTS += $(shell find . -name "*.c") OBJECTS := $(OBJECTS:%.S=%.o) OBJECTS := $(OBJECTS:%.c=%.o) CC := i586-pc-myos-gcc LDFLAGS := -nostdlib -T linkfile.ld LDLIBS := -lkernel kernel: $(OBJECTS) $(LINK.o) $^ $(LOADLIBES) $(LDLIBS) -o $@
and everything else is taken care of by the default rules of gnu make, including preprocessing and assembling .S files.
For executables running under my operating system it's even easier
CC := i586-pc-myos-gcc
That's all.
On a final note, the automatic rules for building object files out of .S
files runs them through CC, which means they are assembled by
i586-pc-myos-as which uses AT&T syntax rather than NASM which uses
Intel syntax. I actually converted all my assembly code to AT&T syntax,
but you might want to use the .intel_syntax directive instead.
The methods described in this post has been implemented in the directory
toolchain of git commit
f09bd57b8e.