All My Brain

I found a few different pages with information on debugging a Linux kernel with kgdb. There wasn’t a single source with all the information I needed to get set up and working though. So here is how I set things up on my Linux host machine to debug a target Linux kernel running in a VirtualBox VM.

1. Set up the virtual machine

   When setting up the VM, add a serial port and set it to use Host Pipe. You can check the box to have VirtualBox create the pipe too. Give a path to the file on the host where you want the pipe. This will become the virtual serial console that gdb connects to on the host machine.

   Install VirtualBox with a VM containing the kernel you want to debug. Build the kernel with debug symbols, kgdb support, sysrq keys

   Kernel hacking-> 
      [*] Magic SysRq Key
      [*] Kernel debugging
      [*] Compile the kernel with debug info
      [*] KGDB: kernel debugging with remote gdb ->
         [*] KGDB: use kgdb over the serial console
   

   Build your kernel and install the modules like you would any other kernel.

   Add kgdboc=ttyS0,115200 to the kernel paremeters (ttyS[0-3] depending on which COM port you choose in the VM setup.)

   Copy the vmlinux (uncompressed version to the host machine).
   Also copy the .ko for the module you want to debug.

   Start VM.

2. Get the target VM ready for debugging

   After your module is inserted, you need its .text address so that you can see the source lines in gdb.

   cat /sys/module/XXX/sections/.text
   0xffffffffa00c0000
   

   When you’re ready to start a debug session:

   echo g > /proc/sysrq-trigger
   

3. Ready the Host to connect to the VM

You need to convert the pipe that VirtualBox created to a serial port like object. socat does that job:

> socat -d -d /path/to/pipe pty &
2010/04/29 12:18:44 socat[4411] N successfully connected from local address AF=1 "\0\0\0\0\0\0\0\0\0\x02\0\0\0\0[\0\0\0|\0\0\0w"
2010/04/29 12:18:44 socat[4411] N successfully connected via C\xE66
2010/04/29 12:18:44 socat[4411] N PTY is /dev/pts/3
2010/04/29 12:18:44 socat[4411] N starting data transfer loop with FDs [3,3] and [4,4]

Notice that your dev is pts/3.

Now we make a .gdbinit file so you can start gdb without typing the commands every time.

file vmlinux
set remotebaud 115200
target remote /dev/pts/3

You’ll change your pts setting to whatever is relevant.

start gdb.

On the target, when you send the g to sysrq-trigger, you should drop to a breakpoint in gdb.

Next job is to load the module symbols.

> add-symbol-file  <.text address>
> b some_file.c:NNN
> c

Now you can set breakpoints or press c to continue. The target should continue until it comes to a breakpoint or encounters an error.

A last tidbit, when I wanted to close gdb on the host, use detach instead of quit.

Happy debugging!