I found a few different pages with information on debugging a Linux kernel with kgdb. There wasn't a single source with all the information I needed to get set up and working though. So here is how I set things up on my Linux host machine to debug a target Linux kernel running in a VirtualBox VM.
Set up the virtual machine
When setting up the VM, add a serial port and set it to use Host Pipe. You can check the box to have VirtualBox create the pipe too. Give a path to the file on the host where you want the pipe. This will become the virtual serial console that gdb connects to on the host machine.
Install VirtualBox with a VM containing the kernel you want to debug. Build the kernel with debug symbols, kgdb support, sysrq keys
- Kernel hacking->
- [*] Magic SysRq Key
- [*] Kernel debugging
- [*] Compile the kernel with debug info
- [*] KGDB: kernel debugging with remote gdb ->
- [*] KGDB: use kgdb over the serial console
Build your kernel and install the modules like you would any other kernel.
Add kgdboc=ttyS0,115200 to the kernel paremeters (ttyS[0-3] depending on which COM port you choose in the VM setup.)
Copy the vmlinux (uncompressed version to the host machine).
Also copy the .ko for the module you want to debug.
Get the target VM ready for debugging
After your module is inserted, you need its .text address so that you can see the source lines in gdb.
- cat /sys/module/XXX/sections/.text
When you're ready to start a debug session:
- echo g > /proc/sysrq-trigger
Ready the Host to connect to the VM
- > socat -d -d /path/to/pipe pty &
- 2010/04/29 12:18:44 socat N successfully connected from local address AF=1 "\0\0\0\0\0\0\0\0\0\x02\0\0\0\0[\0\0\0|\0\0\0w"
- 2010/04/29 12:18:44 socat N successfully connected via C\xE66
- 2010/04/29 12:18:44 socat N PTY is /dev/pts/3
- 2010/04/29 12:18:44 socat N starting data transfer loop with FDs [3,3] and [4,4]
- file vmlinux
- set remotebaud 115200
- target remote /dev/pts/3
- > add-symbol-file <path to module> <.text address>
- > b some_file.c:NNN
- > c
You need to convert the pipe that VirtualBox created to a serial port like object. socat does that job:
Notice that your dev is pts/3.
Now we make a .gdbinit file so you can start gdb without typing the commands every time.
You'll change your pts setting to whatever is relevant.
On the target, when you send the g to sysrq-trigger, you should drop to a breakpoint in gdb.
Next job is to load the module symbols.
Now you can set breakpoints or press c to continue. The target should continue until it comes to a breakpoint or encounters an error.
A last tidbit, when I wanted to close gdb on the host, use detach instead of quit.