Running Applications on the NetBSD Rump Kernel by Justin Cormack
1 like2,634 views
The document describes the NetBSD Rump kernel, which allows running applications without traditional kernel capabilities like executing binaries and scheduling threads. It covers various developments including support for multiple operating systems, implementation of POSIX APIs, and the ability to run unmodified NetBSD binaries, highlighting its use in driver testing and debugging. Additional features discussed include continuous integration for testing, support for microkernel environments, and future improvements needed for broader compatibility and documentation.
Running Applications on the NetBSD Rump Kernel by Justin Cormack
- 1. Running Applications on the NetBSD Rump Kernel Justin Cormack @justincormack
- 2. Rump kernels
- 4. What is a rump kernel? It is the (NetBSD) kernel without support for • executing binaries • scheduling threads • managing hardware privilege levels • most memory management eg virtual memory 4
- 5. Just drivers • Drivers are the bit of the kernel that turns raw hardware into nice abstractions like files and sockets • Get something else (host) to do the other parts (host environment) • Or do without it, just compile in single purpose application. 5
- 6. What use is that? • You get the missing capabilities from a hypercall layer. • Like a very simple virtual machine • man 3 rumpuser • Provides memory allocation, threads, mutexes, console output, random numbers, clock • This can run in userspace for example, but can be implemented in any environment fairy easily 6
- 7. What use is that? • Original use cases around testing drivers. • You can test a kernel driver without booting the OS. • You can test much of the OS without booting into it. • Debugging is easy as it is just a userpsace program • Also for running kernel drivers in userspace eg for file systems 7
- 8. How was it used? • Typically either written for rump like the NetBSD tests rump_init(); rump_sys_mkdir(path, 0777); • or using LD_PRELOAD to change some calls using rumphijack library • The interface was essentially the syscall interface (minus parts not supported by rump) 8
- 10. Xen port • In August last year we got a Xen port of the rump kernel working. In addition to porting the hypercall layer, we also got NetBSD libc to compile against the rump kernel. • We replaced the syscalls in libc with calls to the rump kernel syscall implementations. • This enabled running real applications, in particular we had LuaJIT running, and a simple web server. • Not just syscalls, more of a POSIX API. 10
- 11. Increased portability The POSIX implementation of the rump kernel now runs on • NetBSD • FreeBSD, OpenBSD, DragonflyBSD • Solaris • Linux, including Android • on architectures including arm, x86, amd64, powerpc, sparc64 and mips 11
- 12. Increased testing There is continuous integration using TravisCI and buildbot • Tests all commits to the buildrump.sh script on multiple operating systems and architectures • Tests NetBSD -current hourly on multiple operating systems and architectures • Good test of NetBSD portability issues • Uses ljsyscall to test functionality - good syscall coverage, fast. 12
- 15. rumprun-posix • Late last year I started work on rumprun, which allowed running unmodified NetBSD binaries with the rump kernel. • The main issue with this is that you have two libcs involved, so there are symbol conflicts. However with judicious symbol renaming it turned out to be possible to fix this. • In particular you can compile userspace binaries for the core NetBSD commands like ifconfig, etc. • Can test userspace without booting into OS 15
- 16. How does it work? • Compile NetBSD object code and libraries on host machine • Link code and libraries to new object with -nostdinc • Rename read symbols to rump___sysimpl_read ... • Fix up main and other things which will be called from a stub • Localise symbols to avoid conflicts • Link with host libc and rump libraries • See script for more details, it is messy... 16
- 17. # ./rumpdyn/bin/rump_allserver unix://sock # export RUMP_SERVER=unix://sock # ./bin/ifconfig lo0: flags=8049 mtu 33648 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet 127.0.0.1 netmask 0xff000000 # ./bin/ping -o 127.0.0.1 PING 127.0.0.1 (127.0.0.1): 64 data bytes 64 bytes from 127.0.0.1: icmp_seq=0 ttl=255 time=0.000000 ms ----127.0.0.1 PING Statistics---- 1 packets transmitted, 1 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.000000/0.000000/0.000000/0.000000 ms 17
- 18. Supported commands arp cp dumpfs fsck_msdos ls mount_ext2fs ndp pax reboot sysctl wpa_supplicant cat dd ed halt mkdir mount_ffs newfs pcictl rm umount cgdconfig df fsck ifconfig mknod mount_msdos newfs_ext2fs ping rmdir vnconfig chmod disklabel fsck_ext2fs ktrace modstat mount_tmpfs newfs_msdos ping6 rndctl wlanctl chown dump fsck_ffs ln mount mv npfctl raidctl route wpa_passphrase 18
- 19. What runs? • Almost anything in the NetBSD core can probably be compiled • There is now a cross compiler for rumprun, that should work with most code • There is experimental pthreads support, not yet well tested • Programs that fork or use other things not supported by rump may be a problem • Some of the missing rump features are emulated or partially emulated, eg anon mmap 19
- 20. rumprun - TODOs • Clean up and upstream libc build modifications • Fix other architectures, currently only working on x86, amd64 • Build more of userspace tools and libraries • Use for NetBSD tests - ability to run tests on a kernel without booting it very useful when developing. • Improve build process • Continuous integration and testing on NetBSD current. 20
- 22. rump on green threads • A few months back I added a "green threads" userspace implementation to the userspace rump implementation. • The default implementation uses pthreads. • Uniprocessor, single thread • Useful for embedded implementations • Also useful if you want to make a deterministic implementation 22
- 23. PCI support • Some PCI support has been added • Linux userspace support. • Linux has uio and vfio frameworks for userspace pci drivers. • uio supported now, only works for some cards • vfio may be later; requires an iommu. • Can be used for developing NetBSD drivers even so - used for wireless development • Really want to add BSD support 23
- 24. rump on bare metal • New project to run a rump kernel on (currently x86) bare metal • https://github.com/rumpkernel/rumpuser-baremetal • Includes PCI support, with virtio drivers for KVM, bhyve • Only a few hundred lines of code to deal with interrupts etc, written in a week • Planning ARM port to mbed 24
- 25. rump on microkernels • Genode, a microkernel OS framework, has started using the rump kernel to support NetBSD file system drivers on microkernels. • Less than 3,000 lines of (untrusted) glue code. • See genode.org for more • Minix3 is another potential user, it already uses a lot of NetBSD code. 25
- 26. Logo 26
- 27. Architecture
- 28. Four different environments • hosted, e.g. userspace • paravirtualized, e.g. Xen • "bare metal", e.g. hardware or hypervisor with virtio • microkernels (as servers) 28
- 29. Architecture 29
- 30. Documentation • Documentation is much improved. • All at http://wiki.rumpkernel.org/ • Best places to start are Getting Started and Kernel development tutorials. • A longer introduction is the rump kernel book 30
- 31. Use cases 1. Driver development 2. Tests 3. Drivers for other environments 4. Applications with userspace drivers, eg networking 5. Running code securely eg file system code 6. Very lightweight "containers" with their own OS library 7. ... 31
- 32. What needs doing? • Upstream and improve rumprun • Improve documentation • Portability: native Windows, OSX • Userspace IP stacks: need good performance on 10Gb • dogfooding • ... 32
- 33. Get involved • http://rumpkernel.org/ • Freenode #rumpkernel • Mailing list rumpkernel-users • twitter @rumpkernel • 25 November operatingsystems.io conference in London • 26 November hackday in London • Fosdem 2015 33