Details and and terms for Hurd on L4

Hurd/L4's description

Here's a webpage with a summary.
Here's a pdf built from the tex documentation source in CVS. (compiled 2005/2/13)

semi-Quick reference guide to Hurd on L4

Terms

• "Privileged process"
  This is a process which has access to some extra sys-calls on L4 such as thread scheduling, and memory mapping. These processes still run in ring 3 and have their own virtual memory space.
• "Capability"
  This is a concept used in the abstraction of IPC. L4-Hurd implements RPC as an abstraction layer on IPC using Capabilities. The idea here is that you aquire a capability to a task. This simultaniously implies that messages sent to that task will be allowed to get to that task (not be blocked by the kernel), and that the Task server will not reuse the thread id of the thread that you have the capability too. This guarantees that the thread you think your talking too continues to be the same thread you started talking too. Once I have a capability I am allowed to copy it to another client, in a similar manner to a memory segment on L4.
• "Task info Capability"
  used primarilly for IPC this keeps the task server from re'using the thread id, and implies that the holder will be notified if the task dies (and thus the thread id can be re-used). These can be created any time, by anyone. Quotas or something similar will be used to keep processes from DOS'ing the thread id's.
• "Task control Capability"
  Basically the same as the Task Info Capability excep that it also allows the holder to do something protected over the IPC connection, i.e. somehow control that task over IPC.
• "FPage"
  An Fpage in L4 is a memory region. It must be aligned on a multiple of it's minimum size (meaning it uses paging). These can be copied, or just handed off through capabilities securely. Basically it's a simple paging abstraction supplied by L4
• "IOFPage"
  Similar to an FPage an IOFPage is a memory region, the difference is that an IOFpage refers to memory related to IO (I.E. not normal physical frames). IO Protection is done through IOFpages so that userspace processes device drivers can have access to necicary IOPorts, and not more than that.

What's what

Note that the following list is in approximate boot order

Primary Servers

• Laden -- (written)
  • Laden is the bootloader. When the system boots Grub loads Laden, Laden then loads the L4 kernel into memory and Sigma_0, an initial memory manager. Next it loads Wortel, the first privileged process called the "rootserver" in L4 terminology. Wortel then loads physmem, task, deva, and the root filesystem servers.
• Sigma_0 -- (L4 supplied)
  • Sigma_0 is the initial memory manager, on boot all physical memory is mapped here. After the initial boot though all memory is moved over to the physmem server. Sigma_0 is an L4 concept used only for boottime. This is a priveledged userspace process. This server has a fixed thread id of (UserBase, 1)
• Sigma_1 -- (a nicety for later)
  Allocates Thread Control Blocks. The idea here is that they don't belong to the task, so thread persistence is orthogonal to task servers (I think?). This has a fixed thread id of (UserBase+1, 1) - in L4 thread id's are ordered pairs. This is a priveledged userspace process.
• Wortel -- (written, and to spec, more functionality will probobly be added later though)
  • Wortel is the first privileged process and spawns all other privileged processes. Later in life it serves as a portal to priveledged syscalls, and as a watchdog for forced system shutdown and the like. Wortel retains all physical pages so it has complete control of the system. This is a privileged user-space process. Wortel has a fixed thread id of (UserBase+2, 1).
• Physmem -- (written, no paging support yet)
  • The physical memory server is in charge of handing out physical memory to processes. All physical frames are mapped to physmem on boot, and from then on it hands them out to processes that need memory. Physmem will also at least act as a proxy for the pager. This way the pager doesn't necicarilly have to page out memory on a userspace pageout call if there's still plenty of memory.
• Task -- (partially written)
  • The Task server acts as the scheduler. It hands out thread id's. Capabilities work by making certain that a thread_id is not re-used, so a process can guarantee that another process is still the same one it originally established the connection too. As a result the Task server's other job is to hand out capabilities for the purposes of IPC. The task server is implemented as a privileged process in user-space. The scheduler in Task does not yet support SMP. Most likely the task server will run at extremely high priority.
• Deva -- (doesn't exist yet)
  • Deva, the DEVice Access server, is the device manager and database. This will be implemented as a privileged process running in user-space, which supplies a filesystem interface mounted at /dev on the root filesystem. Basically deva will check that the user has access to this device, get a "capability" from the task server (basically, give the user access to the device directly), and then disappear from the picture. Currently this does not exist.
  • To go along with deva L4/Hurd will also probably require a small library for interfacing to deva and to users for easing the process of writing drivers for L4-Hurd. The most interesting feature of the Deva/device driver design is that device drivers do not need to be privileged processes. Deva will load them, and map whatever "IOFpages" (basically IO memory regions) to them that are necessary. This means that it is safe for normal users to load new device drivers, assuming of course that they have privileges to a given device.
  • The most similar thing to deva is L4env. L4env is a driver under-layer designed for writing other operating systems (such as DROPS). L4env
• Rootfs -- (doesn't exist yet)
  • This is the the root filesystem that all other filesystems will be mounted too. Basically this supplies / and otherwise acts as a multiplexer to other filesystems. Implemented as a normal user process.
• Elf-Loader -- (L4 supplied)
  • Supplied by L4, this is the Elf-Loader used to load anything not loaded by Laden. Again this is a privileged user-space process

Secondary servers and libraries

• libcap -- (exists, and almost tested'ish)
  • Library that basically every program ever will compile against. This library supplies the interface, known as a capability, by which you talk to any server. This means the task server, the filesystem server, deva whatever.
• libcap-server (exist, and almost tested'ish)
  • Library that all servers will compile against. Instead of trying to manage capabilities in some global capability server they are handled by each server individually using this library. It includes such things as automatic thread management for servers, and centralizes sensative code related to talking to untrusted users over capabilities.
• libdev -- (doesn't exist yet)
  • Device driver library, Supplies a set of interfaces for user's to call into the device driver, and calls to Deva to request IO mappings. For how to request IO mappings from the kernel we can look at the libraries written for L4env called the Device Driver Environment (DDE). Initially we will probobly request these at boottime from the kernel, and grant them to the device drivers using wortel or deva. Eventually IO mappings will be granted in some other fashion.
• libfshelp (needs port from hurd)
  • As a serverside Library, this is used by any actual filesystem, filesystem locking, naming starting translators (submounted filesystems), and other basic functionality required of any real filesystem.
• libstore (needs port from hurd)
  • As a serverside library, this is used by filesystems residing on block devices. This allows a filesystem to access a file as a set of blocks. So if a device somewhere exports itself as a file, we can access it usining this library as a block device. Also includes some minimal raid support and junk.
• libdiskfs (needs port from hurd)
  • Serverside library used by a filesystem that are like a normal disk filesystem. Has allocation code, paging stuff, node level control functions, functions for directory change etc.
• libtrivfs (needs port from hurd)
  • Serverside library which gives some simple interfaces for implementing trivial filesystems like /dev/null, /dev/zero etc.
• libiohelp (needs port from hurd)
  • Serverside library for file'ish port io. Much of the io part is actually depricated. This includes the read, write and seek concepts in the io library. This is because it can be better implemented using concepts from other libraries. The functions for creating new "io_users" which basically means a set of GID's and UID's attached to a port are still used though.
• Message wrappers
  • These are the message primatives that, on Mach, are generated by MIG. On L4 though, these must be hand written (for now). L4's message passing system is much more straitforward than Mach's, so hopefully this won't be to horrid.
• Proc server -- (doesn't exist yet)
  • Imposes POSIX process semantics on processes that want it, suppling some of the extra stuff necicary. POSIX Signals are done through the Proc server.
• Policy server -- (doesn't exist yet, still fuzzy)
  • Uses a task-manager capability on the task server to install a policy callback used one creation of new threads etc. Here is where we implement quotas etc.
• Auth server
  • Part of the POSIX psersonality, the Auth server will supply a process with capabilities to other servers based on the user ID. This allows for traditional Unix ACL's based on UID. For efficiency it can hand out passport capabilities, which a task can then show to other servers as proof of identity. Basically this is just a capability to the Auth server with the UID attached to the end, passport capabilities are non-transferable. (I.E. the Auth server will stor the thread id along with the UID as part of the object attached to the capability)

Terminology differences between L4/Hurd and Mach/Hurd

• Capability != Port
  In Mach/Hurd there's the concept of a "port" a port is a handle into another server. All IPC is done through ports. Ports have to chance slightly though in the port to L4. Mach supports fully async IO, but this means queuing the messages in the kernel. L4 is supposed to be as simple as possible, so it does no such queuing. Thus we have capabilities. In Mach/Hurd a file is a port (basically, ports are like file descriptors). In L4/Hurd capabilities are handles to another process, but a file is not a capability. Rather a capability contains a hook into a file object on the fs server, inserting such a hook would equate to opening a file. This means we don't have to throw away and re-create our hard earned connection (guarantee that the other task is who we want) when we open or close files. A port automagically deals with this problem by acting like a file descriptor, if the taks your talking to is gone, you have a dead handle. In L4 though IPC works by Thread ID, so we have to guarantee that someone hasn't replaced the task we want to be talking too.
• Hurd-L4 does not use MIG... yet
  Hurd uses something called MIG building IPC interface stubs. This is due to the complexity of actually setting up a message to be sent over a MACH port. Someday (assuming it makes it) Hurd-L4 will need such a compiler as well, but since no-one has written one, for now they're sticking with defining them by hand.

Code layout (how many lines of what)

• L4Ka::Pistachio (we shouldn't touch this), 71418
  • Lightweight synchronous IPC, 1649
  • Minimal memory abstraction, 2332
  • Scheduling foo, 676
  • IA32 glue: 6893 (idt's, exceptions, tls, mem info, etc.)
  • IA32 arch: 3077 (syscall layout, segmentation fpu stuff)
• L4-Hurd 142441 (most include >5000 lines of malloc)
  • Memory management server (exists), 8851
  • Task server (exists), 7917
  • Bootloader (exists), 5640 (no malloc, but drivers)
  • Driver interface layer (not exist), 7617
  • RPC abstraction of IPC (exist), 1428
  • RPC abstraction serverside (exist), 5977
  • libc (exists, but borken), 17103
  • libl4 (exist), 15556
  • libpthread (exist), 13176
  • Other libraries/datastructures, 7046
• Approximate percentage assembly
  • L4-Hurd is 7% assembly
  • libl4, and crt0 files primarily assembly (one copy per server)
  • A bit of libc, the rest is mutex's and syscall stuff
• Status of compiler linker debugger: gnu toolchain, no dynamic linker yet
• Licence flavor Hurd - GPL L4:Pistachio - two clause BSD (== X11?)

Primary developer/project mappings that I know
(I know there are more):
L4-Hurd people:

• Mattieu Lemerre -- ipc mechanisms for deva
• Daniel Wagner, Peter De Schrijver -- Primary developers for deva, but no recent commits.
• Marcas Brinkman -- Head developer. Has been hacking glibc.
• Niel Walfield -- Server side IPC