divergefs file system for Plan 9
I worked on a diverge file system for Plan 9 operating system which resolves many slightly-irritating situations such as needing shell scripts containing many bind commands when compiling kernel or system programs as an unprivilged user.
Plan 9 uses file metaphor throughout the whole system, and programs communicate through a special file protocol called 9p. As a result, the implementations are cleaner, and users can enjoy uniform operations. Since kernel only needs to know 9p, the job of understanding different file formats, and exposing other resources as files can be easily implemented in the user space. Plan 9 also allows each process to have a separate namespace which means a single file can be different things in different processes. Therefore, many ugly hacks that are seen inside the kernel of normal Unix clones such as /dev/tty, can again be easily implemented in the user space.
Even though Plan 9 is versatile with namespace manipulation through bind and mount, kernel compiling as a normal user can be difficult. In order to compile the kernel, the user has to create several shadow directories for the ones that they don't have write access for and bind all the shadows in front of the old ones.
The matter gets worse when the user wants to modify or apply patch to the underlying read-only files. The current "bind" mechanism isn't smart enough to perform the modification automatically inside the shadow directories when editing files in the read-only directories, so the users are forced to manually copy all the files they need to change into their shadow directories.
The following two scenarios illustrate the problems described above:
I wrote a user-level file system that solves the problems above by allowing files to "diverge" from the original mount point onto user specified directories.
The job of understanding 9p is already done in lib9p which also handles dispatching incoming packets to program specified callbacks and delivering outgoing packets for all the file operations. Therefore, the program needs only to interpose itself on top of the original mount point and properly route the requests to different directories.
File deletion and renaming are more subtle than they first appear. Consider this directory tree:
top/ test.txt base/ test.txt
Assuming that top is layered on top of base,
to the user there is
only one file called text.txt which is served from the top
If the user
In divergefs, the special "hole"
mechanism is used to provide the proper behavior of the layering
file system. When user performs
Instead of having all the changes saved into a single overlay directory, rules give users the flexibility to "diverge" files to different layers of the directory as long as the file satisfies the rule for that layer.
Rules include file extension matching, regular expression file name matching, file attribute matching (readable, writable, directory and etc).
Regression test suite
The test suite helped a lot during development in identifying bugs and preventing introducing bugs. A generic testing framework written in C is included with the source code under the tests subdirectory.
Generic data structures
To unburden future programmers from reinventing wheels, generic implementations of array, set, and dictionary were developed in the course of this project.
Lex/yacc configuration parsing
Since the configuration file syntax is only experimental, scanner and parser are generated from lex and yacc files to provide more flexibility for future modification either by myself or others.
Once divergefs was stable enough, it was used during its own compilation process. The result was significant speed up during development on foreign Plan 9 machine with all the source code located on local Linux machine. Right now, divergefs is stable enough for regular Plan 9 kernel and userland programs building as a normal user. The source code is currently hosted on the "sources" repository at /n/sources/contrib/de0u/divergefs.tar.gz.
[Last modified Tuesday August 23, 2005]