The Clockwise real-time file server for Linux

For the impatient here is Clockwise's installation readme.

Clockwise is a real-time file system that schedules best-effort and real-time disk requests so that (1) real-time disk requests are always served on time (i.e. finish before their deadline), and (2) it tries to schedule best-effort requests as quickly as is possible without violating (1). The basis for this scheduling work is a non-preemptive EDF scheduler from Jeffay et al. This work proved that if a task set satisfies two conditions (which are carefully explained in my Ph.D. thesis), requests can be scheduled by a non-preemptive EDF scheduler. I have extended this theory with slack-time calculations, and this slack time is used to prioritize best-effort requests (such as Unix file-system disk requests).

To demonstrate that this approach is worthwhile, I have made two sample implementations: one for a real-time micro-kernel called Nemesis, which is developed as part of the Esprit Pegasus II project and a Linux version. This page describes the latter.

As said, Clockwise schedules disk requests. Applications that use Clockwise can request periodic (real-time) service, or a best-effort class of service. A real-time application can request bandwidth from Clockwise by establishing a Quality of Service (QoS) contract with Clockwise. The QoS parameters are in terms of used block size (i.e. the buffer size of the malloced buffer) and required bandwidth. Clockwise translates the QoS request into a period and slice (i.e. amount of required disk time) and uses these parameters to execute Jeffay et al.'s schedulability test. When admitted, the application is certain it can request data blocks every period. Best-effort applications are served by Clockwise with a higher priority than the real-time requests. The reason for this is that we believe best-effort applications are served well when the requests are scheduled as quickly as possible.

To map the QoS parameters to a slice, Clockwise needs to know which disk blocks will possibly be requested during a session. For this, Clockwise organizes disk blocks in dynamic partitions. A dynamic partition can best be described as a logical grouping of ranges of disk blocks. The basic unit of a dynamic partition is a consecutive disk range of currently 1MB. We call such a range of 1MB a dynamic partition block. A number of these dynamic partition blocks (possibly from a number of disks) are grouped together and can be accessed as a whole. Note that these dynamic partition blocks do not have to be physically consecutive. When a QoS request arrives in Clockwise for a dynamic partition, Clockwise finds out which physical disk blocks are associated with the dynamic partition and determines the slice parameter. Note that Clockwise needs to know the exact characteristics of a disk (including zoning and such).

For Linux, Clockwise is implemented as an intermediate layer between the Linux file system and the disk device drivers and it behaves exactly like a real disk driver (it has, in fact, its own block and character device number). When a request arrives in Clockwise through the block or character device interface, it maps the device number to a dynamic partition number (based on the minor device number), associates release and deadlines to the request (when applicable), finds the real disk driver and disk block number (through a dynamic partition mapping table), and re-issues the request to a real disk driver by means of a helper process. Each scheduled disk is served by a seperate helper process. Higher layer software does not notice that the Clockwise device driver is only a pseudo-disk driver.

Clockwise dynamic partitions can easily be used for Linux EXT2 file system storage. One first creates an dynamic partition (possibly on multiple disks) and, next, one formats an EXT2 file system on the dynamic partition by executing mkfs -t ext2 /dev/cwb_usr0 if the empty dynamic partition is registered under /dev/cwb_usr0. Next, the dynamic partition can be fscked and mounted into the name space. Since dynamic partitions introduce an extra level of disk addressing, physical disk addresses can be changed without having to notify the higher level application: an EXT2 file system may be moved to entire new disks, without the file system noticing. To demonstrate that our system works, the links to the actual HTML pages describing the movies are served of a Linux EXT2 file system that runs on top of a Clockwise dynamic partition.

To use Clockwise dynamic partitions for real-time service (such as the recording and playback of continuous media), one uses Clockwise's character device driver interface to read and write a dynamic partition. The character device's interface supports ioctls for setting QoS parameters and to use the driver a-synchronously. The latter is important to implement double (or triple, etc...) buffering without having to revert to multi-threaded applications. A sample application that uses Clockwise's real-time functionality and double buffering can be found here. This utility reads data from Clockwise in a real-time manner and sends the data across an ATM network.

The current Linux device drivers do not use DMA functionality well: most device drivers use the CPU to copy data from DMA buffers into user virtual memory. Clockwise for Linux improved on this situation by allowing direct DMAs to user virtual memory. For this it uses memory locking extensively and it uses Linux kernel functionality to obtain user physical addresses. Note that to get this really working, the NCR disk device driver had to be repaired to deal with non-aligned DMA-addresses (such as the addresses that are returned by malloc(3)).

Clockwise for Linux currently runs on linux-2.2.1, with experimental ATM additions that can be obtained here. Our real-time file server is a simple off-the-shelf PPro that runs at 200MHz, is equipped 64MB of EDO DRAM. Currently it schedules 2-6 10,033 Seagate Cheetah disks that are controlled by 2-3 NCR53c875 Ultra-Wide SCSI controllers. Alternatively, we can schedule Quantum Atlas-II disks. We use a ForeRunner LE155 ATM network interface. Running Clockwise is not CPU-bound. We have not measured loads above 10% for a fully loaded Clockwise.

Here are some examples with a number of sample movies on the file server. NOTE THAT DURING OFFICE HOURS THESE LINKS MAY NOT WORK AS THIS IS AN ONGOING PROJECT. OFFICE HOURS ARE BETWEEN 10:30 MET-1DST AND 16:30 MET-1DST.

Both movies are digitized with a Linux Media Labs LML33 motion-JPG compressor and stored in Clockwise. These cards can digitize both NTSC and PAL video. Linux sources for the card are available through the home page of Linux Media Labs. Based on the original implementation, we have re-implemented the driver for this card and we have written utilities to render video in an X-window. We will release this software in a short while. In any case we can recommend the card if you need to digitize video and do not want to spend a bundle. (Note that it seems that older machines have trouble with a 20MB/s video stream from the LML card to a video card -- in our case a Matrox Millenium: half of half (so a quarter) of the video is not displayed. Our current machines have no trouble in rendering full PAL video on a screen.)

I have written software to `find' scenes in the movies automatically by analyzing the red, green and blue values in the movies. Note that this software needs improvement. My current `Friday'-afternoon project (even though it is Wednesday), is to find silence in audio streams. With such a silence detector I hope to improve the scene detector. If you have good scene detectors, please send me e-mail and we can try your code.

Each of the scenes is click-able, but before audio and video can be sent to you I need to know how to get the data to you (otherwise you experience a broken link!). Currently, only raw ATM is supported for audio and video. Changing the code to deal with UDP over fast-Ethernet is trivial, but since I do not have such a network into my office, there is little point in porting the code.

Clockwise for Linux will be formally released shortly. Please send me e-mail to be added to the distribution list. A sample distribution for Linux 2.2.1 (other kernels may work) can be found below. Note that the driver does not compile: you need an implementation of qsort. I used the Berkeley one, but the file has a copyright on it. To fully use Clockwise, you also need my kernel modifications. Also, the utility parselml uses the Independent JPEG Group's JPEG software to decompress LML video.

If you have questions, please do not hesitate to send me e-mail.

Here's a link to my home page, this is a link to research group's home page and this is a link to our faculty's home page.