Friday, December 7, 2001

Question:

"He wants to ensure that, in light of server and network failures, as long as at least one of his servers is accessible to the backbone, exactly one instance of a particular service is available."

What exactly is meant by "available"? Available to *any* client, or to *all* clients? That is, in light of the possibility for backbone partitioning, is it acceptable for one instance of a service available to only one partition, or must the service be available to anyone who connects to any part of the backbone?

Answer:

Available means available to any client.
More than one instance of this service should never be running globally, regardless of partitioning, &c.

Question:

Can a process chdir to a directory that doesn't exist?

Answer:

I can't think of a reason for that to produce anything other than ERROR.

Question:

Is is valid to Write() zero bytes? If so, what behavior should the following series of calls exhibit?
int fd;
fd = Create("/foo");
Seek(fd, 3000);
Write(fd, 0);
Close(fd);
Sync();

Answer:

the file should be totally sparse.

Question:

when 2 proccesses try to write to the same file, is the following true:
1. it is ok to have one write overwrites portions of the other write, as long as they work as if two writes runs sequentially
2. it is wrong to have interlacing data in the file, as a result of concurrent write (e.g 2 blocks of the 1st write, 1 block of the 2nd write..., 3 blocks of the 1st write)

Answer:

Keeping the data of regular files coherent is the user's problem. They can choose to use some locking mechanism or not. As the OS, our concern is making sure none of the meta data such as inode data or directory entries, become confused. The locking should protect this type of thing, not the user data.

Question:

OK...so, say process 2 is a child of process 1. So they share entries into the global open file table. Now Process 1 and process 2 both call read. What's supposed to happen? Should they both read from the same start offset? Then if process 1 reads 5 bytes and finishes second and process 2 reads 10 bytes and finishes first, process 2 will go backwards next time it tries to read. But if I update the offset continuously, neither will read continuous blocks.

Answer:

Here's what should happen --

One of the processes' read() calls is executed first, and the file pointer is updated accordingly. Then, the next process's read() call returns data beginning from the new offset -- the two read() calls return data from different (consecutive) sections of the underlying file.

Question:

There is conflicting information on Seek on the Q&A. One says that Seek MOST DEFINITELY updates the size. The other says that Seek never updates the size. Which is it?

Answer:

We intended for Seek() to work like the real seek() -- only extend the file upon a Write(). But, we realize that there's been some cobnfusion, so we'll take it either way.

	Thursday, December 6, 2001

Question:

If we get a RmDir() call on "/", what is the appropriate action?

Answer:

Ouch. We didn't spec this. In reality, I consider / to be more of a mount point than a vanilla directory. I'd probably fail it...

Question:

what should happen when a user attemps to 1. open a hard link 2. ChDir to a soft link that points to a dir 3. Unlink a hard link or soft link 4. FileSize a hard link or soft link

Answer:

1. should be exactly like opening a file in any other way 2. should be exactly like chdiring to a normal directory. 3. unlinking a hard link should remove it from the directory structure, and if no other hard links exist, return the blocks associated with the file as soon any open file descriptors are closed. 4. A hard link to a file should act exactly the same as if the file was the hard link (if that makes sense).... ie. the user should not be able to tell that in fact it is any different from the actual file. A soft link on the other hand would probably be implementation dependent, but probably should return an error. You cannot just do the size of the file the soft link points to since: a) it can point to a directory b) it can point to something that doesn't exist.

Question:

What is a reasonable way to ensure that the filesystem doesn't become corrupted when the kernel is halted? For that matter, should the kernel halt when, for example, init dies, or no processes are left running? We're using our own kernel, and at the moment it runs idle forever (or until a ctrl-C). But this means we have no clean up; anything that was cached and not written to disk before the ctrl-C is lost and the filesystem is corrupted. Should, then, we have the kernel die under certain circumstances and sync the filesystem before its death? Or should we schedule a sync every n clock cycles (for some number n)?

Answer:

These policy decisions are really up to you. Personally I would find a "sync every n units of time" policy to be pretty hard to justify. A better policy might be to write out dirty blocks any time you have nothing else to do with the disk... From our perspective, your system just needs to act like a real UNIX system. There are no guarantees unless writes basically stop, sync is called, the sync completes, and then the system shuts down. fsck (file system check) on boot is UNIX's answer to the consistency problems that this type of slackness causes :-) As far as halting goes, this is really up to you. I guess I don't see any reason that it couldn't stay alive w/just idle. But, I wouldn't sepnd too much time pondering this one -- it is one fo those edge cases that, by definition, only occurs when it doesn't matter :-)

Question:

Do children inherit their parents' CWD?

Answer:

We didn't spec that -- but it does seem like the reasonable thing to do to me.

	Wednesday, December 5, 2001

Question:

Is my understanding correct in that if we run out of space mid-write, we return ERROR and not the number of bytes actually written? So there should be no case where we ever return a value other than the number of bytes requested or ERROR?

Answer:

if you were able to write "some" blocks, but then ran out of space, you should return the number of bytes written. if you could not write a single byte, you should return error.

Question:

does the offset passed to Seek() _really_ want to always be positive...? it seems like you might kinda want it to be 0 to reset to the beginning of the file...is this correct?

Answer:

The offset passed to Seek() may be 0, in which case the associated file pointer should be reset to the beginning of the file.

	Sunday, December 2, 2001

Question:

Is it okay for sync to get starved while there are ongoing processes starting new writes? (Sync is an expensive call anyway...) If not then we have to have processes that need to write multiple sectors atomically check for adequate space in the buffer cache.

Answer:

Yes. As any system administrator knows, typeing "sync" guarantees nothing. (Although we tend to be very optimistic...)

Question:

each process stores its own current working directory. this works. however, we are wondering if our interpretation of how this works with symlinks is correct. specifically, should we keep track of the symlink, the hardlink or both as the current working directory? currently, we are keeping track of the symlink only, but we have found behavior using bash that we cannot model using this method. since the symlink is dynamically evaluated each time we do an ls, if we cd to a symlinked directory and add a file in the hardlinked directory, our method will give that file when ls occurs; however, this is not the behavior bash gives until the working directory is changed or an ls ../symlinked dir is done. i cannot see this working with our method...

Answer:

Our specs didn't get into enough detail on this one -- so either is acceptable. But, what we are really looking for is something that works like UNIX -- the symbolic link is evaluated and that's the end of it... Otherwise, life could get pretty confusing for some poor process.

	Saturday, December 1, 2001

Question:

We're having trouble linking the provided kernel. We get an error saying that we're missing the LoadInfo function. We made the changes that were stated on the Q & A however when we run "yalnix create" or other such statements it can't load the different init files. Any suggestions?

Answer:

Try chaning the order of the linking. put the -lyalnix-core before the -lkernel. This is because yalnix-core depends on kernel, so if they are included in the other order, the dependencies will never be resolved.

Question:

Should Read and Write update the offset?

Answer:

Yes. Just like the libc read() and write(). You want to make sure that consecutive reads and writes play with different parts of the file.

Question:

Can we store the inode of a process current working directory in the PCB - we realize this inode may become invalid, but would this violate the 6kb limit on caching directorys (we already have a 6kb cache declared elsewhere)?

Answer:

This sounds to me like a _really_ bad idea. I see all sorts of consistency problems arising as he directory file or structure change over time. It makes much more sense to store the inode number -- or even the absolute path. (consider what happens if the directory is deleted and then recreated...)

To answer your original question, it does sound like it violates the rules of the game, because the file represented by the inode isn't necessarily open.

...We'd be willing to reconsider our interpretation of the rules -- but would really like to understand how this would help you, first. Right now, it looks really dangerous.

	Friday, November 30, 2001

Question:

Are shared forks supposed to share a current working directory, or does each have a separate current working directory?

Answer:

We didn't actually specify this...and I think you could probably argue for either.

	Wednesday, November 28, 2001

Question:

just a little clarification on the first question:

are we trying for a total causal ordering among all the machines, or just causal ordering in the sense that if two hosts send messages to a different host, the messages will be processed in the order that they were sent?

Answer:

I am unsure what you mean by a "total causal ordering among all the machines". A causal ordered protocol is a protocol that ensures that the application level cannot process messages in an order that violates causality. That is, a causal ordered protocol will not allow me to process some message M if M may depend upon some other messages I have not yet received. In this case, M is buffered until these other messages arrive.

Lecture 29 describes methods for "Detecting Causality Violations Using Vector Timestamps" after they have occured. However, this is not sufficient to implement a causal ordered unicast protocol -- we need to buffer the messages that arrive too early, not merely detect this condition once we receive a late message. At that point, we have already passed the early messages up to the application and we can't "take them back".

Lecture 29 also describes a method for implementing a "Casual Ordered Multicast Protocol", also using vector timestamps. However, this method relies upon the fact that all messages are sent to every other processor in the multicast group.

Your task is to develop a causal ordered unicast protocol. As a hint, consider this: when your protocol receives a message, it must choose to buffer the message or pass the message up to the application. In order to make this choice, you might find you want your messages to include more information than is included with the multicast protocol variant (which can be thought of as a special case of the unicast protocol).

	Saturday, November 24, 2001

Question:

We are only allowed 48 sectors of cached data. Is it acceptable for each process to allocate a small amount of its own cache as needed?

Answer:

If you are malloc()ing and free()ing just to pass to the process, it isn't really cache - but this is a somewhat inefficient way of dealing with things. It seems that you are proposing copying from one memory region in the kernel to another, and then to the user space... Basically, this doesn't violate the requirements of cache size, but it isn't the best way to do things...

Question:

We were working on implementing some of the symbolic link stuff and ran into a potential problem if we were to create a symbolic link that linked to itself. We tried it out.. % ln -s blah blah % more blah b: Number of symbolic links encountered during path name traversal exceeds MAXSYMLINKS So we're guessing we should set some sort of limit, MAXSYMLINKS, on how far we can follow symlinks? Is this a reasonable assumption? If so, what would be a reasonable value for MAXSYMLINKS?

Answer:

You should set up some maximum limit for symlink dereferencing. Otherwise, your kernel might loop forever attempting to access the target of a symlink cycle. Any large value (1024+) should do.

	Friday, November 23, 2001

Question:

Where is MAXFILENAMESIZE defined?

Answer:

You can define it to anything you want (within reason). Just make sure that you leave it as a macro so that if you and the grader disagree about reasonable, it can be changed on-the-fly.

	Thursday, November 22, 2001

Question:

Are we allowed to use the FD_SET implementation in the standard c library for the close_on_exec bit set and open_files bit set in the the per process struct.

Answer:

You can make use of any defintions contained within the header files, including macros. You can't use the implementations within libc as part of your file system.

	Monday, November 19, 2001

Question:

During an Exec call, the fd table that the child gets has the first three descriptors but loses the remaining ones from the parent. So the first three descriptors in the table are always associated with stdin, out and err. So, in order to save the duped values, we should copy over the first three descriptors from the parent and remove the others. Where in the system wide global table do these file descriptors point to and what would the contents of those entries in the system wide global table look like? Are these just regular files and are we supposed to initiate them?

Question:

We're using your kernel and makefile but we're having difficulty getting it to compile (independent (we believe) of our files). Could you make any suggestions?

Answer:

Sorry, this is a problem with the makefile. Try linking libyalnix-core.a as a dynamic library with -lyalnix-core instead of linking it statically.

Answer:

Your file system does not need to abstract the terminal -- this is handled separately in Yalnix. But, if you want to do this, feel free.

	Saturday, November 17, 2001

Question:

I'm a little confused on the Seek system call. For example, what happens to the filesize when lets say the current filesize is 100 bytes but someone seeks to an offset of 2bytes. Does this make the filesize 2 (or does it remain the same) and does this modify the read offset to 2 as well. If someone seeks to an offset of 200, does this make the filesize 200 and changes the read offset to 200?

Answer:

The only thing that seek really does is change the "offset" associated with the file -- the next location for a read or write.

If you sek beyond the existing end of a file -- and write -- it does make the file larger. The space in between the old end and the place where the write began -- the area the seek skipped over, is called a "hole". Most of it is not allocated (there might be pieces that are part of the block at the beginning and the block at the end). This is what is meant by a "sparse file".

Question:

Should a call to DirEntry with entry position 0 return "." (if our ordering puts "." first in the directory) or should it ignore "." and ".." and return the first "real" entry in the directory?

Answer:

Good question. I think it should probably return "." -- unless there is a reason to do something else.

Question:

How does one shrink a file? If I seek to some huge file size, how can I set the file size back to really small under the current API?

Answer:

You really can't. The same is absically true for traditional UNIXes as well. Sorry!

Comment:

There seems to be a bit of confusion about the semantics of a seek() system call, and our answers to a few questions below may be misleading.

To clarify: a call to seek() never changes a file's size. It simply changes the read/write offset stored in the kernel's underlying open file data structure.

A hole in a file can only be created after seek() sets the read/write offset greater than the filesize and a write() occurs. In this case, any newly created bytes that are not written to by the write() call are assigned the value 0. If these zero'd bytes would extend across an entire block when written to disk, YFS does not allocate this block but instead represents this region of the file (known as a 'hole') with some sentinel value (think: NULL pointer) in that block's appropriate Direct/Indirect entry.

	Friday, November 16, 2001

Question:

In the spec for DirEntry, it says that we have to return the entries in the same order with the possible exception of creating or unlinking files. What about if a subdirectory is removed (or made)? We should be listing subdirectories, right?

Answer:

Yes. Subdirectory entries, like regular file entries, are directory entries. In a real file system, soft link entries, named pipe entries, device special file entries, &c are all directory entries. In other words, a directory entry is any name to inode mapping, regardless of the type of the inode or its use. Things would get really confusing if you changed the order -- how would the programmer know what index to request?

Question:

What happens when you start reading and hit a hole in the file but there is still some data that could be part of the size. Do you count the hole and write nulls in the buffer or skip it or error.

Answer:

If you read from a hole, the read should return a buffer containing all zeros.

Question:

Should read update the file position? i.e. should two consequetive Reads return identical results?

Answer:

Yes, read (and write) should update the file position. There should be no need for the application to explicitly seek each time.

	Thursday, November 15, 2001

Question:

During a seek call, is the hole that is being created by the seek actually part of the file size even though we're not allocating any blocks for it?

Answer:

yes, most definitely. in fact, you can have a filesize of 50gb and not have allocated a single block for it.

Question:

it appears to be that mkyfs.template allocate a fixed amount of inodes at the beginning of the DISK, although most of the inodes are not used. i'm wondering if we can choose not to allocated them like this. Instead, allocate it dynamically when it is needed?

Answer:

Feel free to do whatever you like -- mkyfs is just to help. The only trick is that your fs needs to work with you mkyfs. Dynamically making inodes might sound nice -- but I suspect you'll run into some performance problems. Please be careful!

	Wednesday, November 14, 2001

Question:

During a seek call, is the hole that is being created by the seek actually part of the file size even though we're not allocating any blocks for it?

Answer:

Yes

Question:

If I read sector X from the disk, and then make a request to read sector X+1 at a later time, does the performance of this second request depend upon time between the requests? Does the simulated hardware actually simulate the rotating of the disk, so that if I read sector X, sector X+1 is not necessarily underneath the disk head at the start of a later request? Or can we just assume performance is linear with |X1 - X2| ?

Answer:

In the yalnix disk, there is a fixed cost associated with every call to BlockTransfer, due to the rotation time. Your YFS implementation may choose to read/write multiple sectors with a single BlockTransfer() -- this is the only way your YFS implementation can reduce the delay time due to disk rotation.

A word of advice, however: it is much, much better to have a correct YFS than a broken YFS that attempts this optimization. I recommend worrying about this optimization only after you have a working YFS implementation.

Regardless of how many sectors you read/write with BlockTransfer(), there is a seek cost that is linear in the number of sectors the head must move since the last BlockTransfer(). So, performance is linear with |X1 - X2|.

Question:

What happens in a Seek call where they are trying to seek passed the size of the file and there are no free blocks remaining?

Answer:

The seek call itself doesn't cause a new block to be allocated. Only when data is written to a new part of a file should a block be allocated.

Question:

We were thinking about caching the inode numbers corresponding to pathnames. The packet doesn't say anything about this, and I was wondering if implementing a static sized table mapping pathnames to inode numbers was allowable. The reason I ask is that doing so would sort of be anothing form of caching data on the disk into memory, and could be interpreted as sneaking around the 48 sector max buffer cache size.

Answer:

That is in fact how we might interpret it. To be safe, don't use more than 48 * 512 = 24,576 bytes of memory for your cache. How you use this cache is up to you.

Question:

1. In lecture 21 on the global open file table, what is the difference between file mode (permissions) and the flags passed to open (two of the listed fields in the open file table entry)?
2. We can't have multiple disks right?
3. In lecture 21 on the global open file table, one of the fields in the table entry is "pointer into structure containing pointers to the functions that implement the behaviors like read(), write()..." We don't need this right? When exactly do you need it?
4. So there's the constant MAX_OPEN_FILES...we assume that this refers to the number of entries in the global open file table. Is this correct? In particular, if file A is opened MAX_OPEN_FILES times, can file B be opened as well?

Answer:

1. The open() syscall provides a 'flags' parameter and a 'mode' parameter. The 'flags' parameter is one of O_RDONLY, O_WRONLY, or O_RDWR (read only/write only/read and write), depending upon how the program wishes to use the file. There are a bunch of other flags that can be set by bitwise-oring their representative constants with whichever of {O_RDONLY, O_WRONLY, O_RDWR} is specified. See 'man open' for details on these.

   The 'mode' parameter specifies what permissions will be given to a new file, should open() result in creating a file.

2. not for project 4.
3. You need it when you have a more complicated operating system that possibly supports multiple file systems. The lecture notes mention that it is similar to the context of device drivers - it is. Basically you will have a struct like that associated with every filesystem that you have in your os, and they could potentially reference different functions. one could go for ext2 and another for fat32 or ntfs, for example.
4. max open files is the number of files your filesystem allows to be open at the same time. it is the size of the global file table. if you open file A that number of times, your file table will be full, and file B can no longer be opened (until you close at least one of the currently open files). in a "real" os open() would then return a -1, and errno would probably be set to EMFILE, "Too many open files" (man errno).

	Tuesday, November 13, 2001

Question:

I'm not sure if this has been spotted already, but the specification for the syscall SymLink() says

"Creates a symbolic link from the new file newname to the existing file oldname... The file oldname may be a directory and may not even exist.

This seems a bit contradictory.

Answer:

Yes this is an error in the handout. For symbolic links the link need not exist, nor be a file -- it can be to anything. It is for hardlinks that it is a link to an existing file (and file only). Hope this helps, and thanks for pointing that out.

	Friday, November 9, 2001

Question:

When we are creating what our inode should look like, is there any limit to how many indirect nodes we're allowed to have in the base inode?

Answer:

This is completely up to your design. You should consider things like the maximum file size, access latency, &c.

Question:

When dealing with concurrency within the filesystem, is it ok to lock the filesystem from being accessed by any other processes while the filesystem is being used?

Answer:

This is corect, but not very efficient -- it will become an unnecessary bottleneck under heavy contention. A finer grained lock, such as one at the inode level, is a much better approach -- and will, as a consequence, earn more points.