New Configuration Control Procedures for ABLE

Peter Su

Table of Contents


1. Introduction

With the development of a new prototype proceeding, the project will also be moving to a more effective scheme for source code and configuration management. Our current procedure is based on RCS, and it cumbersome because we must maintain consistently across several RCS repositories, stored in independent directory hierarchies. Our new scheme will be based on the CVS system, which is a front end to RCS that is designed to support large code repositories with modules stored in many directories. A central database tracks version information on all modules in a repository, and allows us to tag the current state of the world with symbolic names, create releases, and do experimental development and maintenance of existing code in parallel with full conflict resolution.

This document will describe the current state of the ABLE CVS repository and will cover how to use CVS to checkout files, edit them and commit your changes. In addition to the command line interface, I'll also go over an emacs based CVS front end called pcl-cvs that makes life easier for those of us who are somewhat shell-impaired.

2. The Basics

CVS maintains a hierarchy of RCS repositories and performs all of its functions relative to the root of this tree. For the ABLE project, I have set up a CVS tree in the directory
/afs/cs/project/able/sunos/cvs. This directory contains a subtree for each component of the able system, plus one directory containing administrative files related to CVS.

Within the administrative directory is a file called modules. This file contains the central database of modules that CVS knows about. For each module, this file contains one line with the module's symbolic name and either a list of files making up the module or a directory name which names the subdirectory in which all of the modules files live. Currently, our modules files looks like this:

# Convenient aliases
world    -a .
# CVSROOT support; run mkmodules whenever anything changes.
CVSROOT    -i mkmodules CVSROOT
modules    -i mkmodules CVSROOT modules
loginfo    -i mkmodules CVSROOT loginfo
commitinfo  -i mkmodules CVSROOT commitinfo
rcsinfo    -i mkmodules CVSROOT rcsinfo
editinfo    -i mkmodules CVSROOT editinfo
# Add other modules here...
fam    fam
doc    doc
The first few lines in the file define modules for editing CVS's administrative files. More on this later. The last two lines define the modules that currently exist in the ABLE repository. The first is for the new FAM code, the second for documentation, like this document.

Before starting, we must set up our environment correctly. First, set the variable CVSROOT to /afs/cs/project/sunos/cvs so that all CVS commands look for the correct repository. I would put this into your .cshrc file, so it gets set whenever you login. Also, you should set the EDITOR environment to something reasonable. Finally, you must make sure that the directory /usr/misc/.cvs/bin is ahead of the directories /usr/misc/bin, /usr/bin and /bin on your PATH. Otherwise, CVS will not find the correct version of diff and some features will break.

In the following examples, we will call the central repository the master collection, and your private copy the local collection.

2.1 Adding a new Module

To add a new module, first make a working directory in /afs/cs/project/able/sunos.

% mkdir /afs/cs/project/able/sunos/psu
Now check out a copy of the modules file:

% cvs checkout modules
CVS will put the copy into ./modules/modules. Edit this file to add a new module line. Say the new module is called `new-ui', then we would add the line:

new-ui     new-ui
to modules/module. Now we check the file back into the main repository:

% cvs commit
At this point CVS will pop an editor for you to type in a log message, such as "Added new module called new-ui." Then you'll get several messages that just mean everything is going OK.

To add files to our new module, we first check out a copy, resulting in an empty working directory:

% cvs checkout new-ui
We then go into new-ui/ and edit our new files. Suppose they are called ui-draw.tk and ui-objects.tk. After making our edits, we can add the files to the repository one by one:

% cvs add ui-draw.tk
% cvs commit ui-draw.tk
% cvs add ui-objects.tk
% cvs commit ui-objects.tk
While CVS allows you to commit files in a big batch, you would then lose the ability to edit individual log messages for each file, so in general, you should commit the files one at a time unless you have several hundred new files to stick in, and appropriate logs already exist somewhere.

2.2 Working with Existing Modules

To check out a current version of the system to make modifications, use the following procedure:

1. Create or cd to a working directory DIR
2. Use the command cvs checkout to check out the modules that you need.
3. Hack away.
4. To commit the all your changes, cd to DIR and type cvs commit. CVS will ask you for log messages through emacs. There may be a bug in how log messages are collected. I'm looking into this and hope to have an answer soon.
5. After your commit, you can continue to hack away, as CVS leaves your copy of all your sources alone. Doing a commit also grabs any files from the repository that were in your collection and had been modified by others.
It is also possible to use the emacs mode pcl-cvs to manage your sources files after you check them out. To do this, instead of using cvs commit, you get into emacs by typing gnu-emacs, load the library pcl-cvs by typing:

 M-x load-library <return> pcl-cvs <return>
and then type

M-x cvs-update.
Emacs will pop a window showing you which files in the local collection have been changed by you, and will let you browse these files, look at diffs, commit changes, write log entries, etc. To get a list of things that you can do, type C-h m for a mode description. Also, there is an info page on pcl-cvs, which you can get to by typing
M-x info, and picking pcl-cvs out of the menu page.

In order to make this work, you have to set things up so that emacs can find the pcl-cvs library. I will give lessons on how to do this for all who want to meet with me in my office.

The cvs-update command is equivalent to using the shell command:

% cvs update.
This command lists the files in the local collection that you have modified, and also checks for files in the master collection that others have modified, but you have not. It it finds any of these, it updates your local copy. Each file that has changed in some way is listed with a single character marker. The markers have the following meanings:

U The file was updated in the master collection and your local copy has been updated.
A You added the file to your local collection, but have not yet committed the change to the master collection.
R You have removed the file from your local collection, but have not committed the change.
M You have updated the file in your local collection, but have not yet committed the change.
C The file has been updated both by you and in the master collection, and the updates conflict. CVS will ask you to take steps to figure out how to merge the two sets of changes.
? The file is unknown.
Note that a commit will only effect files that have been added using cvs add. This is handy because it allows you to keep temporary files around your local collection without being bothered about them when you commit.

2.3 Odds and Ends

For information about how to do other things, such as checking change logs, reverting to the version of a module stored in the master collection and aborting all your current changes, and other things, look at the CVS man page. Summary information can also be found by typing cvs help.

2.4 Releases

CVS also allows for the creation of `releases', to correspond to times when the code for a system is frozen in a certain state to be released to the world. Before, the ABLE system only ever had two releases, alpha and beta, and they weren't really frozen, as things were shifted in and out of the alpha and beta areas as the system materialized. With CVS, we will have a full history of every release made, and will be able to roll back to any state that we marked as frozen. In a sense, this is just an extension of RCS, which allows you to bring a file back to any point in its history. CVS allows you to do this for an arbitrary collection of files.

For our project, we will probably keep a hierarchy of releases. There will be a global release level for the entire collection of tools and libraries (fam, ui, pf, etc) an internal release level for each component of the system. We may perhaps find that it is useful to make internal releases of small collections of tools, such as everything related to the UI.

In general, internal releases are to be made by the people most responsible for the code contained in them. Information about the release and the changes made should then be collected in a log file and posted to the able newsgroup. In addition, our various subgroups can provide experimental releases that are off of the main line of development for people to play with an evaluate before merging back into the core distribution. See the section on branches for more information about this.

Global code freezes will be performed by me and me alone. When everyone claims their code is ready for the freeze, and/or we do a demo and it works [ :-) ], I will collect the most recent version of all modules and freeze them together into a CVS release. This is usually just a matter of tagging a collection of modules with a fixed symbolic name and checking them in at some reasonable version number (1.0, 2.0, 3.0, etc).

2.5 Trunks and Branches

If you remember your RCS, you know that an RCS revision history is actually a tree made up of a series of "trunk" revisions that represent the main line of development on a file, and "branch" revisions that represent experimental revisions that may or may not be merged into the mainstream.

CVS allows the revision history of a module to follow a similar path. In general, one uses trunk revisions for major releases of a component or system and branch revisions for minor changes between releases. The following picture shows and example of this idea:

We will use a scheme like this one to number and name our releases. Each release will be a trunk revision major.minor, where the minor numbers change for small modifications and the Major numbers change for major updates. Between releases, main line development will go in the lowest numbered branch, while experimental development will go in higher numbered branches. When we decide to release, we will take all revisions made in the main branch and merge them back into the trunk, and either increment the minor release number or go to a new major release number at major.0.

In addition, each trunk revision will be tagged with the symbolic tag modulemajorminor, corresponding to the appropriate release. This scheme has the advantage that the tags and the revision numbers for each module always match, even if the revision numbers for the individual files in a module may vary.

Using branch revisions for minor updates also has the advantage that parallel lines of experimental development can occur. This will allow us to experiment with different possible extensions/revisions and only merge the best result back into the main line of releases.

2.6 Making a Release

To make a release, do not use the cvs release command. This is something totally different! The cvs release command tosses out your currently checked out copy of a module or collection of modules. Thus, it probably isn't what you wanted to do.

The two commands that are relevant to making releases are cvs tag and cvs rtag. These commands associate a symbolic name with the current version of a module much in the same way that RCS associates symbolic names with the current version of a file.

To tag the your currently checked out copy of a module with a name, you type:

% cvs update         # make sure we have all current changes
% cvs tag MY_NAME_TAG module
Later on, if you type:

% cvs checkout -rMY_NAME_TAG module
You will get the same versions of all file in module as you had before.

In our project, each component will have a set of permanent tags called "module-major-minor", where module is the module name (i.e. fam), major is the major version number and minor is the minor version number. So, for example, the first major release of the new FAM library will be called "fam-1-0." In addition, there will be a tag called "world-major-minor" to track the current version of the entire world. In addition, each module will have two transient tags called alpha and beta. Whenever we move a new version to alpha or beta, we delete the old alpha and beta tags and replace them with new tags on the new version. Suppose we want to remove the current beta and move the current alpha to beta and the current development version "module-1-3" to alpha:

% cvs update
% cvs tag module-1-3 module
% cvs commit
% cvs tag -d beta
% cvs checkout -ralpha module
% cvs tag -d alpha
% cvs tag beta module
% cvs commit
% cvs checkout module-1-3 module
% cvs tag alpha module
These commands remove the old transient tags and replace them with new ones. Note that the it is vital that the permanent tags never be deleted, since we never know when we might want to roll back to an earlier version of a system. Also note that our release numbers are not related in any way to the RCS revision numbers of any of the files in the module. In general though, whenever we go to a new major release number N-0, we will force all RCS revisions to start again at N.0.

2.7 Working on Released Code

To use the current release of any module, just say cvs checkout module. CVS will find the current trunk revision of each file in the module and give those to you. To use experimental revisions, just use cvs checkout and specify the appropriate branch revision. CVS will then give you the most recent revision on that branch. Finally, to create a branch to work in, you use cvs checkout to checkout the most recent trunk revision, and then you say:

% cvs tag -b BRANCH_NAME
to tell CVS that everything you commit from this module should go into the branch BRANCH_NAME. CVS automatically finds an appropriate set of RCS branch revisions to use. In general, if the current revision of an RCS file is x.y, CVS will put your new branch into x.y.<2*k>.1, where k is the number of branches starting at x.y. For example, if some file in my module is at revision 1.1, then the first branch off of 1.1 goes into 1.1.2.1, the second goes into 1.1.4.1, and so on.

Generally, we should try and give the most recent experimental branches tags of the form module-expr1, module-expr2, and so on. This keeps people from having to remember branch numbers in order to get the newest code.

When it comes time to merge your branch into the trunk, you just say, you first checkout the current trunk revision of the module using cvs checkout. Then, you bring the module up to date using cvs update:

% cvs update -jBRANCH_NAME module
% cvs commit
When you update the module, CVS will apply all changes made in the branch that you specify to the files you checked out from the trunk. If these changes conflict with any updates made to the trunk revision, you will be notified and asked to figure out which updates to keep and which to toss away.

2.8 Summary of Procedures

To sum up:

3. The Current System

The current repository is only sparsely populated. There is this document, a Frame template for future ABLE-related documents that you should use, and the start of the new FAM database library. I haven't bothered to CVS-ify all the third party code that we're using, because I think the changes that we will patch other people's code is small. If we find that we have to, for example, patch tcl in some way, I will make new modules as needed.

3.1 Documents

One more note. Any new documents that we create for the project should be done in Framemaker, if possible, and any Frame documents should use the template doc.template in the doc module. To use this, go to your working directory and type:

% cvs checkout doc
Then, when you make a new Frame document, use the file doc/doc.template as your template. This template has many useful styles for code, lists, sections, and so on. And, it looks nice [:-)].

You can check Frame documents into CVS repositories because newer version of RCS can deal with binary files. To get diffs, you must checkout the two versions that you want to diff and use Frame's diff mechanism. I'm not sure if Frame 3.0 does this, but Frame 4 does, and its coming real soon now.