This file lists some of the main files and directories.  It isn't complete,
as otherwise it'd be too big.  Some undistributed files might have got in
here by mistake - if so, don't panic!

Files marked as DOOMED may be replaced soon.
Files marked as NOTYET probably aren't in the distribution.

Lee

@(#) $Revision: 1.2 $


README	- read the README file first, please!  Then come back here...
GuidedTour	- This file
BugReport	- A template for bug reports sent by mail
doc	- directory containing the documentation for lq-text
src	- directory containing the source for lq-text.
Sample	- A sample empty LQTEXTDIR waiting to have a database created in it :-)
src/TODO	- what I haven't done yet.  Read this before adding features.

Sample/CommonWords
	A list of common words, taken from the Bible.  They are all commented
	out, though.  There are also numbers in there to give you some idea
	of the frequency of the common words.

Sample/*
	A sample empty database:
	    mkdir $HOME/LQTEXTDIR
	    cp Sample/* $HOME/LQTEXTDIR
	    textedit $HOME/LQTEXTDIR/*

Sample/README
	a sample config file.  You should edit this.

src/h/*
	Directory containing all of the lq-text C header files.
	src/h/globals.h is the one you'll have to change.

src/filters/*
	The programs in here are filters that improve indexing quality.  When
	lqaddfile is given what it thinks is a mail folder, for example, it
	starts up MailFilter and reads its output instead of the actual file.
src/sdbm/*
	Ozan Yigit's sdbm package is an ndbm replacement.
src/contrib/*
	Software contributed by various users, but not supported by me.  I do
	not ever expect to support the perl scripts, I'm afraid.  If you use
	any of the others, let me know & I'll consider adding them to the
	main distribution.  Software to go here is always welcomed.
src/TODO
	Things I haven't done yet, wish list
src/menu/*
	This directory contains part of a simple menu package for the Unix
	curses(3X) library, to run on a tty (i.e. not X Windows).
	There are a couple of examples, and "lqtext" is one of them, a
	front end similar in principle to lqtext/lq.sh but more complicated.
src/liblqtext/*
	lq-text architecture is like this: there's a library that provides
	access to the search database, and it's called liblqtext.a, and the
	source for it is in src/liblqtext.  This library uses a couple of
	other libraries, liblqutil.a and liblqerror.a, which have their own
	directories.  An application, such as lqphrase, is linked against all
	of these libraries.  In addition, X Windows clients using the lq-text
	libraries would link against the X libraries, and so on.
src/bin/*
	When you do a make, this is where the binaries go before you do a
	make install.  This directory starts out empty.
src/bsdhash/*
	This is an ndbm replacement with a vengeance.

src/lib/*
	The compiled libraries go here.  This directory starts out empty.

src/liblqerror/*
	source for a simple Error() function.  You might find this useful for
	other programs, too.

src/liblqutil/*
	A library of useful routines.  You might find these useful for other
	programs, too.

src/lqtext/*
	The source for the lq-text programs themselves

src/ozmahash GONE
	replaced by bsdhash and db

src/saber.addfile
	for debugging lqaddfile with sabre C (CentreLine)
src/saber.project
src/saber.project-all
src/sabre.project.menu
	If you don't have Sabre-C, call CentreLine NOW and ask for it.
	You should not be debugging C programs without this under any
	reasonable circumstance.  If you think you have something better,
	TELL ME ABOUT IT!!!!

src/tags
	for use with vi or emacs - "make tags" will create this.

src/test/*
	various badly documented test files.

src/testbin/*
	Where the compiled tests go.  They don't get installed anywhere else.

src/tradeoffs
	Some comparisons of different parameters from globals.h

src/xlqtext	OPEN LOOK GUI (XView) front end to lq-text.
	This is not made by the top level Makefile.
	It's very buggy & untested, it was a quick demo really.  Honest.


More detail:
============

src/filters/FilterMain.c
	Each filter links with this.
src/filters/MailFilter.c
	Filter for /bin/mail-style files (e.g. /usr/spool/mail/lee).
	MailFilter deletes Received-By lines and other irrelevant mail headers.
src/filters/Makefile
	You should not ever need to edit this file.  All of the variables in
	here are actually over-ridden by src/Makefile.
src/filters/NewsFilter.c
	Filter for netnews (usenet) articles.  Deletes the Path header.  Also
	deletes Message-Id at the moment, since lqaddfile couldn't deal with
	it, although that might change in the future.
src/filters/TroffFilter.c
	Filter for troff (or SoftQuad's sqtroff) files.  Not yet used.

src/contrib/boolean.dist
	boolean expressions for lqphrase
src/contrib/README
src/contrib/intersect.pl
	Tom Christiansen's replacement for my intersect.sh - it's slower but
	it does more.
src/contrib/zread.dist
	Not sure this is needed any more.
src/contrib/rs6000.diffs
src/contrib/rs6000-part2
	For use with an IBM system.

src/menu/bcopy.c
	DOOMED
src/menu/examples
	A directory with a couple of examples of using "m", a menu program.
	You'll need to do a "make m" in src/menu first.
src/menu/Makefile
	This is one of the two makefiles that you might want to edit, because
	curses(3X) is not too portable.
src/menu/error.c
	Draw error messages on the screen
src/menu/README
	Out-of-date notes for part of the original menu package I wrote.
src/menu/OldCurses.c
src/menu/oldcurses.h
	The old* files are for Curses(3) instead of curses(3X).
src/menu/character.c
	DOOMED
src/menu/error.h
src/menu/example.c
src/menu/internal.h
src/menu/menu.c
src/menu/menu.h
src/menu/stringbox.c
src/menu/ultrixhack.h
src/menu/liblqmenu.a
	This is the library that gets made...
src/menu/text.c
	This is the source for lqtext itself, the sample client.

doc/*
	the documentation, what there is of it
doc/Admin.Guide
doc/DOCUMENTATION
doc/Mailshot
doc/ProgramStructure
doc/lqtext.1
	The Unix man-page
doc/port
	notes on porting.  DOOMED.
doc/theory
doc/API
	If you get this, you have got the ApplicationC Programmers' Interface
	documentation for the libraries.  If you didn't get it, and would like
	it, contact lee@sq.com; if it isn't included, it's either too big or,
	more likely, too out of date or too short.

src/COPYRIGHT
	this is NOT the Gnu Public Licence.
src/GetIdList
	shell script to get the RCS ident of each file.
src/Makefile
	This is the most important configuration file for lq-text.
	The second-most important one is src/h/globals.h, and you should edit
	them both.
src/PORTING
	Notes on porting.
src/PeterAndJane
	A sample file to index.
src/REVISIONS
	This lists the revisions of each file.  It's useful to help me keep
	track of who has what.

src/bin/*
	When you do a make, this is where the binaries go before you do a
	make install.  This directory starts out empty.

src/bin/intersect	- find files that have two or more phrases in common
src/bin/lines	- prints line numbers and blocks.  Not used yes.  NOTYET
src/bin/lq	- shell script front end to the lq-text package.
src/bin/lqaddfile	- creates an index, or adds files to an index.
src/bin/lqfile	- lists files that you have indexed.
src/bin/lqkwik	- keyword index: lqkwik `lqphrase "phrase"`
src/bin/lqmkfree	- ADMIN: rebuild free bit file by reading data file
src/bin/lqphrase	- match phrases
src/bin/lqrename	ADMIN: rename a file in the index
src/bin/lqshow	- show matches: lqshow `lqphrase "some phrase"`
src/bin/lqtext	- curses-based front end
src/bin/lqword	- lists words in the database.
src/bin/lqwordlist	- print a sorted list of all words in the database.
src/bin/mkwidtable	- ADMIN: rebuild wordlist by looking at data.  DOOMED.

src/bsdhash/*
	This is an ndbm replacement with a vengeance.
src/bsdhash/Makefile
	You might have to edit this one, sorry.

src/db/*
	This is a later version of bsdhash.

src/db/Makefile
	I made this, and it isn't right yet.  Sorry.
src/db/Makefile.inc
	The original makefile, which also wasn't right, mainly because it
	didn't contain any rules!

src/h/*
	Directory containing all of the lq-text C header files.
	src/h/globals.h is the one you'll have to change.
src/h/Liamdbm.h
	Used with gdbm.  Watch out for potential licence conflicts if you
	use gdbm, though.  DOOMED.

src/h/Revision.h
	The lq-text version.

src/h/blkheader.h	low-level blocks
src/h/declare.h		declare things not in stdlib.h for use with gcc
src/h/emalloc.h		for my malloc wrapper which does extra checking
src/h/error.h		for Error()
src/h/fileinfo.h	for storing filenames
src/h/filter.h		defines filter types and magic numbers
src/h/globals.h		CHANGE THIS FILE
src/h/numbers.h		read/write compressed integers
src/h/pblock.h		physical blocked match lists
src/h/phrase.h		for phrases
src/h/smalldb.h		for the ndbm front end liblqtext/smalldbm.c
src/h/wordindex.h	for the Word to WID map
src/h/wordinfo.h	information about words
src/h/wordplace.h	storing matches in memory
src/h/wordrules.h	CHANGE THIS FILE - max word length, etc.

src/lib/*
	The compiled libraries go here.  This directory starts out empty.
	This is also where the filters go before you do a make install.

src/lib/MailFilter	filter for mail
src/lib/NewsFilter	filter for netnews
src/lib/libhash.a	bsdhash package (optional)
src/lib/libsdbm.a	sdbm package (optional)
src/lib/libdb.a		4.4 BSD db package (optional)
src/lib/liblq.a		Liam's C utilities
src/lib/liblqerror.a	error message handler
src/lib/liblqmenu.a	curses menu
src/lib/liblqtext.a	lqtext search & index engine

src/liblqerror/*
	source for a simple Error() function.  You might find this useful for
	other programs, too.
	
src/liblqerror/Makefile
	You don't need to edit this.
src/liblqerror/error.c
src/liblqerror/error.h
	This gets copied into src/h automatically.

src/liblqtext/*
	Source for liblqtext.a
src/liblqtext/Common.c
	handle common words that are not indexed.
src/liblqtext/Defaults.c
	handle user defaults from command line and README
src/liblqtext/DocPath.c
	finds files in the document path
src/liblqtext/FileList.c
	handles storage/retrieval of filenames.
src/liblqtext/FilterType.c
	determine the type of a file so the right filter (if any) gets used.
src/liblqtext/GetWID.c
	do the Word -> WID mapping, i.e. (char *) -> (unsigned long)
src/liblqtext/Makefile
	You should NOT need to change this file.  It is a bug if you do, and
	you can freely report it!
src/liblqtext/Phrase.c
	routines for matching phrases (used mostly by lq-phrase now)
src/liblqtext/Plurals.eg	NOTYET
src/liblqtext/Root.c
	routines for morphological and etymological grouping.
	OK, it strips an s off plurals.
src/liblqtext/SetWID.c
	get/set the largest current word number
src/liblqtext/Unpack.c
	unpack archived or compressed files for retrieval
src/liblqtext/WordInfo.c
	handle the main wordlist
src/liblqtext/asciitrace.c
	defines AsciiTrace, a variable for debugging.
src/liblqtext/bcopy.c
	a bcopy() routine that works.  May not be as fast as the
	system-supplied one, though.  If you're confident that your bcopy()
	is OK, and does overlapping regions, you don't need this file.
src/liblqtext/block.c
	The low-level physical Word Database for lq-text.
src/liblqtext/cmdname.c  DOOMED
	declares cmdname.  If a shell script sets $CMDNAME, all lq-text
	error messages will include it before progname.
src/liblqtext/lastnext.c  DOOMED
	defines a global variable
src/liblqtext/lqsort.c
	4.3 BSD qsort() adapted somewhat.
src/liblqtext/numbers.c
	routines for writing and reading compressed numbers
src/liblqtext/pbcache.c
	physical block cache
src/liblqtext/progname.c  DOOMED
	defines the program name
src/liblqtext/rpblock.c
	routines to read physical data blocks
src/liblqtext/smalldb.c
	front end to dbm/ndbm/bsdhash/db/sdbm... to help portability
src/liblqtext/system.c
	a system(3) routine for use with curses programs.  DOOMED.
src/liblqtext/wpblock.c
	routines to write physical data blocks

src/liblqutil/*
	A library of useful routines.  You might find these useful for other
	programs, too.

src/liblqutil/Makefile
	You should NOT need to edit this.
src/liblqutil/cmdname.c
	see above
src/liblqutil/getline.c
	read a line from a file, using emalloc()
src/liblqutil/isdir.c
	is this path name a directory?
src/liblqutil/isfile.c
	is this path name an ordinary file?
src/liblqutil/malloc.c
	special malloc() with extra checking.  Never returns NULL.
src/liblqutil/progname.c
	see above
src/liblqutil/strings.c
	some string functions -- is a prefix of, etc.

src/lqtext/*
	The source for the lq-text programs themselves

src/lqtext/FindCommon.sh
	make a list of the n most common words in the database.  Can be slow.
src/lqtext/Makefile
	You should NOT need to edit this.  All configuration is in 
	src/Makefile, not here.
src/lqtext/fileindex.c
	for lqfile
src/lqtext/intersect.sh
src/lqtext/lines.c
src/lqtext/lq.sh
src/lqtext/lqaddfile.c
	The most important program - makes the index in the first place!
src/lqtext/lqfile
src/lqtext/lqkwik.c
src/lqtext/lqmkfree.c
src/lqtext/lqphrase.c
	The second most important program - say where a given phrase occurs
	in all of the indexed files in the database.
src/lqtext/lqrename.c
src/lqtext/lqshow.c
src/lqtext/lqword.c
src/lqtext/lqwordlist.c
src/lqtext/matchword.sh
src/lqtext/mkwidtable.c
src/lqtext/sizes.c
src/lqtext/wordtable.c

src/ozmahash GONE
	replaced by bsdhash and db

src/saber.addfile
	for debugging lqaddfile with Sabre-C (now CodeCentre) from CentreLine.
src/saber.project
src/saber.project-all
src/sabre.project.menu
	If you don't have Sabre-C, call CentreLine NOW and ask for it.
	You should not be debugging C programs without this under any
	reasonable circumstance.  If you think you have something better,
	TELL ME ABOUT IT!!!!

src/tags
	for use with vi or emacs - "make tags" will create this.
src/test/*
	various badly documented test files.

src/test/README
	explains the tests
src/test/Makefile
	do not edit this.  Do not run make from this directory.
src/test/MaxWid.c
src/test/NumberTest
src/test/TryHash.c
src/test/TryNum.c
src/test/TryRoot.c
src/test/dbmtry.c
src/test/differ.awk
src/test/malloctrace
src/test/numbers.c
src/test/put.c
src/test/size.c
src/test/trywid.c
src/test/wordlengths

src/testbin/*
	Where the compiled tests go.  They don't get installed anywhere else.

src/testbin/MaxWid
src/testbin/NumberTest
src/testbin/TryHash
src/testbin/TryNum
src/testbin/TryRoot
src/testbin/dbmtry
src/testbin/put
src/testbin/trywid
thesaurus/*	NOTYET

src/xlqtext/Makefile
src/xlqtext/lqtext.info
	put this in $OPENWINHOME/lib/help
src/xlqtext/xmore	yes, it's a silly name.

