Translating Descriptions to C Code
The translator code used to extend Nyquist resides in the
directory. This directory also contains a special
init.lsp, so if
you start XLisp or Nyquist in this directory, it will automatically read
init.lsp, which in turn will load the translator code (which resides
in several files).
Also in the
trnsrc directory are a number of
.alg files, which
contain the source code for the translator (more on these will follow), and
a number of corresponding
To translate a
.alg file to
.h files, you start
XLisp or Nyquist in the
trnsrc directory and type
"prod"should really be replaced by the filename (without a suffix) you want to translate. Be sure you have a saved, working copy of Nyquist or Xlisp before you recompile!
Note: On the Macintosh, just run Nyquist out of the
runtime directory and then use the
Load menu command to load
init.lsp from the
trnsrc directory. This will load the translation code and change Nyquist's current directory to
trnsrc so that commands like
(translate "prod") will work.
prod.h, you need to recompile Nyquist. For Unix systems, you will
want to generate a new Makefile. Modify
transfiles.lsp in your main
Nyquist directory, run Xlisp or Nyquist and load
Follow the instructions to set your machine type, etc., and
Accessing the New Function
The new Lisp function will generally be named with a
snd-prod. You can test this by running Nyquist. Debugging is
usually a combination of calling the code from within the interpreter,
reading the generated code when things go wrong, and using a C debugger to
step through the inner loop of the generated code. An approach I like is to
set the default sample rate to 10 hertz. Then, a one-second sound has only
10 samples, which are easy to print and study on a text console.
For some functions,
you must write some Lisp code to impose
ordinary Nyquist behaviors such as stretching and time shifting. A
good approach is to find some structurally similar functions and see how
they are implemented. Most of the Lisp code for Nyquist is in
Finally, do not forget to write up some documentation. Also, contributions are
welcome. Send your
.alg file, documentation, Lisp support
nyquist.lsp, and examples or test programs
email@example.com. I will either put them in the next release or
make them available at a public ftp site.
Many of the Nyquist signal processing operations are similar in form, but
they differ in details. This code is complicated by many factors: Nyquist
uses lazy evaluation, so the operator must check to see that input samples
are available before trying to access them. Nyquist signals can have
different sample rates, different block sizes, different block boundaries,
and different start times, all of which must be taken into account. The
number of software tests is enormous. (This may sound like a lot of
overhead, but the overhead is amortized over many iterations of the inner
loop. Of course setting up the inner loop to run efficiently is one more
The main idea behind the translation is that all of the checks and setup
code are similar and relatively easy to generate automatically. Programmers
often use macros for this sort of task, but the C macro processor is too
limited for the complex translation required here. To tell the translator
how to generate code, you write
.alg files, which provide many
details about the operation in a declarative style. For example, the code
generator can make some optimizations if you declare that two input signals
are commutative (they can be exchanged with one another). The main part of
.alg file is the inner loop which is the heart of the signal
Writing a .alg File
WARNING: Translation relies heavily on string substitution, which
is fragile. In particular, variables with names that are substrings of
other variables will cause problems. For example if you declare STATE
variables "phase" and "iphase", then the translator will globally
substitute "phase_reg" for "phase", converting "phase" to "phase_reg"
and iphase" to "iphase_reg". Then it will substitute "iphase_reg" for
iphase" which will convert the existing "iphase_reg" to
"iphase_reg_reg". This will be confusing and will not compile.
(End of WARNING)
To give you some idea how functions are specified, here is the
snd-prod, which generates over 250 lines of C code:
(PROD-ALG (NAME "prod") (ARGUMENTS ("sound_type" "s1") ("sound_type" "s2")) (START (MAX s1 s2)) (COMMUTATIVE (s1 s2)) (INNER-LOOP "output = s1 * s2") (LINEAR s1 s2) (TERMINATE (MIN s1 s2)) (LOGICAL-STOP (MIN s1 s2)) )A
.algfile is always of the form:
(name (attribute value) (attribute value) ... )There should be just one of these algorithms descriptions per file. The name field is arbitrary: it is a Lisp symbol whose property list is used to save the following attribute/value pairs. There are many attributes described below. For more examples, see the
.algfiles in the
Understanding what the attributes do is not
easy, so here are three recommendations for implementors. First, if there is
an existing Nyquist operator that is structurally similar to something you
want to implement, make a copy of the corresponding
.alg file and
work from there. In some cases, you can merely rename the parameters and
substitute a new inner loop. Second, read the generated code, especially
the generated inner loop. It may not all make sense, but sometimes you can
spot obvious errors and work your way back to the error in the
file. Third, if you know where something bad is generated, see if you can
find where the code is generated. (The code generator files are listed in
init.lsp.) This code is poorly written and poorly documented, but in
some cases it is fairly straightforward to determine what attribute in the
.alg file is responsible for the erroneous output.
Here are the attributes used for code generation. Attributes and values may
be specified in any order.
.h, and the XLisp function generated will be
(type1 name1) (type2 name2) ..., where type and name are strings in double quotes, e.g. ("sound_type" "s") specifies a SOUND parameter named
s. Note that arglist is not surrounded by parentheses. As seen in this example, the type names and parameter names are C identifiers. Since the parameters are passed in from XLisp, they must be chosen from a restricted set. Valid type names are:
ARGUMENTSabove), and has the form:
(type1 name1 init1 [TEMP]) (type2 name2 init2 [TEMP]) .... The types and names are as in arglist, and the "inits" are double-quoted initial values. Initial values may be any C expression. State is initialized in the order implied by statelist when the operation is first called from XLisp. If
TEMPis omitted the state is preserved in a structure until the sound computation completes. Otherwise, the state variable only exists at state initialization time.
INNER-LOOP-LOCALSattribute instead. Within innerloop-code, each ARGUMENT of type sound_type must be referenced exactly one time. If you need to use a signal value twice, assign it once to a temporary and use the temporary twice. The inner loop must also assign one time to the psuedo-variable output. The model here is that the name of a sound argument denotes the value of the corresponding signal at the current output sample time. The inner loop code will be called once for each output sample. In practice, the code generator will substitute some expression for each signal name. For example,
(INNER-LOOP "output = s1 * s2")
ARGUMENTS.) This expands to the following inner loop in
In cases where arguments have different sample rates, sample interpolation is in-lined, and the expressions can get very complex. The translator is currently very simple-minded about substituting access code in the place of parameter names, and this is a frequent source of bugs. Simple string substitution is performed, so you must not use a parameter or state name that is a substring of another. For example, if two sound parameters were named
*out_ptr_reg++ = *s1_ptr_reg++ * *s2_ptr_reg++;
s2, the translator might substitute for "s" in two places rather than one. If this problem occurs, you will almost certainly get a C compiler syntax error. The fix is to use "more unique" parameter and state variable names.
ARGUMENTSlist. You can also write
(SAMPLE-RATE (MAX name1 name2 ...))where names are unquoted names of arguments.
.hfile. The code typically contains auxiliarly function declarations and definitions of constants.
.cfile. The code typically contains auxiliarly functions and definitions of constants.
(CONSTANT "name1" "name2" ...)
CONSTANTdeclaration can eliminate the overhead of storing these registers.
(MIN name1 name2 ...)means the start time is the minimum of the start times of input signals name1, name2, .... Note that these names are not quoted.
(MIN name1 name2 ...)means the terminate time is the minimum of the terminate times of input arguments name1, name2, .... Note that these names are not quoted. To terminate at the time of a single argument
(MIN s1). To terminate after a specific duration, use
(AFTER "c-expr"), where c-expr is a C variable or expression. To terminate at a particular time, use
(AT "c-expr"). spec may also be
COMPUTED, which means to use the maximum sample rate of any input signal.
TERMINATE. If no
LOGICAL-STOPattribute is present, the logical stop will coincide with the terminate time.
(ALWAYS-SCALE name1 name2 ...)
YESto generate inline interpolation,
NOto disable inline interpolation, and
NILto take the default set by the global variable
*INLINE-INTERPOLATION*. The default is also taken if no INLINE-INTERPOLATION attribute is specified.
(STEP-FUNCTION name1 name2 ...)
(DEPENDS spec1 spec2 ...)
STEP-FUNCTIONlist. Consider a filter coefficient that depends upon an input signal such as bandwidth. In this case, you want to compute the filter coefficient only when the input signal changes rather than every output sample, since output may occur at a much higher sample rate. A spec is of the form
which is interpreted as follows: name depends upon arg; when arg changes, recompute expr and assign it to name. The name must be declared as a
("name" "arg" "expr" [TEMP "type"])
TEMPis present, in which case name is not preserved and is used only to compute other state. Variables are updated in the order of the
(FORCE-INTO-REGISTER name1 name2 ...)
#define'd macro or referenced in a
(NOT-REGISTER name1 name2 ...)
(NOT-IN-INNER-LOOP "name1" "name2" ...)
(MAINTAIN ("name1" "expr1") ("name2" "expr2") ... )
MAINTAINattribute. This suppresses a store of the value of the named variable, making it a dead variable. Where the store would have been, the expression is computed and assigned to the named variable. See
partial.algfor an example. This optimization is never necessary and is only for fine-tuning.
(LINEAR name1 name2 ...)
snd-prod(signal multiplication) are "linear." The inner loop has a single multiplication operator to multiply samples vs. a potential 3 multiplies if each sample were also scaled. To handle scale factors on the input signals, the scale factors are automatically multiplied and the product becomes the scale factor of the resulting output. (This effectively "passes the buck" to some other, or perhaps more than one, signal processing function, which is not always optimal. On the other hand, it works great if you multiply a number of scaled signals together: all the scale factors are ultimately handled with a single multiply.)
(INTERNAL-SCALING name1 name2 ...)
snd-recipoperation computes the reciprocal of the input samples by peforming a division. The simple approach would be to specify an inner loop of
output = 1.0/s1, where
s1is the input. With scaling, this would generate an inner loop something like this:
but a much better approach would be the following:
*output++ = 1.0 / (s1_scale_factor * *s1++);
*output++ = my_scale_factor / *s1++
my_scale_factoris initialized to
1.0 / s1->scale. Working backward from the desired inner loop to the
.alginner loop specification, a first attempt might be to specify:
but this will generate the following:
(INNER-LOOP "output = my_scale_factor / s1")
Since the code generator does not know that scaling is handled elsewhere, the scaling is done twice! The solution is to put
*output++=my_scale_factor/(s1_scale_factor * *s1++);
INTERNAL-SCALINGlist, which essentially means "I've already incorporated scaling into the algorithm, so suppress the multiplication by a scale factor."
(COMMUTATIVE (name1 name2 ...))
downproto.algfor an example where a check is made to guarantee that the output sample rate is not greater than the input sample rate. Otherwise an error is raised.
.c file defines a number of procedures. The procedures
that do actual sample computation are named something like
name_interp-spec_FETCH, where name is the NAME
attribute from the
.alg file, and interp-spec is an interpolation
specification composed of a string of the following letters: n, s, i, and r.
One letter corresponds to each sound argument, indicating no interpolation
(r), scaling only (s), ordinary linear interpolation with scaling (i), and
ramp (incremental) interpolation with scaling (r). The code generator
determines all the combinations of n, s, i, and r that are necessary and
generates a separate fetch function for each.
Another function is name
_toss_fetch, which is called when sounds
are not time-aligned and some initial samples must be discarded from one or
The function that creates a sound is
snd_make_name. This is
where state allocation and initialization takes place. The proper fetch
function is selected based on the sample rates and scale factors of the
sound arguments, and a
sound_type is returned.
Since Nyquist is a functional language, sound operations are not normally allowed to
modify their arguments through side effects, but even reading samples from a
sound_type causes side effects. To hide these from the Nyquist
sound_type arguments are first copied (this only copies a small structure. The samples themselves are on a shared list). The function
snd_name performs the necessary copies and calls
snd_make_name. It is the
snd_name function that is
called by XLisp. The XLisp name for the function is
Notice that the underscore in C is converted to a dash in XLisp. Also,
XLisp converts identifiers to upper case when they are read, so normally,
you would type
snd-name to call the function.
If you want the option of passing either a number (scalar) or a signal as
one of the arguments, you have two choices, neither of which is automated.
Choice 1 is to coerce the constant into a signal from within XLisp. The
naming convention would be to
DEFUN a new function named
S-NAME for ordinary use. The NAME function tests
the arguments using XLisp functions such as
SOUNDP. Any number is converted to a
SOUND, e.g. using
SND-NAME is called with all sound arguments.
The disadvantage of this scheme is that scalars are expanded into a sample
stream, which is slower than having a special inner loop where the scalar
is simply kept in a register, avoiding loads, stores, and addressing
Choice 2 is to generate a different sound operator for each case. The
naming convention here is to append a string of c's and v's, indicating
constant (scalar) or variable (signal) inputs. For example, the
reson operator comes in four variations:
version implements a resonating filter with a variable center frequency (a
sound type) and a constant bandwidth (a
function in Nyquist is an ordinary Lisp function that checks types and calls
Since each of these
SND- functions performs further selection of
implementation based on sample rates and the need for scaling, there are 25
different functions for computing RESON! So far, however, Nyquist is
smaller than Common Lisp and it's about half the size of Microsoft Word.
Hopefully, exponential growth in memory density will outpace linear (as a
function of programming effort) growth of Nyquist.