Developing and Debugging an Audacity Plug-In for Normalization
Using Nyquist
Roger B. Dannenberg
2 July 2010
Introduction
Normalization or any operation that scans an entire sound runs the risk
of allocating lots of sample memory in Nyquist. Here is how to avoid
that problem. For just the results, you can skip ahead to here.
It might seem that if anyone knows how to make Audacity plug-ins using
Nyquist, it's me, but in reality I work almost entirely in Nyquist, and
Audacity is almost entirely managed and maintained by other people now.
I ran into lots of little problems, so I thought I would
describe the process here for my own memory and perhaps to help others.
Getting Started
I downloaded the latest released version of Audacity (1.3.12) and tried
out something simple: I tried to type (peak s) into the Nyquist
prompt.
Lesson 1: You can't run an effect without some
selected audio.
OK, that's easy: I run Generate:Noise... to create noise, then
try again. Typing (peak s)
into Generate:Nyquist Prompt... returns "Nyquist did not return audio."
Well, of course, it's supposed to return a number. The problem is that (peak s) is missing a
parameter. This is verified by clicking the Debug button instead of the
OK button when running Generate:Nyquist Prompt....
Lesson 2: Use the Debug button to run a Nyquist
expression, not the OK button.
I changed the expression to (peak
s
ny:all) and things worked as expected.
Editing Plug-Ins
I fired up Emacs to make a plug-in. After a little searching, I found
plug-ins in the Audacity Plug-In folder. Rather than type a plug-in, I
copied one and edited it. I tried to save it back to the same folder
with a new name, and that seemed to work, but the plug-in did not
appear in the Audacity menu.
Lesson 3: Restart Audacity to rescan the plug-in
folder and pick up new plug-ins.
Unfortunately, things were not all that simple...
Lesson 4: Windows Vista
sometimes pretends to write
to system directories.
From Emacs, it seemed as if I was reading and writing to the Audacity
Plug-In directory, but in reality, the file was going into another
directory. I assume Vista tries to create the illusion of writing for
compatibility with earlier Windows software, but increases security by
preventing the writes. I found the actual location by searching in
Explorer, then copied the file by hand to the Plug-Ins directory. After
that, I was able to edit and modify the file directly from the editor.
Next, I wanted to display some debugging information. After a bit of
search, I discovered you can add a line to the plug-in to enable
printing:
;debugflags
trace
With this in place, I could finally do what I set out to do: print some
heap info, run a plug-in, and look at the heap again.
Lesson 5: Use ";debugflags trace" to get debug info from a plug-in.
Here's the plug-in:
;nyquist
plug-in
;version 1
;type analyze
;categories
"http://audacityteam.org/namespace#OnsetDetector"
;name "rbd test"
;action "Performing rbd test ..."
;info "by Dannenberg for testing"
;debugflags trace
(print "before run")
(info)
(defun rbd-test ()
(format nil "this is a
test: ~A" (peak s ny:all)))
(setf result (rbd-test))
(print "after run")
(info)
result
Note that I defined the action as a function named rbd-test and called it. I could
have just evaluated the expression directly. Note also that I assigned
the result to result so that I could print more information and then
return the value. Finally, I returned a string as an experiment. You
can return either a string or a number.
Lesson 6: You can return and print a value by
returning either a string or a number.
Oddly, the debugflags option seems to work only after writing a new
plug-in. After running the plug-in once, the debug output window does
not appear. When I rewrite the file with the editor and run the
plug-in, the debug output appears again. Running the plug-in again
without rewriting the file generates a result, but no debug output!
This calls for some Audacity debugging...
Debugging Audacity
I have a version of Audacity sources that I'm currently working on, so
that should be good for testing and debugging the "debugflags" option.
I started Visual C++ Express Edition and opened my audacity.sln file.
Then, I used F5 (Debug:Start Debugging) to start Audacity.
Of course, now the rbd-test.ny
plug-in is missing because I'm running a different Audacity. After
copying rbd-test.ny to audacity-src/plug-ins, I still
do not see my "effect" in the Analyze menu. Not surprisingly, I found
another directory:
C:\Users\rbd\audacity\audacity-src\win\Debug\plug-ins
and after moving rbd-test.ny
here, the plug-in works. I still get the behavior where the plug-in
text output from Nyquist
is not displayed the second time I run the plug-in. Debugging can begin.
Nyquist effects are run in src/effects/nyquist/Nyquist.cpp. Each effect
has an mDebug flag that enables the collection of text output from
Nyquist and displays it in a dialog box after the effect runs. For the
Nyquist Prompt... effect, mDebug is set when the user clicks the Debug
button, and mDebug is cleared after the effect runs. The statement that
clears mDebug also clears mDebug set by debugflags, which is why
debugflags only takes effect the first time the effect is executed. (I
fixed this bug so that debugflags is permanent -- maybe we'll see it in
a future release.)
Memory Usage
Now for the real problem: how can we determine the peak value of a
signal without reading in the entire signal? To determine how much
signal memory is in use, run a garbage collection and then call info to
print memory information. Alternatively, you can set *gc-flag* and the garbage
collector will print some information:
(gc)
(info)
or
(setf *gc-flag*
t) (gc)
The rbd-test.ny output
shows that sample memory is allocated for the entire signal that is
analyzed. The output after a couple of runs is:
"before
run"
[ Free: 22627, GC calls: 26,
Total: 1297906; samples 45KB, 43KB free]
"after run"
[ Free: 22613, GC calls: 27,
Total: 1298013; samples 45KB, 0KB free]
The "samples" number is how much sample memory has been allocated. The
last line shows that after running the effect, but before returning the
result, there are 45KB of samples, with 0KB free. I would have expected
4 bytes/sample times 44.1K samples (I analyzed 1s of audio), or about
176KB of samples. Maybe Nyquist is counting samples instead of bytes --
another potential bug.
In any case, the problem must be that the variable s is bound to the signal being
analyzed. We should unbind s
or use a destructive method to compute the peak. As an experiment, I
wrote a function to copy s
to a local, set s to nil,
and return the local. If I compute peak on this function, it appears
that about 100KB of sample memory is used, even on 10 minutes of audio.
The exact amount of memory depends upon when garbage collection is run.
To minimize memory, one should call for garbage collection immediately
after setting s to nil,
but Nyquist will call the garbage collector automatically before memory
allocation becomes a problem.
Normalize
It should now be possible to write a normalize function in Nyquist that
is well-behaved with respect to memory. The function will have to run
in two parts: part 1 will compute the peak value and save it. Part 2
will scale the signal. Two passes are required to avoid saving the
samples in Nyquist memory.
I wrote normalize-part1 to do the analysis and normalize-part2 to alter
the signal. I save the peak value as a property of the symbol *scratch* with the property
symbol normalize-part1.
Unfortunately, this did not work. After some experimentation, it seems
that while the *scratch*
symbol may be retained from one effect to the next, the symbol normalize-part1 is being
removed in the post-effect cleanup. This is a bug we will have to fix.
Breakpoints and Debug Printing
While debugging changes to the Audacity/Nyquist interface in nyx.c
(part of libnyquist), I tried to set breakpoints in nyx.c but got
messages saying "the source code is different from the original
version." I ended up using some print statements to debug (very
annoying). But printf() does not work unless you have a console
application, so I ended up using OutputDebugString(str).
To print a variable in a string, I used strcpy() and strcat() to build up messages
-- what a pain!
Somehow in this process, I must have done more than just add output
statements, because further experimentation showed that changing nyx.c
and running Audacity did not
actually compile and execute the changed code.
Lesson 7: If you make a change to nyx.c and rerun
Audacity from the debugger, it will prompt you to rebuild libnyquist,
but the code changes do not actually affect the application when it
runs. (Perhaps other libraries have this problem too.)
OK, so this indicates some dependencies are not right. I decided
to try increasingly costly builds to see what is required after code
changes. First, I built the Audacity project. Visual C++ said it was
already up-to-date, so I didn't even try running it again. Next, I
tried forcing a link (right click on Audacity in the "Solution
Explorer" panel, and under "Project Only ..." select "Link Only
Audacity". That did the trick.
Lesson 8: After you change nyx.c (and perhaps any
source file in lib-src), you need to relink Audacity manually for the
changes to take effect.
After I relinked Audacity, I found that breakpoints work again as
expected, so the real problem with breakpoints is that I did not have a
consistent application until I manually relinked Audacity.
Further work showed that the Build Solution menu item has the same
problem of not building a consistent application, but that's not too
surprising given that running the debugger does not do a proper build.
Finally, there is a bug in
the garbage collector output messages: it says KB, but the number is
the number of samples divided by 1024. The true memory usage in bytes
is approximately four times greater becaues there are 4 bytes per
sample.
Working Normalize Plug-Ins
After all this, I have two plug-ins: an Analysis plug-in that detects
the peak and an Effect plug-in that scales the audio based on the peak.
These effects depend on a code fix to Audacity that allows properties
to be passed on the *scratch* symbol. The effects are as follows:
normalize-part1.ny
;nyquist plug-in
;version 3
;type analyze
;name "Normalize Part 1"
;categories
"http://lv2plug.in/ns/lv2core#AnalyserPlugin"
;action "Finding peak amplitude
..."
;info "After this analysis step,
run normalize-part2 to normalize the selection."
;; For debugging, we get memory
status before and after running the effect.
(setf *gc-flag* t)
(gc)
;; Get the input signal s, but
set s to nil to avoid retaining samples.
(defun get-signal ()
(let ((local-s s))
(setf s
nil)
local-s))
;; Compute the peak value of s
(setf peak-value (peak
(get-signal) ny:all))
(gc)
;; Save the peak-value on
*scratch* for later use.
;; Assume that
the effect name (normalize-part1) can be safely used
;; as a
property symbol without any conflicts.
(putprop '*scratch* peak-value
'normalize-part1)
;; Return a string:
(format nil "The peak value is:
~A" peak-value)
normalize-part2.ny
;nyquist plug-in
;version 3
;type process
;name "Normalize Part 2 ..."
;categories
"http://lv2plug.in/ns/lv2core#AmplifierPlugin"
;action "Normalizing ..."
;control db "Normalize maximum
amplitude to ..." real "dB" 0.0 -24.0 0.0
;info "You must run Analyze:
Normalize Part 1 before this plug-in."
;; For debugging, we get memory
status before and after running the effect.
(setf *gc-flag* t)
(format t "db is ~A~%" db)
(setf peak-value (get '*scratch*
'normalize-part1))
(format t "peak-value is ~A~%"
peak-value)
(cond ((floatp peak-value)
(mult s (/ (db-to-linear db) peak-value)))
(t
"No peak-value found -- run normalize-part1 to compute the peak value"))