Newsgroups: sci.image.processing,comp.lang.c
Path: cantaloupe.srv.cs.cmu.edu!das-news2.harvard.edu!news2.near.net!news.mathworks.com!gatech!bloom-beacon.mit.edu!usc!crash!rickc
From: rickc@crash.cts.com (Rick Castrapel)
Subject: Re: How to compute covariance & determinant of matrices ?
Organization: CTS Network Services (CTSNET), San Diego, CA
Date: Fri, 2 Jun 1995 02:44:01 GMT
Message-ID: <rickc.802061041@crash.cts.com>
References: <12186@eagle.ukc.ac.uk> <3qejmh$gol@crcnis3.unl.edu> <3qhnnb$auc_001@news.dnai.com>
Sender: news@crash.cts.com (news subsystem)
Nntp-Posting-Host: crash.cts.com
Lines: 92
Xref: glinda.oz.cs.cmu.edu sci.image.processing:14995 comp.lang.c:140800

In <3qhnnb$auc_001@news.dnai.com> vbazarov@imsisoft.com (Victor Bazarov) writes:

>In article <3qejmh$gol@crcnis3.unl.edu>,
>   jwoodfor@unlinfo.unl.edu (Jeffrey N Woodford) wrote:
>>Ferdinand Che (fnc@ukc.ac.uk) wrote:
>>: Hi,
>>
>>: I need c code on how to compute:
>>
>>: [1]	the covariance of a matrix;
>>: [2]	the determinant of a matrix
>>
>>: I have tried using some of the stuff in textbooks like "Numerical Recipes 
>.. "
>>: but because I don't understand much the techniques they are using I am not
>>: very confident with the results I get. In addition to that, I am not too 
>>: certain about debugging the program I have since I don't quite understand
>>: what it is supposed to be doing at each stage.
>>
>>The Numerical Recipes book talks about the covariance of a matrix when
>>they discuss nonlinear curve fitting and the Levenburg-Marquardt
>>Algorithm in Ch.  14 (I think) of the 2nd ed.
>>
>>Doing the determinant of a matrix seems straightforward (although I
>>admit I've not had the need to implement it).  I would do it
>>recursively if I had to do it.

I have recently had the opportunity to help a co-worker write a determinant
calculation program.  She had this as a homework assignment (which I strongly
suspect is also the case for the original poster).  As a homework problem,
posed as it was, it exercised a number of programming concepts: functions;
arguments; pointers; recursion; etc.  However, I felt bound to respond to
this thread because of several misconceptions contained therein.

>The only problem with recursion in calculation the determinant would be 
>necessity to create new matrix on every step down (of course, it depends on 

Actually, the "only problem with recursion" in calculating the determinant,
is that it takes a perfectly good O(n**3) algorithm and makes it O(n!).
This is because submatrix calculations are performed repetitively.  I haven't
found any way around this problem if you are using a recursive definition of
the determinant - even calculating and storing all of the submatrix
determinants would require factorial complexity to figure out where to use
them!

The recursive definition of the determinant can use either a row of elements
as multipliers to the submatrix determinants, or it can use a column.  If,
for example, you use the first column, then there is no need to "create new
matrix on every step down".  You can pass pointers and leave the original
matrix untouched and uncopied through the entire ordeal.

So recursion needn't cause you factorial storage or stack usage, but it
will cause factorial arithmetic.  (Since repeated submatrix calculations
don't start appearing until the submatrix of size n-2, you must try at least
a 4x4 to start seeing the effect.)

>your matrix representation, but if you use vector, or vector of vectors, 
>you'll have to deal with changind dimensions), unless yiou find some twicking 
>algorithm to cheat on the size vs. needed size. I'd suggest you find some fast 
>algorithm in math book. The subject was studied for centuries, there may be 
>several different methods that suit you. And another two cents -- for big 
>matrices recursion may be very expensive in stack use, also there are 
>different schemes of storage especially for big matrices... So all depends on 
>the problem you solve. For example, for the matrix 3x3, there is no need in 
>writing recursive program (or even use cycles), it's one formula with 12 
>multiplications and 5 additions...

>>
>>-Jeff

>Victor.

>+---------------------------------------------------------+
>| Via est vita. (Life is a road)                          |
>| Internet: vbazarov@imsisoft.com, CompuServe: 74633,2430 |
>+---------------------------------------------------------+

Definitions of the determinant are universally recursive as far as I have
seen.  This definition lends itself to proving theorems and teaching
elementary programming.

So how does one calculate a determinant O(n**3)?  It's quite simple, but
requires a little more linear algebra and numerical analysis to understand:

Perform an LU decomposition, then multiply all of the diagonal elements of
L and U together.  This is the approximate determinant of A = LU.

Note that this solution is O(n**3) however, it is not rigorous due to the
divisions (computationally inexact) needed for the LU decomposition.

Rick Castrapel

