Newsgroups: comp.lang.python,comp.lang.perl,comp.lang.java,comp.lang.smalltalk
Path: cantaloupe.srv.cs.cmu.edu!bb3.andrew.cmu.edu!newsfeed.pitt.edu!gatech!newsfeed.internetmci.com!howland.reston.ans.net!torn!nott!cunews!dbuck
From: dbuck@superior.carleton.ca (Dave Buck)
Subject: Re: Speed comparison: Python-Perl-Java-Smalltalk
X-Nntp-Posting-Host: superior.carleton.ca
Message-ID: <DL1wHy.K58@cunews.carleton.ca>
Sender: news@cunews.carleton.ca (News Administrator)
Organization: Carleton University, Ottawa, Canada
References: <4d4h65$626@news1.panix.com>
Date: Fri, 12 Jan 1996 04:29:58 GMT
Lines: 95
Xref: glinda.oz.cs.cmu.edu comp.lang.python:7934 comp.lang.java:15565 comp.lang.smalltalk:33335

In article <4d4h65$626@news1.panix.com>,
trimble <trimble@panix3.panix.com> wrote:
>
>Well, folks, here it is:  a speed comparison of your favorite interpreted
>languages as well as a C++ measurement to compare against.

These are certainly interesting numbers and I appreciate the time and
effort it takes to run this sort of test.  I've made similar
comparisons in the past but mostly between C++ and Smalltalk.

If I may, I think that a few changes in the tests would help us narrow
down the strengths and weaknesses of ther various languages.  I work
in Smalltalk and C++, so I'll comment on these tests.

   1) If you don't want to measure the loop overhead, decrease the
loop count by a factor of 10 and duplicate the lines in the loop 10
times.  This means that less time is spent dealing with the looping
and the tests more accurately represent the lines being tested.
Obviously, this is only necessary when the loop body is very fast so
we have a large number of iterations.

   2) In the Smalltalk tests, use Time millisecondTimeCount to record
the start and end times or Time millisecondsToRun: [] if you can put
everything into a block.  This gives you better precision.

   3) The Smalltalk indexing test is written as follows:

      1 to: INDEX do: [:i | 
        | b |
        b := a at: (i rem: 10) + 1].

It actually takes longer to calculate the (1 rem: 10) + 1 than to
index the array itself.  The rem: method in Smalltalk isn't terribly
efficient.  In a quick test, when I replaced the calculation with just
1, I got a factor of 3.5 speedup.  If you want to test rem: speed,
write a separate test for it.

   4) for the file access test, you probably want to use
       otherStream nextPutAll: 'SM LINE --> '
      rather than:
       'SM LINE --> ' printOn: otherStream
      since the second one writes the quote characters to the stream
as well.  I expect that this won't affect the test very much

   5) One test for C++ was coded as follows:
        for(i=0;i<basearith;i++) {
                float j;
                j = 3.14159 + 2.78;
                j = 3.14159 - 2.78;
                j = 3.14159 * 2.78;
                j = 3.14159 / 2.78;
        }

     Be very careful with this.  It's quite likely that your C++
compiler could virtually eliminate all of the code in the loop with
compile-time optimizations.  I'd suggest that you check the assembly
code generated to verify the the calculations are actually being done.
This test might be fine if you're trying to test how good the compiler
is at eliminating stupid code (no insult intended - all benchmark code
is intentionally stupid) but we're trying to measure the runtime
performance of real calculations so we don't want the compiler to
eliminate the code we're testing.

Several other tests could suffer the same problem.  For example, the
test:
        for(i=0;i<index;i++) {
                int b;
                b = a[i % 10];
        }

Could be totally eliminated by a smart C++ compiler that realizes that
b's value is never used and the array access is never used so it
doesn't need to write code for them.  C++ compilers are notorious for
defeating simple benchmarks.  Be careful.

   6) The C++ code for these tests is really just C code since no
virtual member functions are called, there's no dynamic binding, and
all I/O is done by fread and fwrite instead of << and >>.  Perhaps the
tests should be advertised as a C benchmark rather than a C++
benchmark.  It would be interesting to try out the RogueWave Tools.h++
library for array accessing to compare it to the other languages.

Thanks for the time and effort you put into running these tests.  If
you want revise the test and have someone to look over the Smalltalk
and C++ code, I'd be happy to help out.

David Buck
dbuck@ccs.carleton.ca

_________________________________
| David K. Buck                 |
| dbuck@ccs.carleton.ca         |
| The Object People             |
|_______________________________|

