Newsgroups: comp.lang.lisp,comp.lang.scheme
Path: cantaloupe.srv.cs.cmu.edu!rochester!cornellcs!newsfeed.cit.cornell.edu!newsstand.cit.cornell.edu!travelers.mail.cornell.edu!news.kei.com!news.mathworks.com!tank.news.pipex.net!pipex!howland.reston.ans.net!Germany.EU.net!zib-berlin.de!news.rrz.uni-hamburg.de!news.Hanse.DE!wavehh.hanse.de!cracauer
From: cracauer@wavehh.hanse.de (Martin Cracauer)
Subject: Re: Benchmarking
Message-ID: <1995Oct26.081822.17542@wavehh.hanse.de>
Lines: 52
Organization: BSD User Group Hamburg
References: <jslttery.813682277@GS148>
Date: Mon, 16 Oct 95 18:04:22 GMT
Lines: 53
Xref: glinda.oz.cs.cmu.edu comp.lang.lisp:19727 comp.lang.scheme:14145

jslttery@cs.cmu.edu (Sean Slattery) writes:

>Has anyone out there done benchmarking of various schemes and lisps to get
>an idea of which is faster? I'm primarily interested in Allegro, CMUCL, STk
>and SCM (and compiled SCM). Are there standard benchmark programs out
>there that I could use myself?

The problem with benchmarking is that Lisp application vary too much
in what they do and what the resulting requirements for the
implementation are.

One thing relativly easy to compare is processing of static arrays of
floating-pount numbers. Here, most schemes don't have a chance,
bytecode CL ala clisp are bad, gcl not as worse, native commercial
compilers are in the same ballpark as some static languages and CMU CL
can generate "overhead-free" (in a wider sense) floating-point
array-based code, as fast as C.

Part of CMU's strength is that it has very good compiler messages
informing the user about possible optimization techniques missed in
the compiled code.

Note that Compilers that compile to C doesn't have to produce
efficient code. The problem with such array-based computations is that
the individual numbers have to be stored in the array by themself, not
as pointers. gcl generated efficient C code, but the resulting Code
accesses array elements by pointers, which slows down to a fraction of
possible speed.

One other "benchmark" I did - simply reading in a nested list of one
megabyte of items - shows a completely different picture.

Note also that there is a schem-to-C compiler, BIGLOO, that generates
very (*very*) fast C code, but violates scheme specification to do
so. Also, I didn't check wether it can use pointer-free arrays
(guess not). 

If you really need speed, get CMU CL, use the messages to optimize
your code and then see how fast it runs on CMU CL and the commercial
CLs. 

There is some form of standard benchamrks, the Gabriel Benchmarks, but
they are not too useful to guess from their result to runtimes of
real applications, more to see where the weaknesses of a given system
are. But knowledge of such weaknesses says almost nothing about
overall performance.

Martin
-- 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Martin Cracauer <cracauer@wavehh.hanse.de>  -  Fax +49 40 522 85 36
 BSD User Group Hamburg, Germany   -   No NeXTMail anymore, please.
 Copyright 1995. Redistribution via Microsoft Network is prohibited
