Newsgroups: comp.lang.ada,comp.lang.apl,comp.lang.basic.misc,comp.lang.basic.visual,comp.lang.c,comp.lang.c++,comp.lang.clos,comp.lang.eiffel,comp.lang.forth,comp.lang.fortran,comp.lang.lisp,comp.lang.misc,comp.lang.modula2,comp.lang.oberson,comp.lang.pascal,comp.lang.prolog,comp.lang.smalltalk,comp.lang.objective-c,comp.lang.functional
Path: cantaloupe.srv.cs.cmu.edu!rochester!udel!news.mathworks.com!tank.news.pipex.net!pipex!dish.news.pipex.net!pipex!dircon!rheged!simon
From: simon@rheged.dircon.co.uk (Simon Brooke)
Subject: Criteria for comparing languages (was Re: Please help with research)
Message-ID: <DDLM9M.JD@rheged.dircon.co.uk>
Summary: Not as easy as it sounds
Organization: none. Disorganization: total.
References: <405glu$cv6@ixnews7.ix.netcom.com> <407g6k$705@nova.sti.nasa.gov> <DD207n.8wy@ss3.magec.com>
Date: Sun, 20 Aug 1995 07:32:08 GMT
Lines: 75
Xref: glinda.oz.cs.cmu.edu comp.lang.ada:34176 comp.lang.apl:7248 comp.lang.basic.misc:8344 comp.lang.c:152313 comp.lang.c++:145065 comp.lang.clos:3434 comp.lang.eiffel:10506 comp.lang.forth:23635 comp.lang.fortran:31423 comp.lang.lisp:18884 comp.lang.misc:22786 comp.lang.modula2:12416 comp.lang.prolog:13723 comp.lang.smalltalk:27462 comp.lang.objective-c:4401 comp.lang.functional:6248

In article <DD207n.8wy@ss3.magec.com>,
Steve O'Shaughnessy <smosha@most.magec.com> wrote:
>
>>In article <405glu$cv6@ixnews7.ix.netcom.com>, redmtn@ix.netcom.com says...
>>>
>>>I'm doing comparative language research for a magazine article.

SNIP

>
>There has been a lot of *Flame* type responses to this request, but seems 
>like a reasonable request to me.  Take a trivial specification, give it to a lot of 
>different programmers with different backgrounds using different languages then 
>compare the language used to the quality of code produced.  

SNIP

>There are several pitfalls to be wary of.  For example, as one poster noted, you 
>really can not make a judgement of any languages not included.  The other sticking 
>point is the definition of code quality.  However spelling these details out in the 
>original post would taint the study.  

With the best of good will this sort of study is extremely difficult
to do and proves little. We are all (whether we like it or not)
exceedingly biased in favour of the programming styles and idioms
(if not specific languages) which we find familiar and comfortable. 

The 'objective' (na whatee dat long and difficult word mean?) criteria
on which one might compare programming languages are, in my opinion:

(i) Programmer productivity		;; How long does it take skilled
					;; programmers in each language
					;; to code up the same functionality?

(ii) Computational efficiency		;; How much machine resource (CPU
					;; cycles, memory usage) does each
					;; program thus produced consume?

(iii) Maintainability			;; How long does it take a second
					;; team of skilled programmers,
					;; without access to the first, to
					;; make a specified change in the
					;; functionality of the program?

(iv) Comprehensibility			;; How well can an arts graduate
					;; with no prior exposure to
					;; *any* programming language
					;; explain, after reading the
					;; source code, what the program
					;; is intended to do?

However, even a study based on these criteria would inevitably be
biased. If the task chosen was some simple file filter (such as that
which started this thread), one would expect the simpler conventional
languages such as C and C++ to score highly. If the task chosen
involved sophisticated symbolic reasoning, one would expect LISP-like
languages to do better. After all, some horses (and camels :-}) *are*
better on their home courses.

And how do you ascertain that your programming teams are equally
skilled? One thing is certain: doing this in a half arsed way with
bunches of CS undergraduates would prove less than nothing. 

It *might* be worth trying to set up a competition in which different
language *vendors* were invited to field teams each using their own
particular language (they would have a commercial interest in fielding
good teams), but to make this work the team setting the specification
for the task would have to be seen to be unbiased.

-- 
------- simon@rheged.dircon.co.uk (Simon Brooke)
	"The result is a language that... not even its mother could love.  Like
	the camel, Common Lisp is a horse designed by committee. Camels do have
	their uses."				;; Scott Fahlman,  7 March 1995

