Newsgroups: comp.lang.dylan,comp.lang.misc,comp.lang.lisp,comp.object,comp.arch
Path: cantaloupe.srv.cs.cmu.edu!bb3.andrew.cmu.edu!nntp.sei.cmu.edu!news.psc.edu!hudson.lm.com!news.math.psu.edu!news.cac.psu.edu!newsserver.jvnc.net!newsserver2.jvnc.net!howland.reston.ans.net!swrinde!sdd.hp.com!hpscit.sc.hp.com!hpax!wang
From: wang@cup.hp.com (Thomas Wang)
Subject: Re: allocator and GC locality (was Re: cost of malloc)
Sender: news@hpax (News Admin)
Message-ID: <DCLIwB.D8w@cup.hp.com>
Date: Mon, 31 Jul 1995 19:45:46 GMT
References: <9507261647.AA14556@aruba.apple.com> <3v8g7l$cge@jive.cs.utexas.edu> <3vac07$ptf@info.epfl.ch> <3vb382$dtr@jive.cs.utexas.edu>
Nntp-Posting-Host: capella.cup.hp.com
Organization: Hewlett-Packard
X-Newsreader: TIN [version 1.2 PL0.7]
Followup-To: comp.lang.dylan,comp.lang.misc,comp.lang.lisp,comp.object,comp.arch
Lines: 49
Xref: glinda.oz.cs.cmu.edu comp.lang.dylan:4932 comp.lang.misc:22409 comp.lang.lisp:18495 comp.object:36077 comp.arch:59998

Paul Wilson (wilson@cs.utexas.edu) wrote:

> The memory lost to fragmentation in conventional allocators appears to be
> widely overestimated---at least for the best allocators.  This is partly
> because the allocator research area is such a mess that people use the
> wrong allocators (e.g., next fit) or don't tweak them in the right ways
> (e.g., getting rid of footer overhead for boundary tags).

> Our recent results show good allocators having only about 10% fragmentation.
> This is comparable to the losses you get just from a header word and rounding
> object sizes up to a word or double word for architectural reasons.

> (Some caveats are in order---this is for a set of eight varied C and C++
> programs, but there are lots of kinds of programs that aren't represented
> in *any* small set, and none of our programs runs for a very long time.)

For some contrasting point, our commercial DBMS server initially had some
run-away fragmentation.  We used a work around of keeping an internal free
list that is never freed.  Frequent usage of mega-byte sized buffers
contributed to the memory fragmentation problem.  You know, it is going to be
these large multi-million dollar projects that are going to have this kind
of problems.  On second thought, multi-media applications can run into
fragmention problem as well.

> It's not clear to me that compaction is a win, on average.  If objects
> are placed correctly in the first place, it may not be necessary and it
> may be *bad* for locality on average.  The allocation order and sizes
> of objects are likely to provide very strong hints as to which objects
> will be used together.  This info should be usable for decreasing
> fragmentation and for increasing locality.

I believe the most realistic control on fragmentation can be achieved by
giving language writer access to virtual memory function to free memory pages.
Making large fragments not take up memory space is such an obvious idea.
This solution should knock out the dirty cache problem, as well as be
applicable to conservative collectors.

A master thesis quality paper can be done by implementing this scheme on AIX,
then do some performance analysis.

Regards.


> Paul R. Wilson, Comp. Sci. Dept., U of Texas @ Austin (wilson@cs.utexas.edu)


  -Thomas Wang              (Software Engineer, Enterprise Objects Program)
   wang@cup.hp.com          http://hpeopose.cup.hp.com/~wang/wang.html

