From: Keith Playford <keith@harlequin.co.uk>
Date: Mon, 27 Jun 94 16:49:58 BST
Message-Id: <5815.9406271549@trillian.harlequin.co.uk>
To: dylan-partners@cambridge.apple.com
Subject: General comments on macros
Content-Length: 4809

I'd like to thank Dave Moon for his efforts in getting his head around
and writing up Mike Kahl's work so quickly - it couldn't have been
easy. 

Now I've had some time to read and digest the proposal I have three
main problems with the system as proposed:

  o I don't think it provides a full enough macro capability as it
    stands, even to the rewrite-rules only programmer.

  o It uses an approach to parsing that makes the form, if not the
    content, of macro calls too inflexible.

  o It seems to rely on that simple parsing approach, making it
    difficult if not impossible to extend later on.

--

Taking the first point first, the document says "[...] we have made a
serious attempt to specify the minimum macro facility that meets known
user requirements [...]". Well, maybe but I think at least some more
kinds of macro need to be definable by rewrite-rules only
programmers. For example, one of the most common things programmers do
with macros in Lisp is to use them to define function-like things:

  define transmogrifier foo (form, context, #key descend? (#f))
    [...]
  end transmogrifier;

=>

  begin
    register-transmogrifier(#"foo",
                            method (form, context, #key descend? (#f))
                              [...]
                            end);
  end;

Words or intermediate clauses with parameter list components aren't
supported at all either.

This and some of the other extensions mentioned in the document should
become current rather than future features - or at the very least
feature in current thinking because it's important to prove that they
can actually be added smoothly later (see below).

--

The bottom-up parsed deep syntax described severely limits the form of
macro calls. It may be true that most macros can be written by
squeezing them into one standard form or another but often the result
will be an unintuitive structure that isn't in the same style as the
rest of Dylan. As a simple example, I would write "iterate" (named let
in Scheme) like this naturally:

  iterate walk (l :: <list> = get-list(), count :: <integer> = 0)
    case
      first-case?(l)  => count;
      [...]
    end;
  end iterate;

but have to write something like:

  iterate (walk, l :: <list> = get-list(), count :: <integer> = 0)
    case
      first-case?(l)  => count;
      [...]
    end;
  end iterate;

instead to get it through the parser as a details begin-word. 

I don't think this is a desirable situation long-term which is why I
don't think the programmer interface to the macro system should rely
on this restricted style of parsing now when we may be able to develop
a better solution later.

--

The macro system seems to rely on this restricted parsing approach
because the loose criteria it uses to determine what kind of macro is
being defined don't look to admit new kinds of macro or fragment type
very easily. Take the "transmogrifier" example above:

  define macro define transmogrifier
    { define ?mods transmogrifier ?name (?params) ?body end }
      => { register-transmogrifier(?name,
                                   method (?params) ?body end) }
  end macro;

Even in the current style of parsing, how would the macro system know
to define, say, a "method-define-word" macro rather than a
"definition" were one introduced? Or from this definition:

  define macro tracking-pointer
    { tracking-pointer (?details) ?clause-body end }
      =>
    details: 
      { ?stream, #rest properties } 
        =>
    clause-body:
      { ?body ?event-clauses }
        =>
    event-clauses:
      { on (?params) ?body ... }
        =>
  end macro;

that "on" should be a "parameters-begin-word" rather than a details
"begin-word"? 

This information lives in the constraint types of the pattern
variables of the left hand sides. If it were through these type
constraints that macro-types were determined it would open the door
not just to new kinds of standard macros under the current parsing
scheme (and lighten the pressure on "details" to be all things to all
people) but also, potentially, to more general schemes involving, for
example, statement level macro-directed top down parsing in the
future.

--

I understand that the macro system as proposed may be adequate for
Apple's perceived audience but in all probability Harlequin will want
something more flexible to meet our own requirements and it's
important that such a system could live as a compatible extension on
top of the standard macro system with a view to incorporation into the
language. With that in mind I'd recommend that we think about the
needs of extensions in this area now, perhaps in the same way that
procedural macros have been dealt with, and that we don't accept this
proposal until we're happy that they're possible to add.

-- Keith