Arc Forum | First class macros

Arc Forum

5 points by shader 5388 days ago | 17 comments

I remember reading in one of pg's articles about arc that it supported first class macros, but that doesn't seem to be the case any more. Is there any particular reason this feature was dropped? How hard would it be to add back in?

The main reason that I see first class macros as being useful is that they would be compatible with a user-level library system. I know there was some discussion about this a long time ago, but I don't know how it ended up. While arc may be the hundred year language, this forum doesn't seem to work very well for long term discussions.

Anyway, what are the pros and cons of first class macros? How hard would it be to add them?

I think almkglor suggested macexing function position before applying it, and supposedly someone tried it, but I don't know how that turned out either.

1 point by evanrmurphy 5350 days ago | link

> I remember reading in one of pg's articles about arc that it supported first class macros

Maybe this quote from "Some Work on Arc" [1] is what you were thinking of:

> For example, Arc currently has first class macros. It just seems to be the simplest way to define the language. First-class macros are potentially very inefficient; you could be expanding macro calls at runtime. But I think most users won't want to take advantage of this possibility. They'll ordinarily make some kind of global declaration that macro calls can all be expanded at compile time, and macros won't cost any more than they do in current Lisps.

[1] http://www.paulgraham.com/ilc03.html

-----

1 point by shader 5385 days ago | link

So, does anyone know what happened to the hack of macexing (or something like that) the car of a list before evaluating it? Unless it's obviously a macro of course.

Or any other ideas on how to add first class macros to arc?

-----

3 points by fallintothis 5385 days ago | link

Did you mean this discussion? http://arclanguage.org/item?id=7448

That didn't seem to be about first-class macros, except for a very narrow special case (granted, it's about modules, which you seem most interested in). If you think about it, macro-expanding the car of a list won't really first-class anything -- you'd still need to do

  (if (mem 'x stuff)
      (push 'y stuff)
      (pull 'y stuff))

instead of

  ((if (mem 'x stuff) push pull) 'y stuff)

because macro-expanding the car above doesn't get rid of the macro literals push and pull, which will cause an error on application. vincenz tried to make ac recognize macro literals, but it still amounted to evaling (at least of macros), which he didn't get working. http://arclanguage.org/item?id=804

The most obvious way to handle this is to straight-out interpret. As the most obvious way, I'd guess that's why first-class macros have the reputation of being difficult to compile, since there will be some cases that need run-time expansion.

As I said, to the best of my knowledge it's still an unsolved problem. The most recent information I can find is old: http://lambda-the-ultimate.org/node/2987

-----

2 points by shader 5385 days ago | link

Yes, that was the discussion I was talking about, and I suppose you're correct; it wouldn't really help in most cases.

"As I said, to the best of my knowledge it's still an unsolved problem." Then let's see if we can't solve it ;)

Personally, I think that the advantages in terseness of being able to use first class macros would probably outweigh the slight decrease in performance for edge cases.

Couldn't you use the current system, but if the car is a list, eval it, since you don't know what it is yet? That would only add a performance penalty for complex cars.

Maybe the complex car structures could start with a symbol that signifies that it returns a macro, and otherwise it isn't evaled and the performance penalty isn't incurred.

I.e. add a special form that transforms a function call into a macro call, such as (callmac fn). Then functions and macros could be the same, but you just call them differently. ssyntax could then be used to make it look nicer.

-----

1 point by akkartik 5385 days ago | link

I'm not sure what you're referring to in the history of arc..

Eight has first-class macros (http://arclanguage.org/item?id=10719) but the author seems to agree with fallintothis (http://arclanguage.org/item?id=11534) that it's hard to build a compiler for eight as a result.

-----

1 point by fallintothis 5385 days ago | link

the author seems to agree with shader

Shh, you'll blow my cover!

-----

1 point by akkartik 5385 days ago | link

Oops :)

-----

2 points by aw 5387 days ago | link

What is a first class macro?

-----

2 points by fallintothis 5387 days ago | link

They're what first-class functions are to functions. That is, the ability to create anonymous macros and pass them around as first-class objects. Imagine you could do something like

  (let m (macro (x) `(foo ,x))
    ...)

  (map (macro (x) `(= ,x 'blah)) xs)

It's like the difference between having fn and just having def. Right now we only have mac and no way to reference macros as their own entity.

  arc> ((annotate 'mac (fn (x) x)) 'x)
  Error: "Function call on inappropriate object #(tagged mac #<procedure>) (x)"

Whenever the topic resurfaces, the word seems to be the same: they still make compilation difficult (the most recent research only seems to be from 2000 or so http://lambda-the-ultimate.org/node/2987) and pg has yet to comment on them. So to implement first-class macros currently, you'd probably effectively turn ac into an interpreter so that macros could expand at run-time.

-----

1 point by aw 5387 days ago | link

I don't understand your map example. Can you give an example of what xs could be and what the resulting output would be?

-----

2 points by fallintothis 5387 days ago | link

That example specifically? It was nonsense. My point was that you could, say, map macros (anonymous or not). Those used to inline functions, for instance: you want to use a macro like a function, but it's a macro for efficiency. Currently, you have to wrap it in a function, which is kludgey.

Silly example:

  ; instead of

  (map complement list-of-fns)

  ; you have to do

  (map [complement _] list-of-fns)

Actual example (news.arc and the karma macro):

  (def leading-users ()
    (sort (compare > [karma _]) ; instead of (compare > karma)
          (users [and (> (karma _) leader-threshold*) (~admin _)])))

Again, macros being first-class intuitively works like functions being first-class. So, with first-class macros you could have expressions return macros.

Silly example:

  ((if (mem 'x stuff) pull push) 'y stuff)

Actual example (my trace library http://www.arclanguage.org/item?id=10372):

  (mac m ...)

  ((traced m) ...) ; traced is a higher-order function
                   ; since m is a macro, (traced m) returns a macro

More popularly, Bawden's paper (http://people.csail.mit.edu/alan/mtt/) demonstrates writing a module system with first-class macros in which the modules are also first-class.

-----

2 points by shader 5387 days ago | link

I think the example was supposed to set all of the variable names contained in the list xs to contain the value 'blah.

In reality, you're more likely to want to take in two lists, one of variable names, and one of values, and assign each value to each name.

-----

1 point by fallintothis 5387 days ago | link

I think the example was supposed to set all of the variable names contained in the list xs to contain the value 'blah.

The example wasn't supposed to do anything. I was just foo-bar-baz-ing something. I don't think the example would actually do that anyways, for the same reason the following doesn't work.

  arc> (mac m (x) `(= ,x 'blah))
  #(tagged mac #<procedure: m>)
  arc> (m 'a)
  Error: "reference to undefined identifier: _quote"

I guess it would've alleviated confusion had I said

  (map (macro (x) ...) xs)

-----

2 points by shader 5387 days ago | link

Well, first class macros don't work right now anyway, so I wasn't expecting your example to work. I was just trying to figure out the idea behind the example.

And I think that first class macros would be required to set an arbitrary list of variables to contain an arbitrary list of values via map. Obviously with or let would work in a local scope, but if you want global variables you'd need something like that, or at least a function that wraps the macro.

-----

1 point by aw 5387 days ago | link

Oh, do you mean you want access to the lexical environment at run time? That sounds like to me more like you want the lexical environment to be a first class object.

-----

2 points by shader 5387 days ago | link

The lexical environment is (mostly) accessible at run time. The problem is when you're returning a macro from a function, instead of just renaming it.

Suppose you implement packages as tables of symbols. Suppose also that you have a macro stored in said package. There are two ways to use the macro

  (= newmac package!newmac)
  (newmac args)

  (package!newmac args)

I think that most people would like to use the second form, because it doesn't require importing the macro into the current namespace. Unfortunately, the second version requires first class macros.

Obviously, since the compiler can't easily determine that an expression is going to be a macro, it won't have the advantage of compile time optimization. I'm not sure how important that is. Given that arc is already a "slow" language, why not give it more expressive power at the cost of slightly lower speed? Especially since the user can choose whether or not to use them.

So, as I see it first class macros give you several features:

1. easy to make a package system that can contain macros 2. macros fit in better with functions; no wrapping of macros in functions 3. anonymous macros created for mapping across expressions, etc.

-----

2 points by aw 5387 days ago | link

Oh, I think I sort of get it now...

In an interpreter,

  (expr1 expr2 expr3)

evaluation of expr2 and expr3 can be delayed until after expr1 is evaluated. If expr1 turns out to evaluate to a macro, then the literal expr2 and expr3 can be passed to the macro.

-----