Arc Forumnew | comments | leaders | submit | rocketnia's commentslogin
1 point by rocketnia 4640 days ago | link | parent | on: A module system for Arc

"That discussion is actually a subthread for this discussion"

Er, is it the other way around? It looks like at least this post of mine came after the email discussion: http://arclanguage.org/item?id=16995

---

"Hyper-static scope easily handles all of that"

For whatever it's worth, I still technically disagree with the "all" here. I admit your approach handles the practical cases.

Your approach has developers hardcoding filenames within their source code. If a developer wants to use two files of the same name, they must find a way to segregate the files into subfolders, or they must rename a file and invade some library source code to rewrite the filename occurrences. Please let me know if I'm wrong about this. :)

The LtU post also goes over at least one use case where a library user wants to update the dependencies of the library without also updating the library itself. Fortunately, this time I think we can agree that this isn't the problem we're discussing. :) It's something I care about in a module system, but it's not directly related to name collision.

-----

1 point by Pauan 4640 days ago | link

"Er, is it the other way around? It looks like at least this post of mine came after the email discussion"

Yes, but the discussion started with the Arc topic and then moved to e-mail and then moved back to the Arc topic.

---

"I admit your approach handles the practical cases."

Well then! I'll consider that "all", since I only care about the practical cases.

---

"Your approach has developers hardcoding filenames within their source code. If a developer wants to use two files of the same name, they must find a way to segregate the files into subfolders, or they must rename a file and invade some library source code to rewrite the filename occurrences. Please let me know if I'm wrong about this. :)"

Yes. It is tied to the filesystem, or website URLs, or Git commits, or whatever. But, the filesystem already enforces a "no two files with the same name in the same folder" rule, so no biggie.

If you wanted to create something that manages dependencies at a more abstract level, that's fine, and you can build it on top of my system, but I personally don't see much use for that (yet).

If your worry isn't about filenames at all, and is simply about putting filenames into the source code, I think the answer is really easy: just do something like RubyGems, where you have a standard file called "dependencies" that imports all the dependencies in the right order. And then when you want to load the library, you'd just load the "dependencies" file. This doesn't require any changes to my system, since it's purely user-convention.

This still allows for putting the dependency information straight into the source code, which is useful for quickie scripts and such. But big projects and libraries would use the "dependencies" convention. And as Ruby showed, this kind of user-convention can be applied after the language is already in use. So it doesn't need to be baked in ahead of time, though there might be some minor transition pain. But I'll worry about that once libraries and projects become big enough that a "dependencies" convention becomes useful.

-----

2 points by rocketnia 4643 days ago | link | parent | on: A module system for Arc

"Things unique to the Arcueid implementation such as the marshalling mechanism (CIEL), the Limbo-inspired thread communication and synchronisation primitives, and so forth, must all go into their own modules to limit pollution of the global namespace with incompatible symbols as far as possible."

One of the first things I did in Arc was making a namespace system so I could pile on additional libraries (Lathe) without regret. :)

https://github.com/rocketnia/lathe

In my Lathe namespace system, I used prefix-like macros to rename variables into gensyms:

  my.foo                 -->  gs123-foo
  my!foo                 -->  'gs123-foo
  (my:foo:my:bar a b c)  -->  (gs123-foo (gs123-bar a b c))
Nothing but a global variable can expand as a macro in (typical) Arc, so these gensyms are the way I avoided conflict in the global namespace.

The approach currently present in Lathe has a few disadvantages as a "standard" module system for Arc:

- It's still a bit unhygienic since the prefix-like macros themselves (e.g. 'my) are kept as unmanaged global variables. I've often considered going back and standardizing on just two prefixes: my.foo and yr.foo.

- There are several features I built into the module system at the start and then never used. For instance, if you want to, you can remove a module from the module cache so that it's reloaded the next time someone requires it. If I went back to this system to start from scratch, I wouldn't bother with these features.

- I was never quite satisfied with a way to manage global tables like 'setter or uses of 'extend. In fact, I'm still pursuing a solution to these issues, but it's taking me far outside what's easy to do from within Arc (or any other language I know of).

-----

1 point by Pauan 4643 days ago | link

"I was never quite satisfied with a way to manage global tables like 'setter or uses of 'extend. In fact, I'm still pursuing a solution to these issues, but it's taking me far outside what's easy to do from within Arc (or any other language I know of)."

For what it's worth, Nulan also has that problem with the "syntax-rules" object. The way I decided to solve it is to provide a macro called "w/dict!":

  (var foo (obj a 1 b 2))
  
  (w/dict! foo
    (= foo!c 5))

  foo!c -> nil
Basically, any changes made to the object inside the w/dict! are not seen outside the w/dict!

So in Nulan, if a file creates some new syntax, and you want to load the file but not the syntax, you can use `w/dict! syntax-rules ...`

But, knowing you, you probably meant having these kind of object dependencies automatically detected. I don't have any good ideas for that, sorry.

-----

1 point by rocketnia 4643 days ago | link | parent | on: A module system for Arc

"I just realized: it's possible to add in hyper-static scope to Arc while retaining full backwards compatibility . (Crazy, no?)"

I think you forgot the main reason why backwards-compatibility isn't very feasible: Macros.

  ; ===== util.file ====================================================
  
  (var fun-map
    (fn (func . seqs)
      ...
      ))
  
  ; Lets you write (map x seq (+ 1 x)) in place of
  ; (fun-map (fn (x) (+ 1 x)) seq).
  (var map
    (mc (x seq . body)
      ...
      ))
  
  
  ; ===== this-is-fuuun.file ===========================================
  
  (import util)
  
  (var fun-map (list "  | O | X"
                     "--+---+--"
                     "X | X | O"
                     "--+---+--"
                     "  | X |  "))
  
  (pr (map line fun-map (+ line "\n")))
Arc macros behave as though macroexpansion were simply about constructing some lists of symbols. But we really want each macro-inserted symbol to be looked up in that macro's lexical environment.

I think you've been resolving this by writing macros so that the procedures are inserted directly into the macro result, rather than referred to indirectly by symbols. Right? I don't remember how you succeed at doing this when you compile your code to JavaScript. Maybe I'm thinking of two separate languages? Anyway, some previous discussion of this approach is at http://www.arclanguage.com/item?id=14849.

---

In Penknife, I handled macros by taking advantage of an existing assumption I was making about modules: Assume there are no side effects during the loading of the program, so that we can record the macroexpansion results to a file as a precompiled program without corrupting the program's behavior. Then any code that had a macro in scope at compile time will have a doppelganger of that macro in scope at run time. Whenever we encounter a variable during program execution, we can resolve it by looking up a macro value, accessing its lexical environment, and repeating until the original variable binding is in scope.

Penknife didn't really embrace the hyper-static global environment, but it would have been built upon the same sort of basis: Each file would have started in its own fresh environment, and some commands (namely, imports) would have worked by replacing the current environment.

---

"The definition of "=" is the same: if the variable exists, mutate it, otherwise create a new variable."

The behavior I'd use is that any compile-time variable access (even under a lambda) creates a new, uninitialized variable binding if a binding doesn't already exist.

  ; Create bindings for 'even and 'odd, then set the value of 'even.
  (= even (fn (x) (case x 0 t (~odd:- x 1))))
  
  ; Set the value of 'odd.
  (= odd (fn (x) (case x 1 t (~even:- x 1))))
If you wait to create the variable bindings until assignment time, then even's reference to "odd" is initially unbound, and you have to somehow associate it with the binding of 'odd created in the second line.

-----

2 points by Pauan 4643 days ago | link

"I think you forgot the main reason why backwards-compatibility isn't very feasible: Macros."

I didn't forget: Nulan completely solved the macro hygiene problem after all. But that's a more extensive change so I figured I'd save it for after the basic hyper-static scope system is in place.

In fact, assuming Arcueid does implement my proposal, I would actually go in and make a new version of arc.arc that uses "var" and has hygienic macros by default. Then you could simply load up the new arc.arc to get all the shininess. But loading the old arc.arc would have full compat with existing Arc 3.1 programs.

---

"I think you've been resolving this by writing macros so that the procedures are inserted directly into the macro result, rather than referred to indirectly by symbols. Right?"

Nope. Macro hygiene in Nulan just uses the already existing box implementation. It's really easy, really simple, and really fast. Seriously, boxes are awesome. No need to complicate things.

The way to solve it in Arc: just provide a function called "get-variable-box" which is only available at compile-time and it returns the box for the variable.

Then you change quasiquote so it uses "get-variable-box" rather than inserting the symbol directly. Bam, hygienic macros with no additional runtime cost, and extremely small compile-time cost. And they look and feel just like Arc macros, so you don't lose any power or convenience. No clunky Scheme macros, huzzah!

Once I understood that the fundamental problem was with dynamic scope, and the best way to solve it is with boxes (or similar), everything became super easy and awesome.

---

"The behavior I'd use is that any compile-time variable access (even under a lambda) creates a new, uninitialized variable binding if a binding doesn't already exist."

Yeah I'd do that too, if I wanted to graft dynamic variables onto a hyper-static system. But since Arc uses dynamic variables, I proposed to graft hyper-static onto it instead.

-----

1 point by rocketnia 4640 days ago | link

"But that's a more extensive change so I figured I'd save it for after the basic hyper-static scope system is in place."

The middle ground doesn't seem worthwhile to me. When programmers work with with Arc-style unhygienic macros, at each use site, the variables in scope must (mostly) match the variables the macro author expected. So I think people who like using macros will be happiest if they systematically keep the variable names consistent across all the code in their program (even others' code), at which point namespace mechanisms just get in the way.

---

"Nope. Macro hygiene in Nulan just uses the already existing box implementation . It's really easy, really simple, and really fast. Seriously, boxes are awesome. No need to complicate things."

I think you caught me on a technicality. :) I see "procedures are inserted directly into the macro result" as a general approach. Mutable boxes make it possible for this approach to achieve late binding. Elsewhere in this discussion you tilt the technicality closer to my phrasing, since you recommend to let users build boxes out of getter and setter procedures.

Anyway, I'm a fan of that approach when it works, but it doesn't work so well when compilation is involved: The macroexpanded code contains unserializable values--namely, the procedures or boxes we're talking about. This is a lesson I learned with Penknife, where I at first had macros insert boxes, and then had to reengineer this so macros inserted step-by-step treasure maps for how to find a variable from the run time environment.

---

"Yeah I'd do that too, if I wanted to graft dynamic variables onto a hyper-static system. But since Arc uses dynamic variables, I proposed to graft hyper-static onto it instead."

How do you make the even/odd code work? Under the approach you described, the first line refers to an undefined variable (odd), and I interpret that as an error. I was recommending a fix.

-----

1 point by Pauan 4640 days ago | link

"The middle ground doesn't seem worthwhile to me."

Retaining Arc compatibility in general doesn't seem worthwhile to me, but a lot of people want it, so I gave a system that retains Arc compatibility while tacking on some new shininess. Nulan doesn't have to worry about Arc compatibility, so it has pure hyper-static scope and hygienic macros by default.

---

"How do you make the even/odd code work? Under the approach you described, the first line refers to an undefined variable (odd), and I interpret that as an error. I was recommending a fix."

Easy: I have a macro called "defs" that handles mutual recursion:

  (defs
    even (x)
      (if (is x 0)
        t
        (odd (- x 1)))
    odd (x)
      (if (is x 0)
        nil
        (even (- x 1))))
The above macroexpands into this:

  (var even)
  (var odd)
  (= even (fn (x)
    (if (is x 0)
      t
      (odd (- x 1)))))
  (= odd (fn (x)
    (if (is x 0)
      nil
      (even (- x 1)))))
Basically, it first creates new boxes, and then it assigns the functions to the boxes. This is one of a few reasons why I prefer mutable boxes over immutable boxes. Though, you could probably have "defs" expand to a Y-combinator instead, if you really wanted immutability...

---

"I think you caught me on a technicality. :)"

Maybe it was just a simple miscommunication. What you were talking about sounded exactly like the technique of splicing function values using macros:

http://www.arclanguage.org/item?id=14507

What I'm talking about happens entirely at compile-time using boxes. The effect is very similar, but the implementation is very different.

---

"Anyway, I'm a fan of that approach when it works, but it doesn't work so well when compilation is involved: The macroexpanded code contains unserializable values--namely, the procedures or boxes we're talking about"

Sure, if I cared about serialization, I'd have to make it more complicated. Thankfully, the only serialization I care about right now is compiling to JavaScript code, which is easy enough to do with variable renaming.

Also, what's the point in serializing boxes since functions still can't be serialized? If you found a way to serialize functions, then it'd be much more useful to be able to serialize boxes.

-----

1 point by rocketnia 4638 days ago | link

"Easy: I have a macro called "defs" that handles mutual recursion"

While I appreciate 'defs, it's a non-answer. The even/odd example I posted and the evenp/oddp example dido posted are idiomatic Arc code. While you and I don't care much about Arc compatibility, it's something dido wants for Arcueid, so these examples should work without modification.

---

I'm about to disagree with myself, but first I want to reiterate and clarify what I was saying at "caught me on a technicality":

For this discussion I don't see a much of a reason to distinguish between macros which insert mutable boxes and macros which insert functions. Either system can pretty much support the other as a special case: We can translate spliced boxes into spliced getter/setter functions, and we can translate spliced functions into spliced functions-in-the-box. Because of that equivalence, these systems share the disadvantage of being challenging to serialize.

If dido considers compilation to be important (do you, dido?), then this hygiene approach might be unsuitable, and thus the use of first-class namespaces might be unsuitable. (As I explained at "match the variables the macro author expected," first-class namespaces make hygiene more important.)

---

"What I'm talking about happens entirely at compile-time using boxes."

Ah. I think you have a point!

For compiling Nulan to JavaScript, I guess the boxes you're using aren't arbitrary getter/setter functions, and they aren't merely some mutable container either; they're globally associated with a JavaScript variable name. When you compile the macroexpansion result and it contains a (get-variable-box ...) form, you decide on its JavaScript variable name at that time. If the macroexpansion result contains a box, you use the attached variable name to compile it to JavaScript. Am I getting this right? This sounds very workable. :) And whaddayaknow, Nulan works. ^_-

I seem to remember understanding this before, when you and I talked about Nulan compilation in depth, but I guess I had to retrace the steps just now.

Anyhow, get-variable-box is fantastic IMO, but first-class namespaces still might not be ideal for Arcueid due to Arc's unhygienic macros.

dido, are you comfortable with breaking existing Arc macro idioms in favor of hygiene?

---

I have a convoluted but surprisingly comprehensive idea of how to integrate get-variable-box into a system that's compatible with unhygienic Arc macros, but I've put it in a separate simultaneous post: http://arclanguage.org/item?id=17464

Actually, it's two separate posts, because it's otherwise too long for the forum. If this becomes a tl;dr scenario, I won't be surprised. ^_^

-----

3 points by Pauan 4637 days ago | link

"While I appreciate 'defs, it's a non-answer. The even/odd example I posted and the evenp/oddp example dido posted are idiomatic Arc code. While you and I don't care much about Arc compatibility, it's something dido wants for Arcueid, so these examples should work without modification."

For this example, let's suppose there was a file "foo.arc" that contained idiomatic Arc code that implements evenp/oddp. This code works in Arc 3.1. It will work in my system as well, because undefined symbols automatically create new boxes. Basically, it'll work, but name collisions are possible, just like in Arc 3.1.

If you then write a new file "bar.arc" that uses hyper-static idioms (var, defs, etc.), it can import "foo.arc" and everything will work fine. "foo.arc" will clobber any existing evenp/oddp definitions, but "bar.arc" will not clobber "foo.arc". And of course "bar.arc" can use "w/include" and "w/exclude" to prevent "foo.arc" from clobbering things.

If you wanted to make it so that "foo.arc" behaves correctly without needing to use "w/include" and "w/exclude", you would indeed need to rewrite it to use "defs". But it's still usable even without a rewrite. So it's a perfectly graceful degradation.

My system is designed so that it can correctly use all existing Arc 3.1 code, while new code is written with the hyper-static idioms. Then, slowly, old code can be migrated to use hyper-static scope, until eventually you could make Arc purely hyper-static.

There's three issues I see with my proposal:

1) If you're writing Arc code in a hyper-static fashion, you really want "arc.arc" to be changed to be hyper-static. But old Arc code will need the non-hyper-static "arc.arc". I think the simplest solution to this is to have two versions of "arc.arc", one that uses hyper-static scope, and one that doesn't. Then you would need to make sure to load the non-hyper-static version before loading Arc 3.1 code. This could be automated a tiny bit by using a macro, something like "w/arc3".

2) "load" occurs at run-time, which is why my definition of "w/include" needed to use "eval". Nulan doesn't have this problem because file importing occurs at compile-time. Perhaps the best way to solve this is to keep "load" as-is, and add in a new "import" macro that does all its work at compile-time.

3) If you think (eventually) making Arc purely hyper-static is a bad thing, you won't like my proposal.

---

"Am I getting this right? This sounds very workable. :) And whaddayaknow, Nulan works. ^_-"

Yes, that's more or less correct. The one detail that's different is... Nulan doesn't have a "get-variable-box" function. The reason is because "quote" internally uses (the equivalent of) "get-variable-box". So in Nulan, rather than using "get-variable-box", you'd just use "quote". And if you want to break hygiene, you'd explicitly use the "sym" function.

-----

1 point by rocketnia 4637 days ago | link

I mostly followed along, but I don't understand "It will work in my system as well, because undefined symbols automatically create new boxes." You were talking about having them create new boxes at assignment time, and I was recommending compiling-a-reference-time instead so that we don't get an unbound variable error in the first definition.

-----

1 point by Pauan 4637 days ago | link

How it works is, anytime the compiler sees an undefined symbol, it creates a new box for it like as if it had been created with "var".

Another way to think about it is... the compiler would replace this:

  (= foo (fn () ... bar ...))
  (= bar (fn () ... foo ...))
With this:

  (var bar)
  (var foo)
  (= foo (fn () ... bar ...))
  (= bar (fn () ... foo ...))
What happened is, when it encountered the undefined variable "bar", it created a new box for it. Then it encountered the undefined variable "foo", so it created a new box for it. Then it did the assignments like normal.

Given how you said "compiling-a-reference-time", I think we're talking about the same thing. Why did you mention assignment time?

-----

3 points by rocketnia 4637 days ago | link

"Why did you mention assignment time?"

We've just had a long exchange about you creating boxes at assignment time and me using compiling-a-reference time instead. Here's a recap:

---

You: Here's how you do it. The definition of "=" is the same: if the variable exists, mutate it, otherwise create a new variable. But now you add in a new primitive called "var"[...]

Me: The behavior I'd use is that any compile-time variable access (even under a lambda) creates a new, uninitialized variable binding if a binding doesn't already exist.

You: Yeah I'd do that too, if I wanted to graft dynamic variables onto a hyper-static system. But since Arc uses dynamic variables, I proposed to graft hyper-static onto it instead.

Me: How do you make the even/odd code work? Under the approach you described, the first line refers to an undefined variable (odd), and I interpret that as an error. I was recommending a fix.

You: Easy: I have a macro called "defs" that handles mutual recursion

Me: While I appreciate 'defs, it's a non-answer. The even/odd example [...] should work without modification.

You: It will work in my system as well, because undefined symbols automatically create new boxes.

---

At least we seem to be agreeing now. ^_^;

-----

3 points by Pauan 4637 days ago | link

Ah, sorry, huge miscommunication and misunderstanding on my part. I've actually been agreeing with you all along.

A large part of the problem is that I've been thinking about my proposal as two separate parts: one part deals with backwards compat with Arc, and the other part describes a hyper-static system for Arc.

When I was talking about "defs", I was talking about the hyper-static part. But you were talking about the backwards compat part. Hilarity (?) ensues.

-----

1 point by rocketnia 4637 days ago | link

Okay, we're on the same page now then. ^_^

Having both kinds of scope as options would be great.

-----

2 points by rocketnia 4647 days ago | link | parent | on: Wart update: swapped are args to 'isa'

Reminds me of this old thread you and I participated in: http://arclanguage.org/item?id=10752

aw: "Here's an "as" macro, which swaps the arguments to "coerce", and quotes the type:"

me: "You could do the same thing with isa. [...] This particular example probably isn't quite as useful; if I have a complex expression, I usually want to use the result itself at some point, rather than just passing it to isa."

you: "Languages like haskell do a better job explicitly encouraging this practice - you can curry functions with frequently-used args."

-----

1 point by akkartik 4647 days ago | link

I have no idea what I was saying there, but it took me a couple of years to truly appreciate both your points :)

-----

2 points by rocketnia 4647 days ago | link

"I have no idea what I was saying there"

Oh, well I got a lot from it. :) I may have quoted you out of context just now, but I think your post was the first time I thought about the idea that optional args and Haskell-like currying may encourage opposite argument orders. If a function argument is going to remain mostly constant, it should go last so it can be optional, but it should also go first so we can curry it away.

Since then, I've come to think the conundrum is largely internal to currying itself: Currying is useful when people have a frequent need to insert their own locally constant value. That's a tenuous balance in itself, and only one half of it conflicts with the general idea of putting stable values toward the end.

(Here's another really old exchange about argument order in Haskell: http://arclanguage.org/item?id=4705)

At the moment, I just say no to currying. I even manually eta-expand higher-order applications so it's easier to see what function signatures I expect, and so they're easier to step through in a debugger.

  // No:
  _.arrAll( arr, _.isString )
  
  // Yes:
  _.arrAll( arr, function ( x, i ) {  // or just ( x )
      return _.isString( x );
  } )

-----

2 points by akkartik 4645 days ago | link

After reading http://arclanguage.org/item?id=4703, I wanted to be able to say both:

  (map f (keep f (sort > ..)))
and:

  (map :over seqs
       (fn (f) ..))
But I can't do that. The ways that wart gets in the way are instructive:

a) First I tried adding an alias to https://github.com/akkartik/wart/blob/2fa2a3b1c0/043list.war...:

  def (map f seq|over)
    ..
But it was easy to forget that I extend map later on: https://github.com/akkartik/wart/blob/2fa2a3b1c0/050list2.wa....

Lesson: when adding param aliases we need to update all later extensions. That seems painful.

b) Even after I identified both places to modify, it's unclear how to deal with this declaration:

  def (map f ... seqs)
I could make it:

  def (map f ... seqs|over)
But then this call combines all args into seqs.

  map :over fs
      (fn (f) ..)
Lesson: rest args by keyword sometimes don't work well. Better to try to find the right names for other args.

Maybe something like this?

  map fs :do (fn (f) ..)
I still can't think of the right keyword to make this readable.

Update: I ended up going with

  map fs :with (fn (f) ..)
(https://github.com/akkartik/wart/commit/537fb6d832)

I also made a change to permit keyword args after rest keyword args:

  map :over fs :with (fn (f) ..)
(https://github.com/akkartik/wart/commit/b3667cda49)

Which of these do people prefer? Any other options?

-----

1 point by akkartik 4647 days ago | link

Looking elsewhere in that thread, one idea for really-as is explicit currying:

  (as.int.nil.0 (arg req "id"))
..or something.

Hmm, at the very least, perhaps we can curry as so that we no longer need functions like int (though I know you prefer recognizers to types ^_^)

  (as.int "34")
Now wart's uniform left-associative precedence truly comes into its own:

  (as.int+arg req 'id)

-----

2 points by rocketnia 4647 days ago | link

"Hmm, at the very least, perhaps ... (as.int "34")"

I had thought about that for Penknife (http://www.arclanguage.org/item?id=13715). I was going to call it "ify" and then use it with the reverse application syntax:

  [int'ify:arg req s.id]
(Note Penknife's uniform left-associative precedence.)

Similarly, 'isa was going to be "ish":

  int'ish.x
However, I still don't see any semantic benefit in associating a coercion function with a type tag. This would have been nothing but a way to shove a bunch of different utilities into one organizational unit, like Java's static methods. And an organization style like this can backfire: If the system is trying to be securable according to the principles of object-capability model, a programmer who passes a whole open-ended bundle of utilities into untrusted code might accidentally expose more privileges than they've bargained for.

---

"though I know you prefer recognizers to types"

I was thinking of bringing that up in response to the OP, but I think I'm mostly on the side of type tags now! I use tables with "type" fields all the time. I like the ability to dump these tables at a REPL, serialize them, use 'iso for deep comparison, or pass them between frames (in JavaScript). This could be a mess if I use more of these features than I plan to support, but "adding" support is as easy as changing my mind. :-p

My old argument for maintaining a dedicated (foo? x) procedure was so that the 'foo? symbol could be namespaced just like any other member of the global scope. But if the type needs to go outside the language runtime and into serialized data or other concurrently executing runtimes (namely, browser frames), then runtime-local tools for namespace management aren't much help.

Fully abstraction-leak-proof namespace management amounts to having a secure notion of which programs have privilege over which namespaces. Cryptography makes it possible to achieve a pragmatic degree of security at the level of serialized data. I've been keeping this in mind as a guideline during my recent module system pursuit.

-----

2 points by rocketnia 4653 days ago | link | parent | on: Cyclical data structures in arcueid

Both of the things you're talking about look specialized for cycles in flat lists. I think dido's examples could have just as easily used 'scar instead of 'scdr.

  arc> (= x '(1 2))
  arc> (scar x x)
  ((...) . 2)
  #0=(#0# . 2)
I do like the (1 2 ... (3 4 ...)) notation for cyclic lists though. :)

The Kernel R-1RK has a good approach to treating flat cyclic lists as though they're a usual case. IMO, it means everything that deals with lists is more complicated to explain and arguably doesn't even deal with "lists."

-----

2 points by dido 4651 days ago | link

Well, now that I consider it the real reason why I missed these cycle detection algorithms is that I've never thought of conses as being just lists. Sure, they're general enough to represent lists, and binary trees, but with the presence of scar and scdr, they are also capable of representing directed graphs with vertices of at most degree 2. In general, any Arc variable can be considered as a reference to a general directed graph (hash tables in particular can be considered vertices of arbitrary degree). Thus, pretty-printing an Arc variable boils down to traversing the graph it represents, and the traversal order that seems like it makes best sense here is depth-first search.

To do the #0=(#0# . 2) notation though seems like the traversal needs to be done twice, first to get the numeric assignments, and next to actually print them. Not sure if it can be done in one pass without making a tree of prettyprint fragments. A marshalling algorithm will need to do it that way though, I should think.

-----

2 points by akkartik 4653 days ago | link

You got me rereading R-1RK, and I was reminded again of how much I like the way Shutt connects up his design decisions to design goals and constraints. It's very in the spirit of Christopher Alexander (http://www.amazon.com/Notes-Synthesis-Form-Harvard-Paperback...).

In spite of the level of detail, kernel's design goals diverge so quickly from my own that I can't reuse any of the design work. It's really too bad.

Today's frustrating example: I really couldn't give a rat's ass that Kernel permits cyclical argument lists in operatives but not applicatives. Then in A.4 he talks about what's missing before R0RK, and the support for cycles feels pedantic next to the problem of a fast implementation. If he wasn't so concerned about handling cycles during eval perhaps the code wouldn't be slow. How the heck can a language be slower than wart?!

-----

1 point by akkartik 4653 days ago | link

Yes, the print example was just an illustration. It was also a good test for wart; I caught several bugs :) https://github.com/akkartik/wart/compare/686828edba...0e2388.... The tests are pretty self-contained.

I'd assumed I could apply Floyd's hare to handle cars as well, but you're right, now I'm not so sure. The key difficulty is that there's two dimensions, so does the number of hares double each iteration?

Here's a flat-cycle-detecting iso I've been playing with:

  def (iso a b)
    ((afn ((a harea|through) (b hareb|through))
      (if ~list?.a
            (a = b)
          (addr.a = addr.harea)
            (addr.b = addr.hareb)
          :else
            (and (iso car.a car.b)   # Can't detect cycles through car.
                 (self `(,cdr.a :through ,cdr+cdr.harea)
                       `(,cdr.b :through ,cdr+cdr.hareb)))))
      (list a :through cdr.a)
      (list b :through cdr.b))

-----

2 points by rocketnia 4650 days ago | link

"The key difficulty is that there's two dimensions, so does the number of hares double each iteration?"

Hmm. As the tortoise goes down branches, its location becomes indeterminate. As the hare goes down branches, its location becomes indeterminate relative to the tortoise, so we might need multiple hare markers per branch if we're doing a tortoise-directed search. If we try using the hare to direct the search instead, then we may still need to keep a growing history of hare locations on each branch so the tortoise can follow.

Either way, keeping a hash table sounds more straightforward to me.

Are you using this algorithm properly? I haven't seen any of your example code do a second pass to find out where the cycle begins and what its period is. But maybe you don't need to...?

-----

1 point by akkartik 4650 days ago | link

I'm just taking a constant-factor hit :) A few extra iterations doesn't change the result for either print or iso. And for cycles that loop back all the way to the start it turns out there's zero overhead.

  x <- '(1 2 3)
  lastcdr.x <- x  # 1 2 3 1 2 3 1 ...
  print x
  1
  2
  3
  ...  # exactly one iteration
But:

  x <- '(1 2 3)
  lastcdr.x <- cdr.x   # 1 2 3 2 3 2 3 ...
  print x
  1
  2
  ...

-----

2 points by rocketnia 4650 days ago | link

  You posted:
  
  x <- '(1 2 3)
  lastcdr.x <- cdr.x   # 1 2 3 2 3 2 3 ...
  print x
  1
  2
  ...
Is that the behavior you expect? I would expect "1 2 3 ...", but it looks like the rabbit and the hare meet at the (3 ...) cons and stop.

  1 2 3 2 3 2 3
  *              Print "1".
    * *          Print "2".
      *   *      Print "...".

-----

1 point by akkartik 4650 days ago | link

Yeah I tried it out before posting. I had to take pains in iso to make sure we do one full traversal. With print I didn't care as much.

Update: Ack, I found a bug in iso (http://arclanguage.org/item?id=17365):

  x <- '(1 2 3)
  y <- '(1 2 4)
  (do1 nil (<- lastcdr.x x lastcdr.y y))  # Avoid printing the cycles
  (iso x y)   # not nil!

-----

2 points by rocketnia 4650 days ago | link

Here's a slightly different bug I found before your update. It's another case where I don't know what behavior you expect, but it probably isn't this. :-p

  (do (a <- '(1 2 3)) (lastcdr.a <- cdr.a) nil)
  => nil
  (do (b <- '(1 2 3 2 3)) (lastcdr.b <- cdr.b) nil)
  => nil
  (not+not+iso a b)
  => nil
  (not+not+iso b a)
  => 1
(I'm using not+not+iso so it returns a predictable value rather than an address.)

-----

1 point by rocketnia 4650 days ago | link

"the rabbit and the hare"

Whoops. I didn't mean that. XD

-----

2 points by rocketnia 4654 days ago | link | parent | on: Wart update: compose is now '+'

Here's a discussion I remember every time I think of whether or not to use + as an identifier character: http://arclanguage.org/item?id=10069 "Looks like the answer is to use & instead of +. I like the look of + better, but I can tell people are going to want to use it in names."

-----

1 point by akkartik 4654 days ago | link

..which I've already taken off the table :)

-----

2 points by rocketnia 4653 days ago | link

I don't know what my point was in posting that link--I mulled over it for a long time before just posting what I'd typed in--and now I don't understand your reply. What did you take off the table, and when? ^_^;

-----

2 points by akkartik 4653 days ago | link

:) I assumed you were saying that wart wouldn't be able to include '+' in symbol names, but that was already true before this change. Wart has other constraints, but if you can use '&' for something you can also use '+'.

Does that trigger memories? Did I understand you right? Were you nostalgic for the good old days when wart symbols could include '+'? ^_^

-----

2 points by rocketnia 4653 days ago | link

"I assumed you were saying that wart wouldn't be able to include '+' in symbol names, but that was already true before this change."

Ah, point taken! You were already treating + as infix. Got it. ^_^

-----


I like the way Mezzo deals with memory using "ideas from affine type systems, separation logic, and systems based on regions and capabilities," but my enthusiasm is capped for two reasons.

One of the reasons is the same as one of David Barbour's criticisms of typestate.[1] Why should I care about Mezzo's state handling if it only permits reasoning about state metaphors within the program, and not external resources that are inherently stateful?

Two, this blog post focuses on how Mezzo can express programs without particular classes of errors, rather than with particular conveniences or desirable properties. Eliminating failure doesn't imply that success is easier.

[1] http://awelonblue.wordpress.com/2011/09/26/trouble-with-type...

-----

3 points by rocketnia 4674 days ago | link | parent | on: Remove all duplicates from a list

Not to burst your bubble, but Arc's "dedup" already does this. :) Here's Paul Graham's implementation of 'dedup in Arc 3.1:

  (def dedup (xs)
    (with (h (table) acc nil)
      (each x xs
        (unless (h x)
          (push x acc)
          (set (h x))))
      (rev acc)))
If you indent your code block with two spaces, it'll display as code. I think this might be how you wanted your code to look:

  (def unique_list (ls)
       "Removes all duplicated values in LS. Returns a new list"
       (= ls (sort < ls))
       ((afn (ls uniq_ls x) ;; uniq_ls -> the list we are building, x-> last inserted value
         (let y (car ls) ;; y -> actual value
          (if (no y) 
              uniq_ls ;; we're done, return list
            (is x y)
            (self (cdr ls) uniq_ls y) ;; current val is same than last one, we don't insert it
            (self (cdr ls) (join uniq_ls (list y)) y)))) ;; else we insert it and continue recursion
        ls () nil)) ;; function call

-----

2 points by akkartik 4673 days ago | link

Here's how I would write unique_list without changing your algorithm:

  (def unique_list (ls (o uniq_ls) (o x))
    "Returns ls without duplicates; order unspecified.
    Only works for lists of numbers."
    (if (no ls)
          uniq_ls
        (no uniq_ls)
          (unique_list (sort < cdr.ls) (list car.ls) car.ls)
        (iso car.ls x)
          (unique_list cdr.ls uniq_ls x)
        'else   ; personal idiosyncracy not an arc idiom
          (unique_list cdr.ls (cons car.ls uniq_ls) car.ls)))
1. I assume you only want it applied to lists of numbers? Otherwise sort won't work.

2. Since you're messing up the order anyway, why not return the result in descending rather than ascending order? It's faster.

3. I'd rather just pass car.ls around so I don't need an extra level of indent for y. But slight changes to the code might cause me to revisit that. Arc's ssyntax makes simple expressions look like names for free. (The performance hit is not worth thinking about.)

4. I don't like how the sort is hidden in my solution; perhaps I'm trying too hard to avoid the afn :) I've never liked that idiom, perhaps because it's so hard to indent well.

5. Optional args can often obviate the need for anonymous inner functions.

-----

2 points by Mikechaos 4673 days ago | link

Very interesting! Indeed, I had though about optional args (and that is what I had done first) but didn't want sort called at each rec. I like how you fixed it. Sort might be a bit obfuscated.. But I don't find it really bothering. It's use is straightforward and self explained!

As I see it, the dot (car.ls) is syntactic sugar for applying a one arg function? Pretty clever! It replaces really well the y, no doubt.

The differences are subtle, but it definitely feels better, my personal alarm gets a bit of rest.

Use of iso. Don't know why I used is.. Question I had, can '=' be use as a comparison operator?

Also I get your use of 'else, but not exactly sure how it passes. Does it count as an expression evaluated to true, so then (unique_list...) gets to be it's true clause/case? Or is is simply not counted at all (making (unique_list...) the false clause of (iso car.ls x). Not sure if I'm making my self clear here, so I'll try and rephrase a bit. A -> 'else gets to be an other if expression (if 'else) wich always evaluated to true -> following expression is the true clause of it. (what I think is happening) B -> 'else isn't counted as a full s-expr (not sure it's the right way of saying this) and so it just goes along with following (unique_list...) -> Makes it the false clause of (iso car.ls x) and not interpreted as an other if. (Then, I don't understand why!)

Finally, and that's really a bonus I'm asking, but since I'm looking into making my next side-project (wich could be a long-term project) in Arc-based lisp, it could get me closer, I'll ask it anyway. As of what I know, since PG's fully involved in YC, and so arc's development is in a semi-dead state, I'd better be at least looking at forks of it. Now, if I understood well, there are two major forks, one you are working on, that would be wart, and anarki (mistaken already?).

Would there be any difference in wart's implementation of the same alorigthm (since it shows a bit more use-case than pg's algorithm), or then, what would be wart's more natural form?

Thanks for the time! It's already more than I could ask!

-----

5 points by Pauan 4673 days ago | link

"As I see it, the dot (car.ls) is syntactic sugar for applying a one arg function? Pretty clever!"

Arc has the following ssyntax:

  foo.bar     -> (foo bar)
  foo!bar     -> (foo 'bar)
  
  (~foo 1)    -> (no (foo 1))
  (foo:bar 1) -> (foo (bar 1))
  (foo&bar 1) -> (and (foo 1) (bar 1))
To be honest, I only use the ".", "!", and ":" ssyntax. Also, "~", ":", and "&" work even when they're not in functional position:

  (map ~foo ...)
  (map foo:bar ...)
  (map foo&bar ...)
---

"Use of iso. Don't know why I used is.. Question I had, can '=' be use as a comparison operator?"

The only difference between "is" and "iso" is that "iso" recursively checks lists:

  (is  (list 1 2 3) (list 1 2 3))  ; nil
  (iso (list 1 2 3) (list 1 2 3))  ; t
"=" is always used for assignment, never for comparison. The only built-in equality function is "is", and "iso" is defined using "is".

---

"Also I get your use of 'else, but not exactly sure how it passes."

Like most Lisps, Arc has a datatype called "symbol", which is roughly equivalent to an "identifier" in other languages. Normally, if you use "else" it will refer to the variable "else", but if you use quote, it will be the symbol "else":

  else   ; variable
  'else  ; symbol
The rule for booleans in Arc is that the symbol "nil" is false, and everything else is true. The symbol "else" is not equal to the symbol "nil", therefore it is true.

The reason why akkartik used "else" is a personal style difference. It's idiomatic to just leave it off:

  (if 1    ; if
        2  ; then
      3    ; if
        4  ; then
      5)   ; else
---

"Now, if I understood well, there are two major forks, one you are working on, that would be wart, and anarki (mistaken already?)."

wart isn't a fork of Arc, it's a different language which has some similarities with Arc.

You can find a bunch of Arc info here:

https://sites.google.com/site/arclanguagewiki/

In particular, anarki is a true fork of Arc, which has diverged a lot, so don't expect much compatibility with existing Arc programs. But if you want all kinds of new features, I'd recommend it. I haven't used it myself though, so that's about all I can say about it.

Arc/Nu[1] is a fork of Arc I created, which should be mostly compatible with Arc 3.1, but cleans up the compiler and adds in a few new things. Naturally I recommend using this if you want something similar to Arc 3.1.

Then there's various other implementations of Arc, such as jarc and Rainbow, and an incomplete C implementation of Arc called Arcueid.

---

* [1]: https://github.com/Pauan/ar

-----

3 points by akkartik 4673 days ago | link

  (if 1    ; if
        2  ; then
      3    ; if
        4  ; then
      5)   ; else
Yeah, it's problematic how to indent the "5". The above way makes it look like a test rather than an action, and indenting it further makes it seem like part of the same block as 4.

You can tell that there's no idiomatic way to deal with this because the arc/HN codebase is schizophrenic, using both in different places.

-----

2 points by rocketnia 4673 days ago | link

Well, I always indent it in one of two ways, depending on how much room I have:

  (if 1  2
      3  4
         5)
  
  (if 1
    2
      3
    4
    5)
This style may have drawbacks, but I stubbornly use it anyway. :-p

  (list:if 1   ; misaligned condition
    (do a b c
      (d e f))
      (is x 0)  ; looks like part of the previous expr
    4
    5)
EDIT: Changed "I like it most of the time anyway" to "I stubbornly use it anyway." XD Somebody upvoted me before my edit....

-----

1 point by akkartik 4672 days ago | link

I understood what you meant, rocketnia ^_^

-----

2 points by Pauan 4673 days ago | link

For Nulan, my current thought is that multi-case "if" is too confusing with indentation, so the above would be written like this:

  if 1
    2
    if 3
      4
      5

-----

3 points by akkartik 4673 days ago | link

Pauan already did a great job explaining my use of else; nothing to add there. But I'll show off how it would look in wart:

  def (uniqify ls into|result and|x)
    "Returns ls without duplicates; order unspecified.
    Only works for lists of numbers."
    if (no ls)
         result
       (no result)
         (uniqify sort:cdr.ls :and car.ls :into list:car.ls)
       (car.ls = x)
         (uniqify cdr.ls :and x :into result)
       :else
         (uniqify cdr.ls :and car.ls :into (cons car.ls result))
0. I can choose to skip some of the outer parens if that looks cleaner.

1. The function name for the def goes inside the param list to mimic the call being defined.

2. The '|' operator binds multiple names to the same argument, so that I can refer to the second arg as either result or into, etc. Multiple names might seem like a bad idea, but they allow me to reorganize the later recursive calls to uniqify to look more clean and mnemonic. If you're familiar with python's keyword args, this is the same idea.

(Though maybe I'm getting too clever in this case; if I ever try to use the and operator inside this function I'm liable to confuse myself.)

3. All params are now optional, like in javascript.

4. '=' is for equality, never assignment. And it can be infix.

It's a little confusing at the start to use colons in two different ways so close to each other, but here's how it looks with wart's syntax highlighting for vim: http://i.imgur.com/S7ZROtD.png. Since symbols with a colon at the start always evaluate to themselves just like literal numbers or strings, I highlight them the same as literal numbers or strings.

---

Feel free to ask us more questions regardless of what language you decide to use[1] for your side project. If it isn't one of the above, maybe you can teach us something.

[1] Nu is strictly superior to arc 3.1 in every way.

[Update: there's a bug in the above code. In my original and here I forgot that the call to sort requires a second arg:

  (sort (<) cdr.ls)
We need parens around the '<' to keep wart from parsing it as an infix op.]

-----

3 points by rocketnia 4673 days ago | link

"It's a little confusing at the start to use colons in two different ways so close to each other, but [...]"

I was just thinking that. What would you think about keyword args that started with a hyphen instead of a colon?

  (uniqify (sort (<) cdr.ls) -and car.ls -into list:car.ls)
This way they look like shell command options, and the whole thing looks like a call to "uniquify-X-and-Y-into". Just an idea. :)

-----

2 points by akkartik 4673 days ago | link

I like this idea!

It's too bad this suggestion also uses an operator char, though. Or maybe there's some way to describe keyword args as a macro?!

-----

2 points by Pauan 4673 days ago | link

"perhaps I'm trying too hard to avoid the afn :) I've never liked that idiom, perhaps because it's so hard to indent well."

That's why Arc/Nu provides alet and awith...

  (alet foo 1
    (self (+ foo 1))

  (awith (foo 1)
    (self (+ foo 1))
...which are equivalent to this:

  ((afn (foo)
     (self (+ foo 1)))
   1)

-----

1 point by Mikechaos 4673 days ago | link

Ah! No breaking bubble! I searched for it and couldn't get it (there is no complete doc about all pre built function in arc, out there, heh? You just gotta read the source on and on till you get them all, I guess (wich is, in definite, not a bad thing at all!)).

I actually wanted to re-write it using a hash but couldn't get right away the semantics and syntax they had. This is perfect, It'll get me to discover how they actually work.

Thanks for the tip! It'll render better :)

Though, for the sake of learning (I am, for sure, studying PG's version and learning from it), could anyone quickly comment my code cause and what bad practices I'm doing (I may be a newbie in Lisp, but I still feel when things aren't elegant!).

Thanks to all

-----

1 point by akkartik 4673 days ago | link

(Eek, accidentally clicked down when I meant to vote up.)

-----


Here's a pruned-down summary:

"If we can extend an extensible type and reimplement its interface (and re-prove its invariant) so that the new implementation/proof is observationally equal to the original as far as the original cases are concerned, then our extension may as well have been part of the type all along."

"This take on the expression problem can easily generalize it to cubes, to say the least!"

Solve ALL the expression problems!

---

"I don't particularly intend for my system to support induction or recursion, at least at the proof level. It's great to be able to manage infinite families of value given only finite tools, but I see that as more of a nice-to-have feature, less essential than the modularity and convenience I'm trying to establish."

"I'm interested in demonstrating that this system ties into the foundation of mathematics with multiple relative consistency results, and ironing out any paradoxes along the way. However, I'm even more motivated to expand my model into a full programming language. With any luck, that'll help on the way to those proofs."

-----

1 point by rocketnia 4683 days ago | link | parent | on: Rudimentary pattern-matching in def

"Since I posted this I ended up using backquotes for pattern matching"

I agree with Pauan. This is reasonable. ^_^ Well, beware of corner cases like trying to match literal uses of the symbol 'unquote.

-----

1 point by akkartik 4683 days ago | link

Wart knows knows no symbol unquote, only a literal comma :)

-----

More