What bugs in the last Arc release would you like fixed in the next? For now please stick to bug reports rather than feature requests: i.e. cases where an expression clearly does or returns the wrong thing.
x!y!z is (x 'y 'z) rather than ((x 'y) 'z); likewise x.y.z. While perhaps not a bug per se, this is clearly undesirable (at some point, adding more infix dots is less clear than just parenthesising, so x!y!z for (x 'y 'z) is not very useful, but chaining lookups is common and is much nicer to read as x!y!z than ((x 'y) 'z)) and should be fairly simple to fix.
I would argue that the fact that you can't just say foo!a!x is a borderline bug, as well.
Also, I wish this would work right:
(map `[x _ z] '(1 2 3))
It should give the same result as (map [list 'x _ 'y] '(1 2 3))
Some primitive data types leak scheme internals when printed, such as hash tables- I would call this a bug.
Also, all datatypes should really be readable. For instance,
(read (tostring (pr (obj a 1 b 2))))
should not give an error.
As an aside, if you haven't looked at the map/vector syntax in clojure you really should... The fact that you can write [1 2 (+ x 1)] to yield the same result as (list 1 2 (+ x 1)) or `(1 2 ,(+ x 1)) is very compact, clever and useful for writing succinct code.
(It'll create a vector not a list of course, but that's orthogonal to my point, since all clojure sequence functions can use these two interchangeably)
It would be a mistake to take the atomic out of expansions of =.
Operators that modify things have to be atomic in the stretch between reading the value and writing the modified value.
Suppose x is initially 0 and you have two threads both evaluating (++ x). If the ++s
aren't atomic, you could end up with this sequence:
thread 1 reads the current value of x, 0
thread 2 runs in its entirety, leaving x = 1
thread 1 resumes, setting x to 1 + the 0 read earlier, or 1
You could similarly have two threads evaluating pushes onto the same list that
ended up losing one of the pushed values.
I don't think you should expect to be able to throw control out of an atomic expression-- at least not short of some abort-as-disaster operator. That's the definition of atomicity: it all has to complete.
It would be a mistake to take the atomic out of expansions of =
Right, but it's also a mistake to leave them in. Too much locking is as bad as too little; e.g. (obj a (readfile "foo")) will hang my web server if reading foo happens to take a long time.
Here's my current chain of reasoning around locking...
Arc's approach to programming is exploratory, building larger programs out of small, composable parts.
Locking is a problem with exploratory programming because buggy locking code usually works most of the time, unlike bugs in functional code which are usually visible. With exploratory programming, you try things and see if they work. Sure, with functional code there are the corner cases that you miss and the occasional incorrect algorithm that happens to return the right value for the input you give it, but most of the time with exploratory programming you try things and you get to see that they don't work. But with locking, you throw together some locking code and try it out, and hey, your program runs and doesn't hang and gives the right answer. The bug, if there is one, only bites once in a blue moon when the different threads happen the hit the code in exactly the wrong way.
There's an interesting social aspect to this as well. I've noticed that if I tell someone about a bug in their code, it's less likely to get fixed if it's a threading bug. If their code returns the wrong value for an input, they say "oh my gosh!" and fix it right away :-). But if it's a threading bug, well, yes, it looks like a bug, but the program appears to run ok anyway, so there's little urgency, and how do we know if we've really fixed it or haven't added another threading problem?
The social aspect goes both ways. One of the things I find so delightful about Arc is that because of your work to write concisely, I can look at a function and say "oh, there's a bug". Or, at least that the function isn't doing what I want it to do. Which I can't do with most code, not as easily, because it is surrounded with so much cruft. But I don't have the same feeling of clarity when I look at Arc's locking code. I can read through the code and perhaps pick up on a locking bug or two (e.g. atomic-invoke), but overall, is everything locked that needs to be? Anything locked that shouldn't be? I can't tell. This part of Arc feels like regular software to me... complex enough so that I imagine there are probably bugs, and I don't expect to be able to get them all.
Composibility with locking is a problem too. You have a couple of perfectly good expressions (obj a 1) and (readfile "foo") and you put them together and they break.
My next thought in the chain is, so why use threading anyway? MzScheme only runs on one CPU, so what threading gives us is a) not having to call yield in a long CPU intensive calculation and b) having our program execution randomized for us so that our program doesn't return the same output for the same inputs... unless we very carefully add the right locking in the right places.
So my current inclination is to rewrite the web server to a single threaded event driven model.
Or maybe something like Erlang, where you're not stuck with a single thread, but you're also not trying to deal with sharing modifiable data between threads either.
Hmm, a different way of looking at the issue just occurred to me: pushing atomic inside of expansions of = may make Arc plus News shorter but Arc plus my program longer.
If ++ didn't do locking, then in places where you were using x from multiple threads you'd need to say (atomic (++ x)). Pushing atomic inside ++ makes this shorter because now you can just say (++ x). But pushing atomic into expansions of = also means that locking occurs at other times, and so I can't use it in places where I'm doing things like throw and readfile that break with locking, which makes my program longer.
Which leads to a fascinating idea, if we get a large enough body of open source Arc code that we can start optimizing for code size globally... :)
That would increase the conceptual load of programming in Arc a lot. It would make people have to think about the expansions of operators like ++ to know when to wrap things in atomic and when not to. You need to be able to treat built-in operators like that as black boxes. Once you start thinking about macroexpansions, it's as if you had to write them.
Hmm, well, I can only speak with any knowledge about my own conceptual load... I expect with your background (professor, Lisp book author, tutorial writer, mentor, etc.) you have a much better idea of what other programmers would find easy or difficult.
I know that some things should be atomic, such as accessing shared mutable data structures, and some things that I need to avoid being atomic, such as doing I/O.
I find Arc's making some operations atomic for me doesn't help me all that much, because without knowing the details of the macroexpansion, I don't know if everything that needs to be atomic has been made so. And I find it unhelpful in other cases, when I need an operation to not be atomic, and so I need to look at the macroexpansion of = to find out if that particular expansion is doing something that I need it to not do, or if it's ok.
On the other hand, I have no alternative to offer yet ^_^. I surmise that if I factored Arc + News + my code, perhaps I might come up with a useful suggestion to offer, and if I do, I'll certainly post about it!
This makes it hard to use lists as keys for tables:
arc> (let x (table)
(= (x '(a b)) t)
(x (list 'a 'b)))
nil
As list returns a Scheme list and the key is not ac-niltree'd before being passed to hash-table-get or hash-table-put!, the two lists are not Scheme hash table equal? and so represent different keys in the table.
Ok, this is now fixed. The Arc def of list now makes an explicit copy. Rtm has ambitions of one day making rest args ac-niltreed copies in ac.scm, whereupon we can return to the nice simple def of list.
1. I would like pipe-from on windows to work. "dev/urandom" is a unix thing only. Also maybe a true pipe without file writing would be better?. Not being able to open a connection and hold on to it for reuse is a big road block for me (ie running sql).
2. I would like the path names for file and directory functions to work on windows. ie.e mkdir on windows does not support the "-p" command nor does it support forward slashes. I think the remove directory function also has unix only stuff.
I've noticed it mentioned on this forum (and from reading your code comments) that you use a Mac as do I at home, but I am hoping if there's support for windows right from the beginning that life will be easier for everyone later on.
Running on the current version of mz would be very useful. It's really not fun to have to use a local version when you're used to running on software from a distribution.
First of all, updating to the recent plt version will be good, since there have been a lot of optimizations since then, including a lot of work to the compiler and the jitter.
Arc needs to do some list marshaling in any case to deal with nil -- and as such it reimplements nearly all of the list operations, deferring to mzscheme functions only for a few things. Because of this, using `mcons' shouldn't be too problematic (not even for speed, since arc functions in general are not careful about cyclic lists, a plain implementation will be fast enough -- for example, mzscheme's `map' is now implemented in scheme). I have done some tests in the past to measure
Probably the only major problem is how to deal with a rest argument. One option is to make it illegal to mutate rest arguments. Another is to hook on the fact that rest arguments are not modified often -- so instead of copying them to a mutable list on every call, they can be converted only when needed, by the functions that change the contents of these lists. Given the above (need to implement list functionality anyway), this wouldn't be difficult -- basically some wrapper around a mzscheme list that can do the necessary work. As a side benefit it can be used to deal with nil too, which will make that run faster as well.
(BTW, this will be much faster than using r6rs or r5rs modes.)
One option is to make it illegal to mutate rest arguments.
Currently, in arc.arc, list is defined as
(def list args args)
so this would need to change if rest arguments couldn't be mutated. (Which would be OK with me, I can't think of a single case where I ever mutated a rest argument).
Another is to hook on the fact that rest arguments are not modified often -- ... they can be converted only when needed
How could this possibly work??
((fn args
(= (cadr args) 'X)
args)
'a 'b 'c)
sure, it would be easy for set-cdr! to see that the second pair is immutable and decide to create a new mutable cons. But args is still going to be pointing to the original list of immutable pairs!
As a side benefit it can be used to deal with nil too, which will make that run faster as well.
Can you explain? (I need smaller steps to be able to follow you :-)
(BTW, this will be much faster than using r6rs or r5rs modes.)
Yes, I was getting that impression browsing through the r6rs and r5rs code.
The way I'm leaning right now is to first rewrite the Arc compiler to generate PLT 4 code in the simplest possible way, for example to always convert rest arguments to mutable lists. Then, things like making rest arguments immutable could be done as an optimization if desired.
For the "on-demand" conversion some PLT magic will be needed -- I'm basically talking about doing something at the level of the PLT function that is the result of compiling an arc function. In any case, explaining more through this medium will be hard for me...
Nope. All you can do is use mcons ('mutable cons'), mcar, mcdr, etc. But apparently there are only a few functions that work with these things. It's like using lists in a language that doesn't support lists well. Useless.
Hmmmmmm... I had thought that r6rs compatibility mode wasn't very useful, if it just made car a synonym for mcar etc. But it does more, for example unlike in regular plt-4 where in (lambda args ...) args is an immutable list, in r6rs mode:
#!r6rs
(import (rnrs) (rnrs mutable-pairs (6)))
(define x ((lambda a a) 'a 'b 'c))
(set-car! (cdr x) 'd)
(write x)
(newline)
$ plt-4.1.5/bin/mzscheme -t a.scm
(a d c)
This looks like it could solve a lot of problems with a port to plt-4, since otherwise we'd need to be rewriting the Arc compiler to change the expansion of (fn args ...) etc.
Played around with it a bit more, r6rs appears problematic as apparently ++ is not a legal symbol in r6rs (!)
I took a look at PLT's implementation of lambda for r6rs/r5rs (it's in collects/r5rs/main.ss), and they just simply convert to a mutable list if the lambda has a rest parameter:
(define-syntax (r5rs:lambda stx)
;; Convert rest-arg list to mlist, and use r5rs:body:
(syntax-case stx ()
[(_ (id ...) . body)
(syntax/loc stx (#%plain-lambda (id ...) (r5rs:body . body)))]
[(_ (id ... . rest) . body)
(syntax/loc stx
(#%plain-lambda (id ... . rest)
(let ([rest (list->mlist rest)])
(r5rs:body . body))))]))
(the list->mlist is the part I'm looking at)
So having for example (fn args ...) compile to an (arc-lambda args ...) which does the same thing might be simpler than trying to get Arc to compile and run in the whole complicated r6rs environment.
I participated a bit in the discussion while r6rs was being created, but it quickly became apparent to me that the goals that the editors were striving for weren't things that I personally cared about.
So, anyway, due to the "compromise", can we get mutable pairs if we run MzScheme in r6rs compatibility mode?
a.scm:
#!r6rs
(import (rnrs) (rnrs mutable-pairs (6)))
(define x (cons 'a 'b))
(set-car! x 'c)
(write x)
(newline)
$ ./mzscheme a.scm
(c . b)
So yes, though it turns out that in r6rs compatibility mode cons is really just mcons, so we don't actually gain anything.
Indeed, our experience is that making the result of `cons' immutable
does not create many porting problems. Nevertheless, if your code does
use `set-car!' or `set-cdr!', and if converting to a more functional
style is difficult, then consider using `mcons' and the `scheme/mpair'
library.
If you want to make a good abstract language, you shouldn't care too much about the bugs of this particular implementation. Fixing the bugs of the current version is kind of useless.
If you want to make a good practical language, don't you think it's a bit late to fix the bugs? Why don't you, for instance, just make Anarki the official version?
Anarki has a lot of great stuff in it, but it also has alot of crap in it. I spent 2 weeks just going through the code pulling out the pieces I like from Anarki and not the rest. I certainly hope pg keeps the arc code down to the dozen or so core files forcing the community to build libraries on top rather than integrate adhoc extras at such a low level that it's brutal to separate the good from the bad.
Version n of anything usually consists of incremental improvements to version n-1. Including bug-fixes as well as new features. To deliver a new version written from scratch would be to deliver a new set of bugs written from scratch.
I think that memo should remember the result of the function even if it (the result) was nil. Now it uses hash tables which don't store nils. It would be possible then to be sure that a function won't be called twice it if was wrapped by memo (I used memo for that purpose.)
As a note - It may still be a good idea, but I care less after some thought:
Originally I was trying grab the last number of items in a table (something similar to your code in prompt.arc where you are grabbing/printing the last n items passed into the repl).
When using a list, as you had, we can easily grab the last 'n' items in the list since lists maintain order.
(= records* (list "record 1 text" "record 2 text" "record 3 text" "record 4 text" "record 5 text" "record 6 text"))
(def lastn (n nlist)
(firstn n (rev nlist)))
arc>(each record (lastn 3 records*)
(prn record))
record 6 text
record 5 text
record 4 text
However if storing the data in a table it's not as clear grabbing the last n items since tables have no inherent order.
as you can see this isn't as straight forward:
(= records* (obj 1 "record 1 text" 2 "record 2 text" 3 "record 3 text" 4 "record 4 text" 5 "record 5 text" 6 "record 6 text"))
arc> (each record (firstn 3 (sort > (keys records*)))
(prn (records* record)))
record 6 text
record 5 text
record 4 text
so I had hoped 'for' in reverse would be cleaner...
arc> (for i (len records*)(- (len records*) 3)
(prn (records* i)))
record 6 text
record 5 text
record 4 text
record 3 text
only after re-writing I noticed that I was getting too many results thus ended up having to hack:
If my hope was to make the code more readable, it didn't work.
So.... instead I wrote 'grab':
(def lastn (n nlist)
(firstn n (rev nlist)))
(def grab (which n them (o sop nil))
(case (type them)
table (case which first (if (no sop) (firstn n (sort < (keys them)))(firstn n (sort sop (keys them))))
last (if (no sop) (firstn n (sort > (keys them)))(firstn n (sort sop (keys them)))))
cons (case which first (if (no sop) (firstn n them)(firstn n (sort sop them)))
last (if (no sop) (lastn n them) (lastn n (sort sop them))))))
And now I feel it's much more readable + works on both lists and tables:
arc>(each record (grab 'last 3 records*)
(prn (records* record)))
record 6 text
record 5 text
record 4 text
T.
[EDIT] - Hadn't thought about it until now, but it has re-use even if the table doesn't have nums for keys, it would also work on other keys (example grabbing the last n items of alphabetical order). Anyway... don't spend time on re-working 'for' it it doesn't help anyone else.....
I am not sure if this is a bug or by design, but I will throw it out there. Records stored in tables are not returned in the original order they were loaded (unlike lists are). It would be nice to iterate/process records in the table without having to create code to store the original loaded order.
Further to order type issues shouldn't the 'accum' function return the list of items in the order in which it was accumulated ? Currently it returns the reversed order. it would be nice for this to work as one would expect.
It's intentional. You have to use an explicit rev if you care about the order-- since sometimes you might not. But I'm open to changing that, because I'm often annoyed to have to add the explicit rev. (This is why you have to write apps in a language at the same time you're working on it.)
1. Running on latest MzScheme, possibly as a PLT plugin.
2. Be faster, or at least be consistent. For example, "<=" is an order of a magnitude slower than "<"
