I hope to start building a Lisp soon atop Mu. Maybe Arc? Maybe Wart? Maybe Bel? If the past is any guide it'll be a hybrid of several things. I'll probably also steal a few things from http://www.oilshell.org to try to make it ergonomic from the commandline.
SubX and Mu seem like pretty decent layers over machine code for systems programming, and I appreciate the transparency and control they offer.
How would you rate Mu as a systems programming language compared to say, Rust, for the purposes of implementing a language VM?
And what extensions do you expect to need before it is a good choice for such things?
Also, I'm somewhat curious about what you think about regarding applying SubX/Mu to cross-platform development. Clearly, that would be a challenge because the differences between platforms would be pretty hard to ignore in a transparent system. At the same time, it might be possible to build a code-generation framework on top of Mu that makes language and compiler development easier, because it has better access to the internals and have more control over what is produced on each platform.
Rust and Mu are similar in aiming for memory safety without garbage collection. They differ in how they go about it.
Rust provides better abstractions with higher performance and a more polished experience overall. It's also designed for concurrency. However, linear types are an adjustment, and the compiler is complex and (like other mainstream software) not really intended for users to understand the internals of.
Mu starts by encouraging people to understand it. I have no idea how far I can take it, so I err on the side of being simple to the point of simple-minded in service of an implementation that can fit in a single head. It doesn't require changing habits on memory management. It encourages less performant code (e.g. clearing a register often takes 5 bytes rather than 1 using an xor). It postpones thinking about memory in favor of memory leaks. It totally punts on concurrency for now.
Portability is another area of difference. Rust aims to be portable. Mu is tightly coupled with a single processor architecture (x86). I absolutely want to support other processors -- but they'll be in their own forks, so that each repo stays simple and people who don't run one processor don't have to think about it.
Concurrency is an interesting topic. I hadn't thought of that when reading about SubX, which mostly covered the part of assembly/machine language I'm familiar with. I always wished, though, that my assembly class (or an advanced sequel) had covered some of the more advanced topics for modern processors instead of the 8080, like how to set up memory management (the boot process on newer CPUs is rather involved...) and how multicore computation works. I know all about the higher-level implications, and can use threads etc., but have never actually learned how those things are handled at the bottom. Or syscalls etc.; I've only ever used them, not implemented them. That's a bit out of scope for the discussion, but something I look forward to "seeing-through" your future work on accessible internals. :)
I guess part of my curiosity was targetted at how you would build Mu up to handle performance and portability. Although the way they are usually implemented does obscure the internals, it seems possible that some compromise might be found. Any thoughts?
Perhaps the answer is at the higher lisp (AST / intermediate representation) level. Mu and SubX stay perfectly aligned to the final binary, but a code generator could optimize the AST and generate code in Mu for the target platform.
On that note, have you thought about giving them a more accessible API for such purposes? Or even just an s-expression syntax? Otherwise the target format for generation is limited to text, which would lose a lot of semantics in translation. Also, how are you thinking of maintaining introspection continuity across levels as you add more layers? Maybe something simple like source map support?
At the moment my ideas on all these areas are extremely hand-wavy. Ranging from "you don't really need it," to "somebody else's problem" :)
I feel fairly strongly that I don't want to add very many more layers. Ideally just one more. As more high-level languages get added, I want the lower levels to be integrated into Mu, so that they can in turn become the new HLL. But that's just an aspiration at this point.
One thing somebody else suggested was Plan9's assembly syntax. It may provide some inspiration for making SubX portable.
You might want to chat with @kennethrapp on the GitHub repo. I think that's the only one actively working on the news app. Feel free to open issues and PRs as much as you like. I sadly no longer develop on Arc, but I'm happy to answer any questions if you set about trying to build any of these features.
Regarding backups, all data is stored in a single directory on the disk, so just copying a tarball or zip file of it to some other server or S3 should suffice. The only protocol I'd recommend is being wary of switching back and forth between branches. The master (Anarki) branch has diverged in the file format in some subtle ways, and using the stable or official branch or any other forks may cause data loss. At least make a backup first before you try it.
The input is on the problem page. Scroll to the bottom of https://adventofcode.com/2019/day/3 to 'get your puzzle input'. You should be able to get the output by running the commands I gave. It's just a numeric answer so not super interesting.
But the experience frustrated me. It was hard for me to understand all the software under me as I provided abstractions above me.
So I spent the last 5 years gradually eliminating all the layers of abstraction that add complexity to my Lisp interpreter. The path passed through one other language for teaching programming: http://akkartik.name/post/mu. The sources for it are archived at https://github.com/akkartik/mu1 (there was an earlier prototype in Arc at https://github.com/akkartik/mu0)
At this point I have a very simple syntax for programming pretty much directly in machine code: https://github.com/akkartik/mu#readme. It can be translated to an ELF binary for Linux using about 250KB of static x86 instructions (80% of which are unit tests, and much else of which is duplicated because I built the translator in multiple passes that run in a shell pipeline:
The nice thing about the resulting ELF binaries is that they can be run directly on a Linux kernel with no other dependencies. Not even libc.
There's a script in the repo called `gen_iso` that can take a list of .subx files, translate them into an ELF binary and package up the ELF binary with just a Linux kernel into a bootable disk image. You can then boot this image either in Qemu or on a cloud service like Linode (http://akkartik.name/post/iso-on-linode)
This is what I have so far.
By contrast, the screenshot is quite fake. It's just a program that reads a line of text from the keyboard and prints it out to the screen. You can see it running on an emulated computer in Qemu that has nothing but a Linux kernel.
But I'm going to build up from that core into a high-level language. Maybe an Arc-inspired Lisp. Not a toy this time around.
Just give me 5 more years :D
To reiterate the main project link: https://github.com/akkartik/mu#readme. Should hopefully be pretty easy to get running on Mac or Linux. (Though you're mostly on Windows, right jsgrahamus? I'm really sorry I still don't know Windows well enough to support it :( )
Great! Yeah, I'd love to hear how you fare following the examples in the Readme.
The examples involving gen_iso take a while to run, which may be even greater atop VirtualBox. I'd recommend skipping those for now, particularly the very first one at the top of the Readme.
As of last night, Mu can package up a codebase (Assembly files in my special syntax) with a Linux kernel into a bootable disk image and deploy it to Linode. I've updated the top of https://github.com/akkartik/mu#readme with details.
Have you seen my proposal for infix syntax for Arc? I think it's pretty nice: http://arclanguage.org/item?id=16775. The big drawback: you have to give up '-' in symbol names.
>The big drawback: you have to give up '-' in symbol names.
I wouldn't have a problem with that, but I'm probably of a minority opinion, since that seems to be a Lisp thing. When I started with Arc it took me a while to realize that symbols with asterisks in the name weren't something special like pointers, and using angle brackets just seems wrong after years of writing HTML.
Although if it were possible to do something along these lines, one could have the best of both worlds:
The final straw for me was when I tried to actually _use_ sweet expressions for something. I described it here: http://arclanguage.org/item?id=16699 (Warning: lots of bad ideas in this thread)
