"Because I can't tell if `Pauan` is human or not, I'd like to define exactly how `Pauan` arrived at this answer in the community-managed fork of Arc. Is this possible? (yes no)."

If I understand you correctly, you're asking if we could please open-source Pauan (or a subsystem thereof) for the sake of future development and discussion. No. Please don't undermine the voices of people on this forum.

- You're asking an in-depth question about someone in the third person, when they're right here to explain.

- It seems to me you're discriminating on the basis of someone's involuntary implementation details, without any obvious and respectable reason.

- You're suggesting to "define exactly" something which may be an essential secret ingredient in someone's appearance of individuality. As far as I'm concerned, you might as well suggest we zap someone's brain or hound them with paparazzi.

Also, it's time that I leave the Arc Forum. You all are free to follow my projects on GitHub (https://github.com/Pauan) and can e-mail me at pcxunlimited@gmail.com

I had seen that blog post of yours. I also particularly noticed this tweet you retweeted, which aptly summarizes a common theme between your blog and your Arc Forum posts:

https://twitter.com/garrytan/status/341252234878795777

  Founders: What are you doing right now? Is that thing a 10X
  improvement or a 10% improvement? Always work on 10X. Every
  minute counts.

I think what math people obsess about is always reality. The fact that they are obsessing about it at all is a real phenomenon, and we can think about tracing it back to its most immediate cause. That immediate cause is likely to be a very math-like problem in need of a solution. The not-so-math-like problem which inspired that math-like problem may be an awfully long way from where we started searching, but it's still connected.

Some mathematicians (the pure mathematicians) may pursue math-like curiosities without much regard for where they're going. Still, their own intuition and sense of elegance will constrain their discoveries to forms that can come in handy later on. In this case, mathematics acts as a bridge that starts from today's human intuition and elegance and connects to tomorrow's newfound human problems.

Clearly you're looking for an efficient connection, not just any connection. I suspect the only way to find that is to keep yourself in a state of perpetually looking, even as you continue with what you're doing. Be your own JIT compiler--and yet, don't let a JIT compiler be all that you are.

The Elm project has a lot in common with Flapjax, another FRP language. However, Flapjax automatically lifts expressions like "a + b" to return signals if the arguments are signals, whereas Elm just acts as a Haskell-like FRP library plus a Haskell-like language to use it in. In Elm, there's an explicit family of lift, lift2, lift3, ... operators for mapping an n-arg function over n Signals.

                                   Elm                   Flapjax
                          ---------------------   ---------------------
  Maintained in:           Haskell and some JS     Haskell and some JS
  Compiler runs in:              Haskell                   JS
  Application runs in:             JS                      JS
  Syntax approximates:           Haskell                   JS
  Expressions represent:         Values             Signals or Values

"Elm's interaction with JavaScript code seems like it could use some improvement, which matters because I care about using new Web standards as they come out. Fortunately, it's not so bad that it makes the language unusable for this purpose, and I have an idea for how to prototype a new FFI from within the existing one. I'll elaborate on that idea in a separate post."

Since Elm's evaluation semantics are typical of functional languages, my design in this post could be relevant for just about any language that has an FFI to JavaScript. However, it's focused around what Elm already strives to make available in its FFI.

I've found four ways an Elm program can currently interact with JS code:

- There's a top-level way to associate well-behaved DOM events with Elm signals, described here: http://elm-lang.org/learn/Syntax.elm#javascript-ffi

- Some built-in JavaScript, JavaScript.Experimental, and Json modules are available, as described here: http://elm-lang.org/Documentation.elm

- The built-in functions == and show can operate on JavaScript values. There may be more like these.

- If a variable isn't bound in any imported module, the compiler will translate it directly into a JavaScript variable reference. This makes it possible to call JavaScript's eval() and similar tools. Unfortunately, this process is unhygienic, making it possible to access and assign to(?) the compiler-generated local variables. This feature isn't documented, but I've found no other use values of type JSString, JSFloat, etc., so I conclude it's the trap door to JS that I'm looking for.

When integrating a language like Elm with JS, some semantic awkwardness is to be expected. I don't expect Elm's reactivity to have perfectly elegant integration with all the step-by-step structure of the HTML+JS operational semantics. I don't expect an Elm `Element` to behave well in all the dark corner cases of DOM+CSS layout.

If it were up to me to improve Elm, I would stop relying on this unbound variable feature, since it's probably one of the oddest features as far as the type system is concerned. In place of this, I'd introduce an explicit eval mechanism, and I'd slightly redesign the other basic utilities to be more inclusive of heterogeneous Arrays, and generally to wrap JavaScript's idiosyncrasies more snugly.

As it turns out, I think this can be implemented as a library in Elm.

  module Js where
  
  -- We label something here as "unsafe" iff it makes it possible to
  -- break Elm's usual semantics even when using *safe* JavaScript
  -- techniques. This is meant to indicate that the JavaScript-related
  -- code needs to be extra careful to avoid breaking the Elm program.
  --
  -- For this definition to be meaningful, there has to be some notion
  -- of "safe" JavaScript code. We consider JavaScript code to be
  -- "unsafe" iff it defies JavaScript conventions in such a way that
  -- even an everyday JavaScript program would break. For instance, it's
  -- unsafe to modify basic platform features like Object.prototype and
  -- window.setTimeout. Ultimately, this is subjective; I'm relying on
  -- my own assessment of what an "everyday" JavaScript program is like.
  
  
  -- If a JavaScript Array or string has around 2^53 elements (and if we
  -- can somehow hold it in memory), the indexing functions may not work
  -- very well. Perhaps this issue will be more realistic if Elm ever
  -- supports infinite lists. Either way, I'm using the FixList and
  -- FixString types to make it clear that a list's length must be a
  -- representable integer.
  takeFix : Int -> [a] -> FixList a
  prependFix : FixList a -> [a] -> [a]
  type FixString = FixList Char
  
  
  -- These utilities are actually so safe that their behavior should be
  -- possible even if unsafe JavaScript code has utterly sabotaged
  -- everything else. If only the internal workings of the compiled Elm
  -- code were just as airtight! (Are they?)
  toFloat : JsValue -> Maybe Float
  fromFloat : Float -> JsValue
  toFiniteString : JsValue -> Maybe FixString
  fromFiniteString : FixString -> JsValue
  eq : JsValue -> JsValue -> Bool
  true : JsValue
  false : JsValue
  null : JsValue
  undefined : JsValue
  typeof : JsValue -> FixString
  
  
  -- I'm not sure there's any safe way to detect whether something is an
  -- Array and enumerate its contents. Some objects intentionally
  -- imitate Arrays, so this is a semantic battlefield.
  --
  -- Let's say fromArray checks for a "length" field and then enumerates
  -- numeric properties up to that length. This way it can handle
  -- arguments objects, strings, NodeLists, TouchLists, Uint32Arrays,
  -- jQuery objects, etc. Let's also say that any object that causes
  -- side effects during this operation is so unsafe for JavaScript that
  -- it's not extraordinarily unsafe for Elm.
  --
  fromArray : JsValue -> Maybe (FixList JsValue)
  toArray : FixList JsValue -> JsValue
  
  
  -- Execute arbitrary code as though by using Function( "..." )( x ).
  -- Naturally, the observed value of free variables, primitive-related
  -- prototypes, etc. may vary depending on the frame or Web worker used
  -- for executing the Elm code. If this JavaScript code has side
  -- effects or accesses internal details of Elm-managed objects, it may
  -- interfere with Elm's usual semantics.
  --
  -- Elm's design is oriented around pure manipulation of time-varying
  -- Signals, so significant JavaScript side effects should probably be
  -- reserved for responding to DOM events and Elm Signal updates,
  -- rather than acting at arbitrary moments during Elm's expression
  -- evaluation.
  --
  unsafeEvalFunctionBody : FiniteString -> JsValue -> JsValue
  
  
  -- I'm not sure there's a better alternative to "x instanceof Node" or
  -- "x instanceof Element" for detecting DOM nodes or DOM elements
  -- created in the JS code's frame. I'm not sure which of these Elm
  -- cares about, but let's say JavaScript code is unsafe for Elm if it
  -- causes these checks to fail or perform side effects, or if it gives
  -- a DOM node unusual behavior such as custom property getters.
  domNodeFromElement : Element -> JsValue
  unsafeDomNodeToElement : Int -> Int -> JsValue -> Maybe Element
  
  
  -- These take the place of the entire JavaScript.Experimental module
  -- and the Json module's utilities for JSObjects. I think there's no
  -- safe way to detect whether an object's "own" properties are all
  -- safe to dereference, so the JS-to-Elm conversions are unsafe.
  --
  -- Similarly, I'm not sure object and Array initialization are
  -- necessarily pure if Object.prototype or Array.prototype has been
  -- modified. In my testing, object and Array literals seem to be able
  -- to initialize fields without calling their setters, but it's not
  -- possible to use this technique with programmatic lengths/keys
  -- without resorting to some kind of eval. However, this case is
  -- unsafe for JavaScript code in general, so it isn't extraordinarily
  -- unsafe for Elm.
  --
  fromJson : JsonValue -> Maybe JsValue
  unsafeToJson : JsValue -> Maybe JsonValue
  fromNestedStructure : a -> Maybe JsValue
  unsafeToNestedStructure : JsValue -> Maybe a

Which of these interpretations do you mean?

  (list 1 2 (splice 3 4) 5)
  -->
  (list 1 2 3 4 5)
  
  
  (list 1 2 (splice (reverse (list 4 3))) 5)
  -->
  (list 1 2 3 4 5)

---

"Namely, making the lists that form the code/ast have reverse links, so you can mutate above the macro call level, instead of just insert arbitrary code in place."

I'll make an observation so you can see if it agrees with what you're thinking of: The expression "above the macro call level" will always be a function call or a special form, never a macro call. If it were a macro call, we'd be expanding that call instead of this one.

---

For these purposes, it would be fun to have a cons-cell-like data structure with three accessors: (car x), (cdr x), and (parent x). The parent of x is the most recent cons-with-parent to have been constructed or mutated to have x as its car. If this construction or mutation has never happened, the parent is nil.

Then we can have macros take cons-with-parent values as their argument lists, and your macro would look like this:

  (mac splice list
    (= (cdr list) (cdr (parent list)))
    (= (cdr (parent list)) list))

---

Suppose we have a more comprehensive alternative that lets us manipulate the entire surrounding expression. I'll formulate it without the need to use conses-with-parents or mutation:

  ; We're defining a macro called "splice".
  ; The original code we're replacing is expr.
  ; We affect 1 level of code, and our macro call is at location (i).
  (mac-deep splice expr (i)
    (let (before ((_ . args) . after)) (cut expr i)
      (join before args after)))

  (mac-deep subquote expr (i)
    `(quote ,expr))
  
  
  (list (subquote))
  -->
  (quote (list (subquote)))
  
  
  (do (subquote))
  -->
  ((fn () (subquote)))
  -->
  ((quote (fn () (subquote))))
  
  
  (fn (subquote) (+ subquote subquote))
  -->
  (fn (subquote) (+ subquote subquote))

Doesn't step 1 need to use environment(s)? How else would it turn text into a structure that contains references to previously defined values?

---

"The 'hyperstatic scope' that Pauan keeps mentioning is another way of saying, 'there is no scope', only variables."

The hyper-static global environment (http://c2.com/cgi/wiki?HyperStaticGlobalEnvironment) is essentially a chain of local scopes, each one starting at a variable declaration and continuing for the rest of the commands in the program (or just the file). I think the statement "there is no scope" does a better job of describing languages like Arc, where all global variable references using the same name refer to the same variable.

---

"Flexible scoping becomes possible by specifically referencing the variable in the environment instead of using the default boxed values. They would still be boxes, just referenced by symbol in the environment table."

I don't understand. If we're generating an s-expression and we insert a symbol, that's because we want that symbol to be looked up in the evaluation environment. If we insert a box, that's because we want to look up the box's element during evaluation. Are you suggesting a third thing we could insert here?

Perhaps if the goal is to make this as dynamic as possible, the inserted value should be an object that takes the evaluation environment as a parameter, so that it can do either of the other behaviors as special cases. I did something as generalized as this during Penknife's compilation phase (involving the compilation environment), and this kind of environment-passing is also used by Kernel-like fexprs (vau-calculus) and Christiansen grammars.

---

"Now what I'm wondering is whether there is any useful equivalence to be found between a box and an environment."

I would say yes, but I don't think this is going to be as profound as you're expecting, and it depends on what we mean by this terminology.

I call something an environment when it's commonly used with operations that look vaguely like this:

  String -> VariableName
  (Environment, VariableName) -> Value
  (Environment, Ast) -> Program

  Box -> Value

When each entry of an environment needs to have its own self-contained impure behavior, I typically model the environment as a table which maps variable names to individual boxes. The table's get operation is pure (or at least impure in a simpler way), and the boxes' get operation is is impure.

I think he's talking about run-time environments a la Kernel, Emacs Lisp, etc.

Nulan and Arc/Nu use a compile-time environment to replace symbols with boxes at compile-time. But that feels quite a bit different in practice from run-time environments (it's faster too).

---

"Are you suggesting a third thing we could insert here?"

Once again, I think he's referring to run-time environments. Basically, what he's saying is that you would use boxes at compile-time (like Nulan), but you would also have first-class environments with vau. The benefit of this system is that it's faster than a naive Kernel implementation ('cause of boxes), but you still have the full dynamicism of first-class run-time environments. I suspect there'll be all kinds of strange interactions and corner cases though.

---

Slightly off-topic, but... I would like to point out that if the run-time environments are immutable hash tables of boxes, you effectively create hyper-static scope, even if everything runs at run-time (no compile-time).

It's funny, I've done exactly that recently, with the same goal of getting away from JavaScript with as little cruft as possible. My goal was to make a hackish little language (nevertheless better than JS) and then make a better language on top of it.

Awkwardly, by the time I had a nice reader, I realized I didn't have any particular semantics in mind for the little language! So I decided to work on the semantics I really cared about instead. :-p Although this semantics is still incomplete, the reader is there whenever I'm ready for it. Maybe it could come in handy for Wat.

I've put up a demo so you can see the reader in action: http://rocketnia.github.io/era/demos/reader.html

It's part of Era (https://github.com/rocketnia/era), and specifically it depends on the two files era-misc.js and era-reader.js.

It's yet another read-table-based lisp reader. Every syntax, including lists, symbols, and whitespace, is a reader macro. My implementation of the symbol syntax is a bit nontraditional, because I also intend to use it for string literals.

https://github.com/Pauan/nulan/blob/javascript/src/NULAN.par...

I specifically designed it so it can run stand-alone, without any dependencies on the rest of Nulan. It takes a string and returns a JS array of strings, numbers, and Symbols. You can easily write a simple function to transform the output so that it's compatible with wat.

I've always used use bracket nesting to determine where the syntax's body begins and ends. This way, most code requires no escape sequences, and the few times escape sequences are necessary, at least they stand a chance of being consistent when cutting-and-pasting across staging levels.

  (re /\)*/)       ; Broken.
  (re /(?#()\)*/)  ; Fixed with a regex comment. (Does this work?)
  (re /\>*/)       ; Fixed by redesigning the escape sequence.

(EDIT: Half an hour after I posted this, I realized I had a big mess of previous drafts tacked onto the end. I've deleted those now.)

---

[1] Jisp was one of the first toy languages I made, and it was before I programmed in Arc (or any other lisp). When it encounters (foo a b c), it resolves "foo" to an operator and sends "a b c" to that operator.

  > (if (eq "string (doublequote string)) (exit) 'whoops)
  [The REPL terminates.]

  [groovy
      import com.rocketnia.blade.*
      
      define( [ "out", "sample-var" ], BladeString.of( "sample-val" ) )
  ]

  arc.prn.q[Defining fn-iferr.]
  [fun fn-iferr [body catch]
    [defuse body [hf1:if-maybe err meta-error.it
                   catch.err
                   demeta.it]]]
  
  arc.prn.q[Defining iferr.]
  [mac iferr [var body catch]
    qq.[fn-iferr tf0.\,body [tf [\,var] \,catch]]]

  $rc.rcPage( "/", $cg.parseIn( [str RocketN[i I]A.com] ),
      "19-Nov-2012", $cg.parseIn( [str 2005[en]2010, 2012] ),
      { "title": "RocketNIA.com, Virtual Index of Ross Angle",
          "breadcrumbs": $cg.parseIn(
              [str RocketN[i I]A.com: Virtual Index of Ross Angle] ) },
      $cg.parse( [str
  
  ((This is the open source version of my site.
  [out http://www.rocketnia.com/ The online version] has a bit more
  content.))
  
  ...
  
      ] ) )

Whoops, I can't believe I didn't use the phrase "balanced brackets" instead. ^_^

The following two pieces of text may be similar, but I'd give them significantly different behavior as code:

  (foo a b (bar c d) (baz e) f)
  (foo a b bar c d) (baz e f)

---

"staging levels"

Staged programming is where a program generates some code to run later on, perhaps as a second program in some sense--especially if that boundary is enforced by a need to serialize, transmit, or sandbox the second program rather than executing it here and now. Staged programming has some implications for syntax, since it's valuable to be able to see the code we're generating.

Most languages use " to denote the beginning and end of a string, so they can't also use " to represent the character " inside the string. This can makes it frustrating to nest code within code. I'll use a JavaScript example.

  > eval( "eval( \"1 + 2\" )" )
  3
  > eval( "eval( \"eval( \\\"eval( \\\\\\\"1 + 2\\\\\\\" )\\\" )\" )" )
  3

Suppose we identify the end of the string by looking for a matching bracket, possibly with other pairs of matched brackets in between. I'll use ~< and >~ as example string brackets.

  > eval( ~<eval( ~<1 + 2>~ )>~ )
  3
  > eval( ~<eval( ~<eval( ~<eval( ~<1 + 2>~ )>~ )>~ )>~ )
  3

In JavaScript, technically we can implement delimiters like these if we're open-minded about what a delimiter looks like. We just need a function str() that turns a first-class string into a string literal.

  > str( "abc" )
  "\"abc\""

   Open string:  " + str( "
  Close string:  " ) + "

  > eval( "eval( " + str( "1 + 2" ) + " )" )
  3
  > eval( "eval( " + str( "eval( " + str( "eval( " + str( "1 + 2" ) + " )" ) + " )" ) + " )" )
  3

In Arc, we delimit code using balanced ( ). The code isn't a string this time, but the use of balanced delimiters has the same advantage.

  > (eval '(eval '(eval '(eval '(+ 1 2)))))
  3

Since we're now talking about macros that take strings as input, let's see what happens if Arc syntax is based on strings instead of lists.

  > (let total 0 (each x (list 1 2 3) (++ total x)) total)
  6

  > "let total 0 \"each x \\\"list 1 2 3\\\" \\\"++ total x\\\"\" total"
  6

  > (let total 0 (each x (list 1 2 3) (++ total x)) total)
  6

---

"It also wasn't clear what the escape sequence is in the third example."

Are you talking about this one?

  (re /\>*/)

If you're talking about my Penknife code sample, the "qq." part is quasiquote, and the \, part is unquote. Quasiquotation is relatively complicated here due to the fact that it generates soup, which is like a string with little pieces floating in it. :-p Penknife has no s-expressions, so it was either this foundational kludge or the hard-to-read use of manual AST constructors.

Which of my guesses was correct? :-p

---

"That issue with backslashes you're referring to, I've heard it called leaning toothpick syndrome."

Nice, I hadn't heard of that! http://en.wikipedia.org/wiki/Leaning_toothpick_syndrome

It might be worth pointing out that the LTS appearing in my examples is more pronounced than it needs to be. The usual escape sequence \\ for backslashes creates ridiculous results like \\\\\\\". If we use \- to escape \ we get the more reasonable \--" instead, and then we can see the nonuniform nesting problem without that other distraction:

  > eval( "eval( \"1 + 2\" )" )
  3
  > eval( "eval( \"eval( \-"eval( \--"1 + 2\--" )\-" )\" )" )
  3

I don't think a third-party tool should count as language support, unless (and inasmuch as) the people maintaining the language regularly go out of their way to avoid breaking that tool.[1][2] This is where I draw the line, because of how it affects backwards compatibility.

Everyday language users tend to think of language updates as backwards-compatible as long as their existing code has the same behavior. But tools like eval(), JavaScript minifiers, and Ruby2Ruby place the bar higher, since their input is arbitrary language code, and it might be written in a newer language version than the tool itself. Incidentally, even a program that calls these tools will continue to work after a language update, as long as the input of these calls is all generated programmatically, rather than directly taken from the program's input.

[1] Well, I guess the term "third-party" might not apply in that case!