This is pretty incredible work wrapped in an unassuming name. It basically presents an entirely new way to interact with any programming language. You could make an emacs mode that worked this way, for example.
I could see this being one of the fundamental ways of overcoming Lisp bias. Lisp still isn't mainstream. Clojure was a nice attempt, but it fell short. You can find companies that use Clojure, but it's not the lingua franca of any domain (except perhaps text editors).
If you were to expose a way to write Lisp without dealing with any parens at all, it might have a chance of sparking the interest of younger programers long enough to seduce them to Lisp's other benefits: when you write in Lisp, you're writing in the abstract syntax tree normally generated by other languages. This allows you to write macros, which transform the tree in arbitrary ways. It's trivial to write a program to analyze your entire codebase in arbitrary ways (what are the most popular function calls? what's my dependency graph look like? which functions are unreferenced?) which is normally a herculean effort in other languages.
And it all comes down to syntax. `(if a (b) (or c d))` is so utterly foreign to most programmers compared to `if (a) { return b(); } else { return c || d; }` that it's nearly impossible to overcome the inertia long enough to persuade them that the tradeoff in readability is worth the power you get.
I think this work could be helpful here. When you throw a newcomer in front of a Lisp editor and say "Write a program," what goes through their minds? "How do I write an if statement?" "How do I call a function?" "How do I set a variable?" "How do I loop while a condition is true?" All of these are abstract concepts, not specific to any language, let alone Lisp. That'd let them avoid dealing with parens altogether. Then you can introduce the idea of a macro, which just combines new ways of using existing concepts.
Regarding color: I've often wished for colored quasiquotation. `(foo `(bar `(,baz))) would be more useful if your editor displayed the background color differently for each level of quasiquation. It would also de-mystify statements like (fn (x) `(define-macro foo (y) `(define ,y ,',x))) which can otherwise be incredibly difficult to reason about. (Thankfully nested quasiquotation is rare, but suffice to say I wish editors used color more effectively rather than stylistically.)
Sadly, emacs is one of the only editors flexible enough to implement the concepts presented in this paper, and the intersection between emacs users and novice programmers is about as large as the intersection between hackers and MBAs. But you can flip it around: emacs is flexible enough. That means you can ship a custom preconfigured emacs designed specifically for Lisp hacking, set up to mimic Atom/Coda/Sublime/whatever.
And it all comes down to syntax. `(if a (b) (or c d))` is so utterly foreign to most programmers compared to `if (a) { return b(); } else { return c || d; }`
That's a very strong claim without much evidence. Most programmers routinely deal with multiple syntaxes many of which are weirder than and less readable than lisp. A more likely thing is that there just isn't a sufficiently useful lisp.
I'm fairly sure it's safe to say the C/Algol derivative syntax is more familiar to programmers as the most popular and used programming language in the world uses it.
The claim was not about popularity or even familiarity, it was that Lisp syntax is so utterly alien, it's the primary thing that stands in the way of lisp adoption.
I'm not worried but I appreciate the concern, I guess.
I look at the loader code I use in haskell w/ optparse-applicative and how much better it is than anything I've used in my damn life and I say, "Why is this considered unusable?"
Oh man, I 100% agree about color! I've worked on Snap! (which looks a lot like Scratch) and it has a feature called "zebra coloring" that's very similar. I've seen it be very effective with students.
Hacking up a friendly version of emacs does sound like an interesting project. I suspect it's an uphill battle to convince people to use it, but someone should try. :)
I keep hearing this kind of talk about lisp. I've never actually written anything in it, but if you asked me why, it would have nothing to do with parens or syntax. It has everything to do with the fact that, for most purposes, Python suits me just fine, and for almost all other purposes, I want a language that compiles to machine code.
Lisps just don't bring much to the table. Their dynamic-ness brings overhead that is not justified unless you're doing crazy macro stuff, and ad-hoc crazy macro stuff is not easy to put into code without hurting readability and comprehension.
Plenty of implementations of dialects of Lisp have compilers that produce machine code. I would even say that for Common Lisp implementations it is more common to have a compiler than to not have one. There are several great Scheme compilers too, and Racket has a JIT.
Here's a sample session with SBCL, where I define a function, which is immediately compiled, and then I request a disassembly:
As far as overhead goes, the SHL makes room for tag bits (and notice that's only done once, not per math op), and CLC indicates that the function doesn't return multiple values. That is not significant overhead, and will absolutely destroy reference Python in speed.
Maybe it's worth pointing out that to get that code like that from SBCL it's not enough to declare that m and n are fixnums, you also need to promise the compiler that the multiplication won't overflow:
(defun foo (n m)
(declare (type fixnum m n) (optimize (safety 0)))
(the fixnum (* (1+ m) n)))
The reason I didn't list it is because there's so many different ways to do it.
The optimization declaration can be made globally, or per-file; declaring the type of the foo function (including parameters & return values) externally will fully cover this function without having anything additional in the body; there's helper macros to explicitly manage static types, etc.
So the type declarations don't need to be intrusive into the expressions themselves, and can be as transparent as you want. Of course, type inference also means that you don't have to specify every single type, while still getting the speed & checking benefit of fully typed code.
> Their dynamic-ness brings overhead that is not justified unless you're doing crazy macro stuff, and ad-hoc crazy macro stuff is not easy to put into code without hurting readability and comprehension.
I think that is today's major barrier to Lisp, but not in the way you think.
Most of Lisp's inventive features have crept into other languages by now, except for the in-language compile-time code transformation/generation (ie macros & homoiconicity).
The major phrase I want to point out is "crazy macro stuff". To people from other programming languages, code generation and metaprogramming are way out there concepts, only suitable for insane scenarios. People said the same thing about functional programming, yet any well-seasoned programmer today has a reasonable handle on its usefulness and advantages, and FP style is now used by programmers in many languages to contain complexity.
Lisp-style macro programming is a simple way to do metaprogramming, while in other languages it's an impedance-mismatch-riddled nightmare of separate wonky build steps. Once it is made simple, it's easy to add great conveniences, compile-time optimizations, and architectural support to your programming, with almost no-brainer effort.
It is also much easier to human-parse Lisp code which utilizes a good per-project macro library, than it is to parse other languages with boilerplate and sprawling behavioral dependencies injected all over their usage scenarios.
The catch-22 is that this style of super-easy metaprogramming is linked to code-is-data-is-code homoiconicity, and people still reject that at face value, which slows the propagation of metaprogramming (however, I think there is a continually slow-rising acceptance). Functional programming doesn't require any unique syntax, just semantics, so it was able to spread without that barrier.
Lisp has been compiled since 1960. Every major Common Lisp implementation is compiled, and some are only compiled; as in, every darn thing you type into the REPL, including (+ 2 2) is converted to native machine code and then branched to: there is no interpreter and hence the eval function compiles, too. That kind of thing was done long before "JIT" was a buzzword.
Macros make a tremendous contribution to readability and comprehension. I'm informed by 17 years of Lisp experience versus your zero.
The syntax actually enticed me to play with lisp, and I do like it a lot, but I never found strong reason to prefer using, say, Common Lisp over a more mainstream language/framework stack. I'm just starting to dig into Clojure and I think the combination of lisp syntax and strong support for, and encouragement towards, immutable data really brings a lot to the table in both reason-ability and efficiency.
I could see this being one of the fundamental ways of overcoming Lisp bias. Lisp still isn't mainstream. Clojure was a nice attempt, but it fell short. You can find companies that use Clojure, but it's not the lingua franca of any domain (except perhaps text editors).
If you were to expose a way to write Lisp without dealing with any parens at all, it might have a chance of sparking the interest of younger programers long enough to seduce them to Lisp's other benefits: when you write in Lisp, you're writing in the abstract syntax tree normally generated by other languages. This allows you to write macros, which transform the tree in arbitrary ways. It's trivial to write a program to analyze your entire codebase in arbitrary ways (what are the most popular function calls? what's my dependency graph look like? which functions are unreferenced?) which is normally a herculean effort in other languages.
And it all comes down to syntax. `(if a (b) (or c d))` is so utterly foreign to most programmers compared to `if (a) { return b(); } else { return c || d; }` that it's nearly impossible to overcome the inertia long enough to persuade them that the tradeoff in readability is worth the power you get.
I think this work could be helpful here. When you throw a newcomer in front of a Lisp editor and say "Write a program," what goes through their minds? "How do I write an if statement?" "How do I call a function?" "How do I set a variable?" "How do I loop while a condition is true?" All of these are abstract concepts, not specific to any language, let alone Lisp. That'd let them avoid dealing with parens altogether. Then you can introduce the idea of a macro, which just combines new ways of using existing concepts.
Regarding color: I've often wished for colored quasiquotation. `(foo `(bar `(,baz))) would be more useful if your editor displayed the background color differently for each level of quasiquation. It would also de-mystify statements like (fn (x) `(define-macro foo (y) `(define ,y ,',x))) which can otherwise be incredibly difficult to reason about. (Thankfully nested quasiquotation is rare, but suffice to say I wish editors used color more effectively rather than stylistically.)
Sadly, emacs is one of the only editors flexible enough to implement the concepts presented in this paper, and the intersection between emacs users and novice programmers is about as large as the intersection between hackers and MBAs. But you can flip it around: emacs is flexible enough. That means you can ship a custom preconfigured emacs designed specifically for Lisp hacking, set up to mimic Atom/Coda/Sublime/whatever.