Wednesday, June 18, 2008

Why lisp is *Unpopular*

Lisp is a great programming language which has lot of power; however it is far from being most popular programming language. If you see lisp from a distance you will find:

1. Orthogonal syntax

2. Very quick to learn the basics

3. Very power full when it is used in high level project

4. You write less code as it has most powerful macro system

But still lisp is not the language of choice. It is a fallacy of programming world. Why? This question has stumped me number of times.

This post is my attempt to understand why it is so.

Prefix notation

Lisp is a programming language where prefix notations are used extensively. Prefix notations are good for computer but not for human being. From very old age human civilization and its mathematics is built on top of infix notation. We comprehend mathematical expressions in infix. You can argue with educational system but you cannot ask to change the year old mathematical notation to help a programming language to be understandable. That is simply not possible. This seems to me a perfect road block for lisp and it has to survive with that.

It will not be out of the turn to tell you a story about my friend. She was looking at Practical Common Lisp, and told, “How do you write 4*3+6 in common lisp?”

I replied (in writing), “(+ (* 4 3) 6)”.

She looked at it and closed the browser which was rendering “Practical Common Lisp”.

Mathematics behind lisp

Believe it or not all programmers do not like mathematics. They lie in the pantry, in conferences, in blogs. But the truth is a handful among us really like mathematics and care about it. Moreover it is not necessary to have deep understanding of mathematics to develop programs, or to be a programmer for that matter.

But lisp is an exception to this idea. To have a fair idea about lisp you have to understand some mathematics, like higher order functions. To understand functional programming paradigm one have to devote some weeks to lambda calculus. If one does not understand this he can write programs in lisp but it will be distant from the lisp style. The very idea of Turing machine and lambda calculus is fundamental to computer science but at the real world of software development those ideas do not cross our mind. To be honest with everyone I learnt C programming language without any knowledge of both Turing machine or lambda calculus.

There are self educated programmers who are very good in their trade and they care a damn about those theoretical computer science topics.

So lisp (and proper lisp style) is beyond many programmers and the target audience is less.

Programming language politics

Politics is pervasive; programming languages are not an exception to it. Big corporation look for development environments which are well supported, backward compatible, developers are available, etc. This is true in all large organizations or profit making corporations. When you decide about writing software you try to find out a middle ground. Lisp is powerful, 100 java average programmers = 10 or 2 great lisp hackers, you write less code to get more out programming language – all of them so very true. But when I was asked, “Show me one problem that I cannot solve with Java/Ruby/C++/Python?” I knew as all of them are Turing complete, so there is no problem that cannot be solved by Java/Ruby/C++/Python.

Although you have to learn more lexical separator in those programming languages than lisp still lisp looses. This is *PLP* - Programming Language Politics, we have to live with it. People will reject our ideas of development in lisp and the reason will never be purely technical; rather techno-political.

Code – data Equivalence – Hard to understand

I said, “In lisp code is data and data is code”.

Ron said,”What?”

People do not understand code data equivalence, because they are tuned by other programming languages. Those programming languages are their bread and butter so the immortal suggestion of “Unlearn “is “Unreal”. This is the bottom line of any lisper. This is *THE BASIS* of lisp macros. And it is so away from all other brick-mortar programming languages that no one understands its power without coding for many months and many macros. I got the feel of the matter when I read first ten chapters of Paul Graham’s “On Lisp”. This adds to another stiff cliff in any newbie’s learning curve.


Lisp macros are different beast, a beast when under you command do wonderful job for you. As any other programmer, especially *I know all* programmer like me, need some time to understand that lisp macros are really different.

It took me couple of week to realize that, when I realized it I felt like a stupid school kid with all red score card who has been slapped by his father for his exceptional result. Then I understood,”Oh! Yeah lisp macros are different.”

It opened my mind about programming as whole. Like when I learnt lisp – I learnt map, list, mapcar and all other cool lisp techniques. Before my lisp journey I was struggling with C++ STL concepts, after I learnt lisp all the things fell into place.

But the point here is that lisp macros are the last challenge that one has to understand (if not master).

Run-of-the-mill programmers

In many places it has been mentioned that lispers are not *Run-of-the-mill* programmer. They are different (in some cases arrogant), may be programmer elite. But popularity of a programming language depends on the acceptability of the language not on its power. Natural languages evolve out of existing languages to a simpler but verbose form. The same is not true for programming languages. There is no programming linguistic tree which has a common root node for syntax or semantic or paradigm. Lisp’s elite status may be the singular disadvantage to the general programming cloud.

We all play the role of *run-of-the-mill* programmer in any given project while it comes to coding and debugging. So *run-of-the-mill* programmers are necessary for any project to succeed and sustain. If your target is not *run-of-the-mill* guys you are bound to lose one day or the other.

Undermining Microsoft Windows

Not many lisp software works properly in Microsoft windows. The lion share of desktop application runs on windows box, but many lisp apps do not work.

Lisper world over has to take up the challenge to support Microsoft version of all the application that they are going to release. The word support can be even this one liner: “I do not care about windows. If it works fine else just do not disturb me.” At least some who cares about Microsoft compatibility will not use your stuff in his software.

Undermining Microsoft is bad, because Microsoft to computers is like photosynthesis to trees – it is a necessary evil.


Admin said...

Photosynthesis is a necessary evil? How do you figure?

Anonymous said...

There is an infix maths library for Common Lisp. Several, in fact. Using the primary one noted on , your example would be just #i(4*3+6). Most lispers just do not use it, because once get used to, properly indented prefix notation is much easier to both write and read.

Also, Common Lisp is not a functional language, higher order functions are not a particularly mathematical concept, and I don't even know exactly what lambda calculus is, and yet somehow I manage at least basic competence in CL.

JC said...

This is directed at anonymous.... CL is not a functional language? High order functions aren't mathematical?

The fact that you wrapped that up stating you don't know what lambda calculus is clearly states you should hold off on speaking before you have the facts.

Steve Cooper said...

CL's problem is that it just far too hard to get up and running. There's no major language which is harder to install, has less documentation, or worse editor support.


CL doesn't demand you be functional -- you can write straight procedural code quite happily.

Anonymous said...

If you define functional language as a language that has first class functions, then yes, CL is functional. But using more narrow definition of at least emphasizing referential transparency, like ML/Haskell, it most certainly is not. Scheme is more functional with it's emphasis on recursion and being a Lisp-1, but using typical functional style all the way in CL quickly becomes unwieldy, even with read macros for partial application, function composition and so on.

I meant HOF as they apply to Lisp. Mathematically a function is a mapping, but in CL a function can be an operation as well. In any case I meant that understanding higher order function does not require any more mathematical background than C function pointers.

samik chakraborty said...

To Admin:
Please follow this link if you want to know the answer:
Photosynthesis is necessary evil.
Hope this helps.

Gabriel C. said...

I was listening to SE radio interview to Dick Gabriel and it was very interesting and it got me more interested at Lisp. I know is a extremely powerful language but it seems hard to start with it... I think I can live with the prefix notation, but I was quite annoyed with the operation names: car and cdr? setq? It seems difficult to rememeber car->content address register, far from "head" or "first". Oh, I'm sure there's a library (or very easy to modify) for infix and to use head/tails ... but if I don't need to invent a new langauge... why lisp? why not Haskell, OCaml, or the very practical Scala?
That without mentioning the "balkanization" of Lisp: everybody has it's own flavour of Lisp (or is on the process of writting a new one).. where one should start? CommonLisp? Scheme (and which Scheme)? CLOS? Arc? Defining a new langauge in Lisp is so easy that is its blessing and its curse...
I honestly think Lisp is an extremely interesting and powerful language, but is not for everybody and nto for every task...

Flying Frog Consultancy Ltd. said...

While there is some merit in what you say, you have completely missed the reasons why so many people who are intimately familiar with Lisp still do not choose to use it.

Prefix syntax sucks. The stock Lisper comeback is that prefix syntax is essential for macros to work effectively but Mathematica is an overwhelming counter example.

Code is data is an academic curiosity. Not only does it buy you zero expressiveness (you can always write an interpreter) but it conflicts with static type checking which is useful.

Macros give the programmer the ability to fork the language itself. This quickly leads to totally unmaintainable code and, consequently, virtually all production Lisp code takes only baby steps from core Lisp (e.g. no infix, no pattern matching) and that core language is truly awful compared to any modern alternative.

Your comments regarding "run of the mill programmers" fail to explain why Lisp's unpopularity does not extend to modern functional languages, many of which are more successful than Lisp has even been.

Finally, regarding Microsoft, you might like to ask why Microsoft are now productizing an OCaml derivative (F#) rather a Lisp.