Archive for the ‘lisp’ Category

The Power of Paredit

Thursday, October 1st, 2009

A conversation at ECLM made me look at paredit again. I’ve been consistently using it, but I hadn’t actually been following its evolution. It’s even better than I thought! M-S (look it up) saved me 10 minutes within 60 seconds of noticing it. Even if you’ve been using paredit forever, take another look at what’s available. And if you aren’t using it - WTF? Thank you Taylor ‘riastradh’ Campbell!

eclm09 report

Wednesday, September 16th, 2009

A good time was had by all, including, most importantly, me. The European Common Lisp Meeting has a really nice feel to it, as several people commented to me. Well done Edi and Arthur once again!

The venue was quite stunning, with fabulous views over the Elbe and some of the enormous port. The walking tour on the Saturday was largely interesting (and we didn’t get too wet…). The boat-trip and dinner on the Saturday was fun, too (and we didn’t get wet at all…). The A380 is big. A hangar with multiple A380s is even bigger.

Although not technical, the overall positive theme of the talks - you can actually do something with Common Lisp! - was a refreshing change from the recent “What’s wrong with CL” emphasis. (I’m not saying there shouldn’t be a conversation about CL’s future, just that it shouldn’t be the only topic of conversation.

Dan Weinreb kicked things off with a variation of his Google tech talk. My strong impression was that it featured as a great table-of-contents for a fascinating semester course - which we’re still waiting for. Come on Dan - more! We can take it! He mainly talked about the business rules tier of QRES (the presentation layer being Java and AJAX, the database being Oracle). This is made up of ca. 750k lines of home grown lisp code (including unit tests and tools) and 150k lines of open source libraries (he namechecked Hunchentoot, drakma, usocket, and cffi). There are 50 lispers working on the codebase. Coding standards are important. Code reviews are essential. They’re not completely happy with with the support for large-scale modularity and incremental builds in lisp. Interesting quote: “How to apply experience from previous software projects to new software projects is much more difficult that most people imagine.” Dan finished with three plugs: his CPAN for Lisp proposal; Clojure is interesting; ‘Coders at Work’ is a terrific book.

Paul Tarvydas discussed his Visual Frameworks/CL-VF programming system, which is used in JetLetter, a high performance junk direct mail processor. He made a number of though-provoking claims, including ‘encapsulation should be by process’, ‘we learn from EE that asynchronous components + diagrams = reliable designs’, ’specialization is better than generalization’, ‘not everything should be a diagram (a = b + c); but not everything can be text (a diagram)’, and my favourite ‘in lisp, refactoring means throwing away the code and starting again; code is cheap, thinking and architecture are hard/expensive’.

Dimitri Simon discussed his Piano product. It was a charming presentation, and it’s easy to see how Dimitri has ended up with every major aeronautical outfit as a customer. The piano system was evolved rather than architected or designed, and has a low 5 figures lines of source code. After sharing the religious experience that was Coral Common Lisp on the Mac II, Dimitri shared some thoughts on lisp from his self-declared ‘non-progammer’s point of view:

  • re-focus on the basics
  • the lisp user audience is not necessarily programmers
  • don’t push lisp features too hard - lisp will stand up on its own once discovered
  • simplicity and stability trump features
  • hide things that are scary (like debuggers and steppers)
  • are editors overloaded? (all you need are indenting, eval, retrieve source…)
  • give lots of examples of saving images and creating apps
  • all that is needed is very elementary graphics (x/y plots; sine waves; not file formats)
  • start at (+ 1 2) not (defun factorial (n)
  • let them make their mistakes (e.g. Fortran mindset)
  • examples, examples, examples

We saw what serious industrial customers demand from the GUI of an application with a 5-6 figure licence fee: fixed-width font text output and simple line graphics.

Michael Wessel discussed RacerPro. He said he’d talk about the role of Lisp if he had time. He didn’t. The semantic web is apparently Web 3.0, and with certain classes of description logics you will be able to draw faulty assumptions about old ladies and cats. Unlike every other presentation there wasn’t even a hint of an actual real-world use.

David McClain told us what had happened to SigLab over the last 2.5 years. Well, it moved from synchronous to asynchronous and became GigaDSP - all the eye-candy and 46k LOC, including a liberal use of CLOS and non-standard method-combinations. But wait - there’s more! There is ButterFLY, the best ideas of Erlang (a weak and primitive language otherwise) translated into 40k lines of lisp code for easy control of distributed computing: software transactional memory for queues and system table updates, but not for everything, and home grown security that is explicitly not OpenSSL. And more! OKeaNOs, a massive distributed object repository with exabyte+ individual file size limits, self-describing objects and self-reliant retrieval. All used for developing and running a 10Gbps radio LAN at 140GHz in the jungles of Vietnam. (Or something.) Interesting observation: when developing you can expect one bug per 10 lines of C code versus 50-100 lines of lisp code.

Hisao Kuroda talked about a bunch of products produced by his team (of 12) Common Lispers. His main observation was that the sheer volume of data now having to be analyzed leads to a requirement for statistical analysis/machine learning/visualization/clustering. He addressed the question ‘Why not R?’ by noting that with their extremely domain-specific solutions they have to get below the level R implements to be able to tweak etc. And, of course ‘you think you know when you can learn … but are certain when you can program’. This led to the question ‘Why Lisp?’, to which the answers were maintainability (on industrial strength implementations), ease of integration, platform independence, and runtime efficiency. But he had some observations for compiler vendors (which I reproduce as verbatim as possible):

  • No CL compiler has standard code optimization
    • constant folding (+ 1 2 3) and (+ 1 n 2)
    • constant propagation (let ((a 1)(b 2))(+ a b))
    • common sub-expression elimination (+ (- x y) (- x y) (- x y))
    • loop optimization
  • SMP
    • SBCL (non-thread safe libraries)
    • LispWorks (alpha)
    • ACL (not at all)
  • BLAS/LAPACK
    • F2CL version is slow and buggy
    • displaced array can’t be optimized
    • maybe it’s impossible to beat Intel MKL
  • SSE instruction support

He had a colleague produce a paper on the compiler performance of ACL, LW and SBCL which he will translate from the Japanese and send to the vendors.

And speaking of compiler implementation, Martin Simmons discussed the new SMP features of LispWorks 6. There was a fair amount of detail here, presented in an engagingly low-key way. As someone commented, the key message was ‘don’t use locks’.

Then came the lightning talks. Adlai Chandrasekhar presented sheeple, a prototype-based object system with multiple dispatch, multiple inheritance and a mainly working MOP. Luke Gorrie pointed us to OpenFirmware as a compilation target and/or a source of device drivers. Jeremy Jones demonstrated cross-platform (OSX, Windows, Linux) GUI development using Cocoa (cocotron/GNUStep) and CCL. Nick Levine gave the first of what I imagine will be many talks of his upcoming O’Reilly book ‘Lisp Outside the Box’ aka the Emu book (looks like a dodo but has a hell of a kick?). Stefan Richter tantalised us with his company’s automated software project management AIs ‘The Deadline’ and ‘Captain Feature’, and impressed (me, anyway) with an approach to the business of producing software that actually trusts programmers (and he recommended The Daemon). Tobias Rittweiler highlighted the results of some recent (paid!) work on Slime: improved autodoc highlighting of keywords; support for named readtables; improved hyperdoc support; fancy macroexpansion; and sensible handling of unwound dx objects. Dieter Hoffman presented Preforma - from meaning to machine processing and back; I have no idea.

Most common shout-outs went to LispWorks and Edi Weitz, with SBCL coming in third. The dinner was pleasant. The drinks afterwards were gruelling. I had fun.

eclm2009

Monday, August 31st, 2009

So, I’m going. It seems from this blog that all I do in the Lisp world is attend conferences. Oh well.

The line-up is, as usual, intriguing. I have no idea what to expect from Kuroda Hisao of Mathematical Systems, Inc. David McClain knocked it out of the park last time he spoke, so he might suffer from excessive expectations this time. Martin Simmons of LispWorks is a generally good presenter (I seem to recall). Dimitri Simos is to talk of Piano, which looks interesting, has enviable pricing, and a special secretPaul Tarvydas promises to talk of fun and profit, and who could ask for more. Dan Weinreb will presumably give the convinced lispers version of his tech-talk. And I’ll finally(?) hear about RacerPro from Michael Wessel.

All this without mentioning the traditional ECLM boat trip (sans Kenny this year), dinner and lightning talks. What could go wrong?

P.S. For some reason I’m flying out of Hamburg late afternoon on Monday, so if anyone’s up for lunch that day, let me know.

Old #lisp Logs

Thursday, April 30th, 2009

For reasons which now escape me, I decided at or around ilc09 (how many different abbreviations can I use?) to learn javascript. Someone on #lisp very helpfully said “javascript is scheme”, which was my way in to the language. The baby rhino book from O’Reilly is deeply rubbish. By contrast, the butterfly book, Javascript: The Good Parts by Douglas Crockford, is excellent (thanks, Chris!). And short. Stoyan Stefanov’s Object Oriented Javascript isn’t too bad, either.

So, having discovered the delightful, dynamic, prototype-oriented, functional language hiding inside javascript I have completely ignored what little I’ve learned to scratch an itch of mine - to wit, browsing the #lisp logs from anywhere. After a couple of hours of swearing at $.ajax and mod_rewrite I have oll (see the title of this post) to play with.

ilc11 thoughts

Thursday, April 30th, 2009

I dont know anything about sponsoring (semi)academic conferences, and here is the fruit of my knowledge.

Where the survival of an event depends on its sponsors, then it had better overdeliver to today’s sponsors to ensure there will be some tomorrow. It also helps to be clear about why someone might be interested in becoming (continuing to be) a sponsor. Selling to the people physically at the conference is going to be fairly low down on the list. Making the decision-makers feel good about supporting the lisp community is a perfectly valid reason. So is making employees (current and future) feel good about their employer. This can go further - if competition for talent is tight (think Hyderabad - or the Bay Area - or any top school) a company could distinguish itself as being OK with quirky but seriously powerful stuff by associating itself with Lisp. And if there are other market segments which still have good vibes about Lisp (and let’s face it, a lot of folks think smug lisp weenies have something to be smug about) any organisation interested in those customers would gain from a sponsorship.

As long as the right folks know about the sponsorship. Appearing on the back of the conference program isn’t going to be enough. Of course, much of this is up to the sponsor themselves - they should ‘activate’ the sponsorship by integrating it into their own sales and marketing programs (hence all the ‘a proud sponsor of…’ stuff tacked onto ads). The event can help here by having a nifty logo early enough before the event itself to allow some sort of consumer recognition to develop. The event can also help the sponsors to reach out to (the vast majority of) folks who can’t/won’t attend. A sponsor-recognition strip below the projection screen (and/or on the lectern or the debate table) means all those seeing the talks on youtube (or looking to see what someone they know from irc looks like irl on flikr) will be able to see who supported the event. The AV coverage could even be separately sponsored (”ilc11cast brought to you by NTTDoCoMo, your first choice for a fulfilling engineering career”). And encouraging/providing facilities for podcasts (rooms to interview the celebs etc) would also enable the event to reach out and touch more people than are physically present. (Even something as simple as allowing a sponsor to provide branded lanyards for the badges means that sponsor will appear in every photo and video of the event and its surrounds.)

None of this should be taken as a criticism of ilc09 - I can only imagine the sponsors are very happy to have been associated with such a success. But why keep my ignorance to myself?

Random ilc 2009 thoughts

Thursday, April 30th, 2009

In this age of ubiquitous twitterage a wrap-up of ilc09 (even one as late as this one is) seems not merely self indulgent but positively perverse. In other words, read on!

Overall I thought it was a great success: great venue, great organisation and some great speakers. The only lowlight (for me only) was the banquet, but that was a function of the low price for the event (a good thing in every other way) and the luck of the draw when finding a table.

And now the really random observations. p-cos (Pascal Costanza) is a very good teacher and is posessed of some great insights about the world. In particular he has the ultimate answer to all “how do we make lisp popular” discussions - nobody knows what makes a programming language popular, so all we can do is keep our integrity. (See also William Goldman.) v-diddy (Didier Verna) is also a very entertaining speaker, and someone to keep an eye on. Rich Hickey is a rockstar in spite of himself (who will be next?). I have yet to see an AllegroStore presentation motivated by a real use-case (although I’m sure they exist). The lightning talks were a revelation, kicked off with great energy by Jerry Boetje. Strict time controls, even on longer talks, are vital to keeping the pep in a conference. David Moon was fun. Gerry Sussman would be fun reading the phonebook. The panel session was (predictably) largely a waste of space; the big take-aways for me were (a) Kent Pitman is kinda cool, (b) “let a hundred flowers blossom and a hundred schools of thought contend” and (c) the steam has run out of the Scheme project. The Great Macro Debate reinforced my feeling that the state of the art in large-systems software engineering practice is to treat programmers as a menace (see also XCVB). The CARMA presentation was a highlight - delivering a desktop app to non-techie end-users by using abcl; no muss, no fuss, it’s good enough so let’s ship. Xach is as good on stage as he is on irc. I couldn’t bear to witness the whole of the ALU meeting. Shriram Krishnamurti is an energetic speaker who almost made me care. It’s a pity Olin Shivers gave a talk he’d given before with exactly the same jokes.

ilc2009 - MIT Museum visit (and beer)

Thursday, March 19th, 2009

Seems some folks are going to meet inside the entrance to the MIT Museum at 3pm on Saturday for a (presumably lisp-focused) look around. There is talk of a pre-visit lunch at Miracle of Science around 1:30pm, and post-visit beer at Cambridge Brewing Company around 6pm.

ILC 2009 - I’m going?!

Friday, February 27th, 2009

Quite unexpectedly - to myself, at least - I’ve registered for ILC 2009. The title/theme/tagline is “The Next 50 Years”. Judging by the invited talks, the next 50 years seems to be reinventing CL in Scheme. (Although a 50 year estimate for R8RS doesn’t seem unreasonable.)

Not really sure what to do about the tutorials; I’ll probably end up listening to Pascal all day.

The Great Macro Debate “Are Macros a Menace?” seems a rather desperate attempt to gee up a controversy where there really oughtn’t to be one. On the other hand, if Guy Steele argues for the affirmative in terza rima it could be fun…

Predicted dark-horse highlight: Didier Verna’s CLOS efficiency paper. Predicted dead-horse lowlight: ALU Annual Meeting.

What else should I be doing in Cambridge/Boston while I’m there? 

Stackoverflow answer of the day

Monday, September 29th, 2008

A stackoverflow question entitled “Python Vs. Ruby for Metaprogramming” begins “I’m currently primarily a D programmer and am looking to add another language to my toolbox, preferably one that supports the metaprogramming hacks that just can’t be done in a statically compiled language like D. I’ve read up on Lisp a little and I would love to find a language that allows some of the cool stuff that Lisp does, but without the strange syntax, etc. of Lisp.” and goes on for another 20 lines listing various desiderata.

The currently top-ranked answer, in its entirety: “Grow some and learn Lisp.”

ECLM MMVIII

Tuesday, April 22nd, 2008

Amsterdam turned on its best summer weather for the European Common Lisp Meeting 2008. But enough scene setting. To give a quick rundown of the talks:

InspireData: How We Did It In Lisp Jeremy Jones

A very whizzy, very polished shrinkwrapped Windows and OSX application written in Common Lisp that actually sells well (> 80k seats sold). 12MB of lisp code in 284 source files. Key learnings: specification is hard so incremental refinement is necessary, and “Lisp supports this better than anything I know”. Key quote: “The world needs to use more lisp.”

FEMLISP Nicolas Neuss

A system for solving partial differential equations using finite element methods. 30K lines of lisp code. Key quality: the stability of the base language - “I learned Common Lisp and I’m still up-to-date!” FEMLISP currently has one user, and there seems to be some reluctance to increase that number…

Using Common Lisp For Large Internet Systems Stefan Richter

freiheit.com is a 60 person company that builds large internet systems in 98% Java with the rest being C and C++. (See also  h.w.a.) While it is easy to make mistakes, it is well known how to build large internet systems (1 million users and above). The various bits and pieces are usually in Java, but almost everything could be done in CL today, and certainly tomorrow with a bit of effort. The talk seemed to diverge at this point. On the one hand we should be encouraging people to use lisp in web programming (use weblocks and blog about it; “write a book”). On the other hand we should be introducing call-cc into CL, because (despite seeming to contradict many of the best practices for building scalable internet systems) they are really cool for web apps…

PWGL Killian Sprotte

Killian had a great two-hour presentation on the impressive PWGL/Patchworks with OpenGL musical composition and analysis tool (or “Programming with Boxes”). Unfortunately he only had 45 minutes to present it. Certainly the talk with the highest “wow” factor, this is one to download and try out, not to read (or write) about.

ECL = (not only) Embeddable Comman Lisp Juan José Garcia Ripoll

A presentation of the past, present and future of ECL. A pretty convincing demonstration that ECL is not stripped down, but is a fully fledged Common Lisp (to mix my metaphors).

Selvaag House Designer Kristoffer Kvello

Using Knowledge Based Engineering and CL to automatically design buildings. Six full-time lisp programmers. Impressive application. I’ve always thought lisp uniquely suited to automated compliance applications - turns out I was right. Key observation: “Lisp: all good, no problems.”

HPC Platform Marc Battanyi

Subtitled “CL + FPGA = The Ultimate Processing Platform (or the Worst Case of NIH Syndrome)”, this presentation was another that suffered from being too long for its timeslot. All is forgiven, however, because Marc’s humour is bone dry.

? Ken Tilton

Ummm. This was Cells redux redux. I think. As a proponent (exponent?) of Screenshot Driven Development I can’t disagree with Kenny’s decision to concentrate on (as he put it) “stupid pet tricks”, but with no discernable structure, narrative or point, I really didn’t feel I learned anything…

All in all, the talks were great, the crowd interesting, the venue unique, and, while I won’t be at ELS in Bordeaux, I’m definitely intending to be at ILC ‘09 at MIT (driven by Dan Weinreb - who was in Amsterdam - and Guy Steele - who wasn’t). I might even actually have done some lisping by then…