Feeding Baby Penguins.

Rethinking the MPS II compilation chain

Thu, 30 Sep 2010 20:38:13 +0200

During summer I’ve spent a lot of time looking at how Qi compiles to Common Lisp.

The thing is that I’ve written the MPS II macros to expand into something that looks like it’s been hand coded. That’s been sort of a corner stone in the idea behind MPS and MPS II. The reason is because I wanted to allow easy inspection of the generated code to allow finding bugs and other irregularities easier.

This Qi code study however, changed everything. The generated code is butt ugly but fast as hell and more importantly, the “pretty” Common Lisp code is slow. Goto considered harmful? Sure. No problem, as long as it’s fast.

Obscure Programming Languages Exploration Group

Thu, 27 May 2010 21:52:53 +0200

A couple of days ago @niclasnilsson and I started talking about starting an Obscure[1] Programming Languages Exploration Group here in Gothenburg. Earlier today we met for lunch together with @jorgen99 to discuss how to go about it.

There’s nothing much decided yet about form or content but a few things are worth mentioning anyway. Mostly because I’d love to hear your thoughts on how to go about this.

First of all, we agreed on using GitHub for all the code that we write.

Secondly, we will try to set up a number of problems that are large enough to make a decent code base (we’re thinking slightly larger than a typical code kata) but small enough to be possible to solve alone in a few weeks time without super human powers.

Last, but not least, most of the work will be done homework-style, meaning that we won’t spend much time at the meetings actually writing code. Instead we will focus on discussing differences and similarities between solutions and languages.

There are a few reasons, hopes and assumptions behind these choices:

It allows people to join the fun from anywhere. All you need is a GitHub account and to promise to commit often and to take extra special care when writing your commit messages. We have this idea that if we write really, really detailed commit messages describing not just what has changed but also why and how you think about going about your solution, then going through the commit log should be a decent substitute for attending a meeting where that solution is presented.

Also, since GitHub allows comments on commits, the most interesting discussions about the code won’t necessarily happen at a meeting.

And, if the author doesn’t understand your questions and suggestions or, if you feel that code speaks louder than words, you can fork the repository and implement the solution that you think the author should have done in the first place!

Today we decided to meet again (the three of us) on the 23rd of June and at that time have solutions to a Reversi problem (I’ll post details when I get them) in Scala (Jörgen), Ioke (Niclas) and Prolog (Johan).

And that’s about as far as we’ve discussed things. So. What’s wrong with the above? What will so obviously not work? Which of our assumptions and hopes are completely ridiculous? What else would it take to make the above work for you?

Comments, thoughts, criticism, questions and suggestions welcome.

[1] Obscure might not be the most descriptive word for what we’re looking for. We’re not that interested in languages designed to be obscure, rather languages that haven’t gotten the attention they deserve for some reason or other.

Finishing up MPS II

Fri, 21 May 2010 21:32:26 +0200

Today I added the last of the missing functionality in MPS. With the last commit I’ve got 11 functions in the public API. They are (in alphabetical order): agenda, assert-facts, clear, deffacts, defrule, facts, halt, modify-fact, reset, retract-facts and run. Some of these could have been left out and some should probably work differently from how I’ve implemented them now but that’s all details.

All in all, I’m very pleased with the result. The code base is smaller (roughly 500 LOC, almost half the size of MPS I) and a lot clearer and easier to read and understand. I’m particularly pleased with a few of the public functions. Here’s how the agenda function turned out:

If you know your way around production systems you’ll hopefully agree with me that there’s probably not a simpler description of the agenda functionality (using the conflict-resolution-strategy depth).

The next couple of weeks I’ll try to weed out as many bugs as I can by running a bunch of benchmarks (mostly Ms Manners) and I will hopefully come up with a way to add meta data (for example salience which I’ve left out for now) to rules.

Introducing slime-lispunit

Thu, 15 Apr 2010 12:01:00 +0200

For a couple of weeks now I’ve been playing around with Lisp-unit and Slime. The idea is to provide some basic TDD support for Common Lisp programmers. The result is slime-lispunit.el and the code can be found on GitHub. ~~Unfortunately, at the moment, it requires my fork of lisp-unit to work. I’ll try to either have my changes merged with the original or modify slime-lispunit.el to work with the original.~~ I’ll ~~also~~ try to have slime-lispunit.el included in Slime as a contrib package.

To use slime-lispunit you need to

install Lisp-unit. The easiest way is probably to git clone my this repo and then use ASDF to have it loaded whenever you start Slime (./swank.lisp).
copy the slime-lispunit.el file to the contrib folder of your Slime installation.

Once you’ve got it properly set up you can use C-t C-a to run all tests in a package, C-t C-t to run specific tests in a package and C-t C-r to repeat the last test command without asking any questions.

There’s a screenshot of it here, the code it shows is called the FizzBuzz kata and is mainly used to introduce TDD concepts. I’ve added it as a GitHub gist if anyone is interested in a simple example of how to use TDD to grow tests and code simultaneously.

As always, feedback appreciated.

[Update 2010-04-23] I’ve updated slime-lispunit.el to work with the original version of lisp-unit.

Ada Lovelace day

Wed, 24 Mar 2010 20:56:52 +0100

Today is Ada Lovelace Day, a day to recognize women’s contributions in technology and science. People around the world are contributing blog posts, videos, cartoons and podcasts to draw attention to women and the amazing work they do or have done.

I’m fortunate enough to know several talented women who work in IT and have had the pleasure to work with a few of them. But for today, I especially would like to mention Emily Bache (@emilybache).

Emily is the driving force behind several coding dojo’s here in Gothenburg, Sweden. She’s an amazing speaker and facilitator. As an avid defender of agile methods and test driven development, she’s been pushing, pulling and motivating both herself and others to discuss and learn about what makes good code, and why.

I am very thankful that Emily shares so much of both her time and knowledge with the rest of us. I, for one, am a better developer because of it.

Thank you Emily and happy Ada Lovelace day!

TDD and SLIME

Tue, 23 Mar 2010 21:25:00 +0100

I’ve finally gotten used to the rhythm of test-first development. It took me a while to get here, but now, I wouldn’t dream of writing code without tests.

Even though there are plenty of test frameworks available for Common Lisp, no one has bothered connecting one with SLIME. Personally, I find that very odd since unit tests are meant to be run often. The test suite should evolve at the same pace as the rest of the code base.

For my code, I use Lisp-unit and since I don’t want to have to switch back and forth between code buffers and the REPL just to invoke (lisp-unit:run-all-tests <package>). I decided to write some Emacs Lisp glue to bind the key sequence C-t C-a to lisp-unit’s run-all-tests function. It’s still very rough around the edges, but it works. You can find the code on GitHub.

To install, place the file slime-lispunit.el in your slime/contrib/ directory and update your .emacs to include slime-lispunit in the call to slime-setup.

[Update 2010-03-25] I’ve made some minor improvements of the code. Most importantly, pressing C-t C-a now behaves as if you’ve typed (lisp-unit:run-all-tests <package>) in the REPL.

A walk through of the compilation process

Sat, 06 Mar 2010 15:01:00 +0100

MPS II integrates seamlessly with the Common Lisp compilation chain. This means, among other things, that if your Lisp implementation produce compiled code; then your rules will be compiled as well.

This integration is achieved using macros. Macros make it very easy to transform source code from one shape to another and it all happens before compilation time, so once all macros have been transformed the results are passed on to, and processed by, the compiler leaving us with a fairly efficient solution without having to write too much code.

The individual steps in the chain of macro expansions can be seen by using the macroexpand-1 function. Below is the steps for the defrule example from my previous post. Please, ignore the obvious errors for now.

Using macros to extend Lisp in this way is very common and there are numerous explanations and examples of the technique in several books on Lisp. For example, chapter 24 in Practical Common Lisp by Peter Seibel show how you can extend Common Lisp with functionality for parsing binary files. In chapter 15 of the book ANSI Common Lisp, Paul Graham show how you can add backwards chaining capabilities to Common Lisp with a few lines of code.

Most of the MPS II compiler is written as macros. In fact, I’m hoping that when I’m done, all of it will have fit snugly within macros.

A first look at compiling a defrule

Fri, 05 Mar 2010 22:21:00 +0100

Currently, the code in the MPS II git repository, is only capable of reading a defrule form and printing out the Common Lisp code it compiles into. There are several issues still to fix, but the overall structure of how a defrule is turned into Common Lisp code is there.

Things to fix in MPS II

Thu, 25 Feb 2010 21:02:21 +0100

Apart from some re-structuring and making the code more readable and easier to understand I also want to fix a few things that I’ve left out in MPS. There are several things that are broken or missing, depending on how you choose to see it.

Most notably the packaging is all wrong and the Not-CE (conditional element) can only handle one Pattern-CE. There are a couple of other minor things as well but currently I’m trying to work out a good way of handling several Pattern-CEs in a Not-CE.

My plan is to handle the contents of a Not-CE as a separate subrule, with it’s own Rete network. That’s even fairly easy to accomplish by passing the contents to the compile-LHS macro. I just need to figure out a way to yank out the Production node and replace it with a Not node to place in the network of the original rule.

Since MPS is relying on a mix of macro expansions and function calls to generate the Rete network there are a few timing issues that I need to sort out. But it’s far from impossible. The embryo of the Not-CE compilation can be seen in the Git repo.

MPS II

Tue, 23 Feb 2010 21:36:34 +0100

Minimal Production System (MPS) is an attempt to write a fully functional CLIPS-like Production system in less than 1000 lines of Common Lisp code. As of today (2010-02-23) mps.lisp consists of ~700 lines of code and works pretty well. It’s not fast (compared to CLIPS) but it’s not slow either (compared to Lisa).

MPS II is (going to be) a cleaned up version of MPS. There are a number of improvements I’d like to implement:

Clarify and simplify the code.
Reduce the core needed to run a rules-based program written in MPS.
Add a possibility to use CLOS objects as facts.

The work has begun. Follow my progress on GitHub.