# Debugging Lisp Part 1: Recompilation

This post is the start of a series on how to debug Common Lisp code, specifically with Emacs, Slime, and SBCL. If you do not understand Common Lisp, you should still be able to follow along and recognize just how powerful the facilities provided by the Common Lisp debugger are. Nathan Marz asked me to write these posts since he thought many of the tools for debugging Common Lisp were pretty cool.

The first thing you need to do in order to get started debugging Common Lisp is to set your Lisp’s optimization qualities. Optimization qualities are basically a group of settings which allow you to specify what the compiler should optimize for. These qualities include speed, space, compilation speed, safety, and debugging. If you do not run the code below, which tells the compiler to optimize for debugging, almost none of the examples in this post will work.

```CL-USER> (declaim (optimize (debug 3)))
NIL

CL-USER> (your-program)
...
```

With the compiler optimized for debugging, it becomes possible to do pretty much everything at runtime. This post will show you how Tom, an experienced Lisp developer would debug and patch a buggy function at runtime. Let’s say that Tom has the following code which implements the well known Fibonacci function:

```(defun fib (n)
(if (<= 0 n 1)
(/ 1 0)
(+ (fib (- n 1))
(fib (- n 2)))))
```

There’s just one problem, the code isn’t correct! Instead of returning n in the base case, the code winds up dividing by zero. When Tom tries to calculate the tenth Fibonacci with this code, a debugger window pops up because an error was signaled.

Realizing that he has entered the debugger, Tom wonders what has gone wrong. In order to find the bug, Tom decides to insert a breakpoint into the function.1 In Common Lisp, breakpoints are implemented as a function called ‘break’.2 To insert his breakpoint, Tom adds a call to break at the beginning of fib. After adding the breakpoint, Tom then puts his cursor next to one of the frames and hits the ‘r’ key in order to restart it. In this case, Tom decided to restart the frame where n was three.

By restarting the frame, Tom basically traveled back in time to the beginning of the frame he restarted. After restarting the frame, the debugger immediately hits the breakpoint Tom had just added. From there Tom steps through the code by hitting the ‘s’ key. He eventually realizes that the base case is implemented incorrectly and that that is why he received the error.

After finding the source of the problem, similar to how he had previously inserted the breakpoint, Tom patches the code. He replaces the base case with n and removes the breakpoint he had previously inserted.

After recompiling the code, Tom once again restarts one of the frames. Since he was previously stepping through code, the debugger starts stepping through the frame Tom decided to restart. Tom just taps the ‘0’  (zero) key in order to invoke the step-continue restart3 and continue normal execution. Because Tom restarted a frame which occurred before the bug, and now that the bug is gone, the code runs as if there had never a bug in the first place!

Let’s recap what happened. After the code signaled an error, Tom found himself in the debugger. Tom was able to insert a breakpoint and poke around until he found the source of the problem. After finding the problem, Tom patched the code and restarted the process from a point before it had signaled an error. Because Tom had corrected the code, after he restarted the frame, it acted as if nothing had ever gone wrong!

The ability to recompile code at runtime is just one of the many incredible features provided by Common Lisp. Next time, I’m going to talk about the Slime inspector, which makes it possible to look into and modify objects from within debugger.

1. He may be a great programmer, but he still doesn’t read the error messages.
2. Break is itself implemented in terms of the Common Lisp restart system which I will cover in Debugging Lisp Part 4.
3. Again, restarts will be covered in part four.

## 24 thoughts on “Debugging Lisp Part 1: Recompilation”

1. Chris says:

Great stuff. It always pisses me off in C# when edit-n-continue inevitably doesn’t work in the umpteen cases where it just doesn’t. I want to be able to break anywhere in code, make changes, recompile, and keep on running.

2. Herman van Esch says:

I really like your writings on this subject. There are lots of writings on esoteric lisp subjects, but these hands-on subjects are much more valuable. It’s so hard to start using lisp as a newby. And after all these years, i sometimes still feel as a newby. Thanks for your effort and I am looking forward to the post on slime.

3. David Rysdam says:

This worked great for me until I got to the ‘c’ to continue normal execution. I got errors like

Evaluating call
(- N 1)
with unknown arguments
[Condition of type STEP-FORM-CONDITION]

and there doesn’t seem to be any way to stop this without killing slime + sbcl and starting over. I suspect this has to do with the (break).

Also, how do I “eventually realize” what the bug is? I wasn’t following the ‘s’ stepping part.

1. malisper says:

You should also be able to continue by invoking the ‘step-continue’ restart. You should be able to do this by using the ‘0’ (zero) key on your keyboard. I’m going to change the post to use that instead.

For the realization part, he steps through the code until he sees the line (/ 1 0). Tom realizes that that line is obviously incorrect.

1. David Rysdam says:

Yeah, I saw that step-continue and thought it might work, but it didn’t. Now that I’ve tried the whole tutorial over and done it in the right place, it does. Possibly I’d gotten myself so screwed up that wouldn’t work.

4. Excellent. I look forward to the rest of the posts in this series.

When I was first diving into Lisp, I was very much stuck in the “edit > save > compile > run > think > repeat” cycle. I knew that Lisp supported a much more fluid, interactive style of development, but I couldn’t find any resources that really showed that in action.

Now, there are a few screencasts and such from CEPL and Blocky and such that show some of it in action, but there is still very little (that I know of) in written tutorials like this. This is excellent.

5. Mick Beaver says:

Thanks for the great post!

6. slime-beginner says:

This is exactly what I was looking for. Many thanks for the writeup and I will be eagerly following up for the next posts.
I am having a bit of issue as I just started using SLIME. When you said ” .. Tom adds a call to break at the beginning of fib. After adding the breakpoint, Tom then puts his cursor next to one of the frames and hits the ‘r’ key in order to restart it.”
What key sequence in Emacs/SLIME did you use just after writing (break) to get the fib function recompiled? I downloaded the second gif file and tried to look at it frame by frame but I couldn’t figure it out. A little bit of hand-holding will be highly appreciated.

1. malisper says:

The sequence I pressed was “C-c C-c”. The same way you would compile it normally.

7. jason_m says:

Thanks for the great post! I’ve just recently started using emacs and slime for lisp programming. Restarts and the debugger are mostly a mystery to me thus far.

The first thing we did was that (declaim (optimize …)) bit. Once we’re all done, how do we put things back to “normal” / what is the normal setting?

Also, does C-u C-c C-c give the same amount of debug information?

1. malisper says:

For restoring the normal settings, just use (declaim (optimize (debug 1))).

Yes, C-u C-c C-c is the same thing as setting debug to 3.

8. Might you share your SLIME Emacs configuration please? I would like to reproduce your source level debugging, i.e. the markup/highlighting in the source code to match the current backtrace. I know how to use SLDB to reveal associated source via ‘v’ but that seems crude and is chiefly limited to marking up macro expanded snippets of source with the market ***Here***. Your configuration is remarkably polished.

Chris Kohlhepp

1. malisper says:

I’m not exactly sure what you are asking about, but if you want to know how I got the highlighting in the third image, I did it by stepping through the code with ‘s’.

1. Chris Kohlhepp says:

Yes that was my question. Alas I don’t get this. I get the text label “***HERE***” …and it’s not the original source shown in your examples. Rather it’s the macro expanded named lambda fib, so it’s looking at a very different thing.

(SB-INT:NAMED-LAMBDA FIB
(N)
(BLOCK FIB
(BREAK)
(IF (<= 0 N 1)
(/ 1 0)
(+ (FIB (#:***HERE*** (- N 1))) (FIB (- N 2))))))

1. Chris Kohlhepp says:

To clarify, your example seems to mark up the original source which is really nice. I’d like to know how to do that.

1. Chris Kohlhepp says:

I think I worked it out. In the above example, using fib.lisp, if you send the defun to Slime using eval-defun then the function definition is sent to the REPL. Assuming that the REPL has the correct debug optimizations, still no link between the function and the source is maintained. It’s as if you’d typed it in by hand, the reader macro expanded it to a lambda and that is what you get when you step through. No buffer association, hence to formatting either; it defaults to marking up the text with ***Here***. That’s all different if you use SLIME’s compile/load file feature. Then it knows about the buffer association and consequently marks up the source when stepping through.

9. Hey hey! These are really great! Thanks so much for doing them, they are an immense help :).

I seem to have run into a bit of a problem. When I do debugging, stepping always takes me through SWANK’s code for a while before I finally make it back into the fib function.

which is probably you, and describes my problem exactly. I just don’t know how to “recompile” swank with a lower debug level, since I don’t remember ever compiling swank to begin with.

I don’t mind re-installing swank, so perhaps just a hint on how you did it: I’d really appreciate it.

1. malisper says:

After installing swank (part of slime), the first thing it does is compile. If you have the debug quality optimized for any reason, this will cause the problem you describe (it was optimized for me because I optimize it in my .sbclrc file). What you can do is make one edit to swank, such as deleting a whitespace character and just restart Slime. That should cause swank to recompile. Just make sure that you do not have debug optimized when you do that.

10. Malik Hebert says:

Hi, thank you for your articles and tutorials,

but I have a little problem, I followed your steps one after the others, however when I run the (sldb-restart-frame) command, after I add the break command and recompile the function, i get this error:

error in process filter: Bug: sldb-level is equal but condition differs
(arithmetic error DIVISION-BY-ZERO signalled
Operation was /, operands (1 0). [Condition of type DIVISION-BY-ZERO] nil)
(break [Condition of type SIMPLE-CONDITION] nil)

and then nothing happens

note that I am running latest sbcl version (1.2.14) and latest slime from melpa repo

thank you in advance for any help

1. malisper says:

I’m not sure what the problem is. You might want to bring this up on the Slime mailing list.

11. Great post! I’ve been coding CL for quite some years now, but after reading your post now finally learned how to properly use the debugger and stepper – thanks!

12. glider-gun says:

Hi, I found this series so useful and I want to translate it into Japanese and publish as posts at Qiita, a blog-like Japanese web service to share knowledge of programming.
Could you permit me to do this? I also want to use the figures in entries.

1. malisper says:

Feel free to do so.

1. glider-gun says:

Thank you!