Mon Nov 1

I've posted an assignment due Thursday covering the material in sections 3.2 (local state) and 3.3 (mutable data) in the text.

We've already discussed 3.2 at some length; today we'll go over some of the ideas in 3.3.

And from there I'll give you a pep-talk about where we're headed next.

Questions about anything?

mutable data - section 3.3.

Discuss set-car! and set-cdr! , and the Guile vs DrRacket quirks to look out for, namely that these languages have "mutable pairs" and "immutable pairs", even though there isn't an easy way to test for the difference ... at least not in the "SICP compatible" version of the DrRacket language. From what I can read, the old SICP scheme only had mutable pairs. Lookup "racket mutable" for some of this stuff. In racket, mcons rather than cons is the mutable pair.

The upshot of this is that in the guile scheme, you should use (list 1 2 3) rather than '(1 2 3) to make mutable structures. And grow from (list ), not '() ... at least that's what worked for me.

Exercises to try in class: 3.15 set-to-wow! and 3.16 count-pairs.

coming attractions

Let's think over what we've done so far, and start to talk about the "big picture" of how that fits into the study of programming languages.

• We've studied the "scheme" programming language
• ... which has a particularly simple syntax
• ... and used only a few fundamental ideas to build from :
  • lists within lists using ( ) "forms" our programming language
    • ... with the convention (function arg1 arg2 arg3 ...)
    • ... and an "evaluate" mentality of modeling how program execution proceeds
  • bindings - define (a special form)
  • symbols, numbers, strings : atomic types
  • pairs - cons
    • which gave us data structures
  • lambda - functions
    • ... which gave us scope and local variables (let, environments)
  • conditionals - branching (a special form)
  • recursion
    • ... which gave us loops and iteration
  • local state and mutatable data
    • ... which gave us "objects"
    • ... as contrasted with purely functional (no extra effects) styles

Next I want to pull back and take a look at how these ideas connect to all programming languages ... by talking about how to code a programming language using another programming language, namely "interpreters" and "compilers".

In SICP, some of this is covered in chapter 4 ... and we'll circle back to that, after first looking at the big picture.

I can give you the punch line now : the language we've been doing, lisp, is special in that the syntactic structure of the language makes the semantic structure obvious and trivial. It's really the only language family where the data structures (what we've been doing with list and cons) are made up of the same stuff as the programs (what we've been doing with define and lambda and all that). That means that in lisp, data and code look the same ... which means the tools to have your programming language manipulate your programming language are already right there in front of you.

So : let's start with two concepts.

• An "interpreter" is a program running in language A which can read and execute source code in language B.
• A "compiler" is a program running in language A which can read source code in language B and output source code in language C.

(Language A is itself running on top of an operating system and some sort of assembly language D, but that is not the point here.)

All this is more common that you might think. In many cases you may want to invent your own domain-specific language (DSL) to handle some particular problem. Why do you think that there are 1500 programming languages at 99-bottles-of-beer.net ? Never mind all the non-programming languages like regular expressions, HTML, CSS, JSON, YAML, diff, and so on.

What are the pieces & steps involved in these compilers & interpreters? I'm glad you asked.

• The Grammar of language B , in some sort of BNF form
• A Lexer, to read in the text file of B source code and convert it to a sequence of tokens
• A Parser, to convert that sequence of tokens into an abstract syntax tree (AST)
• Various transformations of that tree into more convenient forms (optimization)
• Finally either executing that tree or converting it into language C, for example with templates (depending on the specifics of what language C can do).

All of these are big topics, and in the month or so left this term we won't fully master them. But we can see some examples and understand how the ideas fit together.

I'll find us some small-ish problems for us to explore these themes, and will post some online resources soon, including

• Matt Might's Compiler's Course
• the classic Dragon Book - Compilers: Principles, Techniques, and Tools

So ... stay tuned. ;)