Structure and
Interpretation
of Computer
Programs

Fall 2021
course
site
-->

Mon Nov 8

I've posted an assignment for Thursday - check it out.

And I have found another source for reading about all this compiler stuff : Basics of Compiler Design , a free online PDF textbook. Chapters 1-3, 5, and 7 cover what we'll be discussing this term.

grammars

First, review what we did last time , looking at pg 62 in that textbook (string "aabbbcc" using grammar 3.4 ; exercise 3.2) . And try exercise 3.6 - you think, then we'll discuss.

lexers

Second, let's start discussing "lexers" - the first step in compiling,

In practice lexing is done with "regular expressions" or (equivalently) "finite state machines" ... both of these have the same expressive power. (Less than context-free-grammars.)

We won't be doing much with DFAs this term, but it's worth understanding (a) the general idea, and (b) the fact that efficient implementations of lexers (and regular expression engines) are DFAs.

Regular expression practice :

A "lexer generator" is a program that creates a program ... given a specification (regular expressions) of a language's keywords and primitives (numbers, strings, operators, parens), it creates a program which takes source code as input and produces a string of tokens [(token-name, value), ...] as output.

example - python

to do :

coming soon : building blocks for lexers in guile & racket

$ scheme
Guile 2.2.7. Enter ',help' for help.
Language: scheme for SICP with some extras.
> (define text (readline))
This is a line of text.
> text
"This is a line of text."
> (string-match "is" text 0)          ;; look for regex "is" , from offset 0
#("This is a line of text." (2 . 4))
;; search "guile readline" and "guile regex"

$ racket
$ racket
Welcome to Racket v8.0 [cs].
> (define text (port->string))
This is some text.
^D> text
"\nThis is some text.\n"
> (regexp-match "is" text)
'("is")

also coming soon

https://cs.bennington.college /courses /fall2021 /sicp /notes /11-08
last modified Mon November 8 2021 4:48 pm