Monday March 1

... start recording.

analysis - O()

Finish up our discussion of this stuff :

• chapter 2 of Skienna's "Algorithm Design Manual"
• chapter 3 in Runestone's online "Problem Solving with Algorithms and Data Structures using Python"

I've posted my answers to the homework due today; we'll go over that. (And oops: there was a formatting typo in my notes; for the four functions I wanted \( n \, \log(n), n^2, 2^n, n! \) ).

I would like to briefly talk about counting things ... and what this has to do with O()

• pairs of N objects
• all sequences of N objects
• all states of N booleans
• number of sets of N objects

Topics to understand and/or discuss depending on your comfort level :

• O() notation
• making plots
• logs
• brute-force : search for something by looking at every
• linear search : an example of brute-force (we did this last time)
• divide-and-conquer : recursively subdividing the problem to get log(n) rather than n behavior
• binary search - an example of divide-and-conquer (we did this last time)
• exponentiation by squaring - another example of divide-and-conquer (in the homework due today

Questions about any of this?

data structures

This week and next we're going to look at some essential data structure building blocks :

• ADT : Abstract Data Type
• indexed arrays
• linked list
• stack & queue
• recursion
• hash tables

You can find this material in

• Skienna's Algorithm Design Manual, chapter 3 "Data Structures"
• Runestone's Problem Solving with Algorithms and Data Structuers using Python
  • chapter 4, Basic Data Structures
  • chapter 5, Recursion
  • chapter 6.5 Hashing

First, what's a "data structure"? See wikipedia's definition

Two ideas :

• An "ADT" (Abstract Data Type) defines an interface (API), some functions or operations
• An "implementation" is some code that realizes an ADT in a programming language

Different data structures are optimized for different tasks, with various strengths. Your mission is to understand what the O() behavior is for different operations, and their pros and cons.

The ideas we'll look at are :

• stack : push, pop ; last in is first out
• queue : push, pop ; first in is first out
• array : a[i] where i is 0,1,2,3,... ; indexed sequential memory
• linked list : a0 -> a1 -> a2 -> a3 -> null ; pointers
• dictionary : (key, value) fast lookup
• hash table : wrap-around array with hash function & collision algorithm
• priority queue : pop always gets biggest (or smallest)
• heap : common array implementation of a priority queue

syntax vs O() behavior

Please note that different languages spell these ideas in different ways. For this course, we care about how fast the operations go, not how they're spelled.

In both Python and C, we can read or write the n'th element of an indexed array with the notation numbers[i] . In lisp, we'd write that as (nth i numbers). We could just as well have numbers.get(i) or whatever.

ADT - objects vs functions

For languages that support objects (which are called "classes" in Python), the typical approach is to write an ADT (Abstract Data Type) as a class, with methods providing the appropriate interface.

C doesn't have objects, but it does allow you to define your own data types. Typically we build up an ADT with a data type and a set of functions, simply passing the data type in rather than using object notation.

All right, enough talking about all this - let's get more specific.

stack

A stack is a "first in, last out" data structure, with a "push" and "pop" method.

I've attached a first python implementation; let's annotate it and test it.

s = Stack()
s.push(1)
s.push(2)
while not s.is_empty():
  print(s.pop())

As a python programming exercise, let's add an __str__ method to this class.

I've also attached a C implementation; let's look at that and discuss.

Both of these implementations use an indexed array as the underlying data store. But that isn't the only way to build it.

linked lists

Another fundamental type of storage is the "linked list". Some languages, lisp in particular, use this idea as one of the core language concepts.

The idea is to create a "node" which has some data and a pointer to the next node. Each one can be anywhere in memory, which avoids allocating large contiguous blocks.

[ node 1 ]  ->  [node 2 ] -> [node 2]

I've attached again both a python and C implementation; we'll walk through the code in class.

These are both "singly linked lists".

Another variation is the "doubly linked list" ... discuss what that would look like.

Many operations on linked lists can be done elegantly with recursion ... such as finding the length of the list.

linked list pros :

• inserting an element is O(1) ... just move links
• swapping two elements is O(1)
• changing either end is O(1) (assuming you keep track of both ends)
• large list does not require contiguous memory

linked list cons :

• counting length of list is O(n) (if you don't keep track of it as the list changes)
• getting n'th element is O(n) (you need to walk the list from one end)

classic job interview question:

• whiteboard reversing a linked list

what are these things good for?

Lots of things. ;)

One example is parsing parens and brackets :

Given a string made up of these characters :  (  )  [ ] {  } and possibly others,
write a function which returns true if the grouping symbols are nested and matched properly,
and false if not.

Let's talk through how that would work. Can you think of another way to do this?

where are we going next?

Your mission for this week is to (a) read about this stuff, (b) review and practice your C and Python, and (c) try out some of these ideas in code.

The specific assignment for next Monday will depend on how far we get and how comfortable you are with all this.

The related coding tasks could include

• Re-implement the stack in Python & C, this time using a linked list as the data store
• Implement a queue with (enqueue, dequeue) operations ; a first-in first-out collection
• Write a recursive function to find the length of a linked list.
• Design a "linked list" ADT, choosing which features (functions or methods) to implement, perhaps some of (insert, swap, push, pop, remove, length, for_each (apply a function to each element), ...).
• Read about "cons" and "cdr" in lisp; the building blocks of the whole language.
• Using nodes with "right" and "left" properties, build a linked binary tree data structure.
• Write a recursive function to count the number of nodes in that linked binary tree.

... though probably not all of that in a week. ;)

The specific coding topics include

• classes in Python
• pointers, structs, and malloc in C
• recursion

There are many sources online to read about these things; check out my resources page. The following pages may be useful :

• my Intro CS notes on python objects
• my C pointers examples
• my word list example

If the recursion ideas aren't familiar, now would be a good time to read about that :

• review recursion if

Other related resources from wikipedia, with lists of other readings :

• Stack (abstract data type)
• Queue (abstract data type)
• Linked list

attachments

		last modified	size
	counting_whiteboard.pdf	Mon Mar 01 2021 06:30 pm	643K
	linked_list.c	Mon Mar 01 2021 12:21 am	909B
	linked_list.py	Mon Mar 01 2021 05:57 pm	1.2K
	stack1.c	Mon Mar 01 2021 12:01 am	1.0K
	stack1.py	Mon Mar 01 2021 12:01 am	603B