March 3

old business

Any questions about anything we've done so far?

Let's start by going over the analysis homework due today.

So far we've covered

• What an "algorithm" is.
• What a "data structure" is : ADT (abstract data type i.e. API) vs implementation.
• Looked at our first example of a data structure, the "indexed array".
• Described a way to analyze the run time of an algorithm using O() notation.

We've also mentioned two algorithm design ideas

• brute force (i.e. loop over every possibility), and
• divide and conquer (reduce the problem by half repeatedly)

and have started to discuss at a few important tasks related to a collection of n items :

• putting thing in
• searching for something
• sorting

new business - preview

I'd like to continue with several related topics this week.

linked lists in python & C

First and foremost, we'll look at another common implementation of a collection, called a "linked list". I'd like to do this in both Python and C, and so this is a good place to dive into C and start to grab hold of its important concepts, especially pointers ... which are crucial to the idea of a linked list.

As I mentioned before, a linked list is made up of smallish pieces often called "nodes", each of which has (a) a data value, and (b) a reference to the next node. So the list [1, 10, 15, 20] might be represented by something like

    [1, ↓ ]
        [ 10, ↓ ]
              [ 15, ↓ ]
                    [ 20, None]

where those downward arrows are some sort of reference to the next part of the list. Different languages have different ways of handling those "references".

Linked lists can be more easily grown, shrunk, and merged than indexed array. An indexed array is one contiguous block of memory; as n gets big, they can be slow and awkward to grow or combine. Linked lists, on the other hand, are made up of small pieces that be spread out throughout memory. Making one bigger means allocating one more small block of memory and referencing it.

However, accessing data from the middle of a linked list is slow - there is no way to find the middle other than traversing the list.

Here's a partial comparison of properties.

                Indexed Array      Linked List
                -------------      ----------- 
 access i'th      O(1)               O(n)
 grow(*)          O(n)               O(1)
 remove           O(n)               O(1)
 merge            O(n)               O(1)

In practice, growing an array isn't always expensive ... because some languages (like Python) will automatically pre-allocate extra space. So if you set a=[1,2,3], it allocate a bit more space. See for example this discussion at towardsdatascience how-lists-are-optimized .

stacks and queues

Second, I'd also like introduce a data structure called a "stack" and another called a "queue"; two important and similar APIs which are used in a number of classic algorithms. These can be implemented with either indexed arrays or with linked lists.

Here's a description of a stack at runestone what is a stack and wikipedia's stack

Stacks have two basic operations:

• push(stack, item)
• pop(stack)

recursion

And third, we're going to start looking at recursion as a way to write our algorithms and to loop over these data structures. Often we can either use a loop or recursion, and the choice comes down to which is easier to describe.

where to read about this stuff

You can find this material in

• Skienna's Algorithm Design Manual, chapter 3 "Data Structures"
• Runestone's Problem Solving with Algorithms and Data Structuers using Python
  • chapter 4, Basic Data Structures
  • chapter 5, Recursion
• Foundations of Computer Science, chapter 6
• A First Course on Data Structures in Python, chapters 6, 7, 8, 9
• learn-c.org , especially the Pointers and LinkedLists sections
• my C pointer examples

time for some code examples

First a discussion of C concepts : pointers, malloc, and all that

Then let's jump in to it all at once: linked lists, a stack, and recursion, in both Python and C .

coming next

I've posted next week's assignment that invites you to explore this ideas, including add new features to my linked list code.

Please come to class on Monday with questions about any of this stuff, and we'll practice and do examples.