March 17

Questions about anything?

loop invariant

First let's discuss something we haven't spent much time on yet: how do you know that an algorithm works?

One way is with a "loop invariant". We'll go over what that's all about.

• wikipedia: loop invariant

In general this is a question about "formal verification" i.e. a "math proof". Let's do an example to illustrate the ideas of (1) a proof, (2) a proof by induction, and (3) a loop invariant.

Example : the sum of 1 to n is n*(n+1)/2.

Proof 1 : finding a clever argument (Effective but can be hard to find ... not a general method.)

  1 + 2 +     3     + ... + n-1 + n    = S     # the forwards sum
  n + (n-1) + (n-2) + ... + 2   + 1    = S     # the backwards sum
  ----------------------------------------
  (1+n) + (1+n) +  ... + (1+n)         = 2*S   # add them together

  So there are n terms, each of which is 1+n. 
  So n * (1+n)  = 2 * S.
  Therefore 1+2+3+....+n = (n)*(n+1)/2. QED.

Proof 2 : by induction (A powerful and common technique related to recursion.)

  Step 0: Explicitly check a base case, for example n=1:   
          n*(n+1)/2 = 1*(1+1)/2 = 1 = sum 1 ..1. Yes.

  Step 1: Assume that it's true for n, and use that to show it's true for n+1.

   Assume 1 + 2 + 3 + .... n =  n * (n+1) /2

   Then for n+1

    1 + 2 + 3 + .... + n + (n+1)  =  n*(n+1)/2  + (n+1)
                                  =  n*n/2 + n/2 + n + 1
                                  =  n*n/2 + 3/2 n + 1

    Now we do some algebra :
       (n+1)*(n+2)/2 = 1/2 (  n*n + n + 2n + 2)
                     =    n*n/2 + 3/2 n + 2/2            
    ... and get the same result. QED.

"Proof 3" : loop invariant (One way that algorithms with loops or recursion can be "verified".)

Now instead of verifying the formula, we want to verify an algorithm that has a loop in it :

total = 0
i = 0
# loop invariant initialize: total is correct 
while i <= N:
  i = i + 1
  total = total + i
  # loop invariant : total is sum up to new i
  #                  since total = (sum up to i-1) + i
  assert i*(i+1)/2 == total     # We can explicitly check loop invariant.
# finished loop: total is sum up to and including N (since i=N).

To be really sure, we can put in an assertion - code that checks to see if the loop invariant is still true. (Of course it isn't always as easy to the verification as it is in this simple case. And this sort of extra check will slow things down, so we'd only do it when convincing ourselves that our algorithm's logic is valid.)

your turn - insertion sort in C

• I'd like to do an in-class coding exercise in smallish (3 people or so) groups.
• This is a chance to look at a specific algorithm and practice your C coding skills.
• I'll give you an explicit template to get you going, which I've attached to this page: sorting_template.c

You can edit and run this at jupyter.bennington.college ... I'll do a demo if you would like.

The algorithm I want you to understand and implement is insertion sort.

Your mission, in these groups:

• Describe (i.e. explain to each other) in words the insertion sort algorithm.
• Implement it within the template. (One person types; others discuss and comment.)
• Explicitly add the appropriate loop invariant check.

We'll come back together to discuss and compare our work before the end of the class.

Questions to discuss:

• What O() is this and why?
• Can this work on a linked list?
• Is this a "stable" sort? (And what does that mean?)
• Other pros and cons?

what's next

If there is still time - either in your groups or all together - we'll look at two more algorithms. And in any case, we'll continue with these on Monday.

• quicksort
• heapsort

attachments

		last modified	size
	jims_insertionsort.c	Thu Mar 17 2022 03:04 pm	4.1K
	sorting_template.c	Thu Mar 17 2022 11:43 am	2.9K