Dan's Thoughts » algorithms

Exercise 1.45 of SICP

dan — Thu, 16 Jul 2009 02:17:16 +0000

Exercise 1.45: We saw in section 1.3.3 that attempting to compute square roots by naively finding a fixed point of does not converge, and that this can be fixed by average damping. The same method works for finding cube roots as fixed points of the average-damped . Unfortunately, the process does not work for fourth roots -- a single average damp is not enough to make a fixed-point search for converge. On the other hand, if we average damp twice (i.e., use the average damp of the average damp of ) the fixed-point search does converge. Do some experiments to determine how many average damps are required to compute nth roots as a fixed-point search based upon repeated average damping of . Use this to implement a simple procedure for computing n^th roots using fixed-point, average-damp, and the repeated procedure of exercise 1.43. Assume that any arithmetic operations you need are available as primitives.

Doing experiments on the average number of damps to get the fixed-point procedure to converge seems to be:

Degree of root	Power of 2	Number of average damps
2	2¹	1
4	2²	2
8	2³	3
16	2⁴	4
32	2⁵	5

Using this basis it makes sense to write a procedure discrete-log that computes the number of powers of 2 in a given number. It's a bit like a very primitive implementation log₂ function hence the name. Discrete-log is because it halves the number of computations every iteration.

(define (root num degree)
  (define (discrete-log n)
    (if (< n 2)
      0
      (+ 1 (discrete-log (/ n 2)))))
  (define (compose f g)
    (lambda (x) (f (g x))))
  (define (repeated-compose f n)
    (if (< n 2)
      f
      (compose f (repeated-compose f (- n 1)))))
  (define (average-damp f)
    (lambda (x) (/ (+ x (f x)) 2)))
  (let ((number-of-compositions (discrete-log degree))
        (initial-function (lambda (x) (/ num (expt x (- degree 1))))))
    (cond ((= degree 0) 1)
          ((= degree 1) num)
          (else
            (fixed-point 
              ((repeated-compose average-damp number-of-compositions) initial-function)
              1.0)))))
(define (fixed-point f guess)
  (define error 0.001)
  (define (close-enough? x y) (< (abs (- 1 (/ x y))) error))
  (define (try guess)
    (let ((new-guess (f guess)))
      (if (close-enough? guess new-guess)
        new-guess
        (try new-guess))))
  (try guess))

> (root 2 0)
1
> (root 2 1)
2
> (root 2 2)
1.4142135623746899
> (root (expt 2 50) 50)
1.9999956962054166
> (root 0.00000001 2)
1.0000000000082464e-4

This program summarizes most of Chapter 1. It has lexical scope, passing functions as parameters, functions that construct and return other functions and the square root algorithm with an improved close-enough? procedure from exercise 1.7.
The repeated-compose and discrete-log could be done iteratively instead of recursively like this.

(define (repeated-compose f n)
  (define (compose f g)
    (lambda (x) (f (g x))))
  (define (iter n func)
    (if (< n 2)
      func
      (iter (- n 1) 
            (compose f func))))
  (iter n f))

(define (discrete-log n)
  (define (iter n exponent)
    (if (< n 2)
      exponent
      (iter (/ n 2) (+ 1 exponent))))
  (iter n 0))

Exercise 1.26 of SICP

dan — Wed, 24 Jun 2009 02:33:11 +0000

Exercise 1.26: Louis Reasoner is having great difficulty doing exercise 1.24. His fast-prime? test seems to run more slowly than his prime? test. Louis calls his friend Eva Lu Ator over to help. When they examine Louis's code, they find that he has rewritten the expmod procedure to use an explicit multiplication, rather than calling square:

(define (expmod base exp m)
  (cond ((= exp 0) 1)
        ((even? exp)
         (remainder (* (expmod base (/ exp 2) m)
                       (expmod base (/ exp 2) m))
                    m))
        (else
         (remainder (* base (expmod base (- exp 1) m))
                    m))))

"I don't see what difference that could make," says Louis. "I do." says Eva. "By writing the procedure like that, you have transformed the process into a process." Explain.

Just like the recursive binary tree computational procedure for Fibonacci numbers was exponential so is this process exponential because of the double call to expmod.
This double call creates a tree of depth log₂(n). The amount of nodes in a tree (the amount of computational steps to perform the algorithm), of depth log₂(n) is . This means the algorithm is indeed .

Exercise 1.25 of SICP

dan — Mon, 22 Jun 2009 11:00:40 +0000

Exercise 1.25: Alyssa P. Hacker complains that we went to a lot of extra work in writing expmod. After all, she says, since we already know how to compute exponentials, we could have simply written

(define (expmod base exp m)
  (remainder (fast-expt base exp) m))

Is she correct? Would this procedure serve as well for our fast prime tester? Explain.

While the procedure is mathematically accurate and would produce the right results, fast-expt would produce enormous numbers that would then have to be reduced modulo m. For example:

(expmod 2 10 7)
(remainder (fast-expt 2 10) 7)
(remainder 1024 7)

The 1024 might not seem so bad at first.

(expmod 2 100 7)
(remainder (fast-expt 2 10) 7)
(remainder 1267650600228229401496703205376 7)
That's a bit excessive.

The version bellow keeps the number in check.

(define (expmod base exp m)
  (cond ((= exp 0) 1)
        ((even? exp)
         (remainder (square (expmod base (/ exp 2) m))
                    m))
        (else
         (remainder (* base (expmod base (- exp 1) m))
                    m))))

I am taking some liberty here in skipping a few minor steps.
(expmod 2 10 7)
(remainder (square (expmod 2 5 7)) 7)
(remainder (square (remainder (* 2 (expmod 2 4 7)) 7)) 7)
(remainder (square (remainder (* 2 (remainder (square (expmod 2 2 7)) 7)) 7)) 7)
(remainder (square (remainder (* 2 (remainder (square (remainder (square (expmod 2 1 7)) 7)) 7) 7) 7)) 7)
(remainder (square (remainder (* 2 (remainder (square (remainder (square (remainder (* 2 1) 7)) 7)) 7)) 7)) 7)
(remainder (square (remainder (* 2 (remainder (square (remainder (square (remainder 2 7)) 7)) 7)) 7)) 7)
(remainder (square (remainder (* 2 (remainder (square (remainder (square 2) 7)) 7)) 7)) 7)
(remainder (square (remainder (* 2 (remainder (square (remainder 4 7)) 7)) 7)) 7)
(remainder (square (remainder (* 2 (remainder (square 4) 7)) 7)) 7)
(remainder (square (remainder (* 2 2) 7)) 7)
(remainder (square (remainder 4 7)) 7)
(remainder (square 4) 7)
2

The key difference is that at no point do any of the numbers grow to be much larger than the base.

Exercise 1.24 of SICP

dan — Sun, 21 Jun 2009 00:33:18 +0000

Exercise 1.24: Modify the timed-prime-test procedure of exercise 1.22 to use fast-prime? (the Fermat method), and test each of the 12 primes you found in that exercise. Since the Fermat test has (log n) growth, how would you expect the time to test primes near 1,000,000 to compare with the time needed to test primes near 1000? Do your data bear this out? Can you explain any discrepancy you find?

I arbitrarily chose to run the fast-prime? test 100 times. As in 1.22 I ran it only larger ranges than the book suggested.

(define (random n) (random-integer n))
(define (runtime) (time->seconds (current-time)))
(define (square x) (* x x))
(define (square-mod n m) (remainder (* n n) m))
(define (expmod base exp m)
  (cond ((= exp 0) 1)
        ((even? exp)
         (remainder (square (expmod base (/ exp 2) m))
                    m))
        (else
         (remainder (* base (expmod base (- exp 1) m))
                    m))))        
(define (fermat-test n)
  (define (try-it a)
    (= (expmod a n n) a))
  (try-it (+ 1 (random (- n 1)))))
 
(define (fast-prime? n times)
  (cond ((= times 0) #t)
        ((fermat-test n) (fast-prime? n (- times 1)))
        (else #f)))
 
(define (timed-prime-test n)
  (newline)
  (display n)
  (start-prime-test n (runtime)))
(define (start-prime-test n start-time)
  (if (fast-prime? n 100)
      (report-prime (- (runtime) start-time))))
(define (report-prime elapsed-time)
  (display " *** ")
  (display elapsed-time))
(define (search-for-primes beg end)
  (define (search beg end num-primes)
    (cond ((= num-primes 3) num-primes)
          ((> beg end) num-primes)
          ((even? beg) (search (+ beg 1) end num-primes))
          ((fast-prime? beg 100) (timed-prime-test beg)
                        (search (+ beg 2) end (+ 1 num-primes)))
          (else
            (search (+ beg 2) end num-primes))))
  (search beg end 0))

> (search-for-primes (expt 10 10) (- (expt 10 11) 1))

10000000019 *** .016000032424926758
10000000033 *** .016000032424926758
10000000061 *** .0160000324249267583

> (search-for-primes (expt 10 11) (- (expt 10 12) 1))

100000000003 *** .016000032424926758
100000000019 *** .016000032424926758
100000000057 *** .0150001049041748053

> (search-for-primes (expt 10 12) (- (expt 10 13) 1))

1000000000039 *** .016000032424926758
1000000000061 *** 0.
1000000000063 *** .0159997940063476563

> (search-for-primes (expt 10 13) (- (expt 10 14) 1))

10000000000037 *** .015999794006347656
10000000000051 *** .016000032424926758
10000000000099 *** .0149998664855957033

The test is too fast to give meaningful times to compare against. I would expect numbers of order 10⁶ to take 2 times longer to compute than 10³. The reasoning follows the property of logarithms. log(10⁶) = 6 and log(10³) = 3.

In my case: log(10¹⁰) = .016
I would predict: log(10²⁰) = .032
> (search-for-primes (expt 10 20) (- (expt 10 21) 1))

100000000000000000039 *** .03099989891052246
100000000000000000129 *** .03099989891052246
100000000000000000151 *** .030999898910522463

Exercise 1.23 of SICP

dan — Fri, 19 Jun 2009 11:52:51 +0000

Exercise 1.23: The smallest-divisor procedure shown at the start of this section does lots of needless testing: After it checks to see if the number is divisible by 2 there is no point in checking to see if it is divisible by any larger even numbers. This suggests that the values used for test-divisor should not be 2, 3, 4, 5, 6, ..., but rather 2, 3, 5, 7, 9, .... To implement this change, define a procedure next that returns 3 if its input is equal to 2 and otherwise returns its input plus 2. Modify the smallest-divisor procedure to use (next test-divisor) instead of (+ test-divisor 1). With timed-prime-test incorporating this modified version of smallest-divisor, run the test for each of the 12 primes found in exercise 1.22. Since this modification halves the number of test steps, you should expect it to run about twice as fast. Is this expectation confirmed? If not, what is the observed ratio of the speeds of the two algorithms, and how do you explain the fact that it is different from 2?

12 Primes from 1.22:

10000000019 *** .14100003242492676
10000000033 *** .1399998664855957
10000000061 *** .14000010490417483
100000000003 *** .5310001373291016
100000000019 *** .5320000648498535
100000000057 *** .54699993133544923
1000000000039 *** 1.7660000324249268
1000000000061 *** 1.75
1000000000063 *** 1.76500010490417483
10000000000037 *** 5.672000169754028
10000000000051 *** 5.672000169754028
10000000000099 *** 5.65599989891052253

Using time-prime-test with the new next function

10000000019 *** .07799983024597168
10000000033 *** .07800006866455078
10000000061 *** .07800006866455078
100000000003 *** .2969999313354492
100000000019 *** .29600000381469727
100000000057 *** .2969999313354492
1000000000039 *** .9849998950958252
1000000000061 *** .9840002059936523
1000000000063 *** .9839999675750732
10000000000037 *** 3.171999931335449
10000000000051 *** 3.1570000648498535
10000000000099 *** 3.171999931335449

Using next	Exercise 1.22	Factor of speedup
0.07799983	0.141000032	1.807696658
0.078000069	0.139999866	1.794868503
0.078000069	0.140000105	1.79487156
0.296999931	0.531000137	1.787879664
0.296000004	0.532000065	1.797297493
0.296999931	0.546999931	1.841751036
0.984999895	1.766000032	1.792893625
0.984000206	1.75	1.778454912
0.983999968	1.765000105	1.793699353
3.171999931	5.67200017	1.788146372
3.157000065	5.67200017	1.796642399
3.171999931	5.655999899	1.78310215

Judging by the speedup it does look like there is about a 2 fold increase in speed of determining if a number is prime.

(define (runtime) (time-&gt;seconds (current-time)))
(define (square x) (* x x))
(define (smallest-divisor n)
  (find-divisor n 2))
(define (next n)
  (if (= n 2)
    3
    (+ n 2)))
(define (find-divisor n test-divisor)
  (cond ((&gt; (square test-divisor) n) n)
        ((divides? test-divisor n) test-divisor)
        (else (find-divisor n (next test-divisor)))))
(define (divides? a b)
  (= (remainder b a) 0))
(define (prime? n)
  (= n (smallest-divisor n)))
 
(define (timed-prime-test n)
  (newline)
  (display n)
  (start-prime-test n (runtime)))
(define (start-prime-test n start-time)
  (if (prime? n)
      (report-prime (- (runtime) start-time))))
(define (report-prime elapsed-time)
  (display " *** ")
  (display elapsed-time))

Iterative version of expmod

dan — Sun, 14 Jun 2009 09:05:07 +0000

Recursive version:

(define (square x) (* x x))
(define (expmod base exp m)
  (cond ((= exp 0) 1)
        ((even? exp)
         (remainder (square (expmod base (/ exp 2) m))
                    m))
        (else
         (remainder (* base (expmod base (- exp 1) m))
                    m))))

I was a bit surprised that the above algorithm didn't come with the problem set of to rewrite it in an iterative form. So here it is.
Iterative Version:

(define (square-mod n m) (remainder (* n n) m))
(define (expmod-iter base n m a)
  (cond ((= n 0) a)
        ((even? n) (expmod-iter (square-mod base m) (/ n 2) m a))
        (else
          (expmod-iter base (- n 1) m (remainder (* a base) m)))))
(define (expmod base n m)
  (expmod-iter base n m 1))

Exercise 1.20 of SICP

dan — Sat, 13 Jun 2009 23:49:45 +0000

Exercise 1.20: The process that a procedure generates is of course dependent on the rules used by the interpreter. As an example, consider the iterative gcd procedure given above. Suppose we were to interpret this procedure using normal-order evaluation, as discussed in section 1.1.5. (The normal-order-evaluation rule for if is described in exercise 1.5.) Using the substitution method (for normal order), illustrate the process generated in evaluating (gcd 206 40) and indicate the remainder operations that are actually performed. How many remainder operations are actually performed in the normal-order evaluation of (gcd 206 40)? In the applicative-order evaluation?

GCD Algorithm:

(define (gcd a b)
  (if (= b 0)
    a
    (gcd b (remainder a b))))

Normal Order:

R1 = (remainder 206 40) ; 6
(gcd 206 40)
(if (= 40 0)
  40
  (gcd 40 R1))
(if (= R1 0)
  40
  (gcd R1 (remainder 40 R1)))
R2 = (remainder 40 R1) ; 4
(if (= R2 0)
    R1
    (gcd R2 (remainder R1 R2)))
R3 = (remainder R1 R2) ; 2
(if (= R3 0)
    R2
    (gcd R3 (remainder R2 R3)))
R4 = (remainder R2 R3) ; 0
(if (= R4 0)
  R3
  (gcd ....)) ; Never called because R4 evaluates to 0

Every time there is an R in an if = statement that R gets evaluated.
#{R1}+#{R2}+#{R3}+#{R4}= 1 + 2 + 4 + 7 = 14 times remainder is run in the if =
R3 is ran once in order to return the actual remainder.
14+#{R3}=18
Number of Times "remainder" function called 18.

Applicative Order:
(gcd 206 40)
(gcd 40 (remainder 206 40))
(gcd 40 6)
(gcd 6 (remainder 40 6))
(gcd 6 4)
(gcd 4 (remainder 6 4))
(gcd 4 2)
(gcd 2 (remainder 4 2))
(gcd 2 0)
2
Number of Times "remainder" function called 4.

Exercise 1.19 of SICP

dan — Fri, 12 Jun 2009 02:53:40 +0000

Exercise 1.19: There is a clever algorithm for computing the Fibonacci numbers in a logarithmic number of steps. Recall the transformation of the state variables a and b in the fib-iter process of section 1.2.2: a <- a + b and b <- a. Call this transformation T, and observe that applying T over and over again n times, starting with 1 and 0, produces the pair Fib(n + 1) and Fib(n). In other words, the Fibonacci numbers are produced by applying Tⁿ, the n^th power of the transformation T, starting with the pair (1,0). Now consider T_pq to be the special case of p = 0 and q = 1 in a family of transformations T_pq , where T_pq transforms the pair (a,b) according to a <- bq + aq + ap and b <- bp + aq. Show that if we apply such a transformation T_pq twice, the effect is the same as using a single transformation T_p'q' of the same form, and compute p' and q' in terms of p and q. This gives us an explicit way to square these transformations, and thus we can compute Tⁿ using successive squaring, as in the fast-expt procedure. Put this all together to complete the following procedure, which runs in a logarithmic number of steps:

(define (fib n)
  (fib-iter 1 0 0 1 n))
(define (fib-iter a b p q count)
  (cond ((= count 0) b)
        ((even? count)
         (fib-iter a
                   b
                   <??>      ; compute p'
                   <??>      ; compute q'
                   (/ count 2)))
        (else (fib-iter (+ (* b q) (* a q) (* a p))
                        (+ (* b p) (* a q))
                        p
                        q
                        (- count 1)))))

Definition of a₁ and b₁:

The key here is to substitute from the definition of a₁ and b₁ into a₂ and b₂ in order to get p' and q'
I start with b₂ because the expression is simpler to work with. the values I get for p' and q' must also work with a₂

a₂ is trickier to check because of the number of terms, I'll take a few short cuts here in showing the math

Finally: where

(define (square x) (* x x))
(define (fib n)
  (fib-iter 1 0 0 1 n))
(define (fib-iter a b p q count)
  (cond ((= count 0) b)
        ((even? count)
         (fib-iter a
                   b
                   (+ (square p) (square q))      ; compute p'
                   (+ (* 2 p q) (square q))       ; compute q'
                   (/ count 2)))
        (else (fib-iter (+ (* b q) (* a q) (* a p))
                        (+ (* b p) (* a q))
                        p
                        q
                        (- count 1)))))

> (fib 10)
55
> (fib 100)
354224848179261915075

Exercise 1.18 of SICP

dan — Wed, 10 Jun 2009 09:06:03 +0000

Exercise 1.18: Using the results of exercises 1.16 and 1.17, devise a procedure that generates an iterative process for multiplying two integers in terms of adding, doubling, and halving and uses a logarithmic number of steps.

(define (double x) (+ x x))
(define (halve x) (/ x 2))
(define (fast-mult-iter a b n)
  (cond ((= b 0) n)
        ((even? b) (fast-mult-iter (double a) (halve b) n))
        (else
          (fast-mult-iter a (- b 1) (+ a n)))))

Exercise 1.17 of SICP

dan — Tue, 09 Jun 2009 00:38:53 +0000

Exercise 1.17: The exponentiation algorithms in this section are based on performing exponentiation by means of repeated multiplication. In a similar way, one can perform integer multiplication by means of repeated addition. The following multiplication procedure (in which it is assumed that our language can only add, not multiply) is analogous to the expt procedure:

(define (mult a b)
  (if (= b 0)
      0
      (+ a (mult a (- b 1)))))

This algorithm takes a number of steps that is linear in b. Now suppose we include, together with addition, operations double, which doubles an integer, and halve, which divides an (even) integer by 2. Using these, design a multiplication procedure analogous to fast-expt that uses a logarithmic number of steps.

(define (double x) (+ x x))
(define (halve x) (/ x 2))
(define (fast-mult a b)
  (cond ((= b 0) 0)
        ((even? b) (double (fast-mult a (halve b))))
        (else
          (+ a (fast-mult a (- b 1))))))

The difference in between and is apparent for large numbers.
On my system:
> (mult 2 (expt 20 8))
Takes forever, I had to interrupt it.
> (fast-mult 2 (expt 20 8))
51200000000

The answer is almost instantaneous.