Here's a really old algorithm (credited to Euclid 325BC--265BC) that is still implemented on computers to this day. The greatest common divisor (GCD) of two non-negative integers (whole numbers) is a non-negative integer that
You might come up with several algorithms for finding the GCD of two non-negative integers. For example, you could find all the candidates by listing all the non-negative integers that divide both numbers, and then choosing the largest. To carry out this plan, you'd need a way to find all the divisors of each non-negative integer, perhaps by trying each smaller non-negative integer in turn to see whether it divides evenly. If our two non-negative integers were large, this approach would take a lot of work.
Euclid solved the same problem in a way that is efficient, both to describe and to carry out. Suppose you want the GCD of m and n, where m and n are symbols that represent non-negative integers. Here's what to do:
Here is a subtle variation of the GCD algorithm.
Another algorithm goes back to our Arabic heritage (where the "al" in algorithm comes from). Usually we multiply small numbers in our head, and for larger numbers we reach for a calculator. However, in a pinch, we might recall our grade-school algorithm for multiplying numbers. It has a lookup table for the products of the integers 0--9 (we keep this lookup table in our heads, usually):
| x | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 1 | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
| 2 | 0 | 2 | 4 | 6 | 8 | 10 | 12 | 14 | 16 | 18 |
| 3 | 0 | 3 | 6 | 9 | 12 | 15 | 18 | 21 | 24 | 27 |
| 4 | 0 | 4 | 8 | 12 | 16 | 20 | 24 | 28 | 32 | 36 |
| 5 | 0 | 5 | 10 | 15 | 20 | 25 | 30 | 35 | 40 | 45 |
| 6 | 0 | 6 | 12 | 18 | 24 | 30 | 36 | 42 | 48 | 54 |
| 7 | 0 | 7 | 14 | 21 | 28 | 35 | 42 | 49 | 56 | 63 |
| 8 | 0 | 8 | 16 | 24 | 32 | 40 | 48 | 56 | 64 | 72 |
| 9 | 0 | 9 | 18 | 27 | 36 | 45 | 54 | 63 | 72 | 81 |
You may fondly recall memorizing all those products (and more, probably, up to 12 times 12) from grade school. Even with a good memory it's not feasible to memorize all the possible products of non-negative integers, so there's another step to the algorithm that extends it. Since we have a positional notation for our number system (the Hindu-Arabic number system), it's possible to line up the factors we want to multiply in columns, multiply the digits individually, and "carry" the excess:
27 x 38 ____ 216 81 ____ 1026
If you don't believe that this algorithm, and the use of number system with positional notation, was a huge breakthrough in multiplication, try the same calculation in roman numerals (a non-positional number system):
XXVII x XXXVIII _________
It would be comforting to be able to solve every problem we face with an algorithm, since even if we couldn't command a computer to carry out the steps, we could put our mind on "auto-pilot" and proceed without worrying about making tricky decisions. There are some general problems with algorithms to consider:
The spectacular success of computers in solving daily problems in fields as diverse as medicine and graphics leads us to suspect that for every problem there is an algorithm that solves it. We just need to be clever enough to discover the algorithm, have enough information, and have enough computer resources to throw at it. This turns out to be untrue in a couple of ways:
I hope the first idea surprises you. Before electronic computers were available, Alonzo Church and Alan Turing were studying the limitations (the flip side of the possibilities) of computers, and independently discovered whole classes of problems that couldn't be solved with an algorithm. If you're comfortable with logic and mathematics, you might look up the halting problem (for example, [Wikipedia]), which has no algorithmic solution.
Here's an informal illustration of an approach to show that there are some problems without an algorithmic solution. Use a technique called proof by contradiction: assume the thing I want to prove is not true, and show that this assumption leads to a logical impossibility. Thus the assumption that the thing I wanted to prove is false, is itself false.
So, let's assume that every problem we can state has an algorithm that solves it (in order to produce a contradiction). So there must be an algorithm that can process any declarative sentence (statement) and tell me whether it's true or false. Feed the following sentence to that algorithm:
...and the algorithm must be able to decide whether the statement is true or false. Ooops!
The amount of information required is limited (it's all contained in the statement itself). What causes the breakdown? The assumption that we can come up with a sequence of steps to decide whether any statement is true or false. There are entire classes of problems that have no algorithm to solve them, but perhaps they seem a bit contrived and artificial. A little more pressing are algorithms that exist, but simply take too long to use.
Suppose you are looking for the page containing somebody's surname in the Toronto phone-book (I know, you use Canada-411). For convenience, we'll assume that the phone book is 2048 leaves (two sides of a page) long. Here are two algorithms that either find a page containing the surname, or conclude that no such page exists:
Linear search: Start at the first leaf
Binary search: (assuming we can accurately tear phone books in half):
The first algorithm requires 2048 repetitions, and the problem doubles each time the size of the phone book doubles. The second algorithm requires 11 steps (provided you have a computer that tears phone books in half). If the phone book doubles, it only increases this algorithm by one step.
For really large phone books (or lists), linear search is an unattractive algorithm...
(material in [Summary of "How to solve it"], [Wikipedia on "How to solve it"]).
So far I have presented ready-made algorithms (good and bad) for you to consider. Considering algorithms in this slick, polished form is a bit dishonest, since many (perhaps most) algorithms were discovered over a long period of time, by many different people, with lots of clumsy modifications before a final, polished version is developed. Although the notion of individual discovery by a "genius" is deeply ingrained in our culture, most important ideas are part of a common, shared intellectual legacy.
As an example, consider the fact that a triangle with sides of length a, b, and c (the symbols a b and c stand for some positive numbers) has a right angle when a^2 + b^2 = c^2. Although the proof of this is attributed to Pythagorus, people were using the idea to survey fields two thousands years before Pythagorus, and various proofs were developed [History of Pythagorean theorem, Wikipedia]. You'll find similar shared authorship for other ideas if you dig enough (check out calculus, for example).
I'm going to take the (perhaps controversial) point of view that ordinary mortals can discover important algorithms, and that this process of discovery can be taught and learned. I'll follow an approach developed by George Polya in "How to solve it". This approach is summarized in four steps (see below). Since one of the steps involves remembering analogous problems, and another step involves looking back on your (perhaps successful) process of problem-solving, keeping a record becomes important. In this course, we'll ask that you keep a journal of your problem-solving for each assignment, as well as an account of one or more problem-solving [sessions]. Here's a sketch of Polya's steps (or see [Polya's own summary]):
You needn't follow the steps in order; you'll probably cycle back and repeat steps, and there's no guarantee of success. However, this has proved to be a good framework for solving some challenging problems, and at least making some progress on some even more challenging problems.