PC clusters as a platform for image processing: promises and pitfalls
Abstract
The growing acceptance and maturity of "desktop supercomputers" consisting of clusters of inexpensive
personal computers brings unlimited image processing capability within the reach of all. Clusters can be
composed of whatever miscellaneous computers are available. New units can be added without
discarding the old, and specialized capabilities residing on only a minority of the computers can be
utilized optimally. However, the technical skills required to configure PC clusters remain an obstacle to
their widespread adoption. This paper provides an overview of fundamental cluster concepts and
introduces a "plug-and-play" cluster implementation designed for use with the Dragon Image Processing
System.
Introduction
Remote sensing and GIS are characterized by a need to perform computationally-intensive operations
on large data sets. Full-scene image processing requires operating on tens of millions of image data
points per scene, while GIS operations such as viewshed analysis or calculating optimal routes may
involve evaluation of candidate solutions whose number increases exponentially with the spatial extent of
the problem area.
The desktop personal computer has grown from a curiosity to become the norm for much of the world's
computing. Today, it offers the lowest cost, on a dollars per floating-point operation basis, of any
computing architecture. Nevertheless, it still can be inadequate for the most demanding tasks. The
difficulty is not a shortage of memory or disk storage, nor is it (in all cases) a lack of suitable software.
The problem is one of elapsed time. A program which takes hours, days, or weeks to execute is simply
too slow to be useful.
The simple answer, of course, is to use a faster computer. However, once one moves beyond the realm
of the personal computer to specialized workstations or mainframes, the price can go up much more
rapidly than the performance.
An attractive alternative, which we examine in this paper, is to group and coordinate a collection, or
cluster, of inexpensive PC's to cooperatively perform the computation. Ideally, the cost of this approach
should be directly proportional to the desired speedup (although in practice this ideal cannot be achieved
completely). Other advantages such as ease of procurement and maintenance, and the possible ability
to reuse existing software, are apparent.
Understanding the problem: Some examples
To put the above into perspective, let us offer some concrete examples. The numbers cited are not too
important; your computers may be faster or slower than ours. It is the comparisons that are important..The First Example
Let's say that I have just bought the fastest, newest PC that I could get. Let's say that I have a collection
of old music (perhaps 1000 pieces) that I want to convert into MPEG format so I can play them while I'm
mountain climbing. However, I rapidly discover that each piece (representing one phonograph record)
takes my computer about half an hour to convert. All told, I'm going to be running that computer full-time,
24 hours per day, for most of a month to convert all that music.
We Westerners are not a patient lot, so what can I do to speed things up? I've already got the fastest PC
available. I can't really afford a Silicon Graphics workstation or IBM mainframe, and even if I could, it
probably wouldn't support my software.
However, if I could just put two or three PC's to work on it, I could (in principle) get it done in a week.
Twenty PC's would get it down to a day. (What's the price of twenty PC's compared to a mainframe? I'm
not sure, but I'll bet the PC's would be cheaper.)
Taking this a step further: suppose that I do purchase a dozen new PC's for this purpose. My results are
so impressive that my friends start coming to me with requests to do these audio conversions for them.
Now, even all my computers together are too slow. However, I can just go buy some more. The old ones
aren't wasted; they're just added into the pile.
A Second Example
Moving into the remote-sensing domain, let's assume that I am doing an astronomy study on some
remote galaxies. Because the light is so faint, I need to extensively pre-process each image. In order to
minimize the chance of getting bad results and having to repeat the measurements, I want to process the
images as they're being received from the sensor. In effect, I want to complete the processing on one
image before starting the next.
Let's assume that I acquire a new image every ten minutes, but processing an image on a single
computer requires an hour. The problem, then, is that I have a single task which takes six times longer
than the time window in which I have to accomplish it. If I can distribute the processing burden of this
task across six (or, to be safe, seven or eight) PC's, I can solve this problem.
