Starfish is a fault tolerant grid computing platform that requires no configuration of members. An easily extendable Algorithm interface allows for the implementation of many types of problems. For example, the base download comes with a large mandelbrot image generator, an MD5 hashed value guesser, and a genetic sequence aligner.
Main features include:
  • Serverless operation: In starfish all members of the grid are equal, there is no centralized server or controlling node. Any node can be removed and processing will continue. You can start a problem and retrieve results from anywhere.
  • Zero config: Starfish is designed to be run with no configuration of grid members. However, a full range of configuration options are available for extreme scale or other specific needs problems.
  • Fault tolerant: Any grid member can leave or join the grid at any time. Algorithm data as well as finished results are distributed redundantly to other grid nodes in real time, so the work of any grid member can't be lost by it leaving the grid.
  • Simple programming interface: Implementing your algorithm involves writing three methods.
  • Dynamic code deployment: Any Algorithm you write for Starfish is distributed dynamically to the grid along with any jars your Algorithm requires.
  • Scalability: Most communication between grid members is direct and switchable so network traffic won't explode with large grid sizes.
  • Built in virtual memory: Problem size is not limited to a grid member's memory allocation. Objects in use overflow onto disk automatically and transparently.
  • Distributed garbage collection: Resources no longer needed by any active Algorithm are removed from all grid members automatically.
  • Problem queueing: Queue multiple problems for execution, they will be executed in order.
  • Platform independence: Take advantage of any machine available, the same Starfish install can run on Windows, Linux, Mac OS X and Solaris.
  • Dynamic GUI: Overall algorithm progress as well as grid health and stats displayed in real time.

Starfish supports three major types of Algorithms:

  1. Pure computation: Generally no input data other than simple parameters, potentially large results. Algorithm must complete all work segments. (see Madelbrot Set)
  2. Analysis: Generally large input data (i.e. genetic data), potentially large resulting data (i.e. list of close matches). Algorithm must complete all work segments. (see Sequence Alignment)
  3. Search: Any size input data, very small resulting data (i.e. found value at location n). Algorithm will terminate as soon as value is found. (see Hashcode Guesser)

Starfish makes uses the following projects:

  • JGroups for grid membership and communication.
  • MegaMap for transparent virtual memory (MegaMap was originally developed as part of StarFish then seperated for others to use).
  • Piccolo provides a low level Java2D framework for the GUI.
  • JAligner is used for the sequence alignment sample algorithm.
  • GUI layout based on research by HCIL, University of Maryland.