Extending the Information Power Grid Throughout the Solar System

Abstract

Introduction

IPG

• Applications -- programs that use IPG resources to do useful work. Many scientific and engineering applications could benefit from the uniformity provided by the IPG. Examples include aerospace design simulations, flight software testing, image analysis, database mining, telescopic asteroid search, etc.
• Programming Tools -- software that makes it easier to develop IPG applications by hiding some of the complexities of the underlying structure. Toolkits provide flexible frameworks for creating applications from modules, some of which may reside on remote computers. Object oriented programming is a particularly powerful mechanism for hiding complex lower-level structure.
• Services -- software  providing uniform security, authentication, resource scheduling, resource access assurances, data management, fault management, and other facilities provided by conventional operating systems on single computers. Grid Common Services are the heart of a "virtual operating system" extending to all IPG physical resources.
• Physical Resources -- computers, visualization environments, mass storage devices, instruments, and networks that provide IPG capabilities. Specialized software on each resource makes it IPG compatible. Current physical resources include seven SGI Origin 2000 supercomputers, a 100 terabyte mass storage system, a LINUX cluster, four Sun SparcStations, and a 250 workstation Condor pool. These resources are distributed over three NASA centers (Ames, Glenn, and Langley). Condor (www.cs.wisc.edu/condor) is a cycle-scavenging batch system which runs jobs on desktop workstations at night, weekends, or other times the workstations aren't in use.

Globus and the IPG

Legion and the IPG

Legion strives to achieve ten design objectives:

1. Site autonomy
2. Extensible core
3. Scalable architecture
4. Easy-to-use, seamless, computational environment
5. High-performance via parallelism
6. Single, persistent, name space
7. Security for users and resource owners
8. Management and exploitation of resource heterogeneity
9. Multiple language support and interoperability
10. Fault tolerance

• Each hardware or software resource is a Legion object. Legion objects are independent, active objects that communicate with one another via asynchronous method calls.
• Class objects are responsible for creating new instances, scheduling them for execution, activating them, and providing metadata about them.
• Users can define and build their own class objects; therefore, Legion programmers can choose and change the mechanisms that support their objects. Legion contains default implementations of a number of classes.

IPG Research

• Co-scheduling -- where several resources must be accessed simultaneously to accomplish a given task.
• Reservations -- where a resource is scheduled for a particular time. Current batch schedulers usually run jobs at the first opportunity.
• Network scheduling -- while CPU and memory resources are routinely scheduled in today's IPG, network resources are not.  Since remote spacecraft must share resources such as the Deep Space Network without communicating directly between themselves, this capability is crucial.

Given the above description of the IPG as a way to make first-class computing, communication, data, and technology resources available to all comers, what do we gain by extending the IPG infrastructure throughout the solar system? The short answer is that it's like extending the Internet to spacecraft and installations throughout the solar systems and making these first-class resources available to both people and machines. However, the high cost of Earth-to-orbit launch prohibits large-scale cost-effective solar system exploration. Fortunately, the IPG may be able to help reduce launch costs.

Launch

Not only is the cost of access to space very high and failure prone, access has not improved very much over the last three decades.  Indeed, measured in person-hours per ton to orbit, the 1960's era Saturn V was significantly less expensive than today's launchers [Wertz and Larson 1996] perhaps because large lift capacity tends to be cheaper per kilogram. The Saturn V lifted the SkyLab space station into orbit with a single launch, in stark contrast to the dozens of launches required to lift the International Space Station (ISS) by today's family of launchers. SkyLab had perhaps half the pressurized volume and a quarter the mass as the ISS will have at completion. This lack of, or even negative, progress in launch vehicles must be decisively reversed for the space program to move beyond a very small number of incredibly expensive missions. Launch vehicle improvement is the issue for space development.

Recent reviews of problems encountered by the space shuttle both before and during launch [SIAT 2000] discovered major opportunities for the application of information technology in general, and Grid capabilities in particular. In addition, a surprisingly large fraction of launch failures are directly attributable to information technology failures.  For example, the destruction of the vehicle and payload in the second Sea Launch mission was apparently caused by a software error.

By 2020, NASA intends to achieve $220/kg to orbit with a 0.01% failure rate. This should be sufficient to support high-end space tourism, driving travel costs to much less than the current ~$20 million for a tourist ticket to the Mir space station. Launch vehicles often operate very near the physical limits of materials and components. To operate near these limits safely and efficiently requires an intimate knowledge of the current state and history of each vehicle and all support systems, which in turn requires first-class, integrated data systems. Recent independent reviews have indicated that trend data are difficult to extract from existing shuttle data systems, and that some data are missing or incomplete [SIAT 2000].  Also, three recent space failures (Sea Launch second flight, Mars Polar Lander, and Mars Climate Orbiter) were caused, in part, by software or information processing failures. There is no doubt that the personnel involved are dedicated and capable, but the data systems are not what they could be.

Potential IPG Contribution

• Human and machine readable digital databases: materials, components and system interactions must be fully characterized and understood to insure launch success and to analyze launch failures. Since most current launch vehicles were developed before the availability of large and inexpensive digital data storage, where these data exist at all they are often scattered among incompatible databases and in some cases are not even machine readable. Consider the experience of the NASA Space Shuttle Independent Assessment Team:


"In order to assess the root cause of wire damage, trends were needed that distinguished the type of wire damage (i.e., insulation damage, exposed conductors, conductor damage) as a function of location of occurrence in the Shuttle. ... To provide this information, a team of 10 engineers and 3 quality inspectors worked for 1 week reviewing Problem Reports in the Kennedy Space Center Problem Resolution and Corrective Action system. This intense effort was needed because the data contained in the system lacked standardization and fidelity and needed to be assessed and interpreted by the engineers in order to be meaningful. In some cases, the data did not exist or could not be resurrected." [SAIC 2000] . Thus, a trend report that a first-class data system should have generated automatically in a few minutes required approximately three person-months to produce.

The space shuttle Challenger was destroyed and seven astronauts killed in an explosion when O-rings in a Solid Rocket Booster failed.  The O-ring material was temperature sensitive and launch took place at ~35 degrees F below any previous launch experiencing no O-ring damage. [Leveson 1995] reports that  O-ring partial failures were increasing over time in the previous 24 flights but this trend was never extracted from the anomaly database.  Of course, the O-ring problem was well known to the SRB engineers but their judgment was overridden. [Tufte 1997] provides evidence that better display of the temperature-failure relationship, which would have been easier with a better data system, might well have convinced management not to launch at such low temperatures.

In a system such as the shuttle involving three major NASA operational centers plus contractors, a single centralized database would be nearly impossible to implement.  Thus, a widely distributed set of archives that can be accessed securely by both people and software, the sort of archives envisioned by the IPG, is necessary. Since NASA personnel occasionally report difficulty getting copies of data developed under NASA contract, integration of data developed with government funds into an accessible (Webbed) distributed archive should perhaps be required by all NASA contracts. IPG can provide enabling infrastructure for this, although this not the major issue. The major issue is metadata standards and standardized data encodings, all packaged in self describing objects such as XML (www.xml.org) provides. If the IPG were adopted, however, it would likely spur standardization.

Even if new RLV efforts are not as widely distributed as the space shuttle project, IPG-like wide area capabilities are important. Crucial data on material and component aging is often gathered in other operational environments, such as commercial and military aircraft.  Also, manufacturers are often reluctant to transfer their data to external archives. Thus, a complete digital database of all relevant data would need to include secure access to databases maintained by component and materials manufacturers and their customers. These are unlikely to be located at the launch site.

• Large computational capabilities: once all relevant data are machine accessible, numerous opportunities for computational analysis will present themselves.  The large computational capabilities being integrated by the current IPG effort can be applied.  Beyond the traditional applications, such as computational fluid dynamics and finite element analysis, automated analysis of failure modes and automated software testing, both discussed below, can benefit from supercomputing capabilities.
• Model based reasoning: model based reasoning (aka model based autonomy) is currently used for real-time control of launch vehicles.  This technique involves modeling launch vehicles as a set of interacting finite state machines.  Model based autonomy simplifies flight control software since launch vehicle engineers build a model of their system that is examined by non-mission specific software to diagnose system state and generate commands to the vehicle (see figure 1). Altairis Aerospace Corporation used model based autonomy for the Delta II cryogenic propellant loading sequence, various aspects of Delta IV, and the Conestoga launch and subsequent failure analysis. NASA Ames is developing X34 and X37 flight experiments using Livingstone, model based autonomy software with more sophisticated diagnostic capabilities. To generate the failure modes used by the models requires large computer resources to simulate the physical operation of components.  Also, given sufficient computer resources, the finite state machine models developed for real-time control could be used for an extensive analysis of failure modes created by global interactions.  Finally, in principle, model based reasoning and other artificial intelligence techniques could be extended to predict component failures in reusable launch vehicles and automatically signal maintenance personnel to replace or refurbish components before failure. These predictions will require very substantial computational resources which could be provided by an IPG.


Figure 1 shows a finite state machine representation of a valve with four states: open, closed, failedOpen and failedClosed:

Figure 1: finite state machine representing a valve

• Wearable computers and augmented reality for technicians: wearable computers are physically integrated directly into a user's clothing.  Augmented reality refers to see-through head mounted display technology that projects computer generated annotation onto the real world.  Hands-on shuttle technicians are cut off from computing technology since loose items are prohibited, in order to avoid dropping things on fragile flight hardware.  Even wedding rings must be taped to technician's fingers.  Wearable computers would give technicians direct access to shuttle databases in the IPG environment.  Shuttle processing also requires many inspection tasks to determine how shuttle hardware state has changed from pre-flight to post-flight. Augmented reality could be used to project preflight status directly onto the hardware being examined, allowing rapid and accurate determination of any changes.  If a video camera were integrated into the augmented reality system, shuttle technicians could take pictures of questionable hardware for automatic analysis and comparison with pre-flight images by IPG computing resources.
• A software agent architecture for continuous examination of the database: once a machine accessible database of all of the data exists, then IPG computing resources can be used to automate inspection, trending, and analysis tasks using software agents.  As the amount of computation that could be consumed by such database examination is essentially unbounded, one approach to gaining the necessary CPU hours without excessive hardware cost is to integrate all NASA desktop computers and workstations into a set of Condor pools integrated with IPG technology.  Since there are many thousands of such computers, and each should be available approximately 17 hours per day based on University of Wisconsin experience, enormous computing resources could be applied to database examination.
• Multiuser virtual reality optimized for launch decision support: launching a rocket requires a series of critical Go/ NoGo decisions, some of which must be made very quickly and require substantial expertise. Such expertise may not always be available at the launch site. When making such decisions it is vital that all decision making participants have a common and accurate view of the current situation. Multiuser, widely distributed, immersive virtual reality technology such as that employed in the NASA Ames Virtual Windtunnel (www.nas.nasa.gov/Software/VWT) can place distributed decision makers in a common information space which, together with voice communication, can assist with critical time-dependent decisions based on data provided by distributed IPG-integrated archives and sensors.
• Automated computationally-intensive software testing: software testing has emerged as a major problem in aerospace design and operations.  Modern software has far too many states to be exhaustively tested. Developing testing schemes is extremely manpower intensive and error prone as the recent Mars Polar Lander loss demonstrated.


One approach to automating software testing is random testing, although only certain aspects of a software system can be exercised.  In this mode, software is subjected to randomly generated inputs.  When the program crashes, the relevant data is collected and supplied to the developer.  There is no practical limit the CPU hours that can be devoted to such testing.  These tests are a good candidate for a NASA-wide set of IPG integrated Condor pools. This would turn every desktop PC and workstation at every NASA center into a software testing resource.  Intel uses a similar scheme to test hardware designs.  The Intel system runs circuit testing software on thousands of CAD workstations distributed around the globe when they are not otherwise engaged.

The effective state space of a control program can be reduced a great deal using model based reasoning because the underlying finite state machines have far fewer states than an equivalent traditional program. Indeed, the state space of some subsystems is small enough for exhaustive testing. Thus, a second computationally intensive software testing task is automated state exploration for model based reasoning programs.  Theorem proving requires large CPU resources and, especially, large memory spaces.  Large single-address space parallel IPG supercomputers such as the SGI Origin 2000 (hundreds of gigabytes of physical memory) should be quite useful.

Solar System Exploration

Model based autonomy has firm theoretical roots (see, for example, [Manna and Pnueli 1991]) and experiments in spacecraft control are progressing.  The NASA Ames on-board Remote Agent Software, including the Livingstone diagnostic engine, controlled the Deep Space 1 spacecraft for approximately two days in 1999. While a thread bug in the executive software cut short the experiment, results suggest that autonomous spacecraft using this technique may be feasible. Also, Altair Aerospace Corporation's implementation of model based autonomy is due to be installed on the WIRE spacecraft for an engineering test in the near future. Autonomous spacecraft, which may occasionally require IPG resources to perform large calculations infeasible for onboard processors or access Earth-bound data, are a significant requirements driver for a solar system wide IPG. A second set of drivers are long latencies, low data rates, and intermittent communications.

Data retrieved from solar system exploration spacecraft are being placed on the World Wide Web. A particularly interesting example is the NASA Ames Lunar Prospector site (lunar.arc.nasa.gov).  NASA intends to extend these beginnings into a "virtual solar system" where researchers and ordinary citizens can examine solar system data in an intuitive and easy to use manner over the Net.  This vision requires integration of the spacecraft, landers and rovers gathering data, Web accessible data archives, and computational facilities for converting raw spacecraft data into four-dimensional (three spatial dimensions plus time) data-driven models of the solar system. This is a computationally intensive task and the IPG may be able to help.

IPG and Solar System Exploration

To minimize exploration spacecraft antenna size and provide communications to spacecraft on the other side of the Sun, we propose a number of communication satellites scattered along Earth's orbit. With a sufficiently large number of communication satellites, line-of-sight laser communication between them can provide communication with Earth and even between exploration satellites.  Thus, the communication satellites form a massive extension of the Deep Space Network and perform a function similar to the Internet backbone.  Scheduled reservations of the communication satellites are necessary to point their high-gain antennas at the exploration satellite needing communication at any given time. Reserved co-scheduling is necessary for communication satellites to point their high-gain antennas at each other to pass messages to and from Earth.

Each spacecraft, lander, and rover may be represented by a software object.  Spacecraft, landers and rovers must be represented by terrestrial mirror objects (proxies) to hide latencies and to represent the vehicles when they are not in communication with Earth. These proxies must know the schedule of their remote reflections so that co-scheduling and reservations may be implemented properly.  Once data have been safely stored in archives, normal Web access should be adequate for browsing, however more controlled access using IPG security will be necessary for applications that read the data, calculate more useful versions (e.g., mosaics of images), and insert the results back into the archive. Thus, the IPG becomes a set of (sometimes) access-restricted high-performance computing, data archive, and special instrument areas of the Web. This amounts to a democratization of high-end resources making them available to a much wider audience, reducing barriers to research and technology advancements, and increasing public support

It is reasonable to assume that the exploration spacecraft will be autonomous but require occasional large-scale processing because of  the limited capacity of their on-board computers.  Large-scale processing needs might include trajectory analysis, rendezvous plan generation, docking plan generation, surface hardness prediction for choosing sampling sites on larger asteroids, etc. After passing a request to Earth for large-scale processing, Earth bound high-performance CPU resources must be reserved to insure that the processing results are available the next time the requesting satellite is in communication. Most large-scale processing should be kept on Earth to take advantage of new developments in computer science and hardware production.

No matter how good the automation software, it is difficult to imagine that no human intervention will be necessary to operate a complex exploration spacecraft, at least in the near future. However, the finite state machines used for model based autonomy may include the unknown state.  Spacecraft may be programmed to go into safe mode and contact Earth when important subsystems enter the unknown state. Indeed, this is normal behavior for Altairis models. This Earth contact could trigger a message to one of three Earth stations staffed by ground control experts.  The Earth stations could be placed around the globe such that each facility is only open during normal working hours while still achieving 24-hour coverage. These Earth stations might effectively appear to spacecraft to be IPG nodes that act as proxies for human controllers.

Images and data captured by exploration spacecraft must be returned to Earth for additional processing and archival.  Exploration spacecraft will spend large periods in transit followed by intense periods of data production during close encounters. Thus, network reservations on the communications satellite infrastructure are necessary as well as a mechanism to insure that archival space is available on Earth when the data arrive.  For prompt processing into usable form, CPU reservations are also necessary.  Thus communication, archival, and CPU co-scheduling is necessary.

Current IPG implementations assume nearly-continuous, high-bandwidth, low-latency communication.  These assumptions are broken in a solar system wide IPG.  Instead, large latencies, low data rates, and intermittent connectivity are typical of deep space communications. The Internet Domain Name system, which is an essential component in the operation of Grids and all other Internet applications, has an architecture that is intended to fail soft in the face of network partitioning: DNS is a partial state data manager capable of autonomous local operation in the face of network failures. This provides an experience base for attacking problems associated with a solar system wide IPG.

Solar System Challenges for the IPG

In the opposite direction, satellites requiring Earth-bound IPG resources, such as large-scale processing, can inform their proxies of the needed computations and the next communication time.  The proxy can then request the calculations and store the results until the satellite has a scheduled communication window.  As the number of spacecraft grows large and operations move farther from Earth, particularly on the other side of the Sun were direct communication is impossible, communication/computation satellites may be placed in various orbits to support exploration spacecraft without sending messages all the way to Earth. The spacecraft may form the Solar System's IPG backbone much as high-speed links and routers form the current Internet backbone. This backbone could also store and forward important information on space weather, for example, solar flares observed by near-Sun satellites.

If these problems are solved and large numbers of inexpensive exploratory spacecraft are integrated into a solar system wide IPG, it may be possible to create a market economy to drive the exploration of the solar system.  A market economy requires a large number of producers and a large number of consumers, no one of which can control prices. If spacecraft and launch are inexpensive, relatively small organizations could operate exploration satellites.  With the IPG keeping track of location and capabilities, a system of large numbers of relatively small university grants could provide the funds to purchase observations.  In this model, the scientists do not purchase entire spacecraft, but rather a portion of a spacecraft's capability. SpaceDev, Inc. (www.spacedev.com) is pursuing a similar, but necessarily more limited, business model. SpaceDev is trying to fund a deep space mission by selling space/power/etc. on the spacecraft buss as well as by selling data.

A typical Solar System IPG interaction might look something like this: Dr. Potter wants a small probe to sequentially visit and sample 10 Near-Earth carbonaceous asteroids with diameter < 100 m and send back to Earth information about the samples. The probe must carry a sample analysis system weighing 10 kg, with volume 0.1 x 0.1 x 0.2 m, drawing 50 w power during operation, 2 hour operation per asteroid, and data transfer of 10 Mbyte per sample. He wants minimum cost for a mission between 6/1/2010 and 12/1/2011. The IPG suggests the following options:

• Option 1:
• Purchase: MagSail3 (with microthruster) by Space Cadets: $100,000
• Launch: Kilamanjaro EM Rail Launcher, 7am on 6/5/2010: $250,000
• Asteroid sampling: 4/7/2011 - 8/26/2011
• Detailed flight animation: <hyperlink>
• Data relay: NASA repeater 14: $10,000
• Total cost: $360,000
• Potential customers for unused capacity: Dr. Dumbledore, Dr. S. Black, and Dr. Granger.
• Option 2:
• Lease: LightSail156 from SpaceDev: $40,000
• Asteroid sampling: 6/7/2010 - 11/3/2011
• Detailed flight animation: <hyperlink>
• Data relay: NASA repeater 14: $8,000
• Total cost: $48,000
• Limitation: only 8 known asteroids may be visited in this period
• Space Telescope 12 can perform a search for additional asteroids along the flight path for $5,000 with a 65% chance of finding three or more asteroids with a diameter < 100 meters. An additional $2,000 will be necessary to characterize each asteroid found.

Conclusion

RLV improvements envisioned by NASA and assisted by the IPG may lead to huge numbers of exploratory robotic spacecraft.  While current information technology approaches are marginally adequate for the small numbers of spacecraft in orbit today, these techniques will not scale well. Spacecraft could be integrated as nodes in the IPG, effectively extending the IPG into the vast reaches of our solar system. This would exercise emerging IPG capabilities such as reservations, network scheduling and co-scheduling, and require the development of proxy and other techniques for hiding the large latencies, low data rates, and intermittent connectivity typical of deep space exploration. This could conceivably lead to a true market economy for the exploration, and perhaps exploitation, of the vast resources of the solar system.

References

[Leveson 1995] SAFEWARE: System Safety and Computers, Nancy G. Leveson, University of Washington, Addison-Wesley Publishing Company.

[Lewis 1996] John S. Lewis, Rain of Iron and Ice: the Very Real Threat of Comet and Asteroid Bombardment, Addison-Wesley Publishing Company.

[Manna and Pnueli 1991] Zohar Manna and Amir Pnueli, "The Temporal Logic of Reactive and Concurrent Systems," Springer-Verlag.

[MCO 2000] Report on Project Management in NASA, Mars Climate Orbiter Mishap Investigation Board, March 13, 2000.

[Nelson 1993] M. L. Nelson, D. T. Britt, and L. A. Lebofsky, "Review of Asteroid Compositions," Resources of Near-Earth Space, John S. Lewis, M. S. Matthews, M. L. Guerrieri, editors, the University of Arizona Press, Tucson and London, pages 493-522.

[Rabinowitz 1997] David L. Rabinowitz, "Are Main-Belt Asteroids a Sufficient Source for the Earth-Approaching Asteroids? Part II. Predicted vs. Observed Size
Distributions," Icarus, V127 N1:33-54, May 1997.

[SIAT 2000] Report to the Associated Administrator Offices Space Flight,  NASA Space Shuttle Independent Assessment Team, 7 March 2000.

[Tufte 1997] Edward R. Turfte, Visual Explanations: Images and Quantities, Evidence and Narrative, Graphics Press, Cheshire, Connecticut.

[Wertz and Larson 1996] James R. Wertz and Wiley J. Larson, Reducing Space Mission Cost, Microcosm/Kluwer.