An agent infrastructure for on-demand processing of remote-sensing archives

Advances in data collection techniques and database technologies, such as remote sensing and satellite telemetry, have led to the collection of huge amounts of data distributed among large databases and heterogeneous remote sites. Intelligent and automatic processing of the distributed data and efficiently supporting scientific collaboration between both professional and casual users is a highly demanding task. It is also particularly challenging when the system must cope with active data that is processed on-demand. These requirements have generated an urgent need for more advanced software infrastructure to create, maintain, evolve, and federate these active digital libraries of scientific data. Traditional models of distributed computing are inadequate to support such complex applications. As part of the ongoing Synthetic Aperture Radar Atlas (SARA) Digital Library project, the research presented here proposes a collaborating mobile agent approach to on-demand processing of remote sensing data. The approach, which is based on autonomous data processing and enables different image analysis algorithms to be wrapped as mobile agents, is expected to be an improvement over the static CGI-based interface and inefficient information discovery that are currently used by SARA. We discuss the agent-based infrastructure we have developed. The SARA system allows users to dispatch their compute-intensive jobs as mobile agents. Since the agents can be programmed to satisfy their specific goals, even if they move and lose contact with their creators they can survive intermittent or unreliable network connections. During their lifetime, the agents can also move themselves autonomously from one server to another for load balancing, and to enhance data locality and fault tolerance. The SARA system relies on XML to support agent communications on clusters of servers. Although the examples presented are based mainly on the SARA system, the proposed techniques are applicable to other active archives. In particular, we believe the proposed agent design can be used to dynamically configure distributed parallel computing resources and automatically integrate data analysis in remote sensing systems.