The basic functionality of the LDA is the server side implementation of the RMI interfaces (services). Each local data agent will provide a uniform external interface and a custom internal representation based on the target database type such as Oracle and Sybase. Each local data agent will also be implemented as an independent component in the broad object-oriented architecture that interacts and cooperates with the other components in the entire system. This will eliminate the problem of requiring the international research community to make double updates, one in their own local format and one in the DIF format. The implementation of local data agents as distributed components will also directly benefit the system by providing the much needed scalability and flexibility.

Table 2 lists the interface’s that describe the functionality of the Local Data Agent (LDA). This interface is implemented in the RMI based server so that the methods can be invoked by the Announcer counterpart.

The Local Data Agents proposed here resemble the Mediators ^[YSH96]. The Local Data Agent in this paper is responsible for the incoming propagated traffic of entries, but does not have an active role without the support of the Announcer. Mediators are also employed for schema translations in other heterogeneous systems.

The Announcer is a Java program that is scheduled to run on a daily basis in each of the mirror sites. While this process will run on the source mirror sites, it will call Java RMI based local data agents in the target mirror sites. The methods are invoked via a Stub that serves as a proxy for the remote object. The Announcer obtains a reference to the remote object by invoking the lookup method of the java.rmi.naming object which uses RMI’s URL based naming scheme to specify the remote object. Figure 4 show separate LDA object exist in the server to service every client (Announcer) in the IDN network. MD8 implements asynchronous transactions by storing a schedule of inserts, updates and deletes and broadcasting them through a remote connection to the respective sites.

Assuming the existence of a single LDA object in each site, in the worst case, (n-1) sites are competing to execute remote methods on a given site. Java’s RMI mechanism runs methods within the same object on different threads for different clients. However this does not preclude from the wait on synchronized methods. To execute methods without loss of atomicity, Java’s synchronized monitor construct is used to gain mutual exclusion. Propagations from one site really do not interfere with a simultaneous propagation from another site, even when the target site is the same. Thus, we want the flexibility to execute methods in a synchronized methods within objects for a fixed pair of source and target (A_i, A_k) sites, but at the same time allow the possibility of simultaneous execution of methods for the pair (A_p, A_k).

The GCMD staff validates DIF entries originating at a given site such as San Diego or Budapest, before they are inserted into the local site’s database. Every entry that gets added into the local site’s database subsequently gets queued for propagation to all remote IDN sites. The Announcer program will broadcast the new entry to all the other sites, and awaits their acknowledgement. Such broadcast entries can either be accepted or rejected by the staff at every IDN site. In both cases, an acknowledgement is sent to the originating IDN site with a reason for rejection, in the case of rejection. Updates to a DIF entry can be done at the site that originated that entry or the master GCMD site at Goddard. However, the GCMD entry cannot propagate an addition of a DIF entry to other sites. Similarly, deleting an entry can be done both by the originating site of the entry as well as the GCMD Goddard site.

Asynchronous ^[LPPB] replication is really the best way to accomplish such an implementation, given the global nature of the system and the frequent occurrences of network failures. Asynchronous transaction model fits in well with our implementation since the data in the IDN sites need not be in sync immediately. Hence the temporarily out-of-sync condition is well tolerated. Further more, our design stipulates that only one site (owner of DIF entry or GCMD trump) can initiate and complete an insert, update or delete of a unique DIF entry. This satisfies the unidirectional update propagation property of asynchronous transaction. However, a locking mechanism needs to be in place to cover for the possibility of not-in-sync updates that happen simultaneously at the GCMD site as well as the originating site. Further asynchronous transactions improve response time and prevent global deadlock. The two broad approaches to handle the problem of replica updates ^[YRRSA] in a distributed database system are eager and lazy. The eager approaches update all the replicas of an item as a part of a single transaction. Under lazy propagation, only one replica is updated by the transaction itself and separate transaction runs on behalf of the original transaction at each site at which update propagation is required. Our implementation uses lazy propagation whose benefits are mentioned in paper ^[YHB97].

Relevant information about the server and the other sites with their I/P locations on each IDN site are stored in serialized local storage. This storage is implemented as a Java serialized file. The number of IDN sites to which a given site and server have access can increase over time. The actual list of sites and the I/P addresses can be input by the GCMD server administrator at the time of system installation or can be added later.

The MD8 discussed in this paper will serve the worldwide environmental research community better than the present GCMD system, because it overcomes the technical difficulties of the present system which included the scalability, automation, distribution and maintainability problems. Most important it is based on a federated architecture which automates the process of sharing change research entries.

The MD8 is a scalable system. For example, if a new country wants to enter the IDN circle, then all that is needed is to install the MD8 components at the new country agency’s site. During such installation, the administrator of the new site will simply need to enter template data that will help the server talk to the local database if one already exists. Then the server will notify all other IDN sites of its existence.

The MD8 is a highly flexible system by design. The flexibility is three fold. First, the translator component of the GCMD handles schema differences that may exist across different IDN sites’ data. The translation of local schemas to the common DIF format adds more flexibility to the system. Second, the Heterogeneity component handles differences across commercial database systems such as Oracle and Sybase. Thus the server software can operate on top of any existing database system. Third, MD8 is implemented using Java which allows portability across different operating systems.

The MD8 design is a fully federated system. There are servers executing at each IDN site. About 70 to 75 % of the metadata is fully replicated across the network. Temporary breakdown of a given server will not affect the IDN as data is updated on every site on a regular basis. The frequency of these updates is site specific. The 70 to 75% data replication is a substantial improvement compared to the current 5% to 10% of global data that the GCMD stores in its database.

Very little maintenance is required for MD8. Data validation is a manual process and does need to be done as an evaluation step to include or exclude an entry in the database. However all sites are automatically updated with new data. The ability of the design to automatically propagate and process database changes on different sites goes a long way in low cost maintenance of the system.

The system was implemented using state of art technology, including Java, XML and

The MD8 implementation also ensures a base standard of SQL which can be executed on several different vendors’ relational database implementations.

The Interbase project ^[JO93] integrates pre-existing systems over a distributed, autonomous and heterogeneous environment via a tool-based interface. It supports heterogeneous applications without violating the local autonomy of component systems.

Interbase has two major components: Distributed Flex Transaction Manager (DFTM) and a set of remote system interfaces (RSI’s). The DFTM interprets and coordinates the reliable execution of global transactions over the entire system. It also provides a unified and flexible interface, the IPL language, with which system programmers can specify the data and control flow of a global transaction. This paper extends the Interbase work. MD8 differs from the Interbase implementation in that it uses an object-oriented approach.

DCOM^[DCOM] ,CORBA ^[CORBA] and Java RMI are all protocols for connecting distributed systems. DCOM will work only across Windows NT or Windows 95/98 platforms. Significant research on implementing CORBA based object-oriented multi-database systems (MOOD, MIND) is discussed in ^{[DDO96, DDO98,EGO95]}. This work employs the CORBA standard for integrating heterogeneous systems across networks. MD8 employs Java RMI as means of connecting IDN sites across networks and exchanging information. Java allows ease of deployment irrespective of the presence of an ORB in a given site.

The papers ^[SDE98] and ^[PADE98] provide insights into addressing the problem of database heterogeneity in a distributed internet-based environment using the XML script language. The latter mentions that the creation of XML wrappers will be necessary to make XML the universal mechanism for addressing heterogeneity. While XML provides a presentation mechanism for query results of heterogeneous sources, the reverse process, which is writing back to heterogeneous sources, is not well established in either research paper. The Garlic project at IBM ^[MPI97] describes wrappers and how they can provide transparency to underlying data. We have used wrappers to encapsulate the DIF information.

proactivity, autonomy and interactiveness is discussed in paper ^[PCL99] . The use of Java technologies gives our implementation the flexibility to run on various platforms while the usage of the DIF schema to encapsulate data provides autonomy to the underlying database system.

Paper ^[LPPB] introduces the need for asynchronous transaction and compares it with synchronous transaction. It goes further in defining the basic requirements for asynchronous transactions based on which our implementation is designed. In our implementation the interaction between LDA and the Announcer is asynchronous. Even though our system uses the asynchronous transaction model, we have made provisions to handle situations of repeated transaction failure.

Author evaluates the performance of 3 gateways - Miracle, DJ, EDI/S mainly for database operations. Our implementation runs every client’s request on its own thread space in the server. This enhances the overall performance of the system both in terms of speed and scalability.

The need for a new version of the GCMD software is motivated by the advances in technology (object-oriented architecture, Java programming language, middleware technologies) and a concerted effort on the part of NASA to expand the scope of GCMD. The MD8 is a federated system that employs some of the latest technological tools. Our solution highlights the use of Java, RMI and other Java technologies to implement a truly scalable, distributive and cost-effective system.

In the development of the MD8 system, the GCMD team considered several development tools very carefully before selecting the Java platform. The disadvantage of Java is in the slow execution speed of the byte code, but for the GCMD application speed is not a major concern. The GCMD application is based on the lazy replication model and more importance is attached to the completeness of a transaction rather than speed. We need a system that is totally heterogeneous and that can reliably do distribution with minimal blocking on method invocations.

One important aspect of the system is the use of serializable files for distribution and propagation. Some real speed issues in traversing through serialized I/O in a Java RMI based server system have been overcome by substituting serialized file I/O with database counterparts in sites where relational databases are available. However, it is important to note that there are some sites in the IDN network that do not use any conventional database systems.

The current implementation utilizes Java’s serialization capability to transport objects across the network. In future, we want to explore the use of XML as a way to encapsulate information and utilize it as a platform for handling heterogeneity. Further we would like to integrate systems developed in a non-java programming environment such as C++.

[DDO98] Building Interoperable Databases on Distributed Object Management Platforms, CM, A.Dogac, C.Dengi and T.Ozsu, Feb 1998.

[EGO95] Experiences in Using CORBA for a Multidatabase Implementation, Ebru Kilic, Gokhan Ozhan, C.Dengi et al, SRDC, METU, Turkey, 1995

[FRK98] The Heterogeneity Problem and Middleware Technology: Experiences with and performance of Database Gateways. Fernando de Ferreira Rezende, Klaudia Hergula, Dept.CAE-Research (FT3/EK), Diamler Benz AG – Ulm-Germany, Proc of 24 VLDB conference New York, USA, 1998.

[GM79] Proving Consistency of Database Transactions. Georges Gardarin and

Michel A. Melkanoff, VLDB 79.

[JO93] USENIX, Journal of USENIX Association, The Implementation of Cooperative Mechanisms among System Components in a Heterogeneous Multidatabase Environment, by Jiansen Chen, Omran Bukhres et al. Computing Systems, vol.6 no.3, Summer 1993.

[PCL97] The Network as a Global Database: Challenges of Interoperability, Proactivity,

interactiveness, Legacy. Peter C. Lockemann et al. Proc of 23 VLDB Conference Athens,Greece, 1997.

[SDE98] Resistance is Futile: The Web will assimilate your database, Susan Malaika, DE Bulletin, June 1998.

[YHB97] Replication and consistency: Being lazy helps sometime -Yuri Breibart and Henry F. Korth, Bell Laboratories, PODS 97 Tucson Arizona USA.

[YRRSA] Update Propagation Protocols for Replicated Databases. –Yuri Brietbart, Raghavan Komondoor, Rajeev Rastogi, S Seshadri, Avi Silberschatz.

[YSH96] Object Fusion in Mediator Systems - Yannis Papakonstantinou and Serge Abiteboul and Hector Garcia-Molina, VLDB 96.