JAVA REAL-TIME DISTRIBUTED PROCESSING OVER ATM NETWORKS WITH CHORUS/OS Christophe Lizzi CS Technologies Informatiques, 3 rue Maryse Basti´e, 69500 Bron (F RANCE) E-mail: [email protected] CNAM, lab. C EDRIC, 292 rue St -Martin, 75141 Paris Cedex 03 (F RANCE) E-mail: [email protected] Abstract - We present an architecture that addresses the heterogeneity of distributed environments by enabling Java distributed real-time processing. Our framework can be used to build real-time Java virtual manufacturing devices that operate over ATM networks. The Java virtual machine is hosted by a custom version of the Chorus distributed real-time operating system. Provided extensions include Earliest Deadline First scheduling and real-time IPC service over ATM. This paper summarizes the problems raised by Java in a real-time context and introduces our architecture. Then, we detail the improvements made to the JVM and the way it uses our real-time services. Transfer rates up to 97 Mbps are substained between distributed Java objects within our testbed cluster.

1. INTRODUCTION Design and deployment of distributed process control systems is complex because of the heterogeneity of the computing nodes and because of the various characteristics of the interconnection networks. By contrast, the hardware abstraction granted by the Java language and its virtual machine, combined to the portability of the Java bytecode allows the introduction of portable Virtual Manufacturing Devices (VMD) [27], as defined by the MMS model [34]. Then, unlike most local area networks, ATM provides a communication service with performance guarantees, associated to high bandwidth availability and enhanced scalability. ATM may thus constitute an alternative to specialized fieldbuses in SCADA (Supervision Control And Data Acquisition) systems. Indeed, its scalability, combined to its ability to convey both real-time supervision commands and multimedia streams, enable the use of high quality audio and video flows useful in quality control and in failure detection. Audio streams may notably help the classification of failures, while video streams contribute to the geographical localization of faulted process components within large automated systems controlled from remote locations. We thus present in this paper a processing architecture that addresses the problem of heterogeneity of hosts and networks by using Java and ATM as core technologies to build distributed real-time systems, while providing a high-level of abstraction, services and performances. This article is structured as follows. The first section introduces Java, our virtual machine and the underlying operating system we use. The second section describes the specific properties of ATM we rely on, while the third one presents our real-time Java framework. Then, pre-

liminary performance results are summarized. 2. JAVA IN REAL-TIME SYSTEMS Java has been introduced by Sun Microsystems in 1995 [11]. Since this date, this programming language has received a very high attention from software industry actors and from researchers. Its portability leads Java to be a privileged technology to develop distributed applications. 2.1 The Java language Java is an object-oriented programming language completed by a set of standard classes that provide the core programming interfaces [15]. One of the most interesting feature of the language is the automatic garbage collection that dispenses the developer to explicitly release instantiated objects at the end of their lifetime, avoiding memory leaks [32]. The Java framework also includes a runtime environment termed as a Java Virtual Machine (JVM) [17]. Java programs are compiled into portable bytecode that can be interpreted by the JVM. The Java programs and the foundation classes shipped with the JVM interact with the underlying operating system using the Java Native Interface (JNI) [30]. The JNI interface grants Java programs to perform calls to legacy code written in other languages than Java. As a consequence of the use of a portable binary format, Java programs can be executed on any hardware architecture, by any JVM that supports the required classes. In order to improve runtime performances, enhanced JVMs also include a just-in-time (JIT) compiler in charge of on-the-fly translation of Java instructions to native operation codes.

2.2 Real-time Java Real-time applications have time constraints and therefore expect performance guarantees from their execution environment. In the context of Java-enabled real-time systems, the embedded JVM also has to exhibit a runtime behavior suitable for real-time applications. Issues related to real-time Java computing are reviewed in [22]. Such problems notably include:

 task scheduling and synchronization. A better model of real-time tasks should be introduced, enabling the applications to express their temporal requirements. A predictable scheduling policy should also be provided and coupled to priority inheritance protocols for thread synchronization;  garbage collection. The garbage collector should ensure that objects can be allocated at any time without interfering with the execution of real-time and be responsible for missed deadlines. As described in [6], a non disruptive collector might use an incremental algorithm that runs for a fixed amount of time;  communications. Support for quality of service requirements should be added to the networking classes. Moreover, in a distributed context, the delay required to perform remote method invocations (RMI) [29] should be predictable. There have been different initiatives to promote the use of Java in real-time systems. The most advanced project is PERC, a commercial product proposed by NewMonics Inc. [23]. PERC uses an extended dialect of Java to enable the specification of execution constraints in the Java code. It also provides a JVM that offers rate monotonic scheduling, priority inheritance, realtime garbage collection and support for runtime execution analysis. Interactions between the real-time Java program and its environment uses proprietary Java classes that enable access to I/O ports and interrupt handling. The stringent need for a clear definition of a real-time Java environment and for a standardization of the related core classes has led to the creation of a Real-Time Java Working Group (RTJWG) hosted by the NIST US federal organization and, more recently, of the J Consortium that aim to federate ongoing works in this direction. As of this writing, the members of the RTJWG are working on the specification of core real-time extensions for the Java platform. These extensions would offer to Java applications the basic services that are generally offered by commercial real-time operating systems. They should also provide a foundation upon which more sophisticated higher level real-time capabilities would be constructed as optional profiles [24]. 3. BACKGROUND In addition to PERC, other commercial efforts, such as MachJ from MachJ Company, Kaffe from Transvirtual

Technologies or JBed from Oberon Microsystems, target the real-time and embedded markets with custom Java virtual machines. JBed is particularly interesting because it strongly integrates Java with the operating system [25], and because it provides an Earliest Deadline First (EDF) [18] thread scheduler. However, all these products currently implement only a subset of the Java specifications. In particular, support for features specific to Java 2 (formerly known as Java Development Kit 1.2) is missing. At the moment, remote method invocation is thus not yet available. Our solution relies on a free virtual machine, GNU Japhar, slightly modified to exhibit real-time capabilities, and hosted by a distributed real-time system, ChorusOS. 3.1 GNU Japhar Japhar is a clean room implementation of the Java virtual machine [26]. As a consequence, it does not include any source code licensed from Sun. Japhar complies with the JNI specifications, and can be embedded in user applications. More than one JVM can even be created within the same process. Japhar is released under the GNU Library General Public License (LGPL), and thus can be embedded into proprietary and commercial products. Japhar does not yet include a JIT compiler but only comes with a bytecode interpreter. Thus, bytecode execution currently suffers some performance penalty. In order to not infringe Sun’s distribution license, the Java core classes are not shipped with Japhar, but have to be separately obtained from a JDK distribution. Alternatively, there are ongoing projects that aim to provide a cleanroom, GPLed, implementation of these classes. Japhar can thus be used either with the foundation classes that come with the Sun JDK 1.1 or 1.2 products, or with the classes supplied by the GNU Classpath project. Thanks to its free nature, Japhar comes with full code source. It can therefore be adapted to fit particular environments, and can be ported to new operating systems such as Chorus. 3.2 The Chorus operating system ChorusOS is a commercial distributed real-time operating system [7]. The last release of the Chorus microkernel is based on a componentized approach. The kernel and the external servers are made of building blocks selected according to application requirements. The selected components determine for instance the programming facilities that remain available, but also the fundamental features of the system, such as the used virtual memory model or the supported scheduling policies. Consequently, the Chorus system is scalable and can be targeted to a wide range of environments, from small embedded systems to large fault-tolerant servers. Since the purchase of Chorus Systems by Sun Microsystems, the Chorus kernel supplies the foundation of the JavaOS for Consumers product from Sun. This product

is broadly an association of the Chorus microkernel, the Sun virtual machine and additional proprietary Java class hierarchies. Scheduling Adequate task scheduling is one of the most important feature required to meet time constraints. Instead of a classical time-sharing scheduling algorithm, real-time kernels provide preemptive schedulers that ensure that, at any time, the processor is allocated to the runnable task which has the highest priority. This priority has to reflect in some manner the time constraints of the task [3]. Methods like the rate monotonic analysis can also be used to affect such fixed priorities [16]. This approach is however not adapted to dynamic environments where timing information about all tasks can not be determinated at application design time. Truly adding deadline concepts to the kernel scheduler should therefore ensure a better meeting of time constraints, because the deadline of a task dynamically shows its urgency. Thus, an Earliest Deadline First (EDF) scheduler has been added into the Chorus kernel [21]. It enables the dynamic scheduling of periodic and aperiodic tasks according to their absolute deadline. Real-time IPC The Chorus system comes with an IPC subsystem that allows communications between threads across the distributed system. This service is location transparent. It is based on a global uniform naming scheme that relies on unique identifiers. These identifiers are guaranteed to be unique in time and space despite site reboots. The Chorus IPC service grants the exchange of messages through ports [7]. A port is a communication endpoint hosted by a Chorus process, termed as an actor. A port is attached to a single actor at a time, but can migrate between actors, locally or across the network, along its lifetime. Migratable ports provide an extra level of indirection between communicating threads, and thus enable the transparent reconfiguration of system and applicative services. Chorus IPC messages can be sent asynchronously, or with a remote procedure call semantics. In this latter case, the sender is blocked until a reply is returned by the receiver, or until an abort event is propagated. This IPC subsystem only provides a best effort message passing service. No quality of service requirements nor temporal constraints can be associated to IPC interactions between threads. We have therefore designed a realtime IPC service that extends the abstractions provided by the regular Chorus IPC service and supplies a communication service that supports end-to-end deadline inheritance. The real-time IPC service, or RT-IPC for short, allows the exchange of real-time messages (RT-messages) between real-time ports (RT-port) [20]. A RT-message is similar to a regular Chorus IPC message, but it additionally conveys a deadline. RT-ports have the same purpose than Chorus

IPC ports but can queue messages according to their reception date or according to their deadline. In contrast to the standard Chorus IPC service, RT-ports have to be explicitly bound prior to message transfer. The binding is instantiated on request from the application. The request contains the quality of service the user expects from the communication service provider. The description of the expected QoS is then submitted to the ATM network. The binding is established only if the ATM call admission controller can guarantee that the requested performance can truly be provided by the network. As a consequence, RT-ports can not migrate while they are connected. ATM support An ATM driver has been added to the I/O subsystem of the Chorus system. The driver is hosted by the Input/Output Manager (IOM), a Chorus actor that provides a standard IFNET interface to peripheral device drivers, identical to the one supplied by BSD kernels. This interface is used by the TCP/IP protocols also hosted by the IOM. The main drawback of the IFNET framework is that it induces an important processing overhead, due to the copy of user data from/to chains of mbuf buffers [33]. The ATM driver thus implements another interface that provides a zero-copy access to the low-level services of the driver [19]. This interface, that additionally grants a direct access to the ATM signalling protocol stack, is preferred by the RT-IPC server to provide a communication service between remote RT-ports. 4. ATM NETWORKING An ATM network consists of a set of switches interconnected by point-to-point links or interfaces [4]. ATM end-systems exchange fixed-size cells that are asynchronously multiplexed across the link. ATM is a connection-oriented network. A virtual circuit (VC) need to be established before any data transfer. Pointto-point virtual circuits are bidirectional, while unidirectional point-to-multipoint virtual connections link a single traffic source (also known as the root of the multipoint tree) to multiple receivers (leaves of the tree). 4.1 Quality of Service ATM natively supports QoS requirements, so that users can describe the nature of the traffic that will be sent and the performance level they expect from the network. In that mind, different service categories are devoted to particular traffic requirements. The following categories are the most suitable to support real-time communications [1]:

 Continuous Bit Rate (CBR) connections carry constant bit rate traffic with a fixed bandwidth and tightly constrained transfer delay and jitter;

 real-time Variable Bit Rate (rt-VBR) also carry data with a constrained delay and jitter, but with a variable bandwidth;  additionally, the Available Bit Rate (ABR) service category only guarantees a minimum bandwidth. The actual rate granted to the traffic source is regulated by periodic feedback informations from the network. This service class is therefore suitable for adaptive distributed applications with soft real-time constraints. 4.2 Call admission A set of attributes is associated to each service category. These attributes define the characteristics of any established VC. They include a description of the traffic being handled by the connection, and a description of the quality of service expected from the network. Traffic parameters include:

 the Peak Cell Rate (PCR) is the cell rate that a traffic source may never exceed. It applies to CBR and VBR connections.  the Substainable Cell Rate (SCR), the upper bound average rate of the traffic, over the lifetime of the connection. It only applies to VBR virtual circuits.

conveys the traffic and QoS descriptions to the nearest switch. ATM switches then dialog using the NNI (Network-Network Interface) signalling protocol to determine whether or not the requested quality of service can be provided. If the call admission control succeeds, the VC is then instantiated. Compliance of the user traffic to the description given to the network is enforced over the lifetime of the VC by the UPC (Usage Parameter Control) function of the ATM switches. Compliance test relies on a Generalized Cell Rate Algorithm (GCRA) [1], also termed as “leaky bucket”. Non conforming cells are then tagged or dropped if they are received at a too higher rate. 5. JAVA REAL-TIME DISTRIBUTED PROCESSING The Java runtime environment provided by the Japhar virtual machine has been enhanced in order to take advantage of our extensions to the Chorus operating system. The new features can be used by the way of a hierarchy of specific classes bundled in a Java package. While some of our custom classes inherit from standard ones, no modification has been required in any Java foundation class. 5.1 Threads

 the Minimum Cell Rate (MCR) is the rate at which a traffic source is always allowed to transmit over an ABR circuit.

Java threads created using the regular Thread class are scheduled by the Chorus kernel according to a pure preemptive priority-based FIFO (First-In, First-Out) policy. On a uniprocessor host, the processor is at any time allocated to the eligible thread with the highest priority, and rescheduling only occurs when the current thread completes, blocks, or when another thread of strictly higher priority is made runnable.

Quality of service parameters that apply to CBR and VBR virtual circuits include:

EDF scheduling

 the maximum Cell Transfer Delay (maxCTD) is the maximum expected delay between the time the first bit of a cell leaves a measurement point to the time its last bit crosses a second measurement point. It expresses the maximum transfer delay experienced by a cell along a VC;

Extended Java classes have been designed to enable the use of threads scheduled according to the EDF policy. These new classes inherit from the Thread class but also provide the methods specifically required to manage the scheduling parameters supported by the EDF scheduler. EDF threads can be created by the way of the following classes:

 the Maximum Burst Size (MBS) is the maximum number of consecutive cells that can be sent at the PCR rate over a VBR connection;

 the peak-to-peak Cell Delay Variation (peak-to-peak CDV) is the difference between maxCTD and the fixed cell transfer delay that could be experienced by any delivered cell on a connection;  the cell loss ratio (CLR) is the ratio of the number of lost cells over the total number of cells transmitted. These parameters are formally defined as objectives of performance that the network agrees to offer over the lifetime of the virtual circuit. The establishment of a switched virtual channel is requested to the network using the UNI (User-Network Interface) signalling protocol [2]. The setup message

public abstract class EdfThread extends Thread; public abstract class OneShotEdfThread extends EdfThread; public abstract class PeriodicEdfThread extends EdfThread;

Instantiating the OneShotEdfThread class grants to create an aperiodic thread, while instantiating the PeriodicEdfThread class created a periodic thread. In both cases, the user must implement the edfRun() abstract method. If the thread is aperiodic, this method will be called once. Otherwise, if the thread is periodic, the method will be called at each release time of the thread.

Methods are defined to set and read the scheduling attributes of an EDF thread. They can be set in the constructor of the thread, and later retrieved or modified while the thread is running. These attributes include:

 the release time, i.e. the date of the first call to edfRun();  the deadline. It can be either specified as an absolute date or as a duration relatively to the release time;  the worst-case execution time. In the case of a periodic thread, this delay should bound the execution time of the thread along one release;  the period. This parameter is only relevant to periodic threads. It defines the elapsed time between two successive releases of the task. All delays and dates are passed as 64 bit integers whose values are expressed in nanoseconds. Dates are relative to the boot time of the system. The same epoch is used by regular methods such as System.currentTimeMillis(). Additionally, the number of times a periodic thread will be released can be specified. The number of the current release, and the cumulative execution time of a thread can also be obtained from the system. Synchronization Java uses monitors [12] to ensure mutual exclusion when entering critical sections, leading to potential priority inversions. A priority inversion occurs when a higher priority task is delayed by the processing of a lower priority task. For instance, a high priority thread may be blocked on a monitor held by a lower priority thread. This blocking increases the response time of the high priority thread, and must be explicitly considered to analyze de feasibility of a schedule. Priority inheritance protocols have been proposed to address such situations. The common approach is to bound in some way the occurrence and the length of priority inversions. The Chorus system provides real-time mutexes [8] that implement the basic Priority Inheritance Protocol (PIP) [28]. The idea behind this protocol is that when a thread locking a resource blocks one or more higher priority threads, it temporarily assumes the highest priority of the blocked threads. In other words, it inherits a higher priority. When the thread unlocks its resource, it resumes its original priority. As the scheduling architecture has been modified to enable EDF scheduling, the implementation of the Chorus real-time mutexes has consequently been updated. Therefore, these mutexes perform deadline inheritance in a way consistent with the modified kernel scheduler. In order to take advantage of this feature, the implementation of the Java monitors now relies on Chorus real-time mutexes.

5.2 Communications As of this writing, the Japhar JVM does not yet support remote method invocation using the JDK 1.2 core classes, because the serialization of composite objects is partially missing. As a consequence, we currently restrict communications over our RT-IPC service to stream-based interactions between Java objects. An overview of the communication system is depicted by the figure 1. Native interface As the RT-IPC code is provided by native methods, interactions between Java and foreign objects make heavy data transfers to cross the JNI interface. The JNI specification theoretically prohibits the direct sharing of memory between native and Java code. The main reason is that a moving garbage collector may address the fragmentation of the memory heap by changing the internal location of Java objects at any time. This is especially true for arrays of composite objects or arrays of primitive types. As a consequence, only non-moveable copies of the arrays involved in a JNI operation are supplied to the native side. If modified data have to be returned to Java, hen another copy has to be performed by the JNI glue code. As the current implementation of the Japhar garbage collector does not move Java objects, we have extended the JNI interface in order to enable the sharing of byte arrays between native and Java objects. Thanks to this extension, serialized data can be exchanged between Java and foreign code without extra processing cost, boosting JNI performances. Consequently, if future versions of the Japhar’s garbage collector handle memory fragmentation by moving objects, then byte arrays shared with native code would have to be tagged as non-moveable. Memory management Real-time messages involved in RT-IPC communication come from user allocated pools of shared buffers. A buffer involved in an output operation is retrieved from its pool, filled by the user, then submitted to the service provided. Then, the buffer will be automatically replaced to its holding pool upon completion of the I/O operation. Similarly, a buffer required to store incoming data is retrieved from its pool by the RT-IPC server and then delivered to the user. The receiver is responsible for the subsequent release of the buffer to its pool after the processing of the data it contains. A shared buffer is an untyped array of bytes whose size is statically determinated at allocation time. To each Buffer object is associated a native RT-IPC shared buffer. Native buffers are page-aligned and furthermore can be allocated using physically contiguous pages. This grants an easier description of the buffer to a DMA engine. A buffer pool can be allocated by creating an instance of the public class BufferPool. The number and the size of the shared buffers being allocated within the

Java Virtual Manufacturing Device

Java Virtual Manufacturing Device End-to-end deadline inheritance

RT-message handler

EDF Java thread RT-port

RT-message

RT-port

EDF Java thread

Pools of shared buffers

Pools of shared buffers

Japhar JVM

Japhar JVM RT-IPC server

RT-IPC server

ATM driver

ATM driver

ChorusOS

ChorusOS

ATM CBR switched virtual circuit

RT-binding

Figure 1. Architecture of the communication system.

pool are parameters of the constructor. As such an instantiation implies the allocation of native resources, the finalizer of the class has also been overridden to ensure that these native resources will truly be released when the BufferPool object is garbaged. Thanks to the modification of the JNI interface described in the preceding section, the content of RT-IPC buffers can be truly shared between Java and foreign objects. The following method just returns the byte array that implements the shared buffer, enabling reading and writing of its content: public byte[] getData();

Additionally, other methods are supplied to handle the content of the buffer with a copy semantics.

public void init( Buffer buffer );

This grants to change the underlying buffer associated to the stream, without having to instantiate a new BufferOutputStream object. The state of the stream is then re-initialized. The major difference between BufferOutputStream and ByteArrayOutputStream classes is that BufferOutputStream objects have a limited size. This fixed size is the length of the associated buffer. Output is truncated if the maximum size is reached. The BufferInputStream class provides functionalities similar to the ByteArrayInputStream core class. Extensions similar than those provided by BufferOutputStream are supplied.

Streams Shared buffers do not provide by themselves the level of abstraction required by distributed applications. We have thus introduced additional wrapping classes that allow stream-based interactions between Java objects: public class BufferOutputStream extends OutputStream; public class BufferInputStream extends InputStream;

The BufferOutputStream provides the same methods than defined by the ByteArrayOutputStream core class. A BufferOutputStream object can be instantiated using the following constructor: public BufferInputStream( Buffer buffer );

The call returns a new stream object that grants to write serialized objects to the supplied shared buffer. Then, the following method is provided:

Real-time ports Communication using the RT-IPC system occurs between RT-ports that are globally designated by unique identifiers allocated by the system. Two pools of shared buffer are associated to an RT-port. The first pool is in charge of output operations, while the second one handles input operations. Nevertheless, the same pool of buffers can be used for both input and output operations and can be associated to several RT-ports at the same time. This enables to pipeline input and output operations involving the same chunk of data, without extra data copy between buffers. A RT-port can be created by instantiating the RtPort class using the following constructor: public RtPort( int flags, BufferPool inputBufferPool, BufferPool outputBufferPool );

The flags determine the behavior of the port. One can precise if an EDF thread waiting for messages on a realtime port may or not inherit the deadline of these messages. Another flag also grant to specify if messages have to be queued by the RT-port according to their deadline of according to their reception order. The RtPort class also exports methods to get the unique identifier of the port and to retrieve free buffers from the associated buffer pools. RT-ports have to be bound before any interaction. A binding request can be performed using the following method: public RtBinding bind( UniqueId tgtUid, QosSpec forwardQos, QosSpec backwardQos ) throws IOException;

tgtUid is the unique identifier of the RT-port targeted by the request. forwardQoS and backwardQoS are the re-

quested QoS specifications for the forward and backward directions of the binding. QoS specification are made of a target specification, that expresses the nominal level of service expected by the sender, and an acceptable specification, that describes the minimum required level of service. Binding requests can be either accepted or rejected by the target port. If a binding is accepted, then the receiver specifies the QoS level that will actually be used. The negotiated QoS is used to issue a setup request to the ATM network, in order to establish a new switched virtual circuit. If the setup succeeds, indicating that the network agrees to provide the requested level of service, then the binding is made operational. Real-time bindings Real-time bindings are intermediate entities placed between RT-ports that wish to communicate. This scheme complies with the Open Distributed Processing reference model [5]. Binding objects are created by real-time upon sending of a binding request or upon acceptance of such a request. Binding objects are thus not supposed to be instantiated by the end user. Like ports, RT-bindings are globally named by a unique identifier. The RtBinding class defines methods to perform the exchange of real-time messages between ports. A realtime message if made of a message body, implemented by a shared buffer, and may be optionally completed by a small fixed-size annex. It also conveys an absolute deadline. This deadline is either specified by the user, or inherited from the one of the sender EDF thread. Real-time messages are referenced by message descriptors implemented by the MsgDesc class. Once a binding is established, messages can be exchanged between peer RT-ports. A real-time message can be asynchronously sent using the following method: public int ipcSend( MsgDesc msgDesc ) throws IOException;

The caller thread is blocked only during the time of the local processing of the message. Upon completion of the output operation, the buffer used as the message body is implicitly released to its holding pool by the RT-IPC system. Then, a thread can wait for the reception of a real-time message on a given binding by invoking the following method: public int ipcReceive( MsgDesc msgDesc, long timeout ) throws IOException;

The thread is there blocked until a message is received, or until the timeout is elapsed. The supplied message descriptor is filled by the RtBinding object upon reception of a real-time message. Alternatively, a message handler can be attached to a realtime port. This handler will be called upon each message reception by the RtPort object. A message handler implements the following interface: public interface MsgHandler { public int handler( RtPort rtPort, MsgDesc msgDesc ); };

The rtPort parameter references the RT-port that has received the message given by msgDesc. This grants to attach the same handler to several ports at the same time. As the message handler is implemented by a separate EDF thread, then this thread may inherit the deadline of the message being processed, enabling end-to-end deadline inheritance from senders to receivers. Synchronous interactions can also be performed using the following method: public int ipcCall( MsgDesc MsgDesc long throws IOException;

reqMsgDesc, repMsgDesc, timeout )

This method sends the real-time message described by reqMsgDesc to the peer port of the binding ad waits for a reply. A transaction semantics guarantees that the received reply, if any, is truly the one expected by the caller. In order to minimize the processing overhead and to minimize the interferences with the garbage collector, the communication process does not require further object instantiations after the establishment of the binding. The design of the RT-IPC classes ensures that the same objects, such as the shared buffers or the stream wrappers, can be reused along the lifetime of the application. 5.3 Resources Allocation The Java language exempts the application programmer to explicitly release allocated memory, as this task is done by the garbage collector. Garbage collection relies on the fact that an allocated memory location which is no longer referenced by any pointer is no longer accessible to the application and is therefore reclaimable [6].

There are basically three kind of garbage collecting techniques: reference counting, mark-and-sweep and copying [35]. Reference counting keeps track of any pointer that references a memory location. When an object is created, its reference counter is set to one. When another object assigns a pointer to that object, then the object count is incremented. When such a reference is removed, the counter is decremented. An object with a null reference count can thus be garbaged. The main problem of reference counting is its inability to reclaim cyclic structures or double-linked collections of objects. By contrast, mark-and-sweep and copying collectors use tracing techniques. They keep track of live objects by tracing out a graph of references, starting from known root objects. Referenced objects are marked in some way during the trace. After completion of this step, unmarked objects are then considered unreachable by a mark-and-sweep collector, and are thus garbaged. A copying collector proceeds differently and moves all live objects to a new area (termed as a semispace) during the trace. All copied objects are thus compacted in the new semispace, while the old area is known to be all free space. An embedded application running Java would require a fast, non-disruptive, garbage collector that also provides predictable performance for future memory allocation. Despite some dedicated techniques can be used to implement such a real-time garbage collector [36][6], the RealTime Java Working Group has not included real-time garbage collection in its draft specifications of the Java real-time core. The rationale is that automatic garbage collection would induce some processing overhead that can be unacceptable for some classes of real-time applications. Thus, garbage collection is treated as an optional enhancement which is not featured by the real-time core but by vendor-specific real-time extensions. In order to address dynamic object allocation and subsequent memory reclamation, alternative solutions have been proposed. These proposals rely on allocation contexts. Allocation Contexts The concept of allocation context has been recently formalized by K. Nilsen and the Real-Time Java Working Group. Allocation contexts can be viewed as partitions of the memory heap. The purpose of an allocation context is to express the lifetime of the objects it contains. Objects are thus automatically reclaimed when the allocation context in which they are allocated is destroyed. By default, objects are instantiated in the allocation context of the thread that performs the allocation. This context is destroyed when the thread terminates. Alternatively, a class can implement the following interface to indicate that all objects instantiated by its methods are to be allocated in a specific allocation context. interface CoreAllocator { public CoreAllocationContext allocationContext(); }

The allocation context to be used is represented by the object returned from the allocationContext() method. The class must provide its own implementation of this method, which should return the same CoreAllocationContext each time it is invoked.

Stackable objects Moreover, one significant performance drawback of Java in comparison with C and C++ is that all dynamically instantiated objects are allocated from the heap instead of from the runtime stack [24]. The Real-Time Java Working Group thus additionally proposes to enable the allocation of stackable objects whole lifetime is limited by the one of their stack frame. Stack frames thus constitute a special kind of allocation contexts, as stackable objects may not be referenced by any other non-stackable objects. The stackable attribute is precised either by calling a specific method exported by the real-time core or by using an extended stackable keyword. Allocation contexts thus provide a convenient way to describe the lifetime of the various objects that are instantiated by a real-time Java application, and we are in the process of implementing these mechanisms in our Java platform. 5.4 Performances Thanks to the optimizations made to the ATM driver and JNI interfaces, output operations can be realized without any data copy across address spaces. User data serialized to shared buffers are directly moved to the memory of the ATM adapter using a DMA engine. By contrast, zero-copy reception generally requires a sophisticated networking hardware able to directly deliver received messages to various memory locations, depending on their actual receiver. In the context of ATM networks, the ATM adapter is then expected to maintain perconnection buffer supply queues. A communication service provider that offers both zero-copy transmission and reception is therefore termed as true zero-copy. A prototype of such an ATM adapter has been designed by CSTI. The hardware of this adapter supports all the service categories described by the ATM specifications. It also provides a specific AAL (ATM Adaptation Layer) [13] devoted real-time traffic. This AALRT, based on the PDU layout of the AAL3/4, introduces the notion of priority to the AAL layer, avoiding priority inversion problems within the ATM adapter. The CSTI ATM adapter additionally performs clock synchronization between the different hosts of the distributed system, using a fault-tolerant protocol that relies on the exchange of Operation And Maintenance (OAM) ATM cells [10]. This adapter will be used in the A NTARA distributed realtime servers produced by CSTI. A NTARA servers are basically clusters of computing nodes running ChorusOS,

and interconnected by a 155 Mbps ATM SAN (System Area Network). The following tests have been run within such an A N TARA cluster. The Java virtual machines are hosted by 300 MHz PowerPC hosts connected by OC-3 links to a 16 port ATM switch. These tests consist in asynchronous transmission operations performed between EDF thread objects across the ATM network, over a CBR virtual circuit to which nearly all the available bandwidth was granted. Msg. size 62 Kbytes 62 Kbytes 62 Kbytes 62 Kbytes

Period of the sender thread 15 ms 10 ms 8 ms 5 ms

Substained rate 33 Mbps 50 Mbps 61 Mbps 97 Mbps

Loss ratio 0.0 % 0.3 % 0.3 % 0.3 %

Then, during synchronous operations, the round-trip delay is about 15 ms. The end-to-end delay between the time a message is sent to the time it is received and retrieved by a receiver is 7 ms. We clearly suffer here the performance penalty induced by the interpreted nature of the Java language. End-to-end delays should be heavily reduced when future releases of the Japhar virtual machine will embed a JIT compiler. Delays also include the time of a data copy, currently required by the reception process. Messages losses are thus mostly induced by buffer overflows within the ATM driver, rather than by cell losses. Then, in order to evaluate the overhead induced by the JVM, one can note that, under similar conditions, rates over 130 Mbps can be substained between C++ programs using the RT-IPC system. The end-to-end delay is in this case less than 300 s. 6. CONCLUSION The system presented in this paper currently solves partially the problems raised by real-time distributed computing using the Java language. We haven’t yet fully addressed the problem of real-time garbage collection, although allocation contexts and stackable objects seem to constitute a promising solution. Then, as soon as the JDK 1.2 classes will be fully supported by Japhar, we will experiment remote method invocations using our RT-IPC system across ATM networks. Finally, in order to enhance the abstraction model and the services provided by our architecture, we intend to integrate the Sun Jini environment [31] and Jonathan [9], a flexible Object Request Broker with real-time communication capabilities that complies with the ODP model [14], and which is developed entirely in Java. We finally plan to use this real-time Java framework to design and implement the core components of an extensible Java-based SCADA system, where data acquisition and processing would be performed by Java virtual manufacturing devices communicating using RMI over our RTIPC service.

REFERENCES [1] ATM Forum, Traffic Management Specification Version 4.0, ATM Forum specification, af–tm–0056-000, April 1996. [2] ATM Forum, ATM user-Network Interface Signalling Specification Version 4.0, ATM Forum specification, af–sig–0061-000, July 1996. [3] N.C. Audsley, “Optimal Priority Assignment and Feasibility of Static Priority Tasks with Arbitrary Start Times”, Technical Report YCS-164, University of York, 1991. [4] U. Black, ATM: Foundations for Broadband Networks, Prentice Hall, 1995. [5] G. Blair, J.B. Stefani, Open Distributed Processing and Multimedia, Addison-Wesley, 1997. [6] A. Petit-Bianco, “Java Garbage Collection for Real-Time Systems”, Dr. Dobbs Journal, October 1998. [7] Chorus Systems, “CHORUS/ClassiX r3 Technical Overview”, Technical Report CS/TR-96-119.14, October 1997. [8] Chorus Systems, “CHORUS/ClassiX Programmer’s Guide”, Technical Report CS/TR-96-188.3, May 1997. [9] B. Dumant, F. Dang Tran, F. Horn, J.-B. Stefani, “Jonathan: an Open Distributed Processing Environment in Java”, IFIP International Conference on Distributed Systems Platforms and Open Distributed Processing, September 1998. [10] L. George, C. Lizzi, J. Montiel, “External/Internal Clock Synchronization in ATM-based Distributed Systems”, 23rd Euromicro Conference, September 1997. [11] J. Gosling, B. Joy, G. Steel, The Java Language Specification, Addison-Wesley, 1996. [12] C.A.R. Hoare, “Monitors: an Operating System Structuring Concept”, Technical Report TR-73-401, CS Dept., Stanford University, 1973. [13] International Telecommunication Union, B-ISDN ATM Adaptation Layer Specification, ITU-T Recommendation I.363, March 1993. [14] International Telecommunication Union, ODP Reference Model: Overview, ITU-T Recommendation X.901, 1995. [15] J. Jaworski, Java Developer’s Guide, Macmillan Computer Publishing, 1996. [16] M.H. Klein, T. Ralya, B. Pollack, R. Obenza, M.G. Harbour, A Practitioner’s Handbook for Real-Time Analysis: Guide to Rate Monotonic Analysis for Real-Time Systems, Kluwer Academic Publishers, 1993. [17] T. Lindholm, F. Yellin, The Java Virtual Machine Specification, Addison-Wesley, 1996. [18] C.L. Liu, J.W. Layland, “Scheduling Algorithms for Multiprogramming in a Hard Real-Time Environment”, Journal of the ACM, 20(1), 1973. [19] C. Lizzi, E. Gressier-Soudan, J. Montiel, “A RealTime Communication Service for ATM-based Distributed Systems”, 1st IEEE International Conference on ATM, June 1998.

[20] C. Lizzi, E. Gressier-Soudan, “A Real-Time IPC Service over ATM Networks for the Chorus Distributed System”, 24th Euromicro Conference, September 1998. [21] C. Lizzi, “Une Architecture de Traitement Temps R´eel R´eparti sur R´eseau ATM pour le Syst`eme Chorus” (“A Real-Time Distributed Processing Architecture over ATM Networks for the Chorus System”), Real-Time & Embedded

Systems Conference, February 1999.

[28] L. Sha, T. Rajkumar, J.P. Lehoczky, “Priority Inheritance Protocols: An Approach to Real-Time Synchronization”, Technical Report CMU-CS-87-181, CS Dept., Carnegie Mellon University, 1987. [29] Sun Microsystems, “Java Remote Method Invocation Specification”, Technical report, May 1996. [30] Sun Microsystems, “Java Native Method Interface Specification”, Technical Report, 1997.

[22] K. Nilsen, “Issues in the Design and Implementation of Real-Time Java”, Technical Report, NewMonics Inc., 1996.

[31] Sun Microsystems, “Jini Architecture Specification”, Technical Report, January 1999.

[23] K. Nilsen, “PERC Real-Time API (Draft 1.3)”, Technical Report, NewMonics Inc., 1997.

[32] D. White, A. Garthwaite, “The GC Interface in the EVM”, Technical Report 98-67, Sun Microsystems Laboratories, December 1998.

[24] K. Nilsen, “Functional Requirements for Core Real-Time Extensions for the Java Platform”, Draft, Real-Time Java Working Group, May 1999.

[33] G.R. Wright, W.R. Stevens, TCP/IP Illustrated, Volume 2, The Implementation, Addison-Wesley, 1995.

[25] Oberon Microsystems, “JBed Real-Time Operating System”, Technical Report, 1998.

[34] A. Valenzano, C. Demartini, L. Ciminiera, MAP and TOP Communications, Addison-Wesley, 1992.

[26] P. Reinholdtsen, “Japhar: the Hungry Programmers Open Source Java”, Talk to the Norwegian Unix Users Group, November 1998.

[35] P.R. Wilson, “Uniprocessor Garbage Collection Techniques”, 1992 International Workshop on Memory Management, Lecture Notes in Computer Science, Vol. 637, Springer-Verlag, September 1992.

[27] L. Seinturier, A. Laurent, B. Dumant, E. Gressier-Soudan, F. Horn, “A Framework for Real-Time Communication Based Object Oriented Industrial Messaging Services”, 7th IEEE International Conference on Emerging Technologies and Factory Automation, October 1999.

[36] P.R. Wilson, M.S. Johnstone, “Real-Time Non-Copying Garbage Collection”, ACM OOPSLA Workshop on Memory Management and Garbage Collection, September 1993.