Technical Publications
QOS-enabled middleware has been applied to a wide range of distributed real-time and embedded (DRE) systems. An emerging class of open DRE systems spans multiple endsystems and although their application tasks have stringent end-to-end QoS requirements (such as timeliness and criticality,) certain aspects of requirements may change dynamically at run-time due to variations in their operating environments or to the loss or failure of individual endsystems. In these open DRE systems there is a need to propagate QoS parameters and enforce task QoS requirements across multiple endsystems in a way that is simultaneously efficient and adaptable. The Object Management Group's (OMG) Real-Time CORBA 2.0 specification (RTC2) defines a dynamic scheduling framework for propagating and enforcing QoS parameters dynamically in standard CORBA middleware.
This paper makes two contributions to research on middleware for open DRE systems. First, it describes the design and capabilities of the RTC2 dynamic scheduling framework provided by TAO, which is our open-source CORBA standards-based Object Request Broker (ORB). Second, it presents empirical studies we have conducted to validate our RTC2 framework in the context of open DRE systems. The results of those experiments show that a range of policies for adaptive scheduling and management of distributable threads can be enforced efficiently in standard middleware for open DRE systems.
Many distributed real-time and embedded (DRE) applications require a scalable event-driven communication model that decouples suppliers from consumers and simultaneously supports advanced quality of service (QoS) properties and event filtering mechanisms. The CORBA Notification Service provides publisher/subscriber capabilities designed to support scalable event-driven communication by routing events efficiently between suppliers and consumers, enforcing QoS properties (such as reliability, priority, ordering, and timeliness), and filtering events at multiple points in a distributed system. The standard CORBA Notification Service is insufficient, however, to enforce predictable communication needed by DRE applications and does not leverage Real-time CORBA capabilities, such as end-to-end priority preservation, priority models, and scheduling.
This paper makes three contributions to the study of scalable real-time notification services for DRE applications. First, we describe the requirements of the OMG Request for Proposals (RFP) on Real-time Notification, which seeks solutions to the problem of enforcing real-time properties by enhancing the standard CORBA Notification Service. Second, we explain how we have addressed key design challenges faced when implementing a Real-time Notification Service for TAO, which is our CORBAcompliant real-time Object Request Broker (ORB). We discuss the optimizations used to improve the scalability of TAO's Real-time Notification Service, which integrates Realtime CORBA features (such as thread pools, thread lanes, and priority models) to provide real-time event communication by dedicating thread resources with minimal locking overhead in the critical path of event propagation. Finally, we analyze the results of empirical benchmarks of the performance and predictability of TAO's Real-time Notification Service. These results show that the static realtime assurances provided by Real-time CORBA are maintained within the more flexible context of TAO's Real- Time Notification Service.
Computing systems are increasingly distributed, real-time, and embedded (DRE) and must operate under highly unpredictable and changeable conditions. To provide predictable mission-critical quality of service (QoS) end-to-end, QoS-enabled middleware services and mechanisms have begun to emerge. However, the current generation of commercial-off-the-shelf middleware lacks adequate support for applications with stringent QoS requirements in changing, dynamic environments. This paper provides two contributions to the study of adaptive middleware to control DRE applications. It first describes how priority- and reservation-based OS and network QoS management mechanisms can be coupled with standards-based, off-the-shelf distributed object computing (DOC) middleware to better support dynamic DRE applications with stringent end-to-end real-time requirements. It then presents the results of experimentation and validation activities we conducted to evaluate these combined OS, network, and middleware capabilities. Our work integrates currently missing low-level resource control capabilities for end-to-end flows with existing capabilities in adaptive DRE middleware and sets the stage for further advances in fine-grained precision management of aggregate flows using dynamic adaptation techniques.
End-to-end predictability of remote operations is essential for many fixed-priority distributed real-time and embedded (DRE) applications, such as command and control systems, manufacturing process control systems, large-scale distributed interactive simulations, and testbeam data acquisition systems. To enhance predictability, the Real-time CORBA specification defines standard middleware features that allow applications to allocate, schedule, and control key CPU, memory, and networking resources necessary to ensure end-to-end quality of service support.
This paper provides two contributions to the study of Real-time CORBA middleware for DRE applications. First, we identify potential problems with ensuring predictable behavior in conventional middleware by examining the end-to-end critical code path of a remote invocation and identifying sources of unbounded priority inversions. Experimental results then illustrate how the problems we identify can yield unpredictable behavior in conventional middleware platforms. Second, we present design techniques for providing real-time QoS guarantees in middleware. We show how middleware can be redesigned to use non-multiplexed resources to eliminate sources of unbounded priority inversion.
Adaptation of distributed software to maintain the best possible application performance in the face of changes in available resources is an increasingly important and complex problem. In this paper, we discuss the application of the QuO adaptive middleware framework and the CORBA A/V Streaming Service to the development of real-time embedded applications. We demonstrate a standards-based middleware platform for developing adaptive applications that are better architected and easier to modify and that can adapt to changes in resource avail-ability to meet QoS requirements. These are presented in the context of an Unmanned Aerial Vehicle (UAV) video distribution application. The UAV application is devel-oped using QuO and the A/V Streaming Service, and uses adaptive behavior to meet timeliness requirements in the face of restrictions in processing power and network bandwidth. We also present some experimental results we have gathered for this application.
Strict control over the scheduling and execution of processor resources is essential for many fixed-priority real-time applications. To facilitate this common requirement, the Real-Time CORBA (RT-CORBA) specification defines standard middleware features that support end-to-end predictability for operations in such applications. One of the most important features in RT-CORBA is thread pools, which allow application developers and end-users to configure and control processor resources.
This paper provides two contributions to the evaluation of techniques for improving the quality of implementation of RT-CORBA thread pools. First, we describe the key patterns underlying common strategies for implementing RT-CORBA thread pools. Second, we evaluate each thread pool strategy in terms of its consequences on (1) feature support, such as request buffering and thread borrowing, (2) scalability in terms of endpoints and event demultiplexers required, (3) efficiency in terms of data movement, context switches, memory allocations, and synchronizations required, (4) optimizations in terms of stack and thread specific storage memory allocations, and (5) bounded and unbounded priority inversion incurred in each implementation. This paper also provides results that illustrate empirically how different thread pool implementation strategies perform in different ORB configurations.
This paper describes the Leader/Followers architectural pattern, that provides an efficient concurrency model where multiple threads take turns sharing a set of event sources in order to detect, de-multiplex, dispatch, and process service requests that occur on these event sources.
To be an effective platform for performance-sensitive real-time systems, commercial-off-the-shelf (COTS) distributed object computing (DOC) middleware must support application quality of service (QoS) requirements end-to-end. However, conventional DOC middleware does not provide this support, which makes it unsuited for applications with stringent latency, determinism, and priority preservation requirements. It is essential, therefore, to develop standards-based, COTS DOC middleware that permits the specification, allocation, and enforcement of application QoS requirements end-to-end.
The Real-time CORBA and Messaging specifications in the forthcoming CORBA 3.0 standard are important steps towards defining standards-based, COTS DOC middleware that can deliver end-to-end QoS support at multiple levels in distributed and embedded real-time systems. However, these specifications lack sufficient detail to portably configure and control processor, communication, and memory resources for applications with stringent QoS requirements.
This paper provides four contributions to research on real-time DOC middleware. First, we characterize the QoS requirements of an important class of applications with stringent real-time requirements and illustrate how the CORBA 3.0 Real-time and Messaging specifications provide a starting point for addressing these requirements. Second, we illustrate how the CORBA 3.0 specifications are not sufficient to solve all the issues within this application domain. Third, we describe how we have implemented portions of these specifications, as well as several enhancements, using TAO, which is our open-source real-time CORBA ORB. Finally, we empirically evaluate the performance of TAO to illustrate how its features address the QoS requirements of certain types of real-time applications.
In many application domains, the distributed object computing (DOC) middleware is responsible for dispatching upcalls to one or more application objects when events or requests arrive from clients. Implementing efficient, predictable, and scalable middleware dispatching mechanisms is hard and implementing them for multi-threaded systems is even harder. In particular, dispatching mechanisms must be prepared to dispatch upcalls to multiple objects, to handle recursive requests originated from application-provided upcalls, and must often collaborate with the application to control object' life-cycle.
In our DOC middleware research we have implemented many dispatching mechanisms that repeatedly apply common solutions to solve the challenges outlined above. Moreover, we have discovered that the forces constraining dispatching mechanisms often differ slightly, thereby requiring alternative solutions. This paper presents two contributions to the design and implementation of efficient, predictable, scalable, and flexible DOC middleware and applications. First, it shows how patterns can be applied to the object-oriented systems to capture key design and performance characteristics of proven dispatching mechanisms. Second, it presents a pattern language that describes successful solutions that are appropriate for key dispatching challenges that arise in various real-time DOC middleware and applications.
First-generation CORBA middleware was relatively successful at meeting the demands of request/response applications with best-effort quality of service (QoS) requirements. However, supporting applications with stringent QoS requirements poses new challenges for next-generation CORBA middleware.
This paper provides three contributions to design and optimization of real-time middleware. First, we outline the challenges faced by real-time ORBs implementers, focusing on techniques to optimize the Object Adapter, ORB Core, and IDL compiler components. Second, we illustrate how TAO, our real-time CORBA implementation, has addressed these challenges. Third, we describe the results of benchmarks that compare the impact of alternative design strategies on the efficiency, predictability, and scalability of real-time ORBs.
Our results indicate that real-time ORBs must be highly configurable and adaptable to meet QoS requirements of a wide range of applications. In addition, we show how TAO can be configured to perform predictably and scalably, which is essential to support real-time applications.
An Object Adapter is an integral part of the Common Object Request Broker Architecture (CORBA). An Object Adapter assists an Object Request Broker (ORB) in delivering client requests to server object implementations (servants). Services provided by an Object Adapter include: (a) generation and interpretation of object references, (b) operation dispatching, (c) object activation and deactivation, (d) mapping object references to servants, and (e) registration of servants.
This paper provides two contributions to the study of Object Adapters. First, it outlines the CORBA Portable Object Adapter (POA) specification, which is a recent addition to the CORBA standard that greatly simplifies the development of portable and extensible servants and server applications. The design goals, architectural components, and semantics of the POA are explained. Second, the paper describes the design choices made to adapt the POA for the TAO Real-time ORB. Key design issues regarding efficient demultiplexing, upcall and collocation optimizations, ORB and POA concurrency configurations, and predictability are covered.
This paper describes the Proactor pattern, which supports the demultiplexing and dispatching of multiple event handlers, which are triggered by the completion of asynchronous events. The Proactor pattern simplifies asynchronous application development by integrating the demultiplexing of completion events and the dispatching of their corresponding event handlers.
This paper describes the Asynchronous Completion Token pattern, which allows applications to efficiently associate state with the completion of asynchronous operations.
Developing efficient and scalable Web servers is essential to meet the increasing demands of WWW traffic. This paper provides two contributions to the study of Web servers. First, it outlines the optimizations necessary to build high-performance Web servers and illustrates how we've applied some of these optimizations to create JAWS. JAWS is a high-performance Web server that is explicitly designed to alleviate overheads incurred by existing Web servers on high-speed networks. It consistently outperforms existing Web servers (such as Apache, Netscape Enterprise, PHTTPD, and Zeus) over 155 Mbps ATM networks on UNIX platforms.
Second, this paper describes how we have customized JAWS to leverage advanced features of Windows NT for multi-processor hardware over ATM. The Windows NT features used in JAWS include asynchronous mechanisms for connection establishment and data transfer. Our previous benchmarking studies demonstrate that once the overhead of disk I/O is reduced to a negligible constant factor (e.g., via memory caches), the primary determinants of Web server performance are the concurrency and event dispatching strategies.
The results reported in this paper indicate that certain Windows NT asynchronous I/O mechanisms (such as I/O completion ports and asynchronous file-to-network I/O) perform no better over a 155 Mbps ATM link than conventional synchronous multi-threaded servers, whereas other mechanism (such as TransmitFile) provide superior performance for large file transfers. However, there is evidence that the asynchronous event dispatching model is less resource intensive and provides more uniform latency when Web servers incur heavy loads over high-speed networks. Thus, to provide optimal performance, Web servers should be adaptive, choosing to use different mechanisms (e.g., TransmitFile) to handle requests for large files, while using alternative I/O mechanisms (e.g., asynchronous event dispatching) for requests for small files.
Modern operating systems provide multiple concurrency mechanisms to develop high-performance Web servers. Synchronous multi-threading is a popular mechanism for developing Web servers that must perform multiple operations simultaneously to meet their performance requirements. In addition, an increasing number of operating systems support asynchronous mechanisms that provide the benefits of concurrency, while alleviating much of the performance overhead of synchronous multi-threading.
This paper provides two contributions to the study of high-performance Web servers. First, it examines how synchronous and asynchronous event dispatching mechanisms impact the design and performance of JAWS, which is our high-performance Web server framework. The results reveal significant performance improvements when a proactive concurrency model is used to combine lightweight concurrency with asynchronous event dispatching.
In general, however, the complexity of the proactive concurrency model makes it harder to program applications that can utilize asynchronous concurrency mechanisms effectively. Therefore, the second contribution of this paper describes how to reduce the software complexity of asynchronous concurrent applications by applying the Proactor pattern. This pattern describes the steps required to structure object-oriented applications that seamlessly combine concurrency with asynchronous event dispatching. The Proactor pattern simplifies concurrent programming and improves performance by allowing concurrent application to have multiple operations running simultaneously without requiring a large number of threads.
This paper describes the design and performance of an object-oriented communication framework being developed by Kodak Health Imaging Systems and the Electronic Radiology Laboratory at Washington University School of Medicine. The framework is designed to meet the demands of next-generation electronic medical imaging systems, which must transfer large quantities of data efficiently and flexibly in a distributed environment. A novel aspect of this framework is its seamless integration of flexible high-level CORBA distributed object computing middleware with efficient low-level socket network programming mechanisms. In the paper, we outline the design goals and software architecture of our framework, illustrate the performance of the framework over high-speed ATM, and describe how we applied design patterns to resolve challenges we faced when developing an object-oriented communication framework for distributed medical imaging.
Trade Journal Publications
This article explores various techniques and patterns for implementing POSIX condition variables correctly and/or fairly on Win32.
Technical Reports