Programming Environments for Grids and Distributed Computing Systems

Vaidy Sunderam

Department of Mathematics and Computer Science
Emory University, USA

Platforms for high-performance concurrent computing range from tightly-coupled multiprocessors to clusters, networks of workstations, and large-scale geographically distributed metacomputing systems. Effective application programming for each type of environment can be a challenge, given the wide variability and heterogeneity of the computational resources, communication interconnects, and level of dedicated access. In this tutorial, we will review and compare programming paradigms and associated runtime systems for high-performance concurrent computing in multiple types of distributed system environments. After a brief discussion of language-based approaches, we will analyze the underlying concurrent programming models embodied in MPI and PVM, with a particular focus on their suitability in loosely coupled distributed systems and grids. The emerging paradigm shift in computational grids to a standards-based service-oriented model will then be explored, and methodologies for developing and programming grid services will be discussed. Examples of writing and deploying grid services as well as clients will be included. The tutorial will conclude by summarizing the essential features and characteristics of programming different distributed computing environments and discussing approaches to application development using a combination of programming paradigms.

Messaging Systems: Parallel Computing the Internet and the Grid

Geoffrey Fox

Indiana University
Community Grids Computing Laboratory
Computer Science, Informatics and Physics
USA

We contrast the requirements and performance of messaging systems in parallel and distributed systems emphasizing the importance of the five orders of magnitude difference in network hardware latencies in the two cases. We note the importance of messaging in Grid and Web service applications in building the integrated system and the architectural advantages of a message based compared to a connection based approach. We illustrate these points using the NaradaBrokering system and its application to Audio-Video conferencing.

Petascale Virtual Machines

Al Geist

Oak Ridge National Laboratory
USA

PVM continues to be a popular software package both for creating personal grids and for building adaptable, fault tolerant applications. We will illustrate this by describing a computational biology environment built on top of PVM that is used by researchers around the world. We will then describe or recent progress in building an even more adaptable distributed virtual machine package called Harness. The Harness project includes research on a scalable, self-adapting core called H2O, and research on fault tolerant MPI. The H2O core can be configured to support distributed web computing like SETI@home, parallel virtual machines like PVM, and OGSA compliant grid environments. The Harness software is being designed to scale to petascale virtual machines. We will describe work at Oak Ridge National Lab on developing algorithms for such petascale virtual machines and the development of a simulator to test these algorithms on simulated 100,000 processor systems.

Future Developments in MPI

William Gropp

Mathematics and Computer Science Division
Argonne National Laboratory
USA

The Message Passing Interface (MPI) has been very successful at providing a programming model for computers from small PC clusters through the world's fastest computers. MPI has succeeded because it successfully addresses many of the requirements of an effective parallel programming model, including portability, performance, modularity, and completeness. But much remains to be done with MPI, both in terms of the performance of MPI and in the supporting the use of MPI in applications. This talk will look at three areas: programming models, implementations, and scalability.
The MPI programming model is often described as a supporting ``only'' basic message passing (point-to-point and collective) and (in MPI-2) simple one-sided communication. Such a description ignores the support in MPI for the creation of effective libraries built using MPI routines. This support has encouraged the development of powerful libraries that, working with MPI, provide a powerful high-level programming environment. This will be illustrated with two examples drawn from computational simulation.
MPI was designed to allow implementations to fully exploit the available hardware. It provides many features that support high performance, including a relaxed memory consistency model. While many MPI implementations take advantage of some of these opportunities, much remains to be done. This talk will describe some of the opportunities for improving the performance of MPI implementations, with particular emphasis on the relaxed memory model and both MPI's one-sided and parallel I/O operations.
Scalability is another goal of the MPI design and many applications have demonstrated scalability to thousands of processors. In the near future, computers with more than 64,000 processors will be built. Barriers to scalability in the definition and the implementation of MPI will be discussed, along with possible future directions for MPI developments. By avoiding a few very low usage routines and with the proper implementation, MPI should scale effectively to the next generation of massively parallel computers.

ASSIST High-performance Programming Environment: Application Experiences and Grid Evolution

Marco Vanneschi

Dipartimento di Informatica
Università di Pisa
Italy

ASSIST (A Software development System based upon Integrated Skeleton Technology) is a new programming environment oriented to the development of parallel and distributed high-performance applications according to a unified approach. The main goals are: high-level programmability and software productivity for complex multidisciplinary applications, including data-intensive and interactive software; performance portability across different platforms, from homogenous parallel machines and possibly heterogeneous clusters to large-scale enabling platforms and computational Grids; effective reuse of parallel and distributed software; efficient evolution of applications through versions scalable according to the underlying technologies.
The programming model of ASSIST has been defined according to two main issues: i) evolution of structured parallel programming, starting from pros/cons of our past experience with the skeletons model, ii) joining the structured programming approach with the objects/components technology.
This talk is organized in two parts. In the first part we discuss the utilization of ASSIST according to a set of application and benchmarking experiences performed during the last year. Such experiences have been done with the ASSIST 1.0 version of compiler and run-time system for Linux clusters; however, the run-time system is based upon ACE portable libraries, thus it is able to support ASSIST programs for heterogeneous clusters too. We show the expressive power and performance measures, and possible weaknesses to be overcame in the next versions of ASSIST.
In the second part we discuss the evolution of ASSIST for large-scale platforms and Grids. The development of Grid applications requires, in general, capabilities and properties beyond those needed in both sequential programming and in "classical" parallel/distributed programming, as it requires the management of computations and environments that are typically dynamic and heterogeneous in their nature and that include resource hierarchies with different features (e.g. memory and network). These issues are investigated in the Italian National Programme in Grid Computing (Grid.it project). Referring to such context, we discuss here some ideas and preliminary results.

Integrating Scalable Process Management into Component-Based Systems Software

Ewing Lusk

Mathematics and Computer Science Division
Argonne National Laboratory
USA

The Scalable Systems Software Project is exploring the design of a systems software architecture based on separate, replaceable components interacting through publicly defined interfaces. This talk will describe how a scalable process manager has provided the implementation of the process management component of that design. We describe a general, implementation-independent definition of process management and how a scalable process management system was used to provide its implementation.

Programming High Performance Applications using Components

Thierry Priol

IRISA/INRIA
Rennes Cedex
France

Computational Grids promise to be the next generation of high-performance computing resources. However, programming such computing infrastructures will be extremely challenging. Current programming practices tend to be based on existing and well understood models such as message-passing and SPMD (single program multiple data). On-going works based on Web services (OGSA) aims at programming Grids by specifying the coordination through the expression of interaction and dependencies between already deployed web services. This talk will focus on another alternative that aims at programming Grids with software components. With such an approach, it will be possible to install components on remote resources and to express interaction between components. This presentation will describe work in progress to develop a component-based software infrastructure, called Padico, for computational Grids based on the CORBA Component Model (CCM) from the Object Management Group (OMG). The objective of Padico is to offer a component model targeting multi-physics simulations or any applications that require the coupling of several codes (simulation or visualisation) within a high-performance environment. Two issues will be addressed during the talk: encapsulation of codes into components and runtime support for efficient communication between components within a Grid. This talk will look at the GridCCM component model, an extension to the CCM model to address the encapsulation of SPMD parallel codes into a component and a communication framework, called PadicoTM, able to efficiently support various communication runtime and middleware in an heterogeneous networking environment.