Open Grid Forum

Standards Home

Why Standards?
Developing Standards

Using OGF Standards
Areas and Groups OGF Document Series

OGF ISV Primer

Running Applications Remotely

The Problem

Running applications remotely – on arbitrary resources – is becoming increasingly common in research, engineering and commerce. The Open Grid Forum (OGF) has identified through its community of end-users, middleware providers, inde-pendent software vendors (ISVs), and resource providers a set of common scenarios that they would like to be supported through standards rather than the current adhoc solutions which are difficult to integrate and coexist in modern networking environments.

Using specifications, that have been developed within and external to the Open Grid Forum, we show these standards can be used to enable desktop access to distributed computing resources. Increasingly, applications are found to have two components – a graphical client that resides on the desktop and a compute intensive server component that can run on a ‘back-end’ compute cluster. Integrating these two components within networks which frequently have firewalls and NATs, across different operating systems and software environments, provides many challenges. A standards based in-terface to this environment would provide ISVs with a solid foundation upon which to build their own applications.

The Scenarios

Scenario Description
Execution of an application on a cluster from a submission host directly using a proprietary distributed resource man-ager and its API where the submission host shares a file system with the head and compute nodes.
Execution of an application on a cluster from a submission host using a standardized API to access a proprietary dis-tributed resource manager to manage the application.
Execution of an application on a cluster from a submission host using a web service protocol and a shared file system.
Execution of an application on a cluster from a client workstation using a web service protocol with explicit file staging instead of a shared file system thereby allowing the client to be disconnected while the application runs.
Execution of an application on a cluster from a client workstation using a web service protocol with two way interaction between the client workstation and the running application.

Further information on these scenarios and how they can be supported through standards can be found in the ‘Independent Software Vendors Remote Compiuting Usage Primer’ -

The Technology

In defining solutions to these scenarios, the Open Grid Forum looks to reuse existing standards before defining new ones. The existing specification pool includes:

Specification Description
WS-Addressing (W3C) Provides an extensible XML data structure called the endpoint reference (EPR) that encapsulates the information needed by a client to message a service, e.g. network protocol address, metadata and session/resource identifiers.
WS-Security (OASIS) General-purpose mechanism for associating security credentials (e.g., X.509, Kerberos, SAML, and Username-token credentials) with message content to provide end-to-end messaging integrity and confidentiality regardless of the underlying communication pro-tocol.
WS-Secure Addressing (GFD 131) Profiles WS-Addressing and WS-Security Policy to securely bind metadata such as keys, type, identity, and security policy into WS-Addressing endpoint references (EPRs). For example, what authentication mechanisms does an endpoint support or require.
WS-Secure Communication (GFD 132) Profiles commonly used security mechanisms defined in the WS-I Basic Security Profile to define a small interoperability core.
Job Submission Description Language (JSDL) (GFD 56) JSDL describes the application and the resources required for an application (e.g., memory, number and speed of CPU’s, etc.), files to stage-in before the application exe-cutes, files to stage-out upon completion, the command line string to be executed, etc.
JSDL Single Process Multi-ple Data Application Exten-sion (GFD 115) Extends JSDL’s Application element to support the definition of parallel applications by including the number of processes, the number of threads per process, how many proc-esses to execute per host and the MPI environment required for execution.
JSDL Parameter Sweep Extension (In public comment) Defines how the values of a JSDL document may be changed to generate a collection of jobs from a single document by substituting values from arrays or loops.
Basic Execution Service (BES) (GFD 108) A web service interface for creating, monitoring, and controlling computational entities such as UNIX or Windows processes, or parallel programs, from JSDL documents on a single computer or clusters through a resource manager such as LSF, SGE, PBS, Windows HPC Server 2008 or Condor; or even another BES implementation.
HPC Profile Application Extension (GFD 111) Extends JSDL’s Application element to define applications for execution on HPC re-sources.
HPC Basic Profile (HPCBP) (GFD 114) Profiles BES and JSDL to use the HPC Profile Application Extension to run HPC appli-cations on a cluster by using username/password or X.509 certificate to identify the cli-ent.
File Staging Extension to the HPC Basic Profile (GFD 135) Defines how the HPC Basic Profile can support the JSDL DataStaging element to copy remote files to or from the compute resource using different protocols and credentials, and if existing files should be overwritten on startup or deleted after the job has com-pleted.
ByteIO (GFD 87) Provides POSIX-like read and write operations on sequences of bytes through two inter-faces. In the RandomByteIO interface, the offset, number of bytes, and data buffers are passed to the operations. In the Streamable ByteIO interface, the operations do not take the offset.
GLUE (In public comment) The GLUE 2.0 specification defines a model for describing cluster resources, and more broadly resources that may be shared between different communities through defined service interfaces.
Distributed Resource Man-agement Application API (DRMAA) (GFD 22) Provides an API around the submission and monitoring of jobs into local distributed re-source management systems. See for more details.
Resource Namespace Service (RNS) (GFD 101) Provides a hierarchical directory structure that maps human comprehensible string paths to EPRs much as a Unix directory maps string paths to inodes.

> login   RSS RSS Contact Webmaster

OGFSM, Open Grid ForumSM, Grid ForumSM, and the OGF Logo are trademarks of OGF