|
HP OpenVMS Systems Documentation |
|
OpenVMS Programming Concepts Manual
3.3 Intra-Cluster CommunicationIntra-cluster communication (ICC), available through ICC system services, forms an application program interface (API) for process-to-process communications. For large data transfers, intra-cluster communication is the highest performance OpenVMS application communication mechanism, better than standard network transports and mailboxes. Intra-cluster communication enables application program developers to create distributed applications with connections between different processes on a single system or between processes on different systems within a single OpenVMS Cluster system. Intra-cluster communication does not require a network product. It uses memory or System Communication Services (SCS). The intra-cluster system services used to implement intra-cluster communication do the following:
Intra-cluster system services provide the following benefits when implementing ICC:
The intra-cluster communication system services are as follows:
See the OpenVMS System Services Reference Manual: GETUTC--Z for additional information about the ICC system
services.
The following sections provide information on how to program with
intra-cluster communictions (ICC) using ICC system services.
The following terms and their definitions are central to creating and using intra-cluster communication. An ASSOCIATION is a named link between an application and ICC. The association name, combined with the node name of the system on which the application is running, identifies this application to other applications that want to communicate within a node or cluster. An ASSOCIATION NAME is a string that identifies an ASSOCIATION. Association Names are 1 to 31 characters in length and must be unique within a node. Association Names are case-sensitive. Associations are created by calling the SYS$ICC_OPEN_ASSOC service (or by the first call to SYS$ICC_CONNECT if OPEN was not previously called) and are identified by an Association Handle. An ASSOCIATION HANDLE is an opaque identifier that represents an association to ICC. Association Handles are passed as unsigned longwords. A NODE is either a standalone system or a member of an OpenVMS Cluster. It is identified by a one-to-six-character case-blind name that matches the SYSGEN parameter SCSNODE. A CONNECTION is a link between two associations created by calling the SYS$ICC_CONNECT or SYS$ICC_ACCEPT service. An association may have multiple simultaneous connections at any given time. A connection is identified by a Connection Handle. A given application may choose to either only initiate connections, or initiate and accept connections as well. Connections support both synchronous and asynchronous communications. A CONNECTION HANDLE is an opaque identifier that represents a connection to ICC. Connection Handles are passed as unsigned longwords. In ICC terminology, a SERVER is an application that has declared to ICC that it is willing to accept connections. A server must call SYS$ICC_OPEN_ASSOC and supply the address of a connection routine. Upon receipt of a connection request AST, a server completes its side of the connection by calling the SYS$ICC_ACCEPT service. It may also choose not to accept the connection request by calling SYS$ICC_REJECT. A CLIENT, in the ICC model, is an application that calls SYS$ICC_CONNECT to request a connection to a server. Note that a server may also be a client. A client need not call SYS$ICC_OPEN_ASSOC (although there are certain benefits to doing so) but may instead call SYS$ICC_CONNECT using the supplied constant Default Association Handle. An association opened in this manner is called the Default Association. Only one default association is permitted per process; however, any number of connections may be established even when there is only one association.
The ICC SIMPLE CLUSTERWIDE REGISTRY provides a method for servers to
register their node and association names under a single logical name,
thus eliminating the need for clients to determine on which node the
server process is running. Multiple servers can register under the same
logical name, and ICC will randomly select one from the list. The
selection algorithm leads to load sharing, but not necessarily to load
balancing.
This section contains information about ICC design considerations.
An ICC server may have two visible names---the association name and, if the ICC registry is used, a registry name. The registry name may be the same as the association name.
If, however, you wish to run multiple copies of the server on a single
node, the registry name should be the more general name, while the
association name should reference a particular instance of the server.
ICC guarantees that messages will be delivered in the order in which
they were transmitted. There is no provision for out-of-order delivery.
ICC implements flow control on a per-association basis. You can change this value by specifying a value for the MAXFLOWBUFCNT parameter to the SYS$ICC_OPEN_ASSOC service. In addition to flow control, ICC uses the value of MAXFLOWBUFCNT to determine preallocation of certain resources cached by the services for performance reasons. Increasing this value results in a larger charge against process quotas. The default value of MAXFLOWBUFCNT is 5.
Because all ICC connections within the association are subject to flow
control, failure to receive data in a timely manner will eventually
cause all connections in the association to stall.
ICC supports two models for receiving data. In the simple receive model, a SYS$ICC_RECEIVE call either completes immediately if data is present, or else the receive buffer is queued up to be filled when data arrives from the sender. SYS$ICC_RECEIVEW does not return to the caller until either data is present, or the connection has disconnected. The major disadvantage to this method is that the buffer you supply to SYS$ICC_RECEIVE must be large enough to receive the largest message the sender might transmit. Receipt of a message larger than the queued buffer causes ICC to disconnect the link to preserve order. There is no provision within ICC to break a single message over multiple receive calls. The second model is data event driven. In this method, the application provides SYS$ICC_OPEN_ASSOC (parameter recv_rtn) the address of a routine to be called whenever a connection established over this particular association is the target of a TRANSMIT. The data routine is called at AST level in the original mode of the caller and supplied with the size of the incoming transfer, the connection handle for this transfer, and a user-supplied context value. Once notified of incoming data, the application must then allocate a sufficiently large buffer and issue a SYS$ICC_RECEIVE call to obtain the data. The maximum transfer size using this method is 1 Mb. The SYS$ICC_RECEIVE call does not have to be made from the data routine; however, a receive request cannot be made before receipt of a data event. Therefore, receive operations are never queued within ICC when you use a data event.
Because there is a single data routine per association, all connections
on the association share the same data routine. To have some
connections use both methods, at least two associations must be opened.
SYS$ICC_TRANSCEIVE is a single service that sends a message and
completes when the other side responds with a call to SYS$ICC_REPLY.
The maximum size of the return message is fixed by the size of the
buffer the caller provides to the transceive service at the time the
call is made. The maximum transmission side for both SYS$ICC_TRANSCEIVE
and SYS$ICC_REPLY is 1 Mb. The minimum length of the reply transmission
is zero bytes.
In a properly coded communications protocol, responsibility for disconnecting the connection should be left to the side of the protocol that last received data. This rule assures that there is no data in transit in which the state may be uncertain at the time the disconnect event occurs. An ICC connection may be disconnected under any of the following circumstances:
No matter what the cause of the disconnection, the application should
be prepared to deal with disconnection other than that as an expected
part of the application communication protocol.
Whenever possible, ICC services attempt to return an appropriate status
value to the caller, either as the routine status value or in the
status fields of the IOSB or IOS_ICC structure. Sometimes the only
recovery method available to ICC is to disconnect the connection. ICC
disconnects only when it lacks sufficient context to return status to
the calling application.
The ICC public structures and constants are declared in the module
$ICCDEF (MACRO), within STARLET.REQ for BLISS and ICCDEF.H in
SYS$LIBRARY:SYS$STARLET_C.TLB for C. STARLET.H provides full system
service prototypes for C.
To open an association to ICC, call ICC$OPEN_ASSOC, which provides the following parameters:
3.3.1.4.1 Connection EventsOnce the association is opened, connection requests are delivered as asynchronous calls to the server's connection routine. The connection routine (and disconnection routine, if supplied) are each called with seven arguments as described in the $ICC_OPEN_ASSOC system service, located in the OpenVMS System Services Reference Manual: GETUTC--Z. A single connection or disconnection routine may distinguish between the events based on the first argument (event_type), which will either be the code ICC$C_EV_CONNECT or ICC$C_EV_DISCONNECT. A connection routine must either call SYS$ICC_ACCEPT to complete the connection or SYS$ICC_REJECT to reject the connection. The EPID and username of the requesting process are supplied by ICC. Any additional information required by the application for verification either must be supplied by the client via the connection data parameter or must be obtainable by the server itself (for example, via SYS$GETJPI.)
Failure to ACCEPT or REJECT the connection will stall all additional
attempts to connect to this server.
A disconnect event signals either that the partner application of the connection has requested a disconnect, the partner process has been deleted, or that the remote node has left the cluster. Upon receiving a disconnect event, the server should call SYS$ICC_DISCONNECT to clean up the server side of the connection.
Failure to call DISCONNECT may leave resources allocated to the
connection that will not be released until the image terminates. In the
case of inner mode associations, resources will be released at process
termination.
A simple client need not call SYS$ICC_OPEN_ASSOC although there may be some benefits for even the simplest clients. For a client, the major benefit of calling OPEN is declaring a data receive routine. In cases where a client may wish to connect to multiple servers, calling OPEN_ASSOC multiple times can help isolate the routines related to data receipt and disconnection. Call SYS$ICC_CONNECT[W] with the following arguments:
When the CONNECT call completes, the connection is established and useable only if success status was returned as both the return value of the service and in the status fields (ios_icc$w_status and ios_icc$l_remstat) of the IOS_ICC.
|