NXM Help RMIF

From ICE Enterprises
Jump to navigation Jump to search
Go to the full list of NXM Help pages.

RMIF - the Remote Midas InterFace protocol.

The RMIF protocol is a take-off on Java's RMI (Remote Method Invocation).

Java's RMI is designed to make objects at a remote site accessible as if they are local objects. A local reference to the remote object is created and method calls on the local reference are passed to the remote object via a transport layer, usually TCPIP. The method name and its arguments are serialized, sent to the remote processor, and executed. The optional resulting object is serialized, sent back to the local processor and reconstructed in the objects local reference.

The main purpose of the RMIF protocol is to allow NeXtMidas GUIs to control Midas applications at remote sites. While the protocol may be applied to other problems, it's primary goal is to minimize link bandwidth for data displays and respond robustly to network drop-outs common to real-world long-distance multi-hop links.

RMI - Java's Remote Method Invocation protocol.

RMI was not used directly for a number of reasons:

  1. the server side applications are not typically Java based (C,C++,FORTRAN).
  2. the standard RMI implementation is layered on TCP, which severely
    limits throughput and robustness over real-world links.
  3. object serialization requires unnecessary over-head and
    data conversions to/from network ordering for data displays.

The RMI reference was kept to make it easier for developers that may have experience with Java RMI. The concepts and methodology were paralleled wherever possible.

The current implementation of RMIF is layered on top of UDP. All routing to and from remote objects is handled by the UDP header. There is no subrouting within the RMIF protocol. An RMIF object is defined by a node name (or network address) and a port number.

Header - The RMIF 8-byte packet header.

The UDP payload begins with an 8-byte RMIF header. The RMIF protocol uses the 8-byte header to, in effect, compress the overhead that would be necessary to serialize a method call. The header bytes (in order) are:

FUNC - defines the method to call
FLAG - is the protocol version number and options mask
INFO - channel number for fast routing of monitored properties
REP  - packet data representation (VAX|IEEE|EEEI)
SEQ  - sequence number (0-127) for Reliable packets
TRY  - number of re-sends for Reliable packets
RPT  - report sequence number
ADJ  - number of bytes in adjunct header

The size of the data portion of the payload is the UDP packet size minus the 8 byte header and the adjunct header if ADJ>0.

AdjunctHeader - The RMIF adjunct header.

The optional adjunct header follows directly after the 8-byte RMIF header. It must be a multiple of 8-bytes as well for alignment purposes. The currently supported adjunct header options are:

ADJ_TIME   - an 8-byte double precision time in J1950 seconds
ADJ_OFFSET - an 8-byte double precision file offset for MFTP

In each of these cases, the ADJ field in the RMIF header equals 8. The data portion of the payload will start at the 16th byte instead of the 8th.

Functions - The currently defined RMIF functions.

The currently supported functions (or method calls) are:

GET  - request the value of a named property
RET  - return the value for a GET request
SET  - set a named property
ACK  - acknowledge the value for a SET request
OPEN - open connection to a remote or a channel to one of it's properties
OPENED - acknowledges an OPEN request
CLOSE - close connection to a remote or a channel to one of it's properties
CLOSED - acknowledges an CLOSE request
MODIFY - modify a channel connection
PING - request keep-alive status
PONG - return keep-alive status
DHDR - process data header
DBUF - process data buffer
DBUFC - process compressed data buffer
DBUFD - process compressed data buffer
GETPROP - request list of published properties
RETPROP - return number of properties or property names
RECEIPT - acknowledge receipt of a reliable packet
MFTP - process the Midas File Transfer Protocol

Methods - How RMIF packets are converted to function calls.

The RMIish components of the protocol are accomplished with the GET, RET, SET, and ACK functions. The Java language suggests the use of public methods beginning with "set" and "get" to provide a common interface to an object's public properties. To allow combining multiple SETs or GETs in a single network packet, the data portion of the GET|RET|SET|ACK packet is a serialized table class. The key for each table entry is the body of the method name, and the value for each key is the method's argument. For example, to call the setFreq, setAmp, and setOptions methods on a remote object the data portion of a RMIF packet with the FUNC=SET would be:

{FREQ=1.3E6,AMP=1,OPTIONS={MODE=1,AXIS=NONE}}

The syntax is all ASCII to allow using off-the-shelf tools or XML parsers to monitoring network traffic. Note that the setOptions() method has more than one argument. To do this, the arguments are expressed as another table, with each argument being named. This again aids in monitoring/debugging.

RMIvsRMIF - Why RMIF and UDP instead of RMI and TCP.

The RMIF protocol uses considerably less bandwidth than RMIs serialization of calls for most functions. Also, TCP socket writes eventually end up in TCP packets that have more overhead than UDP. In addition, when and where socket writes get broken into TCP packets is not under the application's control. If network dropouts cause a TCP packet to be dropped, this may or may not lie on a functional boundary for a method call making graceful recovery difficult.

Another benefit of UDP is throughput. In TCP, system drivers usually limit the number of outstanding packets to 12. This means that until the remote connection receives the packet and acknowledges receipt, the local socket will hold-off outgoing packets. Using the typical maximum transfer unit, MTU, of 1500 bytes and roundtrip delay of 500 ms, we can calculate a maximum sustained transfer rate of 12*1500/.5 = 36kby/sec.

The RMIF protocol implements the reliable portion of the TCP layer at the user level, allowing the application to set the maximum number of outstanding packets (default=12,max=128) and which packets to send reliably or unreliably. Typically, data display packets are sent unreliably, and all others reliably. Unreliable packets set the SEQ header byte to -1. Reliable packets have SEQ set to a one-up counter (0-127) by the sender, along with a retry count TRY. If the sender does not get a receipt after a tunable tolerance plus the last measured round trip delay, it will resend the packet. By combining the reliable and unreliable streams in the same socket, we can implement priority schemes that, for instance, favor controls over data.

Another benefit of UDP is that it is connectionless. A UDP port on a machine is basically a bucket on a node to catch network packets that are thrown at it. They can be thrown from anywhere, from any number of sources, in any order. Each packet contains the network address and port number of who threw the packet, so you can throw something back if called for. Then you're done. No opens, no closes. Network problems cannot cause connections to hang in an unknown state, because there never is a connection. This leaves the timing out and recovery to the application, where appropriate tuning may be applied.

Most of the protocol handling will be administered by an RMIF primitive or component on each end of the connection. This primitive should insulate the framework from the RMIF protocol and convert the method calls to appropriate constructs for that framework. See the RMIF Explain File for more details.

MFTP - The Midas File Transfer Protocol

An RMIF connection also supports file transfers in both directions. The client typically requests a file from the server, however, the client can post a file to the server if allowed.

The low level packets for accessing a file are as follows:

Client: func=MFTP data={FUNC=EXIST,NAME=filename} Server: func=MFTP data={FUNC=EXIST,NAME=filename,EXISTS=Y|N}

Client: func=MFTP data={FUNC=OPEN,NAME=filename,RATE=byte/sec,PKTLEN=bytes} Server: func=MFTP data={FUNC=OPEN,NAME=filename,LENGTH=bytesAtOpen}

Client: func=MFTP data={FUNC=READ,NAME=filename,OFFSET=byte,LENGTH=bytes,RATE=byte/sec,PKTLEN=bytes} Server: func=MFTP data={FUNC=READ,NAME=filename,OFFSET=byte,LENGTH=availbytes} Server: func=DBUF adj=offset data=binaryDataBuffer (multiple packets)

Client: func=MFTP data={FUNC=CLOSE,NAME=filename} Server: func=MFTP data={FUNC=CLOSE,NAME=filename,LENGTH=bytesAtClose}

The RATE and PKTLEN specifiers are optional. They default to RATE=2e6 and PKTLEN=8K.

This level of protocol deals with raw files or an IOResource in NeXtMidas. It does not understand Midas file headers, detached headers, or detached packet files. This must be handled by the calling routine, usually the DataFile class.

For example, an RMIF server on port 9001 can serve a file using the command:

nM> noop/gpw  mftp://remoteHostName:9001/remoteFileName  localFileName

where the /gpw invokes a graphical progress widget.