Port and connection protocols

Table of Contents

• The connection protocol
  • Connection phases
  • The "tcp" carrier
  • The "udp" carrier
  • The "mcast" carrier
  • The "text" carrier
  • The "shmem" carrier
  • The "local" carrier
  • Known protocol specifiers
• Port commands
• YARP URIs
• Carriers supported
• Manually interacting with ports

The connection protocol

This is the protocol used for a single connection from an output port to an input port. We discuss how this process gets initiated in the next section. At the point of creation of a connection, the following information is needed:

• An address – the machine name and socket-port at which the input port is listening.
• The name of the input port.
• The name of the output port associated with the connection. This name needs to be retained for proper disconnection in some cases. If the connection is not actually associated with a port, but is initiated by an external entity, then the name is not important and should be set to "external" (or any name without a leading slash characher).

Connection phases

The connection protocol has several phases – header, index, and body.

• Initiation phase
  • We begin once the sender has successfully opened a tcp socket connection to the receiver (assuming that is the carrier it is registered with).
• Header phase
  • This phase follows immediately after the initiation phase.
  • Transmission of protocol specifier
    • Sender transmits 8 bytes that identify the carrier that will be used. The header may be used to pass a few flags also.
    • Receiver expects 8 bytes, and attempts to find a carrier that is consistent with them.
  • Transmission of sender name
    • Sender transmits the name of the output port it is associated with, in a carrier specific way.
    • Receiver expects the name of the output port, transmitted in a carrier specific way.
  • Transmission of extra header material
    • Sender may transmit extra information, depending on the carrier.
    • Receiver may expect extra information, depending on the carrier.
• Header reply phase
  • This phase follows immediately after the header phase, and concludes the preamble to actual data transmission. After this phase, the two ports are considered connected.
  • Receiver may transmit some data, depending on the carrier. Receiver then may switch from the initial network protocol used to something else (udp, mcast, etc), again depending on the carrier.
  • Sender may expect some data, depending on the carrier. Sender then may switch from the initial network protocol used to something else (udp, mcast, etc), again depending on the carrier.
• Index phase
  • Sender sends carrier-dependent data describing properties of the payload data to come.
  • Receiver expects carrier-dependent data describing properties of the payload data to come.
• Payload data phase
  • Sender sends carrier-dependent expression of user data (maybe none).
  • Receiver expects carrier-dependent expression of user data.
• Acknowledgement phase
  • Receiver sends carrier-dependent acknowledgement of receipt of payload data (maybe none).
  • Sender expects carrier-dependent acknowledgement of receipt of payload data (maybe none).

This is the basic pattern of YARP communication between ports. Clearly different carriers have a lot of freedom in how they operate.

The "tcp" carrier

• Header and header reply
  • The 8-byte protocol specifier for tcp is: {Y A 0xE4 0x1E 0x00 0x00 R P}. Another possible variant is: {Y A 0x64 0x1E 0x00 0x00 R P}. The first version identifies a connection that sends acknowledgements; the second is for connections that omit acknowledgements.
  • The sender name is transmitted and expected to be in the following format: a 4 byte integer (little endian) giving the length of port, followed by the port name in characters, followed by the null character.
  • There is no extra header material for this carrier.
  • The header reply is as 8 bytes long: {Y A B1 B2 0x00 0x00 R P}, where (B1,B2) is a (little-endian) two-byte integer specifying a socket-port number (unused).
  • After the header reply, there is no switch in network protocol, the initial tcp connection continues to be used.
• Index, payload, and acknowledgement
  • The sender transmits 8 bytes: {Y A 0x0A 0x00 0x00 0x00 R P}. This identifies the length of the "index header" as 10 (0x0A).
  • The sender transmits 10 bytes: {LEN 0x01 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF}. LEN is the number of blocks of user data need to be sent. This byte-sequence says there are LEN send blocks, 1 reply block expected, and that the sizes will be listed individually next (this odd format is for backward compatibility with older YARP versions).
  • The sender transmits LEN 4-byte little-endian integers, one for each of the LEN blocks of user data, giving the length of each block.
  • The sender transmits 4 bytes: {0x00 0x00 0x00 0x00}. This asks for a reply length of 0.
  • If this is the variant of the tcp carrier that requires acknowledgments, then the receiver sends 8 bytes: {Y A B1 B2 B3 B4 R P}, where (B1,B2,B3,B4) is a little-endian integer giving a length (could be 0). It then sends that number of extra bytes.

The "udp" carrier

• Header and header reply
  • The 8-byte protocol specifier for udp is: {Y A 0x61 0x1E 0x00 0x00 R P}. The following variant of this should also be accepted: {Y A 0xE1 0x1E 0x00 0x00 R P} (it is the same thing).
  • Otherwise header and header reply are identical to the tcp case.
  • After the header reply, both sides switch to a udp connection to the socket-port specified in the header reply.
• Index, payload, and acknowledgement
  • Identical to tcp. Data is split arbitrarily to fit into datagrams. Datagrams have sequence numbers and checksums so that if part of message is lost or corrupted the entire message can be dropped cleanly.
  • Acknowledgments are not a possibility.

The "mcast" carrier

• Header and header reply
  • The 8-byte protocol specifier for mcast is: {Y A 0x62 0x1E 0x00 0x00 R P}. The following variant of this should also be accepted: {Y A 0xE2 0x1E 0x00 0x00 R P} (it is the same thing).
  • The sender name is sent as for tcp.
  • Extra header material is send (6 bytes). The first 4 bytes specify a multicast IP address. Next 2 bytes are a (bigendian) integer giving a socket-port number. Note that producing these numbers can be helped by side communication with the name server.
  • There is no header reply for mcast.
  • Both sides switch to a multi-cast group on the specified IP and socket-port.
• Index, payload, and acknowledgement
  • Identical to udp.
  • But at most one connection from a given port with an mcast carrier should actually write to the multi-cast group.

The "text" carrier

• This carrier is carefully designed to make it easy to type into a terminal.
• Header and header reply
  • The 8-byte protocol specifier for text is: {C O N N E C T  }.
  • The sender name is sent as plain text followed by the newline character \n.
  • There is no extra material.
  • There is no header reply expected for text.
  • There is no network protocol switch.
• Index, payload, and acknowledgement
  • There is no index.
  • The payload is expected to be a series of lines of text terminated by the newline character \n.
  • There is no acknowledgement expected for text.

The "shmem" carrier

This is essentially the same as the tcp carrier, except that there is no header reply, and there is a shift in protocol after header transmission on both sides to an ACE shared memory stream. This carrier is currently being reworked to make its specification independent of ACE, and to further improve efficiency in an existing implementation.

The advantage of this carrier is that it is fast – the best way to send messages between processes on a single machine. Of course, it doesn't work for processes on different machines.

The "local" carrier

This carrier is designed specifically for communication between threads in a single process. Giving a specification for the protocol it uses has low priority, since two such threads are unlikely to be using different YARP implementations.

Known protocol specifiers

Here are the currently known protocol specifiers. The "shmem" carrier is not yet documented, but is implemented in the C++ version of YARP.

8-byte magic number	protocol	variant
Y A 0x61 0x1E 0x00 0x00 R P	udp
Y A 0xE1 0x1E 0x00 0x00 R P	udp
Y A 0x62 0x1E 0x00 0x00 R P	mcast
Y A 0xE2 0x1E 0x00 0x00 R P	mcast
Y A 0x63 0x1E 0x00 0x00 R P	shmem
Y A 0xE3 0x1E 0x00 0x00 R P	shmem
Y A 0x64 0x1E 0x00 0x00 R P	tcp	without acks
Y A 0xE4 0x1E 0x00 0x00 R P	tcp	with acks
C O N N E C T	text	without acks
C O N N A C K	text	with acks
L O C A L I T Y	local

Port commands

Every port is always available for new connections from external entities - to request that new connections between ports be created, old connections be removed, to inquire after status, etc. % The protocol used for communicating with a port is layered on top of the protocol described in the previous section. Any carrier can be used. The "payload data" is as follows:

• We send an 8 byte header {0x00,0x00,0x00,0x00,~,CHAR,0x00,0x01}.
• CHAR is a character that identifies what the message is about.
• CHAR = d: this header is used to signal that user data is arriving next, as opposed to a port command
• CHAR = anything else: this signals that a port command follows.

for the port command case (CHAR = 0x00) the remainder of the message is interpreted as a string S.

• S begins with /, e.g. /read: this is a request to add a Connection to the named InputPort.
• S begins with !, e.g. !/read: this is a request to remove a Connection to the named InputPort.
• S begins with ~, e.g.~/read`: this is a request to remove a Connection from the named OutputPort.
• S is *: this is a request for the port to dump information about what it is connected to.
• S is q: the specific connection that the command is received on should now shut down.
• S is a: this is like d, signalling that data should be expected, but the data will not be passed on to the client of the port. Instead it will be processed internally according to The administrative interface to YARP ports. This is a more user-friendly interface for communicating with a port.

Alternatively, with the "text" carrier, we send a string terminated in \n. This is the string S. The first letter is copied to be CHAR.

YARP URIs

Port names in YARP can contain multiple special elements. We've seen names such as /write. We can also have names such as udp://write which means "connect to the port named `/write` using the udp carrier".

We can also prepend a network selector of the form /net=NETNAME/. For example, a name such as udp://net=196/write means "connect to the port named /write using the udp carrier, and make the connection on the network with ip addresses beginning with 196". This is useful in scenarios with multiple networks, where it may be desirable to route connections through particular networks (for example, to devote a network to time-critical traffic). This functionality is supported primarily with the help of the name server. The ip it reports for a machine is usually a reasonable default, but the user can choose using "net=" to request a name on a particular network.

Symbolic network names can be configured. This process is not yet specified. You can do it right now by setting properties of a fake port called networks (no leading slash), where the properties are symbolic names and their values are the numeric network IP prefix. But this process will change.

Carriers supported

An implementation of YARP2 must support at least the "tcp" carrier. Other carriers that may be supported: "text", "udp", "mcast", "shmem", "local".

As a place to start an implementation, the "text" carrier is very simple to implement, and can masquerade as "tcp" for the purposes of initial handshaking.

To see this, get the "netcat" program (available as debian package of the same name). In one terminal, run:

nc -l -p 9000

This starts a tcp listener on socket-port 9000, and prints out any data that arrives there. Then tell the name server to create an entry for this listener, and tag it as accepting text:

yarp name register /nc tcp ... 9000
yarp name set /nc accepts text

Now lets write some data to that port.

yarp write /write text://nc

Type something in, such as "hello world", and hit return. On the terminal running netcat you should see:

CONNECT /write
d
0 "hello world"

This is what text mode looks like, for the particular data type used by yarp read and write ("bottles"). As we saw in an earlier section, we can also write to ports in text mode. And if we were to restart nc and then try the following:

yarp connect text://nc /foo

You should see:

CONNECT external
/foo

This is what a command to connect looks like in YARP2. If we omit the text:/ then the tcp carrier may be used, which is compatible with YARP1 but is a bit less trivial to work with. Once our YARP implementations are up to date, the default command carrier will be switched to text.

Manually interacting with ports

Suppose we have created ports as follows by typing the following in different terminals:

  yarp server
  yarp write /write
  yarp read /read
  yarp read /read2

We could connect and disconnect ports using the YARP companion utility, but here's how we could do the same thing "manually":

command:  yarp where
response: Name server is available at ip 192.168.0.3 port 10000

command:  telnet 192.168.0.3 10000
type:     NAME_SERVER query /write
response: registration name /write ip 192.168.0.3 port 10001 type tcp
          *** end of message
          [connection closes]

command:  telnet 192.168.0.3 10001
type:     CONNECT anonymous
response: Welcome anonymous
type:     *
response: This is /write
          There are no outgoing connections
          There is this connection from anonymous to /write using protocol tcp
          *** end of message
type:     /read
response: Connected to /read
type:     *
response: This is /write
          There is a connection from /write to /read using protocol tcp
          There is this connection from anonymous to /write using protocol tcp
          *** end of message
type:     !/read
response: Removing connection from /write to /read
type:     /mcast://read
response: Connected to /read
type:     /read2
response: Connected to /read2
type:     *
response: This is /write
          There is a connection from /write to /read using protocol mcast
          There is a connection from /write to /read2 using protocol tcp
          There is this connection from anonymous to /write using protocol tcp
          *** end of message
type:     q
response: Bye bye
          [connection closes]

command:  telnet 192.168.0.3 10000
type:     NAME_SERVER query /read
response: registration name /write ip 192.168.0.3 port 10002 type tcp
          *** end of message
          [connection closes]

command:  telnet 192.168.0.3 10002
type:     CONNECT anonymous
response: Welcome anonymous
type:     *
response: This is /read
          There are no outgoing connections
          There is a connection from /write to /read using protocol mcast
          There is this connection from anonymous to /read using protocol tcp
          *** end of message
type:     q
response: Bye bye
          [connection closes]