A trip through the guts of YARP

Table of Contents

• Behind the facade
• Summary of important implementation classes
• Port creation
• Connection creation
• Multicast elections

The goal of the tutorial is to help people find their way around the internals of the YARP implementation.

Most people will never need to read it. It could be useful if you want to extend or modify YARP in radical ways.

Beware: classes that are deemed to be "internal" (basically, anything in namespace yarp::os::impl, yarp::sig::impl, etc) are more likely to change as time goes by than other classes.

Behind the facade

Many YARP user-facing classes have private implementations, that is they follow this pattern:

class UserFacingClass {
   ...
private:
  void *implementation;
};

There are two reasons for this. One is ABI stability, so that significant changes can safely be made to the underlying implementation. Another reason is to insulate regular users of YARP from ACE header files. We use the ACE library in the implementation of YARP because of its portability, but it can turn proper compilation into an art-form. We don't want to force our users to deal with that (and don't want to deal with the confused emails they will send us). So we use ACE in the YARP implementation, but do not reference it in user-facing header files.

Summary of important implementation classes

For every port, there is a yarp::os::impl::PortCore object. The user-facing yarp::os::Port class is a superficial wrapper around this.

For every connection between two ports (call them the source port and the target port), there is a yarp::os::impl::PortCoreOutputUnit object held in a list by the source port's PortCore, and a yarp::os::impl::PortCoreInputUnit object held in a list by the target port's PortCore.

Every PortCoreUnit has a yarp::os::InputProtocol or yarp::os::OutputProtocol object that manages the overall phases of YARP's network protocol.

The Protocol object delegates to an object from the yarp::os::Carrier family (yarp::os::impl::TcpCarrier, yarp::os::impl::McastCarrier, ...) to manage the peculiarities of a particular connection type. This carrier can be replaced during the operation of a connection. Typically, connections start with a TcpCarrier, and then switch (after hand-shaking) to whatever carrier the user has selected.

Here's an example diagram of four ports. Port 1 is connected by multicast to ports 2 and 3, and by tcp to port 4.

This diagram shows logical data flow, but omits other relationships. For example, all McastCarrier objects acting as outputs for the same PortCore work together so that data only gets sent once on a shared logical bus. See Multicast elections.

Port creation

Every yarp::os::Port or yarp::os::BufferedPort contains exactly one yarp::os::impl::PortCore object. This is the main coordinator for communication. Every yarp::os::impl::PortCore, on start-up, initiates one yarp::os::impl::Face object. This object is currently always of type yarp::os::impl::TcpFace, which represents a server socket.

The PortCore creates a thread to listen to the Face (the server socket). Any initial communication with the port is made via the Face.

The PortCore maintains a list of yarp::os::impl::PortCoreUnit objects. These represent incoming and outgoing connections. When a request is received via the Face to connect to some other port, the first step is to create a new PortCoreUnit (either a yarp::os::impl::PortCoreInputUnit or a yarp::os::impl::PortCoreOutputUnit). Depending on the configuration of the port, a new thread may be created to service that unit. Inputs always get a new thread. Outputs get a new thread only if requested (see yarp::os::Port::enableBackgroundWrite; this is called for BufferedPorts by default).

Connection creation

When the Face object receives a communication from the outside world, it wraps it in a Protocol object. Concretely, this means that the TcpFace server socket takes each socket it receives and gives it to the PortCore as a Protocol object, which then creates a PortCoreUnit to manage that Protocol object. The abstraction is important here because one of the first steps of the YARP network protocol allows connections to switch from whatever the carrier used to establish the connection is (TCP) to something different for transmitting payloads (multicast, shared memory, etc). The Protocol object is either yarp::os::InputProtocol or yarp::os::OutputProtocol (in fact both of these are interfaces implemented by a single class, yarp::os::impl::Protocol, but most of YARP does not know that).

The Protocol object deals with switching to a particular carrier (implemented by sub-types of yarp::os::Carrier), and the details of the YARP network protocol.

Multicast elections

The two McastCarrier objects owned by Port 1 in the figure above "conspire" behind the scenes to make sure only one of them sends data, but as far as the rest of YARP is concerned there's nothing special about them. This conspiracy is implemented via a global yarp::os::Election object.