YARP data directories

Table of Contents

• Rationale
• Data directories contents
  • Linux
  • Windows
  • MacOS
• ResourceFinder Search policy
  • Contexts and Robots files
    • Shadowing
  • Plugins Manifest files
  • Application Deployment files
• Managing data files: the yarp-config utility
• 3rd party packages (for developers)
  • Installed packages
  • Extending the search path

YARP utilities and modules written by users need to find and access data and configuration files at run-time.

The former use generic configuration files and manifest files for modules and plugins. The latter often require configuration and data files. YARP has an OS-dependent set of default locations where to look for files, and a set of rules that define the search priorities. The default locations can be overridden or extended by modifying the environment variables that YARP relies on. To simplify finding and accessing these files YARP provides a helper class called ResourceFinder.

A full description of the directories and environment variables used by the ResourceFinder is too detailed for most users. We provide here a short and simplified description of the files layout of typical YARP installations for Windows, Linux and MacOs.

If you are confused by this description you can jump directly to the tutorials ResourceFinder Tutorials and Specification.

Rationale

The key points to keep in mind are:

• The software installation should be able to provide reasonable defaults for configuration files so that applications can run out-of-the-box;
• Installation directories are generally non-writable, users without root privileges cannot edit installed configuration files unless they first copy them inside their own private directories, the latter must take precedence and hide the others;
• Therefore it is normal that configuration files can be in multiple places, inside user-specific, private or shared installation directories;
• External packages can install files so that YARP can find them (see here);
• Types of files are divided in families that are placed in specific sub-directories. Current allowed sub-directories are listed here.

File customization can happen by copying files or contexts from the shared installation directories to the ones private to the user. The yarp-config: the yarp-config tool utility simplifies this process.

Data directories contents

Files are normally organized in the following directories:

• contexts
• robots
• plugins
• applications
• modules
• resources
• templates
  • applications
  • modules

These directories can be stored in a directory private for the user (usually in its home) or installed in a shared directory.

Linux

The private directories for the user are in $HOME/.local/share/yarp

The shared directories for installation are usr/local/share/yarp and /usr/share/yarp

The latter can be changed by defining the environment variable YARP_DATA_DIRS or extended through path.d (see below).

Windows

The private directories for the user are in %APPDATA%\yarp\

The shared directory for installation are in %YARP_DIR%\share\yarp\

The latter can be changed by defining the environment variable YARP_DATA_DIRS or extended through path.d (see below).

MacOS

The private directory for the user are in $HOME/Library/Application Support/yarp

The shared directory for installation are in /usr/local/share/yarp and /usr/share/yarp

The latter can be changed by defining the environment variable YARP_DATA_DIRS or extended through path.d (see below).

ResourceFinder Search policy

The resource finder searchers for files in the following order of precedence:

• Looking in the current working directory
• Searches in the robot directory
  • In the user private directory
  • In the shared, installation directory(ies)
• Searches in the context directory
  • In the user private directory
  • In the shared, installation directory(ies)

When searching for files and directories, the ResourceFinder looks inside these directories in the above order, so that files modified by the user take precedence over installed ones.

Contexts and Robots files

The contexts folders contain multiple sub-folders. Each subfolder represents a context, i.e., a collection of configuration files and data that may be used to configure modules at runtime and read/write any type of files. When the resource finder is configured with a specific context-name (either a default, hard-coded one, or provided as command line parameter to the module), contexts/<context-name> is added to the search path in which the initial configuration file and any additional files are sought.

The robots folders contain multiple sub-folders as well. Each subfolder has the name of a robot, and contains robot-specific files. The ResourceFinder adds robots/$YARP_ROBOT_NAME (default value for this environment variable: default) to the search path, before the context-specific paths.

Shadowing

Search of files proceeds from the user private directories to shared installation directories. The same <context-name> directory can appear in multiple places and likely contain files with the same name. In this case the files that are found first take precedence and hide files in the other locations. This shadowing or masking mechanism is useful when the user needs to customize only a subset of the files of a specific context.

Plugins Manifest files

Plugin manifest files are INI files that describe which class implements the plugin, the library it is included in, and how to find such library. These files are needed when loading plugins (e.g. by the yarpdev executable).

Application Deployment files

Applications, modules and resources folders contain XML files that describe programs (modules), the resources available to run the programs (resources), and collections of interconnected modules (applications). The yarpmanager executable allow to run and monitor applications, while the yarpbuilder executables allows to design new ones.

Templates are XML files with a .template extension. Typical templates are applications that need to be configured (i.e., assigning the node where each module should be run) before they can be executed.

Managing data files: the yarp-config utility

Managing configuration and data files requires nothing special, you can copy, edit and remove them using the shell. However YARP provides an utility that simplifies this, see yarp-config: the yarp-config tool.

3rd party packages (for developers)

If you are not a developer you can skip this section.

Other packages need to contribute configuration files for their modules, applications, plugins etc. This can be done in two ways.

• By installing the above files in the YARP directories (YARP exports a set of CMake variables that can be used to this aim);
• By extending the ResourceFinder search path.

YARP provides cmake variables and macros to simplify both the above tasks.

If you are not interested in the details you can skip the following sections and go directly to Configuring your external build.

Installed packages

If yarp is installed the

• YARP_INSTALL_PREFIX will contain the prefix where YARP was installed. If yarp is used from the binary tree, this variable is empty.

YARP exports the following CMake variables that contain relative installation paths:

• YARP_DATA_INSTALL_DIR
• YARP_CONFIG_INSTALL_DIR
• YARP_PLUGIN_MANIFESTS_INSTALL_DIR
• YARP_MODULES_INSTALL_DIR
• YARP_APPLICATIONS_INSTALL_DIR
• YARP_TEMPLATES_INSTALL_DIR
• YARP_CONTEXTS_INSTALL_DIR
• YARP_APPLICATIONS_TEMPLATES_INSTALL_DIR
• YARP_MODULES_TEMPLATES_INSTALL_DIR
• YARP_ROBOTS_INSTALL_DIR
• YARP_STATIC_PLUGINS_INSTALL_DIR
• YARP_DYNAMIC_PLUGINS_INSTALL_DIR

and their corresponding that contain absolute paths:

• YARP_DATA_INSTALL_DIR_FULL
• YARP_CONFIG_INSTALL_DIR_FULL
• YARP_PLUGIN_MANIFESTS_INSTALL_DIR_FULL
• YARP_MODULES_INSTALL_DIR_FULL
• YARP_APPLICATIONS_INSTALL_DIR_FULL
• YARP_TEMPLATES_INSTALL_DIR_FULL
• YARP_CONTEXTS_INSTALL_DIR_FULL
• YARP_APPLICATIONS_TEMPLATES_INSTALL_DIR_FULL
• YARP_MODULES_TEMPLATES_INSTALL_DIR_FULL
• YARP_ROBOTS_INSTALL_DIR_FULL
• YARP_STATIC_PLUGINS_INSTALL_DIR_FULL
• YARP_DYNAMIC_PLUGINS_INSTALL_DIR_FULL

Using the correct CMake variables when installing files will ensure that those files will be in the correct place on every platform, provided CMake is instructed to use the same CMAKE_INSTALL_PREFIX (in Linux this is the case by default).

Also non-root installation is supported, but both YARP and 3rd party software will require to be configured with the same CMAKE_INSTALL_PREFIX, and the environment variables (YARP_DATA_DIRS) will require to be tweaked accordingly (see next section).

Extending the search path

For packages that are not installed, the search path followed by the ResourceFinder can be extended in two ways:

• Through the YARP_DATA_DIRS variable: this is a list of locations, each is used by the ResourceFinder when looking for shared installation directories

Notice: the following should be considered experimental and could be removed in the future if not required)

• By extending the directory path.d

This solution should be adopted for packages that are installed.

path.d is a special folder used by YARP to provide an easy way to install 3rd party applications in separate folders. This folder contains a list of text files that contains extra search paths. The ResourceFinder searches for the directory path.d in the same locations as "Installed" configuration files before checking the same directory for other files. Each file inside path.d indicates an additional path to a directory; the latter is appended to the list of directories used by the ResourceFinder. This mechanisms works similarly to YARP_DATA_DIRS; however it is handy because it allows 3rd party applications to extend the search path without affecting the user environment.

To do so a package should install a .ini file in $YARP_PREFIX/yarp/config/path.d . This can be done using the yarp_configure_external_installation macro (see Configuring your external build)

Example: the iCub package in Linux.

The iCub package contributes a set of application, modules, contexts and robot files to YARP. It does so by installing them in /usr/local/share/iCub to keep them separate from the ones in YARP (/usr/local/share/yarp). To make these files visible to the ResourceFinder make install adds the file iCub.ini in:

/usr/local/share/yarp/config/path.d

The content of the file iCub.ini tells YARP where to look for additional resource files:

  ###### This file is automatically generated by CMake.
  [search iCub]
  path "/usr/local/share/iCub"