Overriding Packages ¶

Assuming you know what a workspace is and how to use multiple workspaces, you may decide you want to change the version of a package in an underlay without rebuilding the whole chain of workspaces. This can be done by overriding the package. It is accomplished by sourcing an existing workspace, then building a different version of that package again in a new overlay workspace.

Overriding a package is not always possible. This page offers tips for avoiding common issues.

Table of Contents

Overriding Packages

How to avoid common issues ¶

These are good practices to avoid common issues. If the advice is too restrictive then see the known issue descriptions for more complex advice.

Use isolated workspaces ¶

If you are building a workspace and suspect you may override a package from it in the future, then use an isolated workspace. This avoids include directory search order issues. See this documentation if you’re unsure what an isolated workspace is.

Override every package that depends on the one you want to override ¶

A leaf package is one that has no other packages that depend on it. Overriding a non-leaf package can be problematic. Packages in the underlay were built against the underlay version of the package, but they will be expected to run with the overlay version. If their build process stores some information about the non-leaf package, such as an expected ABI, then the behavior at runtime is unpredictable.

Problems caused by packages remembering information at build time can be avoided by overriding every package that directly or indirectly depends on the one you actually want to override. The group of overridden packages must span all underlays.

Say there are 3 workspaces, A, B and C, where C overlays B and B overlays A. A contains packages foo and baz where baz depends on foo. B contains packages ping that depends on foo and pong that depends on baz. C is the workspace being built. If you want to override foo then you should also override baz, ping, and pong.

Make sure overridden Python packages do not change entry point specifications ¶

Python packages may have entry point specifications in a setup.py or setup.cfg file. Make sure the package in the overlay has identical specifications to the version in the underlay, or only adds new ones. If any specification has been changed or removed then it may not be possible to override this package.

How to make it easier for your users to override ¶

This section has advice for package authors about how to make easier for your users to use your package and override it or other packages.

Install headers to a unique include directory ¶

Install your package’s headers to a unique folder rather than a shared folder. Say you’re the author of a package foo, and it has a header meant to be included like #include <foo/foo.hpp>. Instead of installing the header to <prefix>/include/foo/foo.hpp, install it to <prefix>/include/foo/foo/foo.hpp.

If you have a CMake package foo with the directory structure include/foo/foo.hpp, then it can install its headers to a unique directory with this:

install(DIRECTORY include/ DESTINATION include/${PROJECT_NAME})

All exported targets in your project need to export the unique include directory too.

target_include_directories(some_library_in_foo INTERFACE
  "$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>"
  "$<INSTALL_INTERFACE:include/${PROJECT_NAME}>")

Dynamically link to libraries outside your package ¶

If your package foo statically links to a library from package bar, then users can’t override bar without also overriding yours. Prefer dynamic linking instead.

Handling Python entry point specifications ¶

If your package loads Python entry points and it encounters two specifications with the same name, then it should use the last specification returned by entry_points(). It should also ignore entry points that can’t be loaded.

Here’s how to do it:

from importlib.metadata import entry_points

# Deduplicate entry point specifications before loading
deduplicated_entry_points = {}
# When faced with duplicates, this loop keeps the last entry point found
for ep in entry_points()['your_group_name']:
    deduplicated_entry_points[ep.name] = ep

for ep in deduplicated_entry_points:
    try:
        inst = ep.load()
    except ImportError:
        # Ignore entry point specifications that can't be loaded
        pass

All Known issues ¶

Include Directory Search Order Problem ¶

When overriding a package, it’s possible for another package to find that package’s headers from the underlay instead of the overlay. This may cause a failure to build or unexpected behavior at runtime depending on the differences between those headers.

Consider an overlay containing package foo and bar, and an underlay containing bar and baz. foo depends on bar and baz. Say the underlay is a merged workspace, and both the overridden bar and baz install their headers to a directory called include/. If any libraries or executables in foo are configured to search for headers in baz’s include directory first, then headers from overridden bar will also be found first.

When it can happen¶

The underlay workspace is a merged workspace
The overridden package installs header files (C/C++)
The overriding package’s headers are different from the overridden package’s
Another package in the underlay is not overridden and installs headers to the same directory as the overridden package (such as include)
A package in the overlay depends on both the package being overridden and the aforementioned additional package in the underlay

How to avoid it¶

Use isolated workspaces¶

If your underlay is an isolated workspace, then no two packages in it will have the same include directory. Using an isolated overlay workspace won’t help if your underlay is already a merged workspace (for example, the default ROS installation when installed from binary packages).

Sort include directories according to the workspace order¶

One solution is to sort include directories before passing them to the compiler such that headers are found in overlay workspaces before underlays. This requires making the build system for every package in an overlay aware of workspaces and the order they were sourced. The only known implementation of this is the find_package(catkin COMPONENTS ...) CMake feature in ROS 1.

Only override packages that install headers to unique include directories¶

If every package in the underlay installs their headers to unique directories, then packages in the overlay cannot accidentally find headers when depending on other packages in the underlay.

Unpredictable behavior when overridden package breaks API ¶

Consider an overlay containing bar, and an underlay containing bar and baz. baz depends on bar. If bar in the overlay changed an API used by baz, then the behavior of baz at runtime can’t be predicted.

When it can happen¶

The overriding package removed or changed APIs compared to the overridden package
A package in the underlay depends on the overridden package

How to avoid it¶

Build everything above the overridden package from source¶

If an API has changed, then every package in the underlay which depends on the overridden package (directly or indirectly) must be overridden too. You will need to find versions of those packages that are compatible with the API changes.

Undefined behavior when overridden package breaks ABI ¶

Consider an overlay containing bar, and an underlay containing bar and baz. baz depends on bar. If bar in the overlay changed ABI, then it is unpredictable what will happen when baz is used at runtime.

When it can happen¶

The overridden package uses a compiled language (C/C++, etc.)
The overriding package is ABI incompatible with the overridden one

How to avoid it¶

Build everything above the overridden package from source¶

If ABI has changed, then every package in the underlay which depends on the overridden package (directly or indirectly) must be overridden too. If only ABI has changed, then no changes to the packages are needed. Compiling them again is enough.

Renamed or deleted Python modules still importable ¶

Consider an overlay containing a Python package pyfoo and an underlay containing a Python package pyfoo. pyfoo in the underlay installs the Python modules foo, foo.bar, and baz. pyfoo in the overlay installs only the Python modules foo.

When the overlay is active, users will still be able to import baz from the underlay version of pyfoo However, they will not be able to import foo.bar because Python will find the foo package in overlay first, and that one does not contain bar.

When it can happen¶

The package being overridden is a Python package
The overridden package installs top level modules not present in the overriding package

How to avoid it¶

No workaround is known yet, but it’s unlikely to cause problems unless combined with another issue.

One-definition rule violations caused by static linking ¶

Consider an overlay containing packages foo and bar, and an underlay containing packages bar and baz. foo depends on bar and baz. baz depends on bar and has a library that statically links to another library in bar. foo has a library depending on both the mentioned library in baz and in bar.

When foo is used there are two definitions for symbols from bar: the ones from the underlay version of bar via baz, and the one from the overlay version of bar. At runtime, the implementations from the underlay version may be used.

When it can happen¶

a package in the underlay statically links to the overridden package
a package in the overlay depends on the overriding package and the package in the underlay

How to avoid it¶

Build everything above the overridden package from source¶

All packages directly or indirectly depending on the overridden package must be added to the overlay. No changes to the packages are needed. Compiling them again is enough.

Python entry_points are duplicated ¶

Consider a package pyfoo that has an entry point specification foobar = pyfoo.bar:baz. If pyfoo is overridden and the overridden version has same specification, then the entry point will be listed twice. Whether or not it is a problem depends on how those entry points are loaded.

If the code loading entry points loads all of them without checking for duplicates, then the same entry points may be used twice.

When it can happen¶

A python package providing entry points is overridden with a version that provides the same specification.

How to avoid it¶

There is no known workaround.

Deleted Python entry_points may still be loaded ¶

Consider a package pyfoo that has an entry point specification foobar = pyfoo.bar:baz. say pyfoo is overridden and the overridden version does not have that specification.

If the specification is still importable, then entry points from the underlay may be run undesirably. If the specification is not importable, then the code loading them must gracefully handle import errors.

When it can happen¶

A python package providing entry points is overridden with a version that omits an entry point available in the underlay.

How to avoid it¶

There is no known workaround.