Using a package after it has been built or building a package on top of its dependencies might require updating environment variables.
For the former, the best option for the user is to have a script perform the environment update for ease of use.
In the latter case, automating the process is necessary to build packages in topological order without user interaction.
E.g. if a package installs an executable which should be invocable by name, the directory containing the executable needs to be part of the
PATH environment variable.
To update environment variables:
- On non-Windows platforms a shell script needs to be
- On Windows a shell script needs to be
Executables are unable to change the environment of the current shell. Therefore this needs to be done from a shell script.
Each package installs its files under its install prefix.
This corresponds to
install/<package_name> except when
colcon build --merge-install is used.
In that case, the install prefix is
For each built package
colcon generates a set of package-level scripts (one for each supported shell type):
These script files update the environment with information specific to this package.
colcon supports multiple different approaches to update environment variables:
- A heuristic checking the installed files for known file types or known destinations.
A few of the supported heuristics:
- An executable under
binadds that directory to the
- An existing Python library directory adds that directory to the
- A file ending in
Config.cmakeadds the directory to the
- A file starting with
Findand ending in
.cmakeadds the directory to the
- A shared library adds the directory to one of the following environment variables (depending on the platform):
- A file ending in
.pcin a directory
lib/pkgconfigadds the directory to the
- An executable under
- Specific package types providing their own scripts to setup the package specific environment.
ament_cmakepackages (supported by
colcon-ros) provide a file named
In the root of the install prefix path,
colcon generates two kinds of scripts:
local_setup.<ext>: updates the environment with information from all packages installed under this prefix path. Since package-level scripts rely on information from their dependencies the package-level scripts must be invoked in topological order. In order to determine the topological order of all packages under the prefix path,
colconstores all run dependencies of a package in a file named
setup.<ext>: first invokes the
local_setup.<ext>files from all parent prefix paths and then invokes the sibling
Each shell has a potentially different syntax and supports a different set of features.
Some environment changes like extending the
PATH are applicable to all shells while others like providing completion functionality are only applicable to some.
If one shell (e.g.
bash) provides a superset of functionality and syntax of another shell (e.g.
sh) the logic of the
.sh scripts doesn’t have to be duplicated for
bash but can be invoked from the package-level setup files.
The latter shell is called a primary shell.
All environment changes necessary to use a package should be expressed in script with the extension of a primary shell. Primary shell extensions are:
.sh: plain shell
.bat: Windows cmd
Non-primary / additional shells¶
Other shells which are able to invoke primary shell scripts within their context commonly don’t duplicate their content and logic. Instead they first invoke the primary shell script and then add additional logic to provide specific functionalities like completion.
Avoid duplicate entries¶
Several environment variables store multiple values separated by a delimiter. Commonly, duplicate values are not useful and only increase length and decrease readability. Therefore, shell scripts try to avoid adding duplicate values where applicable.
With the number of values in such an environment variable the cost to check if a given value is already in the collection increases.
This significantly affects the time it takes to
call the workspace-level setup file since for each attempt to update an environment variable the collection needs to be split and each existing value compared to the to-be-added value.
colcon provides an alternative way to update the environment.
Since shell scripts can contain arbitrary logic it is opaque from the outside how they affect the environment. That makes it impossible to optimize their execution.
Most scripts perform one of the following common operations:
- Set an environment variable to a value.
- Add a value to an environment variable (using a platform specific delimiter) if the value is not already in the collection.
- Source / call another script file.
.dsv file contains the descriptive information about the intended environment change (instead of shell specific logic).
The content of such a file uses a semicolon as the delimiter and contains a single line.
The first value is the
type of the operation followed by a variable number of arguments specific to the operation.
The following list enumerates the supported types and their arguments:
prepend-non-duplicate;<name>;<value>: Prepend a value
<value>to an environment variable
<name>(using a platform specific delimiter) if the value is not already in the collection. The value is considered to be a path. If the value is not an absolute path the prefix path of the
.dsvfile is prepended to the value. An empty value therefore represents the prefix path.
prepend-non-duplicate-if-exists;<name>;<value>: Same as
prepend-non-duplicatebut only if the path described by the value exists.
set;<name>;<value>: Set an environment variable
<name>to a value
<value>. If the value is an existing relative path in the install prefix the install prefix is prepended to the value. Otherwise the value is used as is.
set-if-unset;<name>;<value>: Same as
setbut only if the environment variable is not yet set (or empty).
source;<path>: Source / call another script file
<path>. If the value is not an absolute path the prefix path of the
.dsvfile is prepended.
Implementing the logic to determine the topological order of packages in every primary shell would be a lot of effort and (depending on the shell) cumbersome.
Also parsing and interpreting
.dsv files would likely not be much faster than invoking the native scripts.
Therefore both parts are implemented in a Python script located in the root of the install prefix:
The Python script itself can’t change the environment.
However, it is able to efficiently interpret the operations described by the
.dsv files and generate the shell specific commands necessary to update the environment.
The Python file is templated with information specific to the primary shell it’s used from, hence the
<ext> in the filename.
When sourcing / calling a workspace-level setup file the number of evaluated scripts and / or interpreted
.dsv files can be significant.
To debug what files are being considered in which order and what environment changes are being performed you can prepend the invocation with
As a result each recursively invoked script as well as every generated command will be printed to the terminal.