Tasks¶
OE-lite divides the job of building software into a number of (interdependent) tasks, each with a well-defined purpose. For example, one task is responsible for fetching the source code, another for unpacking it, a third for applying local patches, a fourth for doing the actual compilation, and so on and so forth.
Most recipes end up being split into about 13 tasks. The section Task types below briefly explains the purpose of the various tasks.
Environment¶
The filename of a recipe implicitly defines two variables, PN
and
PV
, which are used in the definition of lots of other
variables. PN
is the name of the recipe, while PV
is the
version. For a recipe file called openssh_7.1p2.oe
, these would be
openssh
and 7.1p2
, respectively. Moreover, P
is a
shorthand for ${PN}-${PV}
. These should never be changed from
within a recipe.
Directories¶
The directory containing the OE-lite manifest
is available as the
variable TOPDIR
. Two other standard variables are defined in terms
of this, INGREDIENTS
(${TOPDIR}/ingredients
) and TMPDIR
(${TOPDIR}/tmp
).
Every recipe gets built in a dedicated subdirectory of
${TMPDIR}/work
, named according to the recipe’s type, the target
architecture and the recipe version. Examples are
${TMPDIR}/work/machine/arm-cortexa9neon-linux-gnueabi/openssh-7.1p2
and
${TMPDIR}/work/native/x86_64-build_unknown-linux-gnu/gmp-6.0.0a
. This
is the contents of the ${WORKDIR}
variable.
There are a number of other standard variables defined in terms of
WORKDIR
, PN
and PV
which one should know about.
- SRCDIR
${WORKDIR}/src
- S
${SRCDIR}/${P}
- B
${S}
- D
${WORKDIR}/install
- T
${WORKDIR}/tmp
- PKGD
${WORKDIR}/packages
Their default values are shown above, but that may be overridden by classes or the recipe itself. We give a few examples of when this might be necessary in the task descriptions below.
Logging¶
The output from each task gets written to a log file in
${WORKDIR}/tmp
. The files are called
<taskname>.<datetime>.log
, e.g. do_compile.20161013074154.log
,
and there is a symbolic link <taskname>.log
pointing to the most
recent log file.
Empty log files get deleted automatically.
Scripts¶
Some tasks are implemented as bash functions. OE-lite runs these by
writing a complete bash script to ${WORKDIR}/tmp
called
<taskname>.<datetime>.run
(again, with <taskname>.run
being a
symlink to the most recent) containing all the necessary environment
settings, function definitions etc., then executes it, with stdout and
stderr redirected to the .log
file.
These scripts can also be run manually, which can be very useful as a debugging tool.
Metadata¶
A task is completely controlled by its associated metadata, which is essentially a set of key-value pairs. This metadata is copied from the metadata for the parent recipe, filtering away variables which are not relevant to the specific task.
Metadata hashing¶
In order to know whether a task needs to be redone and to facilitate
use of prebakes
, OE-lite assigns a hash value to every
task. This hash value is computed from two sources: The hash values of
all tasks which this task depends on, and the set of key-value pairs
constituting the task’s metadata. The former ensures that any change
in the dependency chain (e.g. a change of compiler) causes a rebuild.
A variable can be exempt from affecting the computed hash value by
setting the [nohash]
flag. This should be done with great care,
since it is only safe if it is known not to affect the binaries
generated, and it is only very rarely set in classes or recipes.
Task types¶
Common tasks types¶
These tasks are performed for almost all recipes during a normal
build. Note that for the configure, compile and install tasks, if a
recipe does not define a corresponding do_
function (and does not
inherit a class defining it), it is implicitly assumed that the step
is irrelevant to the recipe, so a dummy no-op function is used.
The listed task dependencies are those that must have completed
succesfully before the task is started. OE-lite does a chdir
to
the given working directory before starting the task.
do_fstage¶
TBD.
do_fetch¶
This task downloads the necessary source code to the local
ingredients directory
. This is typically in the form of
compressed tar-balls, but it can also perform cloning of git
repositories.
Task dependencies: fstage
Working directory: ${INGREDIENTS}
do_unpack¶
This extracts the source code from the local ingredients directory
to ${WORKDIR}/src
. For a tarball, this consists of (uncompressing
and) extracting the file, but it can also consist of checking out a
specific commit from a git repository. It also copies local patches
(files mentioned in SRC_URI
ending with .patch
) to
${WORKDIR}/patches
.
Task dependencies: fetch
Working directory: ${SRCDIR}
do_patch¶
This applies the local patches, if any, to the source code.
Task dependencies: unpack
Working directory: ${PATCHDIR}
do_stage¶
This populates the directory ${WORKDIR}/stage
with all the
necessary build-time dependencies as described by the recipe’s
DEPENDS
variable.
Task dependencies: do_stage depends on the existence of all the
packages
providing the items defined in the
DEPENDS
variable. If a necessary package does not already exist in
the tmp/packages
directory or can be found as a prebake, the
recipe providing that package will automatically get built, in which
case do_stage depends on the do_package task of the other recipe.
Working directory: ${STAGE_DIR}
do_configure¶
This is responsible for configuring the software. In many cases this
is the classic ./configure
step. When a recipe uses an appropriate
class
, OE-lite automatically constructs and passes
the relevant command line parameters to the configure script.
Task dependencies: patch and stage
Working directory: ${B}
do_compile¶
This task is where the software actually gets built. In many cases
this is just calling make
. The working directory is ${S}
.
Task dependencies: configure
Working directory: ${B}
do_install¶
This installs the software under ${WORKDIR}/install
, often just by
invoking make install
. During
Task dependencies: compile
Working directory: ${B}
do_split¶
This splits the files installed under ${WORKDIR}/install
into
packages. Files belonging to the package foo
gets copied to a
directory tree under ${PKGD}/foo
. The splitting is governed by the
FILES_*
variables. These contain space-separated lists of glob
patterns. For example, FILES_${PN}-dev
contain (among other
things) /lib/lib*.so /usr/include
, so all
Task dependencies: install
Working directory: ${D}
do_package¶
This adds some metadata (descripton, license, version etc.) to the packages created by do_split, and then wraps the directories up in a tarball.
Task dependencies: split
Working directory: ${PKGD}
Other tasks¶
These are usually only run when requested explicitly on the command line, e.g.
oe bake openssl -t packageqa
packageqa¶
Perform a number of Quality Assurance checks, for example:
- For shared libraries, check that the so-name matches the
LIBRARY_VERSION
version. - For binaries and shared libraries, check that all
runtime-dependencies are actually listed in the
RDEPENDS
variable.
Task dependencies: package
Working directory: ${PKGD}
clean¶
Remove the entire ${WORKDIR}
as well as the ${STAMPDIR}
– the
former ensures that there are no leftovers from earlier attempts to
build the recipe, while the latter prevents OE-lite from believing
that certain tasks are already succesfully completed and thus eliding
them. Hence a subsequent oe bake foo
should do all tasks related
to the foo recipe.
Task dependencies: none
Working directory: ${TOPDIR}