jim.lai/OpenBMC
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Your Name
2026-04-23 17:07:55 +08:00
commit b7e39e063b
16725 changed files with 1625565 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
poky/documentation/.gitignore
+9
View File
@@ -0,0 +1,9 @@
sphinx/__pycache__
_build/
Pipfile.lock
poky.yaml
sphinx-static/switchers.js
releases.rst
.vscode/
*/svg/*.png
*/svg/*.pdf
poky/documentation/Makefile
+63
View File
@@ -0,0 +1,63 @@
# Minimal makefile for Sphinx documentation
#
# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?= -W --keep-going -j auto
SPHINXBUILD ?= sphinx-build
SOURCEDIR = .
IMAGEDIRS = */svg
BUILDDIR = _build
DESTDIR = final
SVG2PNG = inkscape
SVG2PDF = inkscape
ifeq ($(shell if which $(SPHINXBUILD) >/dev/null 2>&1; then echo 1; else echo 0; fi),0)
$(error "The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed")
endif
# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
.PHONY: all help Makefile clean publish epub latexpdf
publish: Makefile html singlehtml
rm -rf $(BUILDDIR)/$(DESTDIR)/
mkdir -p $(BUILDDIR)/$(DESTDIR)/
cp -r $(BUILDDIR)/html/* $(BUILDDIR)/$(DESTDIR)/
cp $(BUILDDIR)/singlehtml/index.html $(BUILDDIR)/$(DESTDIR)/singleindex.html
sed -i -e 's@index.html#@singleindex.html#@g' $(BUILDDIR)/$(DESTDIR)/singleindex.html
# Build a list of SVG files to convert to PDFs
PDFs := $(foreach dir, $(IMAGEDIRS), $(patsubst %.svg,%.pdf,$(wildcard $(SOURCEDIR)/$(dir)/*.svg)))
# Build a list of SVG files to convert to PNGs
PNGs := $(foreach dir, $(IMAGEDIRS), $(patsubst %.svg,%.png,$(wildcard $(SOURCEDIR)/$(dir)/*.svg)))
# Pattern rule for converting SVG to PDF
%.pdf : %.svg
$(SVG2PDF) --export-filename=$@ $<
# Pattern rule for converting SVG to PNG
%.png : %.svg
$(SVG2PNG) --export-filename=$@ $<
clean:
@rm -rf $(BUILDDIR) $(PNGs) $(PDFs) poky.yaml sphinx-static/switchers.js
epub: $(PNGs)
$(SOURCEDIR)/set_versions.py
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
latexpdf: $(PDFs)
$(SOURCEDIR)/set_versions.py
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
all: html epub latexpdf
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%:
$(SOURCEDIR)/set_versions.py
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
poky/documentation/Pipfile
+14
View File
@@ -0,0 +1,14 @@
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true
[dev-packages]
[packages]
sphinx = "*"
sphinx-rtd-theme = "*"
pyyaml = "*"
[requires]
python_version = "3"
poky/documentation/README
+387
View File
@@ -0,0 +1,387 @@
documentation
=============
This is the directory that contains the Yocto Project documentation. The Yocto
Project source repositories at https://git.yoctoproject.org/cgit.cgi have two
instances of the "documentation" directory. You should understand each of
these instances.
poky/documentation - The directory within the poky Git repository containing
the set of Yocto Project manuals. When you clone the
poky Git repository, the documentation directory
contains the manuals. The state of the manuals in this
directory is guaranteed to reflect the latest Yocto
Project release. The manuals at the tip of this
directory will also likely contain most manual
development changes.
yocto-docs/documentation - The Git repository for the Yocto Project manuals.
This repository is where manual development
occurs. If you plan on contributing back to the
Yocto Project documentation, you should set up
a local Git repository based on this upstream
repository as follows:
git clone git://git.yoctoproject.org/yocto-docs
Changes and patches are first pushed to the
yocto-docs Git repository. Later, they make it
into the poky Git repository found at
git://git.yoctoproject.org/poky.
Manual Organization
===================
Here the folders corresponding to individual manuals:
* overview-manual - Yocto Project Overview and Concepts Manual
* sdk-manual - Yocto Project Software Development Kit (SDK) Developer's Guide.
* bsp-guide - Yocto Project Board Support Package (BSP) Developer's Guide
* dev-manual - Yocto Project Development Tasks Manual
* kernel-dev - Yocto Project Linux Kernel Development Manual
* ref-manual - Yocto Project Reference Manual
* brief-yoctoprojectqs - Yocto Project Quick Start
* profile-manual - Yocto Project Profiling and Tracing Manual
* toaster-manual - Toaster User Manual
* test-manual - Yocto Project Test Environment Manual
Each folder is self-contained regarding content and figures.
If you want to find HTML versions of the Yocto Project manuals on the web,
the current versions reside at https://docs.yoctoproject.org.
poky.yaml
=========
This file defines variables used for documentation production. The variables
are used to define release pathnames, URLs for the published manuals, etc.
standards.md
============
This file specifies some rules to follow when contributing to the documentation.
template/
=========
Contains a template.svg, to make it easier to create consistent
SVG diagrams.
Sphinx
======
The Yocto Project documentation was migrated from the original DocBook
format to Sphinx based documentation for the Yocto Project 3.2
release. This section will provide additional information related to
the Sphinx migration, and guidelines for developers willing to
contribute to the Yocto Project documentation.
Sphinx is a tool that makes it easy to create intelligent and
beautiful documentation, written by Georg Brandl and licensed under
the BSD license. It was originally created for the Python
documentation.
Extensive documentation is available on the Sphinx website:
https://www.sphinx-doc.org/en/master/. Sphinx is designed to be
extensible thanks to the ability to write our own custom extensions,
as Python modules, which will be executed during the generation of the
documentation.
Yocto Project documentation website
===================================
The website hosting the Yocto Project documentation, can be found
at: https://docs.yoctoproject.org/.
The entire Yocto Project documentation, as well as the BitBake manual,
is published on this website, including all previously released
versions. A version switcher was added, as a drop-down menu on the top
of the page to switch back and forth between the various versions of
the current active Yocto Project releases.
Transition pages have been added (as rst files) to show links to old
versions of the Yocto Project documentation with links to each manual
generated with DocBook.
How to build the Yocto Project documentation
============================================
Sphinx is written in Python. While it might work with Python2, for
obvious reasons, we will only support building the Yocto Project
documentation with Python3.
Sphinx might be available in your Linux distro packages repositories,
however it is not recommended to use distro packages, as they might be
old versions, especially if you are using an LTS version of your
distro. The recommended method to install the latest versions of Sphinx
and of its required dependencies is to use the Python Package Index (pip).
To install all required packages run:
$ pip3 install sphinx sphinx_rtd_theme pyyaml
To make sure you always have the latest versions of such packages, you
should regularly run the same command with an added "--upgrade" option:
$ pip3 install --upgrade sphinx sphinx_rtd_theme pyyaml
Also install the "inkscape" package from your distribution.
Inkscape is need to convert SVG graphics to PNG (for EPUB
export) and to PDF (for PDF export).
Additionally install "fncychap.sty" TeX font if you want to build PDFs. Debian
and Ubuntu have it in "texlive-latex-extra" package while RedHat distributions
and OpenSUSE have it in "texlive-fncychap" package for example.
To build the documentation locally, run:
$ cd documentation
$ make html
The resulting HTML index page will be _build/html/index.html, and you
can browse your own copy of the locally generated documentation with
your browser.
Alternatively, you can use Pipenv to automatically install all required
dependencies in a virtual environment:
$ cd documentation
$ pipenv install
$ pipenv run make html
Sphinx theme and CSS customization
==================================
The Yocto Project documentation is currently based on the "Read the
Docs" Sphinx theme, with a few changes to make sure the look and feel
of the project documentation is preserved.
Most of the theme changes can be done using the file
'sphinx-static/theme_overrides.css'. Most CSS changes in this file
were inherited from the DocBook CSS stylesheets.
Sphinx design guidelines and principles
=======================================
The initial Docbook to Sphinx migration was done with an automated
tool called Pandoc (https://pandoc.org/). The tool produced some clean
output markdown text files. After the initial automated conversion
additional changes were done to fix up headings, images and links. In
addition Sphinx has built in mechanisms (directives) which were used
to replace similar functions implemented in Docbook such as glossary,
variables substitutions, notes and references.
Headings
========
The layout of the Yocto Project manuals is organized as follows
Book
Chapter
Section
Subsection
Subsubsection
Subsubsubsection
Here are the heading styles that we use in the manuals:
Book => overline ===
Chapter => overline ***
Section => ====
Subsection => ----
Subsubsection => ~~~~
Subsubsubsection => ^^^^
With this proposal, we preserve the same TOCs between Sphinx and Docbook.
Built-in glossary
=================
Sphinx has a glossary directive. From
https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#glossary:
This directive must contain a reST definition list with terms and
definitions. It's then possible to refer to each definition through the
[https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#role-term
'term' role].
So anywhere in any of the Yocto Project manuals, :term:`VAR` can be
used to refer to an item from the glossary, and a link is created
automatically. A general index of terms is also generated by Sphinx
automatically.
Global substitutions
====================
The Yocto Project documentation makes heavy use of global
variables. In Docbook these variables are stored in the file
poky.ent. This Docbook feature is not handled automatically with
Pandoc. Sphinx has builtin support for substitutions
(https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#substitutions),
however there are important shortcomings. For example they cannot be
used/nested inside code-block sections.
A Sphinx extension was implemented to support variable substitutions
to mimic the DocBook based documentation behavior. Variable
substitutions are done while reading/parsing the .rst files. The
pattern for variables substitutions is the same as with DocBook,
e.g. `&VAR;`.
The implementation of the extension can be found here in the file
documentation/sphinx/yocto-vars.py, this extension is enabled by
default when building the Yocto Project documentation. All variables
are set in a file call poky.yaml, which was initially generated from
poky.ent. The file was converted into YAML so that it is easier to
process by the custom Sphinx extension (which is a Python module).
For example, the following .rst content will produce the 'expected'
content:
.. code-block::
$ mkdir poky-&DISTRO;
or
$ git clone &YOCTO_GIT_URL;/git/poky -b &DISTRO_NAME_NO_CAP;
Variables can be nested, like it was the case for DocBook:
YOCTO_HOME_URL : "https://www.yoctoproject.org"
YOCTO_DOCS_URL : "&YOCTO_HOME_URL;/docs"
Note directive
==============
Sphinx has a builtin 'note' directive that produces clean Note section
in the output file. There are various types of directives such as
"attention", "caution", "danger", "error", "hint", "important", "tip",
"warning", "admonition" that are supported, and additional directives
can be added as Sphinx extension if needed.
Figures
=======
The Yocto Project documentation has many figures/images. Sphinx has a
'figure' directive which is straightforward to use. To include a
figure in the body of the documentation:
.. image:: figures/YP-flow-diagram.png
Links and References
====================
The following types of links can be used: links to other locations in
the same document, to locations in other documents and to external
websites.
More information can be found here:
https://sublime-and-sphinx-guide.readthedocs.io/en/latest/references.html.
For external links, we use this syntax:
`link text <link URL>`__
instead of:
`link text <link URL>`_
Both syntaxes work, but the latter also creates a "link text" reference
target which could conflict with other references with the same name.
So, only use this variant when you wish to make multiple references
to this link, reusing only the target name.
See https://stackoverflow.com/questions/27420317/restructured-text-rst-http-links-underscore-vs-use
Anchor (<#link>) links are forbidden as they are not checked by Sphinx during
the build and may be broken without knowing about it.
References
==========
The following extension is enabled by default:
sphinx.ext.autosectionlabel
(https://www.sphinx-doc.org/en/master/usage/extensions/autosectionlabel.html).
This extension allows you to refer sections by their titles. Note that
autosectionlabel_prefix_document is enabled by default, so that we can
insert references from any document.
For example, to insert an HTML link to a section from
documentation/manual/intro.rst, use:
Please check this :ref:`manual/intro:Cross-References to Locations in the Same Document`
Alternatively a custom text can be used instead of using the section
text:
Please check this :ref:`section <manual/intro:Cross-References to Locations in the Same Document>`
TIP: The following command can be used to dump all the references that
are defined in the project documentation:
python -msphinx.ext.intersphinx <path to build folder>/html/objects.inv
This dump contains all links and for each link it shows the default
"Link Text" that Sphinx would use. If the default link text is not
appropriate, a custom link text can be used in the ':ref:' directive.
Extlinks
========
The sphinx.ext.extlinks extension is enabled by default
(https://sublime-and-sphinx-guide.readthedocs.io/en/latest/references.html#use-the-external-links-extension),
and it is configured with the 'extlinks' definitions in
the 'documentation/conf.py' file:
'yocto_home': ('https://yoctoproject.org%s', None),
'yocto_wiki': ('https://wiki.yoctoproject.org%s', None),
'yocto_dl': ('https://downloads.yoctoproject.org%s', None),
'yocto_lists': ('https://lists.yoctoproject.org%s', None),
'yocto_bugs': ('https://bugzilla.yoctoproject.org%s', None),
'yocto_ab': ('https://autobuilder.yoctoproject.org%s', None),
'yocto_docs': ('https://docs.yoctoproject.org%s', None),
'yocto_git': ('https://git.yoctoproject.org%s', None),
'oe_home': ('https://www.openembedded.org%s', None),
'oe_lists': ('https://lists.openembedded.org%s', None),
'oe_git': ('https://git.openembedded.org%s', None),
'oe_wiki': ('https://www.openembedded.org/wiki%s', None),
'oe_layerindex': ('https://layers.openembedded.org%s', None),
'oe_layer': ('https://layers.openembedded.org/layerindex/branch/master/layer%s', None),
It creates convenient shortcuts which can be used throughout the
documentation rst files, as:
Please check this :yocto_wiki:`wiki page </Weekly_Status>`
Intersphinx links
=================
The sphinx.ext.intersphinx extension is enabled by default
(https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html),
so that we can cross reference content from other Sphinx based
documentation projects, such as the BitBake manual.
References to the BitBake manual can directly be done:
- With a specific description instead of the section name:
:ref:`Azure Storage fetcher (az://) <bitbake-user-manual/bitbake-user-manual-fetching:fetchers>`
- With the section name:
:ref:`bitbake-user-manual/bitbake-user-manual-intro:usage and syntax` option
If you want to refer to an entire document (or chapter) in the BitBake manual,
you have to use the ":doc:" macro with the "bitbake:" prefix:
- :doc:`BitBake User Manual <bitbake:index>`
- :doc:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata`" chapter
Note that a reference to a variable (:term:`VARIABLE`) automatically points to
the BitBake manual if the variable is not described in the Reference Manual's Variable Glossary.
However, if you need to bypass this, you can explicitely refer to a description in the
BitBake manual as follows:
:term:`bitbake:BB_NUMBER_PARSE_THREADS`
This would be the same if we had identical document filenames in
both the Yocto Project and BitBake manuals:
:ref:`bitbake:directory/file:section title`
Submitting documentation changes
================================
Please see the top level README file in this repository for details of where
to send patches.
poky/documentation/bitbake.rst
+19
View File
@@ -0,0 +1,19 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
=====================
BitBake Documentation
=====================
|
BitBake was originally a part of the OpenEmbedded project. It was inspired by
the Portage package management system used by the Gentoo Linux distribution. In
2004, the OpenEmbedded project was split the project into two distinct pieces:
- BitBake, a generic task executor
- OpenEmbedded, a metadata set utilized by BitBake
Today, BitBake is the primary build tool of OpenEmbedded based projects, such as
the Yocto Project.
The BitBake documentation can be found :doc:`here <bitbake:index>`.
poky/documentation/boilerplate.rst
+20
View File
@@ -0,0 +1,20 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
.. include:: <xhtml1-lat1.txt>
.. include:: <xhtml1-symbol.txt>
----
| |project_name|
| <docs@lists.yoctoproject.org>
Permission is granted to copy, distribute and/or modify this document under the
terms of the `Creative Commons Attribution-Share Alike 2.0 UK: England & Wales
<https://creativecommons.org/licenses/by-sa/2.0/uk/>`__ as published by Creative
Commons.
To report any inaccuracies or problems with this (or any other Yocto Project)
manual, or to send additions or changes, please send email/patches to the Yocto
Project documentation mailing list at ``docs@lists.yoctoproject.org`` or
log into the `Libera Chat <https://libera.chat/>`__ ``#yocto`` channel.
poky/documentation/brief-yoctoprojectqs/figures/bypqs-title.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 14 KiB

poky/documentation/brief-yoctoprojectqs/figures/yocto-project-transp.png
Executable
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 8.4 KiB

poky/documentation/brief-yoctoprojectqs/index.rst
+449
View File
@@ -0,0 +1,449 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
=========================
Yocto Project Quick Build
=========================
Welcome!
========
This short document steps you through the process for a typical
image build using the Yocto Project. The document also introduces how to
configure a build for specific hardware. You will use Yocto Project to
build a reference embedded OS called Poky.
.. note::
- The examples in this paper assume you are using a native Linux
system running a recent Ubuntu Linux distribution. If the machine
you want to use Yocto Project on to build an image
(:term:`Build Host`) is not
a native Linux system, you can still perform these steps by using
CROss PlatformS (CROPS) and setting up a Poky container. See the
:ref:`dev-manual/start:setting up to use cross platforms (crops)`
section
in the Yocto Project Development Tasks Manual for more
information.
- You may use version 2 of Windows Subsystem For Linux (WSL 2) to set
up a build host using Windows 10 or later, Windows Server 2019 or later.
See the :ref:`dev-manual/start:setting up to use windows subsystem for
linux (wsl 2)` section in the Yocto Project Development Tasks Manual
for more information.
If you want more conceptual or background information on the Yocto
Project, see the :doc:`/overview-manual/index`.
Compatible Linux Distribution
=============================
Make sure your :term:`Build Host` meets the
following requirements:
- At least &MIN_DISK_SPACE; Gbytes of free disk space, though
much more will help to run multiple builds and increase
performance by reusing build artifacts.
- At least &MIN_RAM; Gbytes of RAM, though a modern modern build host with as
much RAM and as many CPU cores as possible is strongly recommended to
maximize build performance.
- Runs a supported Linux distribution (i.e. recent releases of Fedora,
openSUSE, CentOS, Debian, or Ubuntu). For a list of Linux
distributions that support the Yocto Project, see the
:ref:`ref-manual/system-requirements:supported linux distributions`
section in the Yocto Project Reference Manual. For detailed
information on preparing your build host, see the
:ref:`dev-manual/start:preparing the build host`
section in the Yocto Project Development Tasks Manual.
-
- Git &MIN_GIT_VERSION; or greater
- tar &MIN_TAR_VERSION; or greater
- Python &MIN_PYTHON_VERSION; or greater.
- gcc &MIN_GCC_VERSION; or greater.
- GNU make &MIN_MAKE_VERSION; or greater
If your build host does not meet any of these three listed version
requirements, you can take steps to prepare the system so that you
can still use the Yocto Project. See the
:ref:`ref-manual/system-requirements:required git, tar, python, make and gcc versions`
section in the Yocto Project Reference Manual for information.
Build Host Packages
===================
You must install essential host packages on your build host. The
following command installs the host packages based on an Ubuntu
distribution::
$ sudo apt install &UBUNTU_HOST_PACKAGES_ESSENTIAL;
.. note::
For host package requirements on all supported Linux distributions,
see the :ref:`ref-manual/system-requirements:required packages for the build host`
section in the Yocto Project Reference Manual.
Use Git to Clone Poky
=====================
Once you complete the setup instructions for your machine, you need to
get a copy of the Poky repository on your build host. Use the following
commands to clone the Poky repository.
.. code-block:: shell
$ git clone git://git.yoctoproject.org/poky
Cloning into 'poky'...
remote: Counting
objects: 432160, done. remote: Compressing objects: 100%
(102056/102056), done. remote: Total 432160 (delta 323116), reused
432037 (delta 323000) Receiving objects: 100% (432160/432160), 153.81 MiB | 8.54 MiB/s, done.
Resolving deltas: 100% (323116/323116), done.
Checking connectivity... done.
Go to :yocto_wiki:`Releases wiki page </Releases>`, and choose a release
codename (such as ``&DISTRO_NAME_NO_CAP;``), corresponding to either the
latest stable release or a Long Term Support release.
Then move to the ``poky`` directory and take a look at existing branches:
.. code-block:: shell
$ cd poky
$ git branch -a
.
.
.
remotes/origin/HEAD -> origin/master
remotes/origin/dunfell
remotes/origin/dunfell-next
.
.
.
remotes/origin/gatesgarth
remotes/origin/gatesgarth-next
.
.
.
remotes/origin/master
remotes/origin/master-next
.
.
.
For this example, check out the ``&DISTRO_NAME_NO_CAP;`` branch based on the
``&DISTRO_NAME;`` release:
.. code-block:: shell
$ git checkout -t origin/&DISTRO_NAME_NO_CAP; -b my-&DISTRO_NAME_NO_CAP;
Branch 'my-&DISTRO_NAME_NO_CAP;' set up to track remote branch '&DISTRO_NAME_NO_CAP;' from 'origin'.
Switched to a new branch 'my-&DISTRO_NAME_NO_CAP;'
The previous Git checkout command creates a local branch named
``my-&DISTRO_NAME_NO_CAP;``. The files available to you in that branch
exactly match the repository's files in the ``&DISTRO_NAME_NO_CAP;``
release branch.
Note that you can regularly type the following command in the same directory
to keep your local files in sync with the release branch:
.. code-block:: shell
$ git pull
For more options and information about accessing Yocto Project related
repositories, see the
:ref:`dev-manual/start:locating yocto project source files`
section in the Yocto Project Development Tasks Manual.
Building Your Image
===================
Use the following steps to build your image. The build process creates
an entire Linux distribution, including the toolchain, from source.
.. note::
- If you are working behind a firewall and your build host is not
set up for proxies, you could encounter problems with the build
process when fetching source code (e.g. fetcher failures or Git
failures).
- If you do not know your proxy settings, consult your local network
infrastructure resources and get that information. A good starting
point could also be to check your web browser settings. Finally,
you can find more information on the
":yocto_wiki:`Working Behind a Network Proxy </Working_Behind_a_Network_Proxy>`"
page of the Yocto Project Wiki.
#. **Initialize the Build Environment:** From within the ``poky``
directory, run the :ref:`ref-manual/structure:\`\`oe-init-build-env\`\``
environment
setup script to define Yocto Project's build environment on your
build host.
.. code-block:: shell
$ cd poky
$ source oe-init-build-env
You had no conf/local.conf file. This configuration file has therefore been
created for you with some default values. You may wish to edit it to, for
example, select a different MACHINE (target hardware). See conf/local.conf
for more information as common configuration options are commented.
You had no conf/bblayers.conf file. This configuration file has therefore
been created for you with some default values. To add additional metadata
layers into your configuration please add entries to conf/bblayers.conf.
The Yocto Project has extensive documentation about OE including a reference
manual which can be found at:
https://docs.yoctoproject.org
For more information about OpenEmbedded see their website:
https://www.openembedded.org/
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-full-cmdline
core-image-sato
core-image-weston
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
Other commonly useful commands are:
- 'devtool' and 'recipetool' handle common recipe tasks
- 'bitbake-layers' handles common layer tasks
- 'oe-pkgdata-util' handles common target package tasks
Among other things, the script creates the :term:`Build Directory`, which is
``build`` in this case and is located in the :term:`Source Directory`. After
the script runs, your current working directory is set to the
:term:`Build Directory`. Later, when the build completes, the
:term:`Build Directory` contains all the files created during the build.
#. **Examine Your Local Configuration File:** When you set up the build
environment, a local configuration file named ``local.conf`` becomes
available in a ``conf`` subdirectory of the :term:`Build Directory`. For this
example, the defaults are set to build for a ``qemux86`` target,
which is suitable for emulation. The package manager used is set to
the RPM package manager.
.. tip::
You can significantly speed up your build and guard against fetcher
failures by using :ref:`overview-manual/concepts:shared state cache`
mirrors and enabling :ref:`overview-manual/concepts:hash equivalence`.
This way, you can use pre-built artifacts rather than building them.
This is relevant only when your network and the server that you use
can download these artifacts faster than you would be able to build them.
To use such mirrors, uncomment the below lines in your ``conf/local.conf``
file in the :term:`Build Directory`::
BB_SIGNATURE_HANDLER = "OEEquivHash"
BB_HASHSERVE = "auto"
BB_HASHSERVE_UPSTREAM = "hashserv.yocto.io:8687"
SSTATE_MIRRORS ?= "file://.* https://sstate.yoctoproject.org/all/PATH;downloadfilename=PATH"
#. **Start the Build:** Continue with the following command to build an OS
image for the target, which is ``core-image-sato`` in this example:
.. code-block:: shell
$ bitbake core-image-sato
For information on using the ``bitbake`` command, see the
:ref:`overview-manual/concepts:bitbake` section in the Yocto Project Overview and
Concepts Manual, or see
:ref:`bitbake-user-manual/bitbake-user-manual-intro:the bitbake command`
in the BitBake User Manual.
#. **Simulate Your Image Using QEMU:** Once this particular image is
built, you can start QEMU, which is a Quick EMUlator that ships with
the Yocto Project:
.. code-block:: shell
$ runqemu qemux86-64
If you want to learn more about running QEMU, see the
:ref:`dev-manual/qemu:using the quick emulator (qemu)` chapter in
the Yocto Project Development Tasks Manual.
#. **Exit QEMU:** Exit QEMU by either clicking on the shutdown icon or by typing
``Ctrl-C`` in the QEMU transcript window from which you evoked QEMU.
Customizing Your Build for Specific Hardware
============================================
So far, all you have done is quickly built an image suitable for
emulation only. This section shows you how to customize your build for
specific hardware by adding a hardware layer into the Yocto Project
development environment.
In general, layers are repositories that contain related sets of
instructions and configurations that tell the Yocto Project what to do.
Isolating related metadata into functionally specific layers facilitates
modular development and makes it easier to reuse the layer metadata.
.. note::
By convention, layer names start with the string "meta-".
Follow these steps to add a hardware layer:
#. **Find a Layer:** Many hardware layers are available. The Yocto Project
:yocto_git:`Source Repositories <>` has many hardware layers.
This example adds the
`meta-altera <https://github.com/kraj/meta-altera>`__ hardware layer.
#. **Clone the Layer:** Use Git to make a local copy of the layer on your
machine. You can put the copy in the top level of the copy of the
Poky repository created earlier:
.. code-block:: shell
$ cd poky
$ git clone https://github.com/kraj/meta-altera.git
Cloning into 'meta-altera'...
remote: Counting objects: 25170, done.
remote: Compressing objects: 100% (350/350), done.
remote: Total 25170 (delta 645), reused 719 (delta 538), pack-reused 24219
Receiving objects: 100% (25170/25170), 41.02 MiB | 1.64 MiB/s, done.
Resolving deltas: 100% (13385/13385), done.
Checking connectivity... done.
The hardware layer is now available
next to other layers inside the Poky reference repository on your build
host as ``meta-altera`` and contains all the metadata needed to
support hardware from Altera, which is owned by Intel.
.. note::
It is recommended for layers to have a branch per Yocto Project release.
Please make sure to checkout the layer branch supporting the Yocto Project
release you're using.
#. **Change the Configuration to Build for a Specific Machine:** The
:term:`MACHINE` variable in the
``local.conf`` file specifies the machine for the build. For this
example, set the :term:`MACHINE` variable to ``cyclone5``. These
configurations are used:
https://github.com/kraj/meta-altera/blob/master/conf/machine/cyclone5.conf.
.. note::
See the "Examine Your Local Configuration File" step earlier for more
information on configuring the build.
#. **Add Your Layer to the Layer Configuration File:** Before you can use
a layer during a build, you must add it to your ``bblayers.conf``
file, which is found in the :term:`Build Directory` ``conf`` directory.
Use the ``bitbake-layers add-layer`` command to add the layer to the
configuration file:
.. code-block:: shell
$ cd poky/build
$ bitbake-layers add-layer ../meta-altera
NOTE: Starting bitbake server...
Parsing recipes: 100% |##################################################################| Time: 0:00:32
Parsing of 918 .bb files complete (0 cached, 918 parsed). 1401 targets,
123 skipped, 0 masked, 0 errors.
You can find
more information on adding layers in the
:ref:`dev-manual/layers:adding a layer using the \`\`bitbake-layers\`\` script`
section.
Completing these steps has added the ``meta-altera`` layer to your Yocto
Project development environment and configured it to build for the
``cyclone5`` machine.
.. note::
The previous steps are for demonstration purposes only. If you were
to attempt to build an image for the ``cyclone5`` machine, you should
read the Altera ``README``.
Creating Your Own General Layer
===============================
Maybe you have an application or specific set of behaviors you need to
isolate. You can create your own general layer using the
``bitbake-layers create-layer`` command. The tool automates layer
creation by setting up a subdirectory with a ``layer.conf``
configuration file, a ``recipes-example`` subdirectory that contains an
``example.bb`` recipe, a licensing file, and a ``README``.
The following commands run the tool to create a layer named
``meta-mylayer`` in the ``poky`` directory:
.. code-block:: shell
$ cd poky
$ bitbake-layers create-layer meta-mylayer
NOTE: Starting bitbake server...
Add your new layer with 'bitbake-layers add-layer meta-mylayer'
For more information
on layers and how to create them, see the
:ref:`dev-manual/layers:creating a general layer using the \`\`bitbake-layers\`\` script`
section in the Yocto Project Development Tasks Manual.
Where To Go Next
================
Now that you have experienced using the Yocto Project, you might be
asking yourself "What now?". The Yocto Project has many sources of
information including the website, wiki pages, and user manuals:
- **Website:** The :yocto_home:`Yocto Project Website <>` provides
background information, the latest builds, breaking news, full
development documentation, and access to a rich Yocto Project
Development Community into which you can tap.
- **Video Seminar:** The `Introduction to the Yocto Project and BitBake, Part 1
<https://youtu.be/yuE7my3KOpo>`__ and
`Introduction to the Yocto Project and BitBake, Part 2
<https://youtu.be/iZ05TTyzGHk>`__ videos offer a video seminar
introducing you to the most important aspects of developing a
custom embedded Linux distribution with the Yocto Project.
- **Yocto Project Overview and Concepts Manual:** The
:doc:`/overview-manual/index` is a great
place to start to learn about the Yocto Project. This manual
introduces you to the Yocto Project and its development environment.
The manual also provides conceptual information for various aspects
of the Yocto Project.
- **Yocto Project Wiki:** The :yocto_wiki:`Yocto Project Wiki <>`
provides additional information on where to go next when ramping up
with the Yocto Project, release information, project planning, and QA
information.
- **Yocto Project Mailing Lists:** Related mailing lists provide a forum
for discussion, patch submission and announcements. There are several
mailing lists grouped by topic. See the
:ref:`ref-manual/resources:mailing lists`
section in the Yocto Project Reference Manual for a complete list of
Yocto Project mailing lists.
- **Comprehensive List of Links and Other Documentation:** The
:ref:`ref-manual/resources:links and related documentation`
section in the Yocto Project Reference Manual provides a
comprehensive list of all related links and other user documentation.
.. include:: /boilerplate.rst
poky/documentation/bsp-guide/bsp.rst
+1509
View File
File diff suppressed because it is too large Load Diff
poky/documentation/bsp-guide/figures/bsp-dev-flow.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 51 KiB

poky/documentation/bsp-guide/figures/bsp-title.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 17 KiB

poky/documentation/bsp-guide/index.rst
+15
View File
@@ -0,0 +1,15 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
=====================================================
Yocto Project Board Support Package Developer's Guide
=====================================================
|
.. toctree::
:caption: Table of Contents
:numbered:
bsp
.. include:: /boilerplate.rst
poky/documentation/conf.py
+168
View File
@@ -0,0 +1,168 @@
# Configuration file for the Sphinx documentation builder.
#
# SPDX-License-Identifier: CC-BY-SA-2.0-UK
#
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html
# -- Path setup --------------------------------------------------------------
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
import os
import sys
import datetime
try:
import yaml
except ImportError:
sys.stderr.write("The Yocto Project Sphinx documentation requires PyYAML.\
\nPlease make sure to install pyyaml Python package.\n")
sys.exit(1)
# current_version = "dev"
# bitbake_version = "" # Leave empty for development branch
# Obtain versions from poky.yaml instead
with open("poky.yaml") as data:
buff = data.read()
subst_vars = yaml.safe_load(buff)
if "DOCCONF_VERSION" not in subst_vars:
sys.stderr.write("Please set DOCCONF_VERSION in poky.yaml")
sys.exit(1)
current_version = subst_vars["DOCCONF_VERSION"]
if "BITBAKE_SERIES" not in subst_vars:
sys.stderr.write("Please set BITBAKE_SERIES in poky.yaml")
sys.exit(1)
bitbake_version = subst_vars["BITBAKE_SERIES"]
# String used in sidebar
version = 'Version: ' + current_version
if current_version == 'dev':
version = 'Version: Current Development'
# Version seen in documentation_options.js and hence in js switchers code
release = current_version
# -- Project information -----------------------------------------------------
project = 'The Yocto Project \xae'
copyright = '2010-%s, The Linux Foundation, CC-BY-SA-2.0-UK license' % datetime.datetime.now().year
author = 'The Linux Foundation'
# -- General configuration ---------------------------------------------------
# Prevent building with an outdated version of sphinx
needs_sphinx = "4.0"
# to load local extension from the folder 'sphinx'
sys.path.insert(0, os.path.abspath('sphinx'))
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx.ext.autosectionlabel',
'sphinx.ext.extlinks',
'sphinx.ext.intersphinx',
'yocto-vars'
]
autosectionlabel_prefix_document = True
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', 'boilerplate.rst']
# master document name. The default changed from contents to index. so better
# set it ourselves.
master_doc = 'index'
# create substitution for project configuration variables
rst_prolog = """
.. |project_name| replace:: %s
.. |copyright| replace:: %s
.. |author| replace:: %s
""" % (project, copyright, author)
# external links and substitutions
extlinks = {
'cve': ('https://nvd.nist.gov/vuln/detail/CVE-%s', 'CVE-%s'),
'cve_mitre': ('https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-%s', 'CVE-%s'),
'yocto_home': ('https://www.yoctoproject.org%s', None),
'yocto_wiki': ('https://wiki.yoctoproject.org/wiki%s', None),
'yocto_dl': ('https://downloads.yoctoproject.org%s', None),
'yocto_lists': ('https://lists.yoctoproject.org%s', None),
'yocto_bugs': ('https://bugzilla.yoctoproject.org%s', None),
'yocto_ab': ('https://autobuilder.yoctoproject.org%s', None),
'yocto_docs': ('https://docs.yoctoproject.org%s', None),
'yocto_git': ('https://git.yoctoproject.org%s', None),
'yocto_sstate': ('http://sstate.yoctoproject.org%s', None),
'oe_home': ('https://www.openembedded.org%s', None),
'oe_lists': ('https://lists.openembedded.org%s', None),
'oe_git': ('https://git.openembedded.org%s', None),
'oe_wiki': ('https://www.openembedded.org/wiki%s', None),
'oe_layerindex': ('https://layers.openembedded.org%s', None),
'oe_layer': ('https://layers.openembedded.org/layerindex/branch/master/layer%s', None),
'wikipedia': ('https://en.wikipedia.org/wiki/%s', None),
}
# Intersphinx config to use cross reference with BitBake user manual
intersphinx_mapping = {
'bitbake': ('https://docs.yoctoproject.org/bitbake/' + bitbake_version, None)
}
# Suppress "WARNING: unknown mimetype for ..."
suppress_warnings = ['epub.unknown_project_files']
# -- Options for HTML output -------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
try:
import sphinx_rtd_theme
html_theme = 'sphinx_rtd_theme'
html_theme_options = {
'sticky_navigation': False,
}
except ImportError:
sys.stderr.write("The Sphinx sphinx_rtd_theme HTML theme was not found.\
\nPlease make sure to install the sphinx_rtd_theme Python package.\n")
sys.exit(1)
html_logo = 'sphinx-static/YoctoProject_Logo_RGB.jpg'
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['sphinx-static']
html_context = {
'current_version': current_version,
}
# Add customm CSS and JS files
html_css_files = ['theme_overrides.css']
html_js_files = ['switchers.js']
# Hide 'Created using Sphinx' text
html_show_sphinx = False
# Add 'Last updated' on each page
html_last_updated_fmt = '%b %d, %Y'
# Remove the trailing 'dot' in section numbers
html_secnumber_suffix = " "
latex_elements = {
'passoptionstopackages': '\PassOptionsToPackage{bookmarksdepth=5}{hyperref}',
'preamble': '\setcounter{tocdepth}{2}',
}
# Make the EPUB builder prefer PNG to SVG because of issues rendering Inkscape SVG
from sphinx.builders.epub3 import Epub3Builder
Epub3Builder.supported_image_types = ['image/png', 'image/gif', 'image/jpeg']
poky/documentation/dev-manual/bmaptool.rst
+59
View File
@@ -0,0 +1,59 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Flashing Images Using ``bmaptool``
**********************************
A fast and easy way to flash an image to a bootable device is to use
Bmaptool, which is integrated into the OpenEmbedded build system.
Bmaptool is a generic tool that creates a file's block map (bmap) and
then uses that map to copy the file. As compared to traditional tools
such as dd or cp, Bmaptool can copy (or flash) large files like raw
system image files much faster.
.. note::
- If you are using Ubuntu or Debian distributions, you can install
the ``bmap-tools`` package using the following command and then
use the tool without specifying ``PATH`` even from the root
account::
$ sudo apt install bmap-tools
- If you are unable to install the ``bmap-tools`` package, you will
need to build Bmaptool before using it. Use the following command::
$ bitbake bmap-tools-native
Following, is an example that shows how to flash a Wic image. Realize
that while this example uses a Wic image, you can use Bmaptool to flash
any type of image. Use these steps to flash an image using Bmaptool:
#. *Update your local.conf File:* You need to have the following set
in your ``local.conf`` file before building your image::
IMAGE_FSTYPES += "wic wic.bmap"
#. *Get Your Image:* Either have your image ready (pre-built with the
:term:`IMAGE_FSTYPES`
setting previously mentioned) or take the step to build the image::
$ bitbake image
#. *Flash the Device:* Flash the device with the image by using Bmaptool
depending on your particular setup. The following commands assume the
image resides in the :term:`Build Directory`'s ``deploy/images/`` area:
- If you have write access to the media, use this command form::
$ oe-run-native bmap-tools-native bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX
- If you do not have write access to the media, set your permissions
first and then use the same command form::
$ sudo chmod 666 /dev/sdX
$ oe-run-native bmap-tools-native bmaptool copy build-directory/tmp/deploy/images/machine/image.wic /dev/sdX
For help on the ``bmaptool`` command, use the following command::
$ bmaptool --help
poky/documentation/dev-manual/build-quality.rst
+409
View File
@@ -0,0 +1,409 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Maintaining Build Output Quality
********************************
Many factors can influence the quality of a build. For example, if you
upgrade a recipe to use a new version of an upstream software package or
you experiment with some new configuration options, subtle changes can
occur that you might not detect until later. Consider the case where
your recipe is using a newer version of an upstream package. In this
case, a new version of a piece of software might introduce an optional
dependency on another library, which is auto-detected. If that library
has already been built when the software is building, the software will
link to the built library and that library will be pulled into your
image along with the new software even if you did not want the library.
The :ref:`ref-classes-buildhistory` class helps you maintain the quality of
your build output. You can use the class to highlight unexpected and possibly
unwanted changes in the build output. When you enable build history, it records
information about the contents of each package and image and then commits that
information to a local Git repository where you can examine the information.
The remainder of this section describes the following:
- :ref:`How you can enable and disable build history <dev-manual/build-quality:enabling and disabling build history>`
- :ref:`How to understand what the build history contains <dev-manual/build-quality:understanding what the build history contains>`
- :ref:`How to limit the information used for build history <dev-manual/build-quality:using build history to gather image information only>`
- :ref:`How to examine the build history from both a command-line and web interface <dev-manual/build-quality:examining build history information>`
Enabling and Disabling Build History
====================================
Build history is disabled by default. To enable it, add the following
:term:`INHERIT` statement and set the :term:`BUILDHISTORY_COMMIT` variable to
"1" at the end of your ``conf/local.conf`` file found in the
:term:`Build Directory`::
INHERIT += "buildhistory"
BUILDHISTORY_COMMIT = "1"
Enabling build history as
previously described causes the OpenEmbedded build system to collect
build output information and commit it as a single commit to a local
:ref:`overview-manual/development-environment:git` repository.
.. note::
Enabling build history increases your build times slightly,
particularly for images, and increases the amount of disk space used
during the build.
You can disable build history by removing the previous statements from
your ``conf/local.conf`` file.
Understanding What the Build History Contains
=============================================
Build history information is kept in ``${``\ :term:`TOPDIR`\ ``}/buildhistory``
in the :term:`Build Directory` as defined by the :term:`BUILDHISTORY_DIR`
variable. Here is an example abbreviated listing:
.. image:: figures/buildhistory.png
:align: center
:width: 50%
At the top level, there is a ``metadata-revs`` file that lists the
revisions of the repositories for the enabled layers when the build was
produced. The rest of the data splits into separate ``packages``,
``images`` and ``sdk`` directories, the contents of which are described
as follows.
Build History Package Information
---------------------------------
The history for each package contains a text file that has name-value
pairs with information about the package. For example,
``buildhistory/packages/i586-poky-linux/busybox/busybox/latest``
contains the following:
.. code-block:: none
PV = 1.22.1
PR = r32
RPROVIDES =
RDEPENDS = glibc (>= 2.20) update-alternatives-opkg
RRECOMMENDS = busybox-syslog busybox-udhcpc update-rc.d
PKGSIZE = 540168
FILES = /usr/bin/* /usr/sbin/* /usr/lib/busybox/* /usr/lib/lib*.so.* \
/etc /com /var /bin/* /sbin/* /lib/*.so.* /lib/udev/rules.d \
/usr/lib/udev/rules.d /usr/share/busybox /usr/lib/busybox/* \
/usr/share/pixmaps /usr/share/applications /usr/share/idl \
/usr/share/omf /usr/share/sounds /usr/lib/bonobo/servers
FILELIST = /bin/busybox /bin/busybox.nosuid /bin/busybox.suid /bin/sh \
/etc/busybox.links.nosuid /etc/busybox.links.suid
Most of these
name-value pairs correspond to variables used to produce the package.
The exceptions are ``FILELIST``, which is the actual list of files in
the package, and ``PKGSIZE``, which is the total size of files in the
package in bytes.
There is also a file that corresponds to the recipe from which the package
came (e.g. ``buildhistory/packages/i586-poky-linux/busybox/latest``):
.. code-block:: none
PV = 1.22.1
PR = r32
DEPENDS = initscripts kern-tools-native update-rc.d-native \
virtual/i586-poky-linux-compilerlibs virtual/i586-poky-linux-gcc \
virtual/libc virtual/update-alternatives
PACKAGES = busybox-ptest busybox-httpd busybox-udhcpd busybox-udhcpc \
busybox-syslog busybox-mdev busybox-hwclock busybox-dbg \
busybox-staticdev busybox-dev busybox-doc busybox-locale busybox
Finally, for those recipes fetched from a version control system (e.g.,
Git), there is a file that lists source revisions that are specified in
the recipe and the actual revisions used during the build. Listed
and actual revisions might differ when
:term:`SRCREV` is set to
${:term:`AUTOREV`}. Here is an
example assuming
``buildhistory/packages/qemux86-poky-linux/linux-yocto/latest_srcrev``)::
# SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
# SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f"
SRCREV_meta ="a227f20eff056e511d504b2e490f3774ab260d6f"
You can use the
``buildhistory-collect-srcrevs`` command with the ``-a`` option to
collect the stored :term:`SRCREV` values from build history and report them
in a format suitable for use in global configuration (e.g.,
``local.conf`` or a distro include file) to override floating
:term:`AUTOREV` values to a fixed set of revisions. Here is some example
output from this command::
$ buildhistory-collect-srcrevs -a
# all-poky-linux
SRCREV:pn-ca-certificates = "07de54fdcc5806bde549e1edf60738c6bccf50e8"
SRCREV:pn-update-rc.d = "8636cf478d426b568c1be11dbd9346f67e03adac"
# core2-64-poky-linux
SRCREV:pn-binutils = "87d4632d36323091e731eb07b8aa65f90293da66"
SRCREV:pn-btrfs-tools = "8ad326b2f28c044cb6ed9016d7c3285e23b673c8"
SRCREV_bzip2-tests:pn-bzip2 = "f9061c030a25de5b6829e1abf373057309c734c0"
SRCREV:pn-e2fsprogs = "02540dedd3ddc52c6ae8aaa8a95ce75c3f8be1c0"
SRCREV:pn-file = "504206e53a89fd6eed71aeaf878aa3512418eab1"
SRCREV_glibc:pn-glibc = "24962427071fa532c3c48c918e9d64d719cc8a6c"
SRCREV:pn-gnome-desktop-testing = "e346cd4ed2e2102c9b195b614f3c642d23f5f6e7"
SRCREV:pn-init-system-helpers = "dbd9197569c0935029acd5c9b02b84c68fd937ee"
SRCREV:pn-kmod = "b6ecfc916a17eab8f93be5b09f4e4f845aabd3d1"
SRCREV:pn-libnsl2 = "82245c0c58add79a8e34ab0917358217a70e5100"
SRCREV:pn-libseccomp = "57357d2741a3b3d3e8425889a6b79a130e0fa2f3"
SRCREV:pn-libxcrypt = "50cf2b6dd4fdf04309445f2eec8de7051d953abf"
SRCREV:pn-ncurses = "51d0fd9cc3edb975f04224f29f777f8f448e8ced"
SRCREV:pn-procps = "19a508ea121c0c4ac6d0224575a036de745eaaf8"
SRCREV:pn-psmisc = "5fab6b7ab385080f1db725d6803136ec1841a15f"
SRCREV:pn-ptest-runner = "bcb82804daa8f725b6add259dcef2067e61a75aa"
SRCREV:pn-shared-mime-info = "18e558fa1c8b90b86757ade09a4ba4d6a6cf8f70"
SRCREV:pn-zstd = "e47e674cd09583ff0503f0f6defd6d23d8b718d3"
# qemux86_64-poky-linux
SRCREV_machine:pn-linux-yocto = "20301aeb1a64164b72bc72af58802b315e025c9c"
SRCREV_meta:pn-linux-yocto = "2d38a472b21ae343707c8bd64ac68a9eaca066a0"
# x86_64-linux
SRCREV:pn-binutils-cross-x86_64 = "87d4632d36323091e731eb07b8aa65f90293da66"
SRCREV_glibc:pn-cross-localedef-native = "24962427071fa532c3c48c918e9d64d719cc8a6c"
SRCREV_localedef:pn-cross-localedef-native = "794da69788cbf9bf57b59a852f9f11307663fa87"
SRCREV:pn-debianutils-native = "de14223e5bffe15e374a441302c528ffc1cbed57"
SRCREV:pn-libmodulemd-native = "ee80309bc766d781a144e6879419b29f444d94eb"
SRCREV:pn-virglrenderer-native = "363915595e05fb252e70d6514be2f0c0b5ca312b"
SRCREV:pn-zstd-native = "e47e674cd09583ff0503f0f6defd6d23d8b718d3"
.. note::
Here are some notes on using the ``buildhistory-collect-srcrevs`` command:
- By default, only values where the :term:`SRCREV` was not hardcoded
(usually when :term:`AUTOREV` is used) are reported. Use the ``-a``
option to see all :term:`SRCREV` values.
- The output statements might not have any effect if overrides are
applied elsewhere in the build system configuration. Use the
``-f`` option to add the ``forcevariable`` override to each output
line if you need to work around this restriction.
- The script does apply special handling when building for multiple
machines. However, the script does place a comment before each set
of values that specifies which triplet to which they belong as
previously shown (e.g., ``i586-poky-linux``).
Build History Image Information
-------------------------------
The files produced for each image are as follows:
- ``image-files:`` A directory containing selected files from the root
filesystem. The files are defined by
:term:`BUILDHISTORY_IMAGE_FILES`.
- ``build-id.txt:`` Human-readable information about the build
configuration and metadata source revisions. This file contains the
full build header as printed by BitBake.
- ``*.dot:`` Dependency graphs for the image that are compatible with
``graphviz``.
- ``files-in-image.txt:`` A list of files in the image with
permissions, owner, group, size, and symlink information.
- ``image-info.txt:`` A text file containing name-value pairs with
information about the image. See the following listing example for
more information.
- ``installed-package-names.txt:`` A list of installed packages by name
only.
- ``installed-package-sizes.txt:`` A list of installed packages ordered
by size.
- ``installed-packages.txt:`` A list of installed packages with full
package filenames.
.. note::
Installed package information is able to be gathered and produced
even if package management is disabled for the final image.
Here is an example of ``image-info.txt``:
.. code-block:: none
DISTRO = poky
DISTRO_VERSION = 3.4+snapshot-a0245d7be08f3d24ea1875e9f8872aa6bbff93be
USER_CLASSES = buildstats
IMAGE_CLASSES = qemuboot qemuboot license_image
IMAGE_FEATURES = debug-tweaks
IMAGE_LINGUAS =
IMAGE_INSTALL = packagegroup-core-boot speex speexdsp
BAD_RECOMMENDATIONS =
NO_RECOMMENDATIONS =
PACKAGE_EXCLUDE =
ROOTFS_POSTPROCESS_COMMAND = write_package_manifest; license_create_manifest; cve_check_write_rootfs_manifest; ssh_allow_empty_password; ssh_allow_root_login; postinst_enable_logging; rootfs_update_timestamp; write_image_test_data; empty_var_volatile; sort_passwd; rootfs_reproducible;
IMAGE_POSTPROCESS_COMMAND = buildhistory_get_imageinfo ;
IMAGESIZE = 9265
Other than ``IMAGESIZE``,
which is the total size of the files in the image in Kbytes, the
name-value pairs are variables that may have influenced the content of
the image. This information is often useful when you are trying to
determine why a change in the package or file listings has occurred.
Using Build History to Gather Image Information Only
----------------------------------------------------
As you can see, build history produces image information, including
dependency graphs, so you can see why something was pulled into the
image. If you are just interested in this information and not interested
in collecting specific package or SDK information, you can enable
writing only image information without any history by adding the
following to your ``conf/local.conf`` file found in the
:term:`Build Directory`::
INHERIT += "buildhistory"
BUILDHISTORY_COMMIT = "0"
BUILDHISTORY_FEATURES = "image"
Here, you set the
:term:`BUILDHISTORY_FEATURES`
variable to use the image feature only.
Build History SDK Information
-----------------------------
Build history collects similar information on the contents of SDKs (e.g.
``bitbake -c populate_sdk imagename``) as compared to information it
collects for images. Furthermore, this information differs depending on
whether an extensible or standard SDK is being produced.
The following list shows the files produced for SDKs:
- ``files-in-sdk.txt:`` A list of files in the SDK with permissions,
owner, group, size, and symlink information. This list includes both
the host and target parts of the SDK.
- ``sdk-info.txt:`` A text file containing name-value pairs with
information about the SDK. See the following listing example for more
information.
- ``sstate-task-sizes.txt:`` A text file containing name-value pairs
with information about task group sizes (e.g. :ref:`ref-tasks-populate_sysroot`
tasks have a total size). The ``sstate-task-sizes.txt`` file exists
only when an extensible SDK is created.
- ``sstate-package-sizes.txt:`` A text file containing name-value pairs
with information for the shared-state packages and sizes in the SDK.
The ``sstate-package-sizes.txt`` file exists only when an extensible
SDK is created.
- ``sdk-files:`` A folder that contains copies of the files mentioned
in ``BUILDHISTORY_SDK_FILES`` if the files are present in the output.
Additionally, the default value of ``BUILDHISTORY_SDK_FILES`` is
specific to the extensible SDK although you can set it differently if
you would like to pull in specific files from the standard SDK.
The default files are ``conf/local.conf``, ``conf/bblayers.conf``,
``conf/auto.conf``, ``conf/locked-sigs.inc``, and
``conf/devtool.conf``. Thus, for an extensible SDK, these files get
copied into the ``sdk-files`` directory.
- The following information appears under each of the ``host`` and
``target`` directories for the portions of the SDK that run on the
host and on the target, respectively:
.. note::
The following files for the most part are empty when producing an
extensible SDK because this type of SDK is not constructed from
packages as is the standard SDK.
- ``depends.dot:`` Dependency graph for the SDK that is compatible
with ``graphviz``.
- ``installed-package-names.txt:`` A list of installed packages by
name only.
- ``installed-package-sizes.txt:`` A list of installed packages
ordered by size.
- ``installed-packages.txt:`` A list of installed packages with full
package filenames.
Here is an example of ``sdk-info.txt``:
.. code-block:: none
DISTRO = poky
DISTRO_VERSION = 1.3+snapshot-20130327
SDK_NAME = poky-glibc-i686-arm
SDK_VERSION = 1.3+snapshot
SDKMACHINE =
SDKIMAGE_FEATURES = dev-pkgs dbg-pkgs
BAD_RECOMMENDATIONS =
SDKSIZE = 352712
Other than ``SDKSIZE``, which is
the total size of the files in the SDK in Kbytes, the name-value pairs
are variables that might have influenced the content of the SDK. This
information is often useful when you are trying to determine why a
change in the package or file listings has occurred.
Examining Build History Information
-----------------------------------
You can examine build history output from the command line or from a web
interface.
To see any changes that have occurred (assuming you have
:term:`BUILDHISTORY_COMMIT` = "1"),
you can simply use any Git command that allows you to view the history
of a repository. Here is one method::
$ git log -p
You need to realize,
however, that this method does show changes that are not significant
(e.g. a package's size changing by a few bytes).
There is a command-line tool called ``buildhistory-diff``, though,
that queries the Git repository and prints just the differences that
might be significant in human-readable form. Here is an example::
$ poky/poky/scripts/buildhistory-diff . HEAD^
Changes to images/qemux86_64/glibc/core-image-minimal (files-in-image.txt):
/etc/anotherpkg.conf was added
/sbin/anotherpkg was added
* (installed-package-names.txt):
* anotherpkg was added
Changes to images/qemux86_64/glibc/core-image-minimal (installed-package-names.txt):
anotherpkg was added
packages/qemux86_64-poky-linux/v86d: PACKAGES: added "v86d-extras"
* PR changed from "r0" to "r1"
* PV changed from "0.1.10" to "0.1.12"
packages/qemux86_64-poky-linux/v86d/v86d: PKGSIZE changed from 110579 to 144381 (+30%)
* PR changed from "r0" to "r1"
* PV changed from "0.1.10" to "0.1.12"
.. note::
The ``buildhistory-diff`` tool requires the ``GitPython``
package. Be sure to install it using Pip3 as follows::
$ pip3 install GitPython --user
Alternatively, you can install ``python3-git`` using the appropriate
distribution package manager (e.g. ``apt``, ``dnf``, or ``zipper``).
To see changes to the build history using a web interface, follow the
instruction in the ``README`` file
:yocto_git:`here </buildhistory-web/>`.
Here is a sample screenshot of the interface:
.. image:: figures/buildhistory-web.png
:width: 100%
poky/documentation/dev-manual/building.rst
+939
View File
@@ -0,0 +1,939 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Building
********
This section describes various build procedures, such as the steps
needed for a simple build, building a target for multiple configurations,
generating an image for more than one machine, and so forth.
Building a Simple Image
=======================
In the development environment, you need to build an image whenever you
change hardware support, add or change system libraries, or add or
change services that have dependencies. There are several methods that allow
you to build an image within the Yocto Project. This section presents
the basic steps you need to build a simple image using BitBake from a
build host running Linux.
.. note::
- For information on how to build an image using
:term:`Toaster`, see the
:doc:`/toaster-manual/index`.
- For information on how to use ``devtool`` to build images, see the
":ref:`sdk-manual/extensible:using \`\`devtool\`\` in your sdk workflow`"
section in the Yocto Project Application Development and the
Extensible Software Development Kit (eSDK) manual.
- For a quick example on how to build an image using the
OpenEmbedded build system, see the
:doc:`/brief-yoctoprojectqs/index` document.
The build process creates an entire Linux distribution from source and
places it in your :term:`Build Directory` under ``tmp/deploy/images``. For
detailed information on the build process using BitBake, see the
":ref:`overview-manual/concepts:images`" section in the Yocto Project Overview
and Concepts Manual.
The following figure and list overviews the build process:
.. image:: figures/bitbake-build-flow.png
:width: 100%
#. *Set up Your Host Development System to Support Development Using the
Yocto Project*: See the ":doc:`start`" section for options on how to get a
build host ready to use the Yocto Project.
#. *Initialize the Build Environment:* Initialize the build environment
by sourcing the build environment script (i.e.
:ref:`structure-core-script`)::
$ source oe-init-build-env [build_dir]
When you use the initialization script, the OpenEmbedded build system
uses ``build`` as the default :term:`Build Directory` in your current work
directory. You can use a `build_dir` argument with the script to
specify a different :term:`Build Directory`.
.. note::
A common practice is to use a different :term:`Build Directory` for
different targets; for example, ``~/build/x86`` for a ``qemux86``
target, and ``~/build/arm`` for a ``qemuarm`` target. In any
event, it's typically cleaner to locate the :term:`Build Directory`
somewhere outside of your source directory.
#. *Make Sure Your* ``local.conf`` *File is Correct*: Ensure the
``conf/local.conf`` configuration file, which is found in the
:term:`Build Directory`, is set up how you want it. This file defines many
aspects of the build environment including the target machine architecture
through the :term:`MACHINE` variable, the packaging format used during
the build (:term:`PACKAGE_CLASSES`), and a centralized tarball download
directory through the :term:`DL_DIR` variable.
#. *Build the Image:* Build the image using the ``bitbake`` command::
$ bitbake target
.. note::
For information on BitBake, see the :doc:`bitbake:index`.
The target is the name of the recipe you want to build. Common
targets are the images in ``meta/recipes-core/images``,
``meta/recipes-sato/images``, and so forth all found in the
:term:`Source Directory`. Alternatively, the target
can be the name of a recipe for a specific piece of software such as
BusyBox. For more details about the images the OpenEmbedded build
system supports, see the
":ref:`ref-manual/images:Images`" chapter in the Yocto
Project Reference Manual.
As an example, the following command builds the
``core-image-minimal`` image::
$ bitbake core-image-minimal
Once an
image has been built, it often needs to be installed. The images and
kernels built by the OpenEmbedded build system are placed in the
:term:`Build Directory` in ``tmp/deploy/images``. For information on how to
run pre-built images such as ``qemux86`` and ``qemuarm``, see the
:doc:`/sdk-manual/index` manual. For
information about how to install these images, see the documentation
for your particular board or machine.
Building Images for Multiple Targets Using Multiple Configurations
==================================================================
You can use a single ``bitbake`` command to build multiple images or
packages for different targets where each image or package requires a
different configuration (multiple configuration builds). The builds, in
this scenario, are sometimes referred to as "multiconfigs", and this
section uses that term throughout.
This section describes how to set up for multiple configuration builds
and how to account for cross-build dependencies between the
multiconfigs.
Setting Up and Running a Multiple Configuration Build
-----------------------------------------------------
To accomplish a multiple configuration build, you must define each
target's configuration separately using a parallel configuration file in
the :term:`Build Directory` or configuration directory within a layer, and you
must follow a required file hierarchy. Additionally, you must enable the
multiple configuration builds in your ``local.conf`` file.
Follow these steps to set up and execute multiple configuration builds:
- *Create Separate Configuration Files*: You need to create a single
configuration file for each build target (each multiconfig).
The configuration definitions are implementation dependent but often
each configuration file will define the machine and the
temporary directory BitBake uses for the build. Whether the same
temporary directory (:term:`TMPDIR`) can be shared will depend on what is
similar and what is different between the configurations. Multiple MACHINE
targets can share the same (:term:`TMPDIR`) as long as the rest of the
configuration is the same, multiple :term:`DISTRO` settings would need separate
(:term:`TMPDIR`) directories.
For example, consider a scenario with two different multiconfigs for the same
:term:`MACHINE`: "qemux86" built
for two distributions such as "poky" and "poky-lsb". In this case,
you would need to use the different :term:`TMPDIR`.
Here is an example showing the minimal statements needed in a
configuration file for a "qemux86" target whose temporary build
directory is ``tmpmultix86``::
MACHINE = "qemux86"
TMPDIR = "${TOPDIR}/tmpmultix86"
The location for these multiconfig configuration files is specific.
They must reside in the current :term:`Build Directory` in a sub-directory of
``conf`` named ``multiconfig`` or within a layer's ``conf`` directory
under a directory named ``multiconfig``. Following is an example that defines
two configuration files for the "x86" and "arm" multiconfigs:
.. image:: figures/multiconfig_files.png
:align: center
:width: 50%
The usual :term:`BBPATH` search path is used to locate multiconfig files in
a similar way to other conf files.
- *Add the BitBake Multi-configuration Variable to the Local
Configuration File*: Use the
:term:`BBMULTICONFIG`
variable in your ``conf/local.conf`` configuration file to specify
each multiconfig. Continuing with the example from the previous
figure, the :term:`BBMULTICONFIG` variable needs to enable two
multiconfigs: "x86" and "arm" by specifying each configuration file::
BBMULTICONFIG = "x86 arm"
.. note::
A "default" configuration already exists by definition. This
configuration is named: "" (i.e. empty string) and is defined by
the variables coming from your ``local.conf``
file. Consequently, the previous example actually adds two
additional configurations to your build: "arm" and "x86" along
with "".
- *Launch BitBake*: Use the following BitBake command form to launch
the multiple configuration build::
$ bitbake [mc:multiconfigname:]target [[[mc:multiconfigname:]target] ... ]
For the example in this section, the following command applies::
$ bitbake mc:x86:core-image-minimal mc:arm:core-image-sato mc::core-image-base
The previous BitBake command builds a ``core-image-minimal`` image
that is configured through the ``x86.conf`` configuration file, a
``core-image-sato`` image that is configured through the ``arm.conf``
configuration file and a ``core-image-base`` that is configured
through your ``local.conf`` configuration file.
.. note::
Support for multiple configuration builds in the Yocto Project &DISTRO;
(&DISTRO_NAME;) Release does not include Shared State (sstate)
optimizations. Consequently, if a build uses the same object twice
in, for example, two different :term:`TMPDIR`
directories, the build either loads from an existing sstate cache for
that build at the start or builds the object fresh.
Enabling Multiple Configuration Build Dependencies
--------------------------------------------------
Sometimes dependencies can exist between targets (multiconfigs) in a
multiple configuration build. For example, suppose that in order to
build a ``core-image-sato`` image for an "x86" multiconfig, the root
filesystem of an "arm" multiconfig must exist. This dependency is
essentially that the
:ref:`ref-tasks-image` task in the
``core-image-sato`` recipe depends on the completion of the
:ref:`ref-tasks-rootfs` task of the
``core-image-minimal`` recipe.
To enable dependencies in a multiple configuration build, you must
declare the dependencies in the recipe using the following statement
form::
task_or_package[mcdepends] = "mc:from_multiconfig:to_multiconfig:recipe_name:task_on_which_to_depend"
To better show how to use this statement, consider the example scenario
from the first paragraph of this section. The following statement needs
to be added to the recipe that builds the ``core-image-sato`` image::
do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs"
In this example, the `from_multiconfig` is "x86". The `to_multiconfig` is "arm". The
task on which the :ref:`ref-tasks-image` task in the recipe depends is the
:ref:`ref-tasks-rootfs` task from the ``core-image-minimal`` recipe associated
with the "arm" multiconfig.
Once you set up this dependency, you can build the "x86" multiconfig
using a BitBake command as follows::
$ bitbake mc:x86:core-image-sato
This command executes all the tasks needed to create the
``core-image-sato`` image for the "x86" multiconfig. Because of the
dependency, BitBake also executes through the :ref:`ref-tasks-rootfs` task for the
"arm" multiconfig build.
Having a recipe depend on the root filesystem of another build might not
seem that useful. Consider this change to the statement in the
``core-image-sato`` recipe::
do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_image"
In this case, BitBake must
create the ``core-image-minimal`` image for the "arm" build since the
"x86" build depends on it.
Because "x86" and "arm" are enabled for multiple configuration builds
and have separate configuration files, BitBake places the artifacts for
each build in the respective temporary build directories (i.e.
:term:`TMPDIR`).
Building an Initial RAM Filesystem (Initramfs) Image
====================================================
An initial RAM filesystem (:term:`Initramfs`) image provides a temporary root
filesystem used for early system initialization, typically providing tools and
loading modules needed to locate and mount the final root filesystem.
Follow these steps to create an :term:`Initramfs` image:
#. *Create the :term:`Initramfs` Image Recipe:* You can reference the
``core-image-minimal-initramfs.bb`` recipe found in the
``meta/recipes-core`` directory of the :term:`Source Directory`
as an example from which to work.
#. *Decide if You Need to Bundle the :term:`Initramfs` Image Into the Kernel
Image:* If you want the :term:`Initramfs` image that is built to be bundled
in with the kernel image, set the :term:`INITRAMFS_IMAGE_BUNDLE`
variable to ``"1"`` in your ``local.conf`` configuration file and set the
:term:`INITRAMFS_IMAGE` variable in the recipe that builds the kernel image.
Setting the :term:`INITRAMFS_IMAGE_BUNDLE` flag causes the :term:`Initramfs`
image to be unpacked into the ``${B}/usr/`` directory. The unpacked
:term:`Initramfs` image is then passed to the kernel's ``Makefile`` using the
:term:`CONFIG_INITRAMFS_SOURCE` variable, allowing the :term:`Initramfs`
image to be built into the kernel normally.
#. *Optionally Add Items to the Initramfs Image Through the Initramfs
Image Recipe:* If you add items to the :term:`Initramfs` image by way of its
recipe, you should use :term:`PACKAGE_INSTALL` rather than
:term:`IMAGE_INSTALL`. :term:`PACKAGE_INSTALL` gives more direct control of
what is added to the image as compared to the defaults you might not
necessarily want that are set by the :ref:`ref-classes-image`
or :ref:`ref-classes-core-image` classes.
#. *Build the Kernel Image and the Initramfs Image:* Build your kernel
image using BitBake. Because the :term:`Initramfs` image recipe is a
dependency of the kernel image, the :term:`Initramfs` image is built as well
and bundled with the kernel image if you used the
:term:`INITRAMFS_IMAGE_BUNDLE` variable described earlier.
Bundling an Initramfs Image From a Separate Multiconfig
-------------------------------------------------------
There may be a case where we want to build an :term:`Initramfs` image which does not
inherit the same distro policy as our main image, for example, we may want
our main image to use ``TCLIBC="glibc"``, but to use ``TCLIBC="musl"`` in our :term:`Initramfs`
image to keep a smaller footprint. However, by performing the steps mentioned
above the :term:`Initramfs` image will inherit ``TCLIBC="glibc"`` without allowing us
to override it.
To achieve this, you need to perform some additional steps:
#. *Create a multiconfig for your Initramfs image:* You can perform the steps
on ":ref:`dev-manual/building:building images for multiple targets using multiple configurations`" to create a separate multiconfig.
For the sake of simplicity let's assume such multiconfig is called: ``initramfscfg.conf`` and
contains the variables::
TMPDIR="${TOPDIR}/tmp-initramfscfg"
TCLIBC="musl"
#. *Set additional Initramfs variables on your main configuration:*
Additionally, on your main configuration (``local.conf``) you need to set the
variables::
INITRAMFS_MULTICONFIG = "initramfscfg"
INITRAMFS_DEPLOY_DIR_IMAGE = "${TOPDIR}/tmp-initramfscfg/deploy/images/${MACHINE}"
The variables :term:`INITRAMFS_MULTICONFIG` and :term:`INITRAMFS_DEPLOY_DIR_IMAGE`
are used to create a multiconfig dependency from the kernel to the :term:`INITRAMFS_IMAGE`
to be built coming from the ``initramfscfg`` multiconfig, and to let the
buildsystem know where the :term:`INITRAMFS_IMAGE` will be located.
Building a system with such configuration will build the kernel using the
main configuration but the :ref:`ref-tasks-bundle_initramfs` task will grab the
selected :term:`INITRAMFS_IMAGE` from :term:`INITRAMFS_DEPLOY_DIR_IMAGE`
instead, resulting in a musl based :term:`Initramfs` image bundled in the kernel
but a glibc based main image.
The same is applicable to avoid inheriting :term:`DISTRO_FEATURES` on :term:`INITRAMFS_IMAGE`
or to build a different :term:`DISTRO` for it such as ``poky-tiny``.
Building a Tiny System
======================
Very small distributions have some significant advantages such as
requiring less on-die or in-package memory (cheaper), better performance
through efficient cache usage, lower power requirements due to less
memory, faster boot times, and reduced development overhead. Some
real-world examples where a very small distribution gives you distinct
advantages are digital cameras, medical devices, and small headless
systems.
This section presents information that shows you how you can trim your
distribution to even smaller sizes than the ``poky-tiny`` distribution,
which is around 5 Mbytes, that can be built out-of-the-box using the
Yocto Project.
Tiny System Overview
--------------------
The following list presents the overall steps you need to consider and
perform to create distributions with smaller root filesystems, achieve
faster boot times, maintain your critical functionality, and avoid
initial RAM disks:
- :ref:`Determine your goals and guiding principles
<dev-manual/building:goals and guiding principles>`
- :ref:`dev-manual/building:understand what contributes to your image size`
- :ref:`Reduce the size of the root filesystem
<dev-manual/building:trim the root filesystem>`
- :ref:`Reduce the size of the kernel <dev-manual/building:trim the kernel>`
- :ref:`dev-manual/building:remove package management requirements`
- :ref:`dev-manual/building:look for other ways to minimize size`
- :ref:`dev-manual/building:iterate on the process`
Goals and Guiding Principles
----------------------------
Before you can reach your destination, you need to know where you are
going. Here is an example list that you can use as a guide when creating
very small distributions:
- Determine how much space you need (e.g. a kernel that is 1 Mbyte or
less and a root filesystem that is 3 Mbytes or less).
- Find the areas that are currently taking 90% of the space and
concentrate on reducing those areas.
- Do not create any difficult "hacks" to achieve your goals.
- Leverage the device-specific options.
- Work in a separate layer so that you keep changes isolated. For
information on how to create layers, see the
":ref:`dev-manual/layers:understanding and creating layers`" section.
Understand What Contributes to Your Image Size
----------------------------------------------
It is easiest to have something to start with when creating your own
distribution. You can use the Yocto Project out-of-the-box to create the
``poky-tiny`` distribution. Ultimately, you will want to make changes in
your own distribution that are likely modeled after ``poky-tiny``.
.. note::
To use ``poky-tiny`` in your build, set the :term:`DISTRO` variable in your
``local.conf`` file to "poky-tiny" as described in the
":ref:`dev-manual/custom-distribution:creating your own distribution`"
section.
Understanding some memory concepts will help you reduce the system size.
Memory consists of static, dynamic, and temporary memory. Static memory
is the TEXT (code), DATA (initialized data in the code), and BSS
(uninitialized data) sections. Dynamic memory represents memory that is
allocated at runtime: stacks, hash tables, and so forth. Temporary
memory is recovered after the boot process. This memory consists of
memory used for decompressing the kernel and for the ``__init__``
functions.
To help you see where you currently are with kernel and root filesystem
sizes, you can use two tools found in the :term:`Source Directory`
in the
``scripts/tiny/`` directory:
- ``ksize.py``: Reports component sizes for the kernel build objects.
- ``dirsize.py``: Reports component sizes for the root filesystem.
This next tool and command help you organize configuration fragments and
view file dependencies in a human-readable form:
- ``merge_config.sh``: Helps you manage configuration files and
fragments within the kernel. With this tool, you can merge individual
configuration fragments together. The tool allows you to make
overrides and warns you of any missing configuration options. The
tool is ideal for allowing you to iterate on configurations, create
minimal configurations, and create configuration files for different
machines without having to duplicate your process.
The ``merge_config.sh`` script is part of the Linux Yocto kernel Git
repositories (i.e. ``linux-yocto-3.14``, ``linux-yocto-3.10``,
``linux-yocto-3.8``, and so forth) in the ``scripts/kconfig``
directory.
For more information on configuration fragments, see the
":ref:`kernel-dev/common:creating configuration fragments`"
section in the Yocto Project Linux Kernel Development Manual.
- ``bitbake -u taskexp -g bitbake_target``: Using the BitBake command
with these options brings up a Dependency Explorer from which you can
view file dependencies. Understanding these dependencies allows you
to make informed decisions when cutting out various pieces of the
kernel and root filesystem.
Trim the Root Filesystem
------------------------
The root filesystem is made up of packages for booting, libraries, and
applications. To change things, you can configure how the packaging
happens, which changes the way you build them. You can also modify the
filesystem itself or select a different filesystem.
First, find out what is hogging your root filesystem by running the
``dirsize.py`` script from your root directory::
$ cd root-directory-of-image
$ dirsize.py 100000 > dirsize-100k.log
$ cat dirsize-100k.log
You can apply a filter to the script to ignore files
under a certain size. The previous example filters out any files below
100 Kbytes. The sizes reported by the tool are uncompressed, and thus
will be smaller by a relatively constant factor in a compressed root
filesystem. When you examine your log file, you can focus on areas of
the root filesystem that take up large amounts of memory.
You need to be sure that what you eliminate does not cripple the
functionality you need. One way to see how packages relate to each other
is by using the Dependency Explorer UI with the BitBake command::
$ cd image-directory
$ bitbake -u taskexp -g image
Use the interface to
select potential packages you wish to eliminate and see their dependency
relationships.
When deciding how to reduce the size, get rid of packages that result in
minimal impact on the feature set. For example, you might not need a VGA
display. Or, you might be able to get by with ``devtmpfs`` and ``mdev``
instead of ``udev``.
Use your ``local.conf`` file to make changes. For example, to eliminate
``udev`` and ``glib``, set the following in the local configuration
file::
VIRTUAL-RUNTIME_dev_manager = ""
Finally, you should consider exactly the type of root filesystem you
need to meet your needs while also reducing its size. For example,
consider ``cramfs``, ``squashfs``, ``ubifs``, ``ext2``, or an
:term:`Initramfs` using ``initramfs``. Be aware that ``ext3`` requires a 1
Mbyte journal. If you are okay with running read-only, you do not need
this journal.
.. note::
After each round of elimination, you need to rebuild your system and
then use the tools to see the effects of your reductions.
Trim the Kernel
---------------
The kernel is built by including policies for hardware-independent
aspects. What subsystems do you enable? For what architecture are you
building? Which drivers do you build by default?
.. note::
You can modify the kernel source if you want to help with boot time.
Run the ``ksize.py`` script from the top-level Linux build directory to
get an idea of what is making up the kernel::
$ cd top-level-linux-build-directory
$ ksize.py > ksize.log
$ cat ksize.log
When you examine the log, you will see how much space is taken up with
the built-in ``.o`` files for drivers, networking, core kernel files,
filesystem, sound, and so forth. The sizes reported by the tool are
uncompressed, and thus will be smaller by a relatively constant factor
in a compressed kernel image. Look to reduce the areas that are large
and taking up around the "90% rule."
To examine, or drill down, into any particular area, use the ``-d``
option with the script::
$ ksize.py -d > ksize.log
Using this option
breaks out the individual file information for each area of the kernel
(e.g. drivers, networking, and so forth).
Use your log file to see what you can eliminate from the kernel based on
features you can let go. For example, if you are not going to need
sound, you do not need any drivers that support sound.
After figuring out what to eliminate, you need to reconfigure the kernel
to reflect those changes during the next build. You could run
``menuconfig`` and make all your changes at once. However, that makes it
difficult to see the effects of your individual eliminations and also
makes it difficult to replicate the changes for perhaps another target
device. A better method is to start with no configurations using
``allnoconfig``, create configuration fragments for individual changes,
and then manage the fragments into a single configuration file using
``merge_config.sh``. The tool makes it easy for you to iterate using the
configuration change and build cycle.
Each time you make configuration changes, you need to rebuild the kernel
and check to see what impact your changes had on the overall size.
Remove Package Management Requirements
--------------------------------------
Packaging requirements add size to the image. One way to reduce the size
of the image is to remove all the packaging requirements from the image.
This reduction includes both removing the package manager and its unique
dependencies as well as removing the package management data itself.
To eliminate all the packaging requirements for an image, be sure that
"package-management" is not part of your
:term:`IMAGE_FEATURES`
statement for the image. When you remove this feature, you are removing
the package manager as well as its dependencies from the root
filesystem.
Look for Other Ways to Minimize Size
------------------------------------
Depending on your particular circumstances, other areas that you can
trim likely exist. The key to finding these areas is through tools and
methods described here combined with experimentation and iteration. Here
are a couple of areas to experiment with:
- ``glibc``: In general, follow this process:
#. Remove ``glibc`` features from
:term:`DISTRO_FEATURES`
that you think you do not need.
#. Build your distribution.
#. If the build fails due to missing symbols in a package, determine
if you can reconfigure the package to not need those features. For
example, change the configuration to not support wide character
support as is done for ``ncurses``. Or, if support for those
characters is needed, determine what ``glibc`` features provide
the support and restore the configuration.
4. Rebuild and repeat the process.
- ``busybox``: For BusyBox, use a process similar as described for
``glibc``. A difference is you will need to boot the resulting system
to see if you are able to do everything you expect from the running
system. You need to be sure to integrate configuration fragments into
Busybox because BusyBox handles its own core features and then allows
you to add configuration fragments on top.
Iterate on the Process
----------------------
If you have not reached your goals on system size, you need to iterate
on the process. The process is the same. Use the tools and see just what
is taking up 90% of the root filesystem and the kernel. Decide what you
can eliminate without limiting your device beyond what you need.
Depending on your system, a good place to look might be Busybox, which
provides a stripped down version of Unix tools in a single, executable
file. You might be able to drop virtual terminal services or perhaps
ipv6.
Building Images for More than One Machine
=========================================
A common scenario developers face is creating images for several
different machines that use the same software environment. In this
situation, it is tempting to set the tunings and optimization flags for
each build specifically for the targeted hardware (i.e. "maxing out" the
tunings). Doing so can considerably add to build times and package feed
maintenance collectively for the machines. For example, selecting tunes
that are extremely specific to a CPU core used in a system might enable
some micro optimizations in GCC for that particular system but would
otherwise not gain you much of a performance difference across the other
systems as compared to using a more general tuning across all the builds
(e.g. setting :term:`DEFAULTTUNE`
specifically for each machine's build). Rather than "max out" each
build's tunings, you can take steps that cause the OpenEmbedded build
system to reuse software across the various machines where it makes
sense.
If build speed and package feed maintenance are considerations, you
should consider the points in this section that can help you optimize
your tunings to best consider build times and package feed maintenance.
- *Share the :term:`Build Directory`:* If at all possible, share the
:term:`TMPDIR` across builds. The Yocto Project supports switching between
different :term:`MACHINE` values in the same :term:`TMPDIR`. This practice
is well supported and regularly used by developers when building for
multiple machines. When you use the same :term:`TMPDIR` for multiple
machine builds, the OpenEmbedded build system can reuse the existing native
and often cross-recipes for multiple machines. Thus, build time decreases.
.. note::
If :term:`DISTRO` settings change or fundamental configuration settings
such as the filesystem layout, you need to work with a clean :term:`TMPDIR`.
Sharing :term:`TMPDIR` under these circumstances might work but since it is
not guaranteed, you should use a clean :term:`TMPDIR`.
- *Enable the Appropriate Package Architecture:* By default, the
OpenEmbedded build system enables three levels of package
architectures: "all", "tune" or "package", and "machine". Any given
recipe usually selects one of these package architectures (types) for
its output. Depending for what a given recipe creates packages,
making sure you enable the appropriate package architecture can
directly impact the build time.
A recipe that just generates scripts can enable "all" architecture
because there are no binaries to build. To specifically enable "all"
architecture, be sure your recipe inherits the
:ref:`ref-classes-allarch` class.
This class is useful for "all" architectures because it configures
many variables so packages can be used across multiple architectures.
If your recipe needs to generate packages that are machine-specific
or when one of the build or runtime dependencies is already
machine-architecture dependent, which makes your recipe also
machine-architecture dependent, make sure your recipe enables the
"machine" package architecture through the
:term:`MACHINE_ARCH`
variable::
PACKAGE_ARCH = "${MACHINE_ARCH}"
When you do not
specifically enable a package architecture through the
:term:`PACKAGE_ARCH`, The
OpenEmbedded build system defaults to the
:term:`TUNE_PKGARCH` setting::
PACKAGE_ARCH = "${TUNE_PKGARCH}"
- *Choose a Generic Tuning File if Possible:* Some tunes are more
generic and can run on multiple targets (e.g. an ``armv5`` set of
packages could run on ``armv6`` and ``armv7`` processors in most
cases). Similarly, ``i486`` binaries could work on ``i586`` and
higher processors. You should realize, however, that advances on
newer processor versions would not be used.
If you select the same tune for several different machines, the
OpenEmbedded build system reuses software previously built, thus
speeding up the overall build time. Realize that even though a new
sysroot for each machine is generated, the software is not recompiled
and only one package feed exists.
- *Manage Granular Level Packaging:* Sometimes there are cases where
injecting another level of package architecture beyond the three
higher levels noted earlier can be useful. For example, consider how
NXP (formerly Freescale) allows for the easy reuse of binary packages
in their layer
:yocto_git:`meta-freescale </meta-freescale/>`.
In this example, the
:yocto_git:`fsl-dynamic-packagearch </meta-freescale/tree/classes/fsl-dynamic-packagearch.bbclass>`
class shares GPU packages for i.MX53 boards because all boards share
the AMD GPU. The i.MX6-based boards can do the same because all
boards share the Vivante GPU. This class inspects the BitBake
datastore to identify if the package provides or depends on one of
the sub-architecture values. If so, the class sets the
:term:`PACKAGE_ARCH` value
based on the ``MACHINE_SUBARCH`` value. If the package does not
provide or depend on one of the sub-architecture values but it
matches a value in the machine-specific filter, it sets
:term:`MACHINE_ARCH`. This
behavior reduces the number of packages built and saves build time by
reusing binaries.
- *Use Tools to Debug Issues:* Sometimes you can run into situations
where software is being rebuilt when you think it should not be. For
example, the OpenEmbedded build system might not be using shared
state between machines when you think it should be. These types of
situations are usually due to references to machine-specific
variables such as :term:`MACHINE`,
:term:`SERIAL_CONSOLES`,
:term:`XSERVER`,
:term:`MACHINE_FEATURES`,
and so forth in code that is supposed to only be tune-specific or
when the recipe depends
(:term:`DEPENDS`,
:term:`RDEPENDS`,
:term:`RRECOMMENDS`,
:term:`RSUGGESTS`, and so forth)
on some other recipe that already has
:term:`PACKAGE_ARCH` defined
as "${MACHINE_ARCH}".
.. note::
Patches to fix any issues identified are most welcome as these
issues occasionally do occur.
For such cases, you can use some tools to help you sort out the
situation:
- ``state-diff-machines.sh``*:* You can find this tool in the
``scripts`` directory of the Source Repositories. See the comments
in the script for information on how to use the tool.
- *BitBake's "-S printdiff" Option:* Using this option causes
BitBake to try to establish the closest signature match it can
(e.g. in the shared state cache) and then run ``bitbake-diffsigs``
over the matches to determine the stamps and delta where these two
stamp trees diverge.
Building Software from an External Source
=========================================
By default, the OpenEmbedded build system uses the :term:`Build Directory`
when building source code. The build process involves fetching the source
files, unpacking them, and then patching them if necessary before the build
takes place.
There are situations where you might want to build software from source
files that are external to and thus outside of the OpenEmbedded build
system. For example, suppose you have a project that includes a new BSP
with a heavily customized kernel. And, you want to minimize exposing the
build system to the development team so that they can focus on their
project and maintain everyone's workflow as much as possible. In this
case, you want a kernel source directory on the development machine
where the development occurs. You want the recipe's
:term:`SRC_URI` variable to point to
the external directory and use it as is, not copy it.
To build from software that comes from an external source, all you need to do
is inherit the :ref:`ref-classes-externalsrc` class and then set
the :term:`EXTERNALSRC` variable to point to your external source code. Here
are the statements to put in your ``local.conf`` file::
INHERIT += "externalsrc"
EXTERNALSRC:pn-myrecipe = "path-to-your-source-tree"
This next example shows how to accomplish the same thing by setting
:term:`EXTERNALSRC` in the recipe itself or in the recipe's append file::
EXTERNALSRC = "path"
EXTERNALSRC_BUILD = "path"
.. note::
In order for these settings to take effect, you must globally or
locally inherit the :ref:`ref-classes-externalsrc` class.
By default, :ref:`ref-classes-externalsrc` builds the source code in a
directory separate from the external source directory as specified by
:term:`EXTERNALSRC`. If you need
to have the source built in the same directory in which it resides, or
some other nominated directory, you can set
:term:`EXTERNALSRC_BUILD`
to point to that directory::
EXTERNALSRC_BUILD:pn-myrecipe = "path-to-your-source-tree"
Replicating a Build Offline
===========================
It can be useful to take a "snapshot" of upstream sources used in a
build and then use that "snapshot" later to replicate the build offline.
To do so, you need to first prepare and populate your downloads
directory your "snapshot" of files. Once your downloads directory is
ready, you can use it at any time and from any machine to replicate your
build.
Follow these steps to populate your Downloads directory:
#. *Create a Clean Downloads Directory:* Start with an empty downloads
directory (:term:`DL_DIR`). You
start with an empty downloads directory by either removing the files
in the existing directory or by setting :term:`DL_DIR` to point to either
an empty location or one that does not yet exist.
#. *Generate Tarballs of the Source Git Repositories:* Edit your
``local.conf`` configuration file as follows::
DL_DIR = "/home/your-download-dir/"
BB_GENERATE_MIRROR_TARBALLS = "1"
During
the fetch process in the next step, BitBake gathers the source files
and creates tarballs in the directory pointed to by :term:`DL_DIR`. See
the
:term:`BB_GENERATE_MIRROR_TARBALLS`
variable for more information.
#. *Populate Your Downloads Directory Without Building:* Use BitBake to
fetch your sources but inhibit the build::
$ bitbake target --runonly=fetch
The downloads directory (i.e. ``${DL_DIR}``) now has
a "snapshot" of the source files in the form of tarballs, which can
be used for the build.
#. *Optionally Remove Any Git or other SCM Subdirectories From the
Downloads Directory:* If you want, you can clean up your downloads
directory by removing any Git or other Source Control Management
(SCM) subdirectories such as ``${DL_DIR}/git2/*``. The tarballs
already contain these subdirectories.
Once your downloads directory has everything it needs regarding source
files, you can create your "own-mirror" and build your target.
Understand that you can use the files to build the target offline from
any machine and at any time.
Follow these steps to build your target using the files in the downloads
directory:
#. *Using Local Files Only:* Inside your ``local.conf`` file, add the
:term:`SOURCE_MIRROR_URL` variable, inherit the
:ref:`ref-classes-own-mirrors` class, and use the
:term:`BB_NO_NETWORK` variable to your ``local.conf``::
SOURCE_MIRROR_URL ?= "file:///home/your-download-dir/"
INHERIT += "own-mirrors"
BB_NO_NETWORK = "1"
The :term:`SOURCE_MIRROR_URL` and :ref:`ref-classes-own-mirrors`
class set up the system to use the downloads directory as your "own
mirror". Using the :term:`BB_NO_NETWORK` variable makes sure that
BitBake's fetching process in step 3 stays local, which means files
from your "own-mirror" are used.
#. *Start With a Clean Build:* You can start with a clean build by
removing the ``${``\ :term:`TMPDIR`\ ``}`` directory or using a new
:term:`Build Directory`.
#. *Build Your Target:* Use BitBake to build your target::
$ bitbake target
The build completes using the known local "snapshot" of source
files from your mirror. The resulting tarballs for your "snapshot" of
source files are in the downloads directory.
.. note::
The offline build does not work if recipes attempt to find the
latest version of software by setting
:term:`SRCREV` to
``${``\ :term:`AUTOREV`\ ``}``::
SRCREV = "${AUTOREV}"
When a recipe sets :term:`SRCREV` to
``${``\ :term:`AUTOREV`\ ``}``, the build system accesses the network in an
attempt to determine the latest version of software from the SCM.
Typically, recipes that use :term:`AUTOREV` are custom or modified
recipes. Recipes that reside in public repositories usually do not
use :term:`AUTOREV`.
If you do have recipes that use :term:`AUTOREV`, you can take steps to
still use the recipes in an offline build. Do the following:
#. Use a configuration generated by enabling :ref:`build
history <dev-manual/build-quality:maintaining build output quality>`.
#. Use the ``buildhistory-collect-srcrevs`` command to collect the
stored :term:`SRCREV` values from the build's history. For more
information on collecting these values, see the
":ref:`dev-manual/build-quality:build history package information`"
section.
#. Once you have the correct source revisions, you can modify
those recipes to set :term:`SRCREV` to specific versions of the
software.
poky/documentation/dev-manual/changes.rst
+525
View File
@@ -0,0 +1,525 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Making Changes to the Yocto Project
***********************************
Because the Yocto Project is an open-source, community-based project,
you can effect changes to the project. This section presents procedures
that show you how to submit a defect against the project and how to
submit a change.
Submitting a Defect Against the Yocto Project
=============================================
Use the Yocto Project implementation of
`Bugzilla <https://www.bugzilla.org/about/>`__ to submit a defect (bug)
against the Yocto Project. For additional information on this
implementation of Bugzilla see the ":ref:`Yocto Project
Bugzilla <resources-bugtracker>`" section in the
Yocto Project Reference Manual. For more detail on any of the following
steps, see the Yocto Project
:yocto_wiki:`Bugzilla wiki page </Bugzilla_Configuration_and_Bug_Tracking>`.
Use the following general steps to submit a bug:
#. Open the Yocto Project implementation of :yocto_bugs:`Bugzilla <>`.
#. Click "File a Bug" to enter a new bug.
#. Choose the appropriate "Classification", "Product", and "Component"
for which the bug was found. Bugs for the Yocto Project fall into
one of several classifications, which in turn break down into
several products and components. For example, for a bug against the
``meta-intel`` layer, you would choose "Build System, Metadata &
Runtime", "BSPs", and "bsps-meta-intel", respectively.
#. Choose the "Version" of the Yocto Project for which you found the
bug (e.g. &DISTRO;).
#. Determine and select the "Severity" of the bug. The severity
indicates how the bug impacted your work.
#. Choose the "Hardware" that the bug impacts.
#. Choose the "Architecture" that the bug impacts.
#. Choose a "Documentation change" item for the bug. Fixing a bug might
or might not affect the Yocto Project documentation. If you are
unsure of the impact to the documentation, select "Don't Know".
#. Provide a brief "Summary" of the bug. Try to limit your summary to
just a line or two and be sure to capture the essence of the bug.
#. Provide a detailed "Description" of the bug. You should provide as
much detail as you can about the context, behavior, output, and so
forth that surrounds the bug. You can even attach supporting files
for output from logs by using the "Add an attachment" button.
#. Click the "Submit Bug" button submit the bug. A new Bugzilla number
is assigned to the bug and the defect is logged in the bug tracking
system.
Once you file a bug, the bug is processed by the Yocto Project Bug
Triage Team and further details concerning the bug are assigned (e.g.
priority and owner). You are the "Submitter" of the bug and any further
categorization, progress, or comments on the bug result in Bugzilla
sending you an automated email concerning the particular change or
progress to the bug.
Submitting a Change to the Yocto Project
========================================
Contributions to the Yocto Project and OpenEmbedded are very welcome.
Because the system is extremely configurable and flexible, we recognize
that developers will want to extend, configure or optimize it for their
specific uses.
The Yocto Project uses a mailing list and a patch-based workflow that is
similar to the Linux kernel but contains important differences. In
general, there is a mailing list through which you can submit patches. You
should send patches to the appropriate mailing list so that they can be
reviewed and merged by the appropriate maintainer. The specific mailing
list you need to use depends on the location of the code you are
changing. Each component (e.g. layer) should have a ``README`` file that
indicates where to send the changes and which process to follow.
You can send the patch to the mailing list using whichever approach you
feel comfortable with to generate the patch. Once sent, the patch is
usually reviewed by the community at large. If somebody has concerns
with the patch, they will usually voice their concern over the mailing
list. If a patch does not receive any negative reviews, the maintainer
of the affected layer typically takes the patch, tests it, and then
based on successful testing, merges the patch.
The "poky" repository, which is the Yocto Project's reference build
environment, is a hybrid repository that contains several individual
pieces (e.g. BitBake, Metadata, documentation, and so forth) built using
the combo-layer tool. The upstream location used for submitting changes
varies by component:
- *Core Metadata:* Send your patch to the
:oe_lists:`openembedded-core </g/openembedded-core>`
mailing list. For example, a change to anything under the ``meta`` or
``scripts`` directories should be sent to this mailing list.
- *BitBake:* For changes to BitBake (i.e. anything under the
``bitbake`` directory), send your patch to the
:oe_lists:`bitbake-devel </g/bitbake-devel>`
mailing list.
- *"meta-\*" trees:* These trees contain Metadata. Use the
:yocto_lists:`poky </g/poky>` mailing list.
- *Documentation*: For changes to the Yocto Project documentation, use the
:yocto_lists:`docs </g/docs>` mailing list.
For changes to other layers hosted in the Yocto Project source
repositories (i.e. ``yoctoproject.org``) and tools use the
:yocto_lists:`Yocto Project </g/yocto/>` general mailing list.
.. note::
Sometimes a layer's documentation specifies to use a particular
mailing list. If so, use that list.
For additional recipes that do not fit into the core Metadata, you
should determine which layer the recipe should go into and submit the
change in the manner recommended by the documentation (e.g. the
``README`` file) supplied with the layer. If in doubt, please ask on the
Yocto general mailing list or on the openembedded-devel mailing list.
You can also push a change upstream and request a maintainer to pull the
change into the component's upstream repository. You do this by pushing
to a contribution repository that is upstream. See the
":ref:`overview-manual/development-environment:git workflows and the yocto project`"
section in the Yocto Project Overview and Concepts Manual for additional
concepts on working in the Yocto Project development environment.
Maintainers commonly use ``-next`` branches to test submissions prior to
merging patches. Thus, you can get an idea of the status of a patch based on
whether the patch has been merged into one of these branches. The commonly
used testing branches for OpenEmbedded-Core are as follows:
- *openembedded-core "master-next" branch:* This branch is part of the
:oe_git:`openembedded-core </openembedded-core/>` repository and contains
proposed changes to the core metadata.
- *poky "master-next" branch:* This branch is part of the
:yocto_git:`poky </poky/>` repository and combines proposed
changes to BitBake, the core metadata and the poky distro.
Similarly, stable branches maintained by the project may have corresponding
``-next`` branches which collect proposed changes. For example,
``&DISTRO_NAME_NO_CAP;-next`` and ``&DISTRO_NAME_NO_CAP_MINUS_ONE;-next``
branches in both the "openembdedded-core" and "poky" repositories.
Other layers may have similar testing branches but there is no formal
requirement or standard for these so please check the documentation for the
layers you are contributing to.
The following sections provide procedures for submitting a change.
Preparing Changes for Submission
--------------------------------
#. *Make Your Changes Locally:* Make your changes in your local Git
repository. You should make small, controlled, isolated changes.
Keeping changes small and isolated aids review, makes
merging/rebasing easier and keeps the change history clean should
anyone need to refer to it in future.
#. *Stage Your Changes:* Stage your changes by using the ``git add``
command on each file you changed.
#. *Commit Your Changes:* Commit the change by using the ``git commit``
command. Make sure your commit information follows standards by
following these accepted conventions:
- Be sure to include a "Signed-off-by:" line in the same style as
required by the Linux kernel. This can be done by using the
``git commit -s`` command. Adding this line signifies that you,
the submitter, have agreed to the Developer's Certificate of
Origin 1.1 as follows:
.. code-block:: none
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
- Provide a single-line summary of the change and, if more
explanation is needed, provide more detail in the body of the
commit. This summary is typically viewable in the "shortlist" of
changes. Thus, providing something short and descriptive that
gives the reader a summary of the change is useful when viewing a
list of many commits. You should prefix this short description
with the recipe name (if changing a recipe), or else with the
short form path to the file being changed.
- For the body of the commit message, provide detailed information
that describes what you changed, why you made the change, and the
approach you used. It might also be helpful if you mention how you
tested the change. Provide as much detail as you can in the body
of the commit message.
.. note::
You do not need to provide a more detailed explanation of a
change if the change is minor to the point of the single line
summary providing all the information.
- If the change addresses a specific bug or issue that is associated
with a bug-tracking ID, include a reference to that ID in your
detailed description. For example, the Yocto Project uses a
specific convention for bug references --- any commit that addresses
a specific bug should use the following form for the detailed
description. Be sure to use the actual bug-tracking ID from
Bugzilla for bug-id::
Fixes [YOCTO #bug-id]
detailed description of change
Using Email to Submit a Patch
-----------------------------
Depending on the components changed, you need to submit the email to a
specific mailing list. For some guidance on which mailing list to use,
see the
:ref:`list <dev-manual/changes:submitting a change to the yocto project>`
at the beginning of this section. For a description of all the available
mailing lists, see the ":ref:`Mailing Lists <resources-mailinglist>`" section in the
Yocto Project Reference Manual.
Here is the general procedure on how to submit a patch through email
without using the scripts once the steps in
:ref:`dev-manual/changes:preparing changes for submission` have been followed:
#. *Format the Commit:* Format the commit into an email message. To
format commits, use the ``git format-patch`` command. When you
provide the command, you must include a revision list or a number of
patches as part of the command. For example, either of these two
commands takes your most recent single commit and formats it as an
email message in the current directory::
$ git format-patch -1
or ::
$ git format-patch HEAD~
After the command is run, the current directory contains a numbered
``.patch`` file for the commit.
If you provide several commits as part of the command, the
``git format-patch`` command produces a series of numbered files in
the current directory – one for each commit. If you have more than
one patch, you should also use the ``--cover`` option with the
command, which generates a cover letter as the first "patch" in the
series. You can then edit the cover letter to provide a description
for the series of patches. For information on the
``git format-patch`` command, see ``GIT_FORMAT_PATCH(1)`` displayed
using the ``man git-format-patch`` command.
.. note::
If you are or will be a frequent contributor to the Yocto Project
or to OpenEmbedded, you might consider requesting a contrib area
and the necessary associated rights.
#. *Send the patches via email:* Send the patches to the recipients and
relevant mailing lists by using the ``git send-email`` command.
.. note::
In order to use ``git send-email``, you must have the proper Git packages
installed on your host.
For Ubuntu, Debian, and Fedora the package is ``git-email``.
The ``git send-email`` command sends email by using a local or remote
Mail Transport Agent (MTA) such as ``msmtp``, ``sendmail``, or
through a direct ``smtp`` configuration in your Git ``~/.gitconfig``
file. If you are submitting patches through email only, it is very
important that you submit them without any whitespace or HTML
formatting that either you or your mailer introduces. The maintainer
that receives your patches needs to be able to save and apply them
directly from your emails. A good way to verify that what you are
sending will be applicable by the maintainer is to do a dry run and
send them to yourself and then save and apply them as the maintainer
would.
The ``git send-email`` command is the preferred method for sending
your patches using email since there is no risk of compromising
whitespace in the body of the message, which can occur when you use
your own mail client. The command also has several options that let
you specify recipients and perform further editing of the email
message. For information on how to use the ``git send-email``
command, see ``GIT-SEND-EMAIL(1)`` displayed using the
``man git-send-email`` command.
The Yocto Project uses a `Patchwork instance <https://patchwork.yoctoproject.org/>`__
to track the status of patches submitted to the various mailing lists and to
support automated patch testing. Each submitted patch is checked for common
mistakes and deviations from the expected patch format and submitters are
notified by patchtest if such mistakes are found. This process helps to
reduce the burden of patch review on maintainers.
.. note::
This system is imperfect and changes can sometimes get lost in the flow.
Asking about the status of a patch or change is reasonable if the change
has been idle for a while with no feedback.
Using Scripts to Push a Change Upstream and Request a Pull
----------------------------------------------------------
For larger patch series it is preferable to send a pull request which not
only includes the patch but also a pointer to a branch that can be pulled
from. This involves making a local branch for your changes, pushing this
branch to an accessible repository and then using the ``create-pull-request``
and ``send-pull-request`` scripts from openembedded-core to create and send a
patch series with a link to the branch for review.
Follow this procedure to push a change to an upstream "contrib" Git
repository once the steps in :ref:`dev-manual/changes:preparing changes for submission` have
been followed:
.. note::
You can find general Git information on how to push a change upstream
in the
`Git Community Book <https://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows>`__.
#. *Push Your Commits to a "Contrib" Upstream:* If you have arranged for
permissions to push to an upstream contrib repository, push the
change to that repository::
$ git push upstream_remote_repo local_branch_name
For example, suppose you have permissions to push
into the upstream ``meta-intel-contrib`` repository and you are
working in a local branch named `your_name`\ ``/README``. The following
command pushes your local commits to the ``meta-intel-contrib``
upstream repository and puts the commit in a branch named
`your_name`\ ``/README``::
$ git push meta-intel-contrib your_name/README
#. *Determine Who to Notify:* Determine the maintainer or the mailing
list that you need to notify for the change.
Before submitting any change, you need to be sure who the maintainer
is or what mailing list that you need to notify. Use either these
methods to find out:
- *Maintenance File:* Examine the ``maintainers.inc`` file, which is
located in the :term:`Source Directory` at
``meta/conf/distro/include``, to see who is responsible for code.
- *Search by File:* Using :ref:`overview-manual/development-environment:git`, you can
enter the following command to bring up a short list of all
commits against a specific file::
git shortlog -- filename
Just provide the name of the file for which you are interested. The
information returned is not ordered by history but does include a
list of everyone who has committed grouped by name. From the list,
you can see who is responsible for the bulk of the changes against
the file.
- *Examine the List of Mailing Lists:* For a list of the Yocto
Project and related mailing lists, see the ":ref:`Mailing
lists <resources-mailinglist>`" section in
the Yocto Project Reference Manual.
#. *Make a Pull Request:* Notify the maintainer or the mailing list that
you have pushed a change by making a pull request.
The Yocto Project provides two scripts that conveniently let you
generate and send pull requests to the Yocto Project. These scripts
are ``create-pull-request`` and ``send-pull-request``. You can find
these scripts in the ``scripts`` directory within the
:term:`Source Directory` (e.g.
``poky/scripts``).
Using these scripts correctly formats the requests without
introducing any whitespace or HTML formatting. The maintainer that
receives your patches either directly or through the mailing list
needs to be able to save and apply them directly from your emails.
Using these scripts is the preferred method for sending patches.
First, create the pull request. For example, the following command
runs the script, specifies the upstream repository in the contrib
directory into which you pushed the change, and provides a subject
line in the created patch files::
$ poky/scripts/create-pull-request -u meta-intel-contrib -s "Updated Manual Section Reference in README"
Running this script forms ``*.patch`` files in a folder named
``pull-``\ `PID` in the current directory. One of the patch files is a
cover letter.
Before running the ``send-pull-request`` script, you must edit the
cover letter patch to insert information about your change. After
editing the cover letter, send the pull request. For example, the
following command runs the script and specifies the patch directory
and email address. In this example, the email address is a mailing
list::
$ poky/scripts/send-pull-request -p ~/meta-intel/pull-10565 -t meta-intel@lists.yoctoproject.org
You need to follow the prompts as the script is interactive.
.. note::
For help on using these scripts, simply provide the ``-h``
argument as follows::
$ poky/scripts/create-pull-request -h
$ poky/scripts/send-pull-request -h
Responding to Patch Review
--------------------------
You may get feedback on your submitted patches from other community members
or from the automated patchtest service. If issues are identified in your
patch then it is usually necessary to address these before the patch will be
accepted into the project. In this case you should amend the patch according
to the feedback and submit an updated version to the relevant mailing list,
copying in the reviewers who provided feedback to the previous version of the
patch.
The patch should be amended using ``git commit --amend`` or perhaps ``git
rebase`` for more expert git users. You should also modify the ``[PATCH]``
tag in the email subject line when sending the revised patch to mark the new
iteration as ``[PATCH v2]``, ``[PATCH v3]``, etc as appropriate. This can be
done by passing the ``-v`` argument to ``git format-patch`` with a version
number.
Lastly please ensure that you also test your revised changes. In particular
please don't just edit the patch file written out by ``git format-patch`` and
resend it.
Submitting Changes to Stable Release Branches
---------------------------------------------
The process for proposing changes to a Yocto Project stable branch differs
from the steps described above. Changes to a stable branch must address
identified bugs or CVEs and should be made carefully in order to avoid the
risk of introducing new bugs or breaking backwards compatibility. Typically
bug fixes must already be accepted into the master branch before they can be
backported to a stable branch unless the bug in question does not affect the
master branch or the fix on the master branch is unsuitable for backporting.
The list of stable branches along with the status and maintainer for each
branch can be obtained from the
:yocto_wiki:`Releases wiki page </Releases>`.
.. note::
Changes will not typically be accepted for branches which are marked as
End-Of-Life (EOL).
With this in mind, the steps to submit a change for a stable branch are as
follows:
#. *Identify the bug or CVE to be fixed:* This information should be
collected so that it can be included in your submission.
See :ref:`dev-manual/vulnerabilities:checking for vulnerabilities`
for details about CVE tracking.
#. *Check if the fix is already present in the master branch:* This will
result in the most straightforward path into the stable branch for the
fix.
#. *If the fix is present in the master branch --- submit a backport request
by email:* You should send an email to the relevant stable branch
maintainer and the mailing list with details of the bug or CVE to be
fixed, the commit hash on the master branch that fixes the issue and
the stable branches which you would like this fix to be backported to.
#. *If the fix is not present in the master branch --- submit the fix to the
master branch first:* This will ensure that the fix passes through the
project's usual patch review and test processes before being accepted.
It will also ensure that bugs are not left unresolved in the master
branch itself. Once the fix is accepted in the master branch a backport
request can be submitted as above.
#. *If the fix is unsuitable for the master branch --- submit a patch
directly for the stable branch:* This method should be considered as a
last resort. It is typically necessary when the master branch is using
a newer version of the software which includes an upstream fix for the
issue or when the issue has been fixed on the master branch in a way
that introduces backwards incompatible changes. In this case follow the
steps in :ref:`dev-manual/changes:preparing changes for submission` and
:ref:`dev-manual/changes:using email to submit a patch` but modify the subject header of your patch
email to include the name of the stable branch which you are
targetting. This can be done using the ``--subject-prefix`` argument to
``git format-patch``, for example to submit a patch to the dunfell
branch use
``git format-patch --subject-prefix='&DISTRO_NAME_NO_CAP_MINUS_ONE;][PATCH' ...``.
poky/documentation/dev-manual/custom-distribution.rst
+109
View File
@@ -0,0 +1,109 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Creating Your Own Distribution
******************************
When you build an image using the Yocto Project and do not alter any
distribution :term:`Metadata`, you are
creating a Poky distribution. If you wish to gain more control over
package alternative selections, compile-time options, and other
low-level configurations, you can create your own distribution.
To create your own distribution, the basic steps consist of creating
your own distribution layer, creating your own distribution
configuration file, and then adding any needed code and Metadata to the
layer. The following steps provide some more detail:
- *Create a layer for your new distro:* Create your distribution layer
so that you can keep your Metadata and code for the distribution
separate. It is strongly recommended that you create and use your own
layer for configuration and code. Using your own layer as compared to
just placing configurations in a ``local.conf`` configuration file
makes it easier to reproduce the same build configuration when using
multiple build machines. See the
":ref:`dev-manual/layers:creating a general layer using the \`\`bitbake-layers\`\` script`"
section for information on how to quickly set up a layer.
- *Create the distribution configuration file:* The distribution
configuration file needs to be created in the ``conf/distro``
directory of your layer. You need to name it using your distribution
name (e.g. ``mydistro.conf``).
.. note::
The :term:`DISTRO` variable in your ``local.conf`` file determines the
name of your distribution.
You can split out parts of your configuration file into include files
and then "require" them from within your distribution configuration
file. Be sure to place the include files in the
``conf/distro/include`` directory of your layer. A common example
usage of include files would be to separate out the selection of
desired version and revisions for individual recipes.
Your configuration file needs to set the following required
variables:
- :term:`DISTRO_NAME`
- :term:`DISTRO_VERSION`
These following variables are optional and you typically set them
from the distribution configuration file:
- :term:`DISTRO_FEATURES`
- :term:`DISTRO_EXTRA_RDEPENDS`
- :term:`DISTRO_EXTRA_RRECOMMENDS`
- :term:`TCLIBC`
.. tip::
If you want to base your distribution configuration file on the
very basic configuration from OE-Core, you can use
``conf/distro/defaultsetup.conf`` as a reference and just include
variables that differ as compared to ``defaultsetup.conf``.
Alternatively, you can create a distribution configuration file
from scratch using the ``defaultsetup.conf`` file or configuration files
from another distribution such as Poky as a reference.
- *Provide miscellaneous variables:* Be sure to define any other
variables for which you want to create a default or enforce as part
of the distribution configuration. You can include nearly any
variable from the ``local.conf`` file. The variables you use are not
limited to the list in the previous bulleted item.
- *Point to Your distribution configuration file:* In your ``local.conf``
file in the :term:`Build Directory`, set your :term:`DISTRO` variable to
point to your distribution's configuration file. For example, if your
distribution's configuration file is named ``mydistro.conf``, then
you point to it as follows::
DISTRO = "mydistro"
- *Add more to the layer if necessary:* Use your layer to hold other
information needed for the distribution:
- Add recipes for installing distro-specific configuration files
that are not already installed by another recipe. If you have
distro-specific configuration files that are included by an
existing recipe, you should add an append file (``.bbappend``) for
those. For general information and recommendations on how to add
recipes to your layer, see the
":ref:`dev-manual/layers:creating your own layer`" and
":ref:`dev-manual/layers:following best practices when creating layers`"
sections.
- Add any image recipes that are specific to your distribution.
- Add a ``psplash`` append file for a branded splash screen, using
the :term:`SPLASH_IMAGES` variable.
- Add any other append files to make custom changes that are
specific to individual recipes.
For information on append files, see the
":ref:`dev-manual/layers:appending other layers metadata with your layer`"
section.
poky/documentation/dev-manual/custom-template-configuration-directory.rst
+52
View File
@@ -0,0 +1,52 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Creating a Custom Template Configuration Directory
**************************************************
If you are producing your own customized version of the build system for
use by other users, you might want to provide a custom build configuration
that includes all the necessary settings and layers (i.e. ``local.conf`` and
``bblayers.conf`` that are created in a new :term:`Build Directory`) and a custom
message that is shown when setting up the build. This can be done by
creating one or more template configuration directories in your
custom distribution layer.
This can be done by using ``bitbake-layers save-build-conf``::
$ bitbake-layers save-build-conf ../../meta-alex/ test-1
NOTE: Starting bitbake server...
NOTE: Configuration template placed into /srv/work/alex/meta-alex/conf/templates/test-1
Please review the files in there, and particularly provide a configuration description in /srv/work/alex/meta-alex/conf/templates/test-1/conf-notes.txt
You can try out the configuration with
TEMPLATECONF=/srv/work/alex/meta-alex/conf/templates/test-1 . /srv/work/alex/poky/oe-init-build-env build-try-test-1
The above command takes the config files from the currently active :term:`Build Directory` under ``conf``,
replaces site-specific paths in ``bblayers.conf`` with ``##OECORE##``-relative paths, and copies
the config files into a specified layer under a specified template name.
To use those saved templates as a starting point for a build, users should point
to one of them with :term:`TEMPLATECONF` environment variable::
TEMPLATECONF=/srv/work/alex/meta-alex/conf/templates/test-1 . /srv/work/alex/poky/oe-init-build-env build-try-test-1
The OpenEmbedded build system uses the environment variable
:term:`TEMPLATECONF` to locate the directory from which it gathers
configuration information that ultimately ends up in the
:term:`Build Directory` ``conf`` directory.
If :term:`TEMPLATECONF` is not set, the default value is obtained
from ``.templateconf`` file that is read from the same directory as
``oe-init-build-env`` script. For the Poky reference distribution this
would be::
TEMPLATECONF=${TEMPLATECONF:-meta-poky/conf/templates/default}
If you look at a configuration template directory, you will
see the ``bblayers.conf.sample``, ``local.conf.sample``, and
``conf-notes.txt`` files. The build system uses these files to form the
respective ``bblayers.conf`` file, ``local.conf`` file, and show
users a note about the build they're setting up
when running the ``oe-init-build-env`` setup script. These can be
edited further if needed to improve or change the build configurations
available to the users.
poky/documentation/dev-manual/customizing-images.rst
+223
View File
@@ -0,0 +1,223 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Customizing Images
******************
You can customize images to satisfy particular requirements. This
section describes several methods and provides guidelines for each.
Customizing Images Using ``local.conf``
=======================================
Probably the easiest way to customize an image is to add a package by
way of the ``local.conf`` configuration file. Because it is limited to
local use, this method generally only allows you to add packages and is
not as flexible as creating your own customized image. When you add
packages using local variables this way, you need to realize that these
variable changes are in effect for every build and consequently affect
all images, which might not be what you require.
To add a package to your image using the local configuration file, use
the :term:`IMAGE_INSTALL` variable with the ``:append`` operator::
IMAGE_INSTALL:append = " strace"
Use of the syntax is important; specifically, the leading space
after the opening quote and before the package name, which is
``strace`` in this example. This space is required since the ``:append``
operator does not add the space.
Furthermore, you must use ``:append`` instead of the ``+=`` operator if
you want to avoid ordering issues. The reason for this is because doing
so unconditionally appends to the variable and avoids ordering problems
due to the variable being set in image recipes and ``.bbclass`` files
with operators like ``?=``. Using ``:append`` ensures the operation
takes effect.
As shown in its simplest use, ``IMAGE_INSTALL:append`` affects all
images. It is possible to extend the syntax so that the variable applies
to a specific image only. Here is an example::
IMAGE_INSTALL:append:pn-core-image-minimal = " strace"
This example adds ``strace`` to the ``core-image-minimal`` image only.
You can add packages using a similar approach through the
:term:`CORE_IMAGE_EXTRA_INSTALL` variable. If you use this variable, only
``core-image-*`` images are affected.
Customizing Images Using Custom ``IMAGE_FEATURES`` and ``EXTRA_IMAGE_FEATURES``
===============================================================================
Another method for customizing your image is to enable or disable
high-level image features by using the
:term:`IMAGE_FEATURES` and
:term:`EXTRA_IMAGE_FEATURES`
variables. Although the functions for both variables are nearly
equivalent, best practices dictate using :term:`IMAGE_FEATURES` from within
a recipe and using :term:`EXTRA_IMAGE_FEATURES` from within your
``local.conf`` file, which is found in the :term:`Build Directory`.
To understand how these features work, the best reference is
:ref:`meta/classes-recipe/image.bbclass <ref-classes-image>`.
This class lists out the available
:term:`IMAGE_FEATURES` of which most map to package groups while some, such
as ``debug-tweaks`` and ``read-only-rootfs``, resolve as general
configuration settings.
In summary, the file looks at the contents of the :term:`IMAGE_FEATURES`
variable and then maps or configures the feature accordingly. Based on
this information, the build system automatically adds the appropriate
packages or configurations to the
:term:`IMAGE_INSTALL` variable.
Effectively, you are enabling extra features by extending the class or
creating a custom class for use with specialized image ``.bb`` files.
Use the :term:`EXTRA_IMAGE_FEATURES` variable from within your local
configuration file. Using a separate area from which to enable features
with this variable helps you avoid overwriting the features in the image
recipe that are enabled with :term:`IMAGE_FEATURES`. The value of
:term:`EXTRA_IMAGE_FEATURES` is added to :term:`IMAGE_FEATURES` within
``meta/conf/bitbake.conf``.
To illustrate how you can use these variables to modify your image,
consider an example that selects the SSH server. The Yocto Project ships
with two SSH servers you can use with your images: Dropbear and OpenSSH.
Dropbear is a minimal SSH server appropriate for resource-constrained
environments, while OpenSSH is a well-known standard SSH server
implementation. By default, the ``core-image-sato`` image is configured
to use Dropbear. The ``core-image-full-cmdline`` and ``core-image-lsb``
images both include OpenSSH. The ``core-image-minimal`` image does not
contain an SSH server.
You can customize your image and change these defaults. Edit the
:term:`IMAGE_FEATURES` variable in your recipe or use the
:term:`EXTRA_IMAGE_FEATURES` in your ``local.conf`` file so that it
configures the image you are working with to include
``ssh-server-dropbear`` or ``ssh-server-openssh``.
.. note::
See the ":ref:`ref-manual/features:image features`" section in the Yocto
Project Reference Manual for a complete list of image features that ship
with the Yocto Project.
Customizing Images Using Custom .bb Files
=========================================
You can also customize an image by creating a custom recipe that defines
additional software as part of the image. The following example shows
the form for the two lines you need::
IMAGE_INSTALL = "packagegroup-core-x11-base package1 package2"
inherit core-image
Defining the software using a custom recipe gives you total control over
the contents of the image. It is important to use the correct names of
packages in the :term:`IMAGE_INSTALL` variable. You must use the
OpenEmbedded notation and not the Debian notation for the names (e.g.
``glibc-dev`` instead of ``libc6-dev``).
The other method for creating a custom image is to base it on an
existing image. For example, if you want to create an image based on
``core-image-sato`` but add the additional package ``strace`` to the
image, copy the ``meta/recipes-sato/images/core-image-sato.bb`` to a new
``.bb`` and add the following line to the end of the copy::
IMAGE_INSTALL += "strace"
Customizing Images Using Custom Package Groups
==============================================
For complex custom images, the best approach for customizing an image is
to create a custom package group recipe that is used to build the image
or images. A good example of a package group recipe is
``meta/recipes-core/packagegroups/packagegroup-base.bb``.
If you examine that recipe, you see that the :term:`PACKAGES` variable lists
the package group packages to produce. The ``inherit packagegroup``
statement sets appropriate default values and automatically adds
``-dev``, ``-dbg``, and ``-ptest`` complementary packages for each
package specified in the :term:`PACKAGES` statement.
.. note::
The ``inherit packagegroup`` line should be located near the top of the
recipe, certainly before the :term:`PACKAGES` statement.
For each package you specify in :term:`PACKAGES`, you can use :term:`RDEPENDS`
and :term:`RRECOMMENDS` entries to provide a list of packages the parent
task package should contain. You can see examples of these further down
in the ``packagegroup-base.bb`` recipe.
Here is a short, fabricated example showing the same basic pieces for a
hypothetical packagegroup defined in ``packagegroup-custom.bb``, where
the variable :term:`PN` is the standard way to abbreviate the reference to
the full packagegroup name ``packagegroup-custom``::
DESCRIPTION = "My Custom Package Groups"
inherit packagegroup
PACKAGES = "\
${PN}-apps \
${PN}-tools \
"
RDEPENDS:${PN}-apps = "\
dropbear \
portmap \
psplash"
RDEPENDS:${PN}-tools = "\
oprofile \
oprofileui-server \
lttng-tools"
RRECOMMENDS:${PN}-tools = "\
kernel-module-oprofile"
In the previous example, two package group packages are created with
their dependencies and their recommended package dependencies listed:
``packagegroup-custom-apps``, and ``packagegroup-custom-tools``. To
build an image using these package group packages, you need to add
``packagegroup-custom-apps`` and/or ``packagegroup-custom-tools`` to
:term:`IMAGE_INSTALL`. For other forms of image dependencies see the other
areas of this section.
Customizing an Image Hostname
=============================
By default, the configured hostname (i.e. ``/etc/hostname``) in an image
is the same as the machine name. For example, if
:term:`MACHINE` equals "qemux86", the
configured hostname written to ``/etc/hostname`` is "qemux86".
You can customize this name by altering the value of the "hostname"
variable in the ``base-files`` recipe using either an append file or a
configuration file. Use the following in an append file::
hostname = "myhostname"
Use the following in a configuration file::
hostname:pn-base-files = "myhostname"
Changing the default value of the variable "hostname" can be useful in
certain situations. For example, suppose you need to do extensive
testing on an image and you would like to easily identify the image
under test from existing images with typical default hostnames. In this
situation, you could change the default hostname to "testme", which
results in all the images using the name "testme". Once testing is
complete and you do not need to rebuild the image for test any longer,
you can easily reset the default hostname.
Another point of interest is that if you unset the variable, the image
will have no default hostname in the filesystem. Here is an example that
unsets the variable in a configuration file::
hostname:pn-base-files = ""
Having no default hostname in the filesystem is suitable for
environments that use dynamic hostnames such as virtual machines.
poky/documentation/dev-manual/debugging.rst
+1248
View File
File diff suppressed because it is too large Load Diff
poky/documentation/dev-manual/development-shell.rst
+82
View File
@@ -0,0 +1,82 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Using a Development Shell
*************************
When debugging certain commands or even when just editing packages,
``devshell`` can be a useful tool. When you invoke ``devshell``, all
tasks up to and including
:ref:`ref-tasks-patch` are run for the
specified target. Then, a new terminal is opened and you are placed in
``${``\ :term:`S`\ ``}``, the source
directory. In the new terminal, all the OpenEmbedded build-related
environment variables are still defined so you can use commands such as
``configure`` and ``make``. The commands execute just as if the
OpenEmbedded build system were executing them. Consequently, working
this way can be helpful when debugging a build or preparing software to
be used with the OpenEmbedded build system.
Following is an example that uses ``devshell`` on a target named
``matchbox-desktop``::
$ bitbake matchbox-desktop -c devshell
This command spawns a terminal with a shell prompt within the
OpenEmbedded build environment. The
:term:`OE_TERMINAL` variable
controls what type of shell is opened.
For spawned terminals, the following occurs:
- The ``PATH`` variable includes the cross-toolchain.
- The ``pkgconfig`` variables find the correct ``.pc`` files.
- The ``configure`` command finds the Yocto Project site files as well
as any other necessary files.
Within this environment, you can run configure or compile commands as if
they were being run by the OpenEmbedded build system itself. As noted
earlier, the working directory also automatically changes to the Source
Directory (:term:`S`).
To manually run a specific task using ``devshell``, run the
corresponding ``run.*`` script in the
``${``\ :term:`WORKDIR`\ ``}/temp``
directory (e.g., ``run.do_configure.``\ `pid`). If a task's script does
not exist, which would be the case if the task was skipped by way of the
sstate cache, you can create the task by first running it outside of the
``devshell``::
$ bitbake -c task
.. note::
- Execution of a task's ``run.*`` script and BitBake's execution of
a task are identical. In other words, running the script re-runs
the task just as it would be run using the ``bitbake -c`` command.
- Any ``run.*`` file that does not have a ``.pid`` extension is a
symbolic link (symlink) to the most recent version of that file.
Remember, that the ``devshell`` is a mechanism that allows you to get
into the BitBake task execution environment. And as such, all commands
must be called just as BitBake would call them. That means you need to
provide the appropriate options for cross-compilation and so forth as
applicable.
When you are finished using ``devshell``, exit the shell or close the
terminal window.
.. note::
- It is worth remembering that when using ``devshell`` you need to
use the full compiler name such as ``arm-poky-linux-gnueabi-gcc``
instead of just using ``gcc``. The same applies to other
applications such as ``binutils``, ``libtool`` and so forth.
BitBake sets up environment variables such as :term:`CC` to assist
applications, such as ``make`` to find the correct tools.
- It is also worth noting that ``devshell`` still works over X11
forwarding and similar situations.
poky/documentation/dev-manual/device-manager.rst
+74
View File
@@ -0,0 +1,74 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
.. _device-manager:
Selecting a Device Manager
**************************
The Yocto Project provides multiple ways to manage the device manager
(``/dev``):
- Persistent and Pre-Populated ``/dev``: For this case, the ``/dev``
directory is persistent and the required device nodes are created
during the build.
- Use ``devtmpfs`` with a Device Manager: For this case, the ``/dev``
directory is provided by the kernel as an in-memory file system and
is automatically populated by the kernel at runtime. Additional
configuration of device nodes is done in user space by a device
manager like ``udev`` or ``busybox-mdev``.
Using Persistent and Pre-Populated ``/dev``
===========================================
To use the static method for device population, you need to set the
:term:`USE_DEVFS` variable to "0"
as follows::
USE_DEVFS = "0"
The content of the resulting ``/dev`` directory is defined in a Device
Table file. The
:term:`IMAGE_DEVICE_TABLES`
variable defines the Device Table to use and should be set in the
machine or distro configuration file. Alternatively, you can set this
variable in your ``local.conf`` configuration file.
If you do not define the :term:`IMAGE_DEVICE_TABLES` variable, the default
``device_table-minimal.txt`` is used::
IMAGE_DEVICE_TABLES = "device_table-mymachine.txt"
The population is handled by the ``makedevs`` utility during image
creation:
Using ``devtmpfs`` and a Device Manager
=======================================
To use the dynamic method for device population, you need to use (or be
sure to set) the :term:`USE_DEVFS`
variable to "1", which is the default::
USE_DEVFS = "1"
With this
setting, the resulting ``/dev`` directory is populated by the kernel
using ``devtmpfs``. Make sure the corresponding kernel configuration
variable ``CONFIG_DEVTMPFS`` is set when building you build a Linux
kernel.
All devices created by ``devtmpfs`` will be owned by ``root`` and have
permissions ``0600``.
To have more control over the device nodes, you can use a device manager
like ``udev`` or ``busybox-mdev``. You choose the device manager by
defining the ``VIRTUAL-RUNTIME_dev_manager`` variable in your machine or
distro configuration file. Alternatively, you can set this variable in
your ``local.conf`` configuration file::
VIRTUAL-RUNTIME_dev_manager = "udev"
# Some alternative values
# VIRTUAL-RUNTIME_dev_manager = "busybox-mdev"
# VIRTUAL-RUNTIME_dev_manager = "systemd"
poky/documentation/dev-manual/disk-space.rst
+45
View File
@@ -0,0 +1,45 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Conserving Disk Space
*********************
Conserving Disk Space During Builds
===================================
To help conserve disk space during builds, you can add the following
statement to your project's ``local.conf`` configuration file found in
the :term:`Build Directory`::
INHERIT += "rm_work"
Adding this statement deletes the work directory used for
building a recipe once the recipe is built. For more information on
"rm_work", see the :ref:`ref-classes-rm-work` class in the
Yocto Project Reference Manual.
When you inherit this class and build a ``core-image-sato`` image for a
``qemux86-64`` machine from an Ubuntu 22.04 x86-64 system, you end up with a
final disk usage of 22 Gbytes instead of &MIN_DISK_SPACE; Gbytes. However,
&MIN_DISK_SPACE_RM_WORK; Gbytes of initial free disk space are still needed to
create temporary files before they can be deleted.
Purging Duplicate Shared State Cache Files
==========================================
After multiple build iterations, the Shared State (sstate) cache can contain
duplicate cache files for a given package, while only the most recent one
is likely to be reusable. The following command purges all but the
newest sstate cache file for each package::
sstate-cache-management.sh --remove-duplicated --cache-dir=build/sstate-cache
This command will ask you to confirm the deletions it identifies.
.. note::
The duplicated sstate cache files of one package must have the same
architecture, which means that sstate cache files with multiple
architectures are not considered as duplicate.
Run ``sstate-cache-management.sh`` for more details about this script.
poky/documentation/dev-manual/efficiently-fetching-sources.rst
+68
View File
@@ -0,0 +1,68 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Efficiently Fetching Source Files During a Build
************************************************
The OpenEmbedded build system works with source files located through
the :term:`SRC_URI` variable. When
you build something using BitBake, a big part of the operation is
locating and downloading all the source tarballs. For images,
downloading all the source for various packages can take a significant
amount of time.
This section shows you how you can use mirrors to speed up fetching
source files and how you can pre-fetch files all of which leads to more
efficient use of resources and time.
Setting up Effective Mirrors
============================
A good deal that goes into a Yocto Project build is simply downloading
all of the source tarballs. Maybe you have been working with another
build system for which you have built up a
sizable directory of source tarballs. Or, perhaps someone else has such
a directory for which you have read access. If so, you can save time by
adding statements to your configuration file so that the build process
checks local directories first for existing tarballs before checking the
Internet.
Here is an efficient way to set it up in your ``local.conf`` file::
SOURCE_MIRROR_URL ?= "file:///home/you/your-download-dir/"
INHERIT += "own-mirrors"
BB_GENERATE_MIRROR_TARBALLS = "1"
# BB_NO_NETWORK = "1"
In the previous example, the
:term:`BB_GENERATE_MIRROR_TARBALLS`
variable causes the OpenEmbedded build system to generate tarballs of
the Git repositories and store them in the
:term:`DL_DIR` directory. Due to
performance reasons, generating and storing these tarballs is not the
build system's default behavior.
You can also use the
:term:`PREMIRRORS` variable. For
an example, see the variable's glossary entry in the Yocto Project
Reference Manual.
Getting Source Files and Suppressing the Build
==============================================
Another technique you can use to ready yourself for a successive string
of build operations, is to pre-fetch all the source files without
actually starting a build. This technique lets you work through any
download issues and ultimately gathers all the source files into your
download directory :ref:`structure-build-downloads`,
which is located with :term:`DL_DIR`.
Use the following BitBake command form to fetch all the necessary
sources without starting the build::
$ bitbake target --runall=fetch
This
variation of the BitBake command guarantees that you have all the
sources for that BitBake target should you disconnect from the Internet
and want to do the build later offline.
poky/documentation/dev-manual/error-reporting-tool.rst
+84
View File
@@ -0,0 +1,84 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Using the Error Reporting Tool
******************************
The error reporting tool allows you to submit errors encountered during
builds to a central database. Outside of the build environment, you can
use a web interface to browse errors, view statistics, and query for
errors. The tool works using a client-server system where the client
portion is integrated with the installed Yocto Project
:term:`Source Directory` (e.g. ``poky``).
The server receives the information collected and saves it in a
database.
There is a live instance of the error reporting server at
https://errors.yoctoproject.org.
When you want to get help with build failures, you can submit all of the
information on the failure easily and then point to the URL in your bug
report or send an email to the mailing list.
.. note::
If you send error reports to this server, the reports become publicly
visible.
Enabling and Using the Tool
===========================
By default, the error reporting tool is disabled. You can enable it by
inheriting the :ref:`ref-classes-report-error` class by adding the
following statement to the end of your ``local.conf`` file in your
:term:`Build Directory`::
INHERIT += "report-error"
By default, the error reporting feature stores information in
``${``\ :term:`LOG_DIR`\ ``}/error-report``.
However, you can specify a directory to use by adding the following to
your ``local.conf`` file::
ERR_REPORT_DIR = "path"
Enabling error
reporting causes the build process to collect the errors and store them
in a file as previously described. When the build system encounters an
error, it includes a command as part of the console output. You can run
the command to send the error file to the server. For example, the
following command sends the errors to an upstream server::
$ send-error-report /home/brandusa/project/poky/build/tmp/log/error-report/error_report_201403141617.txt
In the previous example, the errors are sent to a public database
available at https://errors.yoctoproject.org, which is used by the
entire community. If you specify a particular server, you can send the
errors to a different database. Use the following command for more
information on available options::
$ send-error-report --help
When sending the error file, you are prompted to review the data being
sent as well as to provide a name and optional email address. Once you
satisfy these prompts, the command returns a link from the server that
corresponds to your entry in the database. For example, here is a
typical link: https://errors.yoctoproject.org/Errors/Details/9522/
Following the link takes you to a web interface where you can browse,
query the errors, and view statistics.
Disabling the Tool
==================
To disable the error reporting feature, simply remove or comment out the
following statement from the end of your ``local.conf`` file in your
:term:`Build Directory`::
INHERIT += "report-error"
Setting Up Your Own Error Reporting Server
==========================================
If you want to set up your own error reporting server, you can obtain
the code from the Git repository at :yocto_git:`/error-report-web/`.
Instructions on how to set it up are in the README document.
poky/documentation/dev-manual/external-scm.rst
+67
View File
@@ -0,0 +1,67 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Using an External SCM
*********************
If you're working on a recipe that pulls from an external Source Code
Manager (SCM), it is possible to have the OpenEmbedded build system
notice new recipe changes added to the SCM and then build the resulting
packages that depend on the new recipes by using the latest versions.
This only works for SCMs from which it is possible to get a sensible
revision number for changes. Currently, you can do this with Apache
Subversion (SVN), Git, and Bazaar (BZR) repositories.
To enable this behavior, the :term:`PV` of
the recipe needs to reference
:term:`SRCPV`. Here is an example::
PV = "1.2.3+git${SRCPV}"
Then, you can add the following to your
``local.conf``::
SRCREV:pn-PN = "${AUTOREV}"
:term:`PN` is the name of the recipe for
which you want to enable automatic source revision updating.
If you do not want to update your local configuration file, you can add
the following directly to the recipe to finish enabling the feature::
SRCREV = "${AUTOREV}"
The Yocto Project provides a distribution named ``poky-bleeding``, whose
configuration file contains the line::
require conf/distro/include/poky-floating-revisions.inc
This line pulls in the
listed include file that contains numerous lines of exactly that form::
#SRCREV:pn-opkg-native ?= "${AUTOREV}"
#SRCREV:pn-opkg-sdk ?= "${AUTOREV}"
#SRCREV:pn-opkg ?= "${AUTOREV}"
#SRCREV:pn-opkg-utils-native ?= "${AUTOREV}"
#SRCREV:pn-opkg-utils ?= "${AUTOREV}"
SRCREV:pn-gconf-dbus ?= "${AUTOREV}"
SRCREV:pn-matchbox-common ?= "${AUTOREV}"
SRCREV:pn-matchbox-config-gtk ?= "${AUTOREV}"
SRCREV:pn-matchbox-desktop ?= "${AUTOREV}"
SRCREV:pn-matchbox-keyboard ?= "${AUTOREV}"
SRCREV:pn-matchbox-panel-2 ?= "${AUTOREV}"
SRCREV:pn-matchbox-themes-extra ?= "${AUTOREV}"
SRCREV:pn-matchbox-terminal ?= "${AUTOREV}"
SRCREV:pn-matchbox-wm ?= "${AUTOREV}"
SRCREV:pn-settings-daemon ?= "${AUTOREV}"
SRCREV:pn-screenshot ?= "${AUTOREV}"
. . .
These lines allow you to
experiment with building a distribution that tracks the latest
development source for numerous packages.
.. note::
The ``poky-bleeding`` distribution is not tested on a regular basis. Keep
this in mind if you use it.
poky/documentation/dev-manual/external-toolchain.rst
+40
View File
@@ -0,0 +1,40 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Optionally Using an External Toolchain
**************************************
You might want to use an external toolchain as part of your development.
If this is the case, the fundamental steps you need to accomplish are as
follows:
- Understand where the installed toolchain resides. For cases where you
need to build the external toolchain, you would need to take separate
steps to build and install the toolchain.
- Make sure you add the layer that contains the toolchain to your
``bblayers.conf`` file through the
:term:`BBLAYERS` variable.
- Set the :term:`EXTERNAL_TOOLCHAIN` variable in your ``local.conf`` file
to the location in which you installed the toolchain.
The toolchain configuration is very flexible and customizable. It
is primarily controlled with the :term:`TCMODE` variable. This variable
controls which ``tcmode-*.inc`` file to include from the
``meta/conf/distro/include`` directory within the :term:`Source Directory`.
The default value of :term:`TCMODE` is "default", which tells the
OpenEmbedded build system to use its internally built toolchain (i.e.
``tcmode-default.inc``). However, other patterns are accepted. In
particular, "external-\*" refers to external toolchains. One example is
the Mentor Graphics Sourcery G++ Toolchain. Support for this toolchain resides
in the separate ``meta-sourcery`` layer at
https://github.com/MentorEmbedded/meta-sourcery/.
See its ``README`` file for details about how to use this layer.
Another example of external toolchain layer is
:yocto_git:`meta-arm-toolchain </meta-arm/tree/meta-arm-toolchain/>`
supporting GNU toolchains released by ARM.
You can find further information by reading about the :term:`TCMODE` variable
in the Yocto Project Reference Manual's variable glossary.
poky/documentation/dev-manual/figures/bitbake-build-flow.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 49 KiB

poky/documentation/dev-manual/figures/buildhistory-web.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 49 KiB

poky/documentation/dev-manual/figures/buildhistory.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 44 KiB

poky/documentation/dev-manual/figures/cute-files-npm-example.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 72 KiB

poky/documentation/dev-manual/figures/dev-title.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 16 KiB

poky/documentation/dev-manual/figures/multiconfig_files.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 18 KiB

poky/documentation/dev-manual/figures/recipe-workflow.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 47 KiB

poky/documentation/dev-manual/gobject-introspection.rst
+155
View File
@@ -0,0 +1,155 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Enabling GObject Introspection Support
**************************************
`GObject introspection <https://gi.readthedocs.io/en/latest/>`__
is the standard mechanism for accessing GObject-based software from
runtime environments. GObject is a feature of the GLib library that
provides an object framework for the GNOME desktop and related software.
GObject Introspection adds information to GObject that allows objects
created within it to be represented across different programming
languages. If you want to construct GStreamer pipelines using Python, or
control UPnP infrastructure using Javascript and GUPnP, GObject
introspection is the only way to do it.
This section describes the Yocto Project support for generating and
packaging GObject introspection data. GObject introspection data is a
description of the API provided by libraries built on top of the GLib
framework, and, in particular, that framework's GObject mechanism.
GObject Introspection Repository (GIR) files go to ``-dev`` packages,
``typelib`` files go to main packages as they are packaged together with
libraries that are introspected.
The data is generated when building such a library, by linking the
library with a small executable binary that asks the library to describe
itself, and then executing the binary and processing its output.
Generating this data in a cross-compilation environment is difficult
because the library is produced for the target architecture, but its
code needs to be executed on the build host. This problem is solved with
the OpenEmbedded build system by running the code through QEMU, which
allows precisely that. Unfortunately, QEMU does not always work
perfectly as mentioned in the ":ref:`dev-manual/gobject-introspection:known issues`"
section.
Enabling the Generation of Introspection Data
=============================================
Enabling the generation of introspection data (GIR files) in your
library package involves the following:
#. Inherit the :ref:`ref-classes-gobject-introspection` class.
#. Make sure introspection is not disabled anywhere in the recipe or
from anything the recipe includes. Also, make sure that
"gobject-introspection-data" is not in
:term:`DISTRO_FEATURES_BACKFILL_CONSIDERED`
and that "qemu-usermode" is not in
:term:`MACHINE_FEATURES_BACKFILL_CONSIDERED`.
In either of these conditions, nothing will happen.
#. Try to build the recipe. If you encounter build errors that look like
something is unable to find ``.so`` libraries, check where these
libraries are located in the source tree and add the following to the
recipe::
GIR_EXTRA_LIBS_PATH = "${B}/something/.libs"
.. note::
See recipes in the ``oe-core`` repository that use that
:term:`GIR_EXTRA_LIBS_PATH` variable as an example.
#. Look for any other errors, which probably mean that introspection
support in a package is not entirely standard, and thus breaks down
in a cross-compilation environment. For such cases, custom-made fixes
are needed. A good place to ask and receive help in these cases is
the :ref:`Yocto Project mailing
lists <resources-mailinglist>`.
.. note::
Using a library that no longer builds against the latest Yocto
Project release and prints introspection related errors is a good
candidate for the previous procedure.
Disabling the Generation of Introspection Data
==============================================
You might find that you do not want to generate introspection data. Or,
perhaps QEMU does not work on your build host and target architecture
combination. If so, you can use either of the following methods to
disable GIR file generations:
- Add the following to your distro configuration::
DISTRO_FEATURES_BACKFILL_CONSIDERED = "gobject-introspection-data"
Adding this statement disables generating introspection data using
QEMU but will still enable building introspection tools and libraries
(i.e. building them does not require the use of QEMU).
- Add the following to your machine configuration::
MACHINE_FEATURES_BACKFILL_CONSIDERED = "qemu-usermode"
Adding this statement disables the use of QEMU when building packages for your
machine. Currently, this feature is used only by introspection
recipes and has the same effect as the previously described option.
.. note::
Future releases of the Yocto Project might have other features
affected by this option.
If you disable introspection data, you can still obtain it through other
means such as copying the data from a suitable sysroot, or by generating
it on the target hardware. The OpenEmbedded build system does not
currently provide specific support for these techniques.
Testing that Introspection Works in an Image
============================================
Use the following procedure to test if generating introspection data is
working in an image:
#. Make sure that "gobject-introspection-data" is not in
:term:`DISTRO_FEATURES_BACKFILL_CONSIDERED`
and that "qemu-usermode" is not in
:term:`MACHINE_FEATURES_BACKFILL_CONSIDERED`.
#. Build ``core-image-sato``.
#. Launch a Terminal and then start Python in the terminal.
#. Enter the following in the terminal::
>>> from gi.repository import GLib
>>> GLib.get_host_name()
#. For something a little more advanced, enter the following see:
https://python-gtk-3-tutorial.readthedocs.io/en/latest/introduction.html
Known Issues
============
Here are know issues in GObject Introspection Support:
- ``qemu-ppc64`` immediately crashes. Consequently, you cannot build
introspection data on that architecture.
- x32 is not supported by QEMU. Consequently, introspection data is
disabled.
- musl causes transient GLib binaries to crash on assertion failures.
Consequently, generating introspection data is disabled.
- Because QEMU is not able to run the binaries correctly, introspection
is disabled for some specific packages under specific architectures
(e.g. ``gcr``, ``libsecret``, and ``webkit``).
- QEMU usermode might not work properly when running 64-bit binaries
under 32-bit host machines. In particular, "qemumips64" is known to
not work under i686.
poky/documentation/dev-manual/index.rst
+54
View File
@@ -0,0 +1,54 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
======================================
Yocto Project Development Tasks Manual
======================================
|
.. toctree::
:caption: Table of Contents
:numbered:
intro
start
layers
customizing-images
new-recipe
new-machine
upgrading-recipes
temporary-source-code
quilt.rst
development-shell
python-development-shell
building
speeding-up-build
libraries
prebuilt-libraries
x32-psabi
gobject-introspection
external-toolchain
wic
bmaptool
securing-images
custom-distribution
custom-template-configuration-directory
disk-space
packages
efficiently-fetching-sources
init-manager
device-manager
external-scm
read-only-rootfs
build-quality
runtime-testing
debugging
changes
licenses
vulnerabilities
sbom
error-reporting-tool
wayland
qemu
.. include:: /boilerplate.rst
poky/documentation/dev-manual/init-manager.rst
+162
View File
@@ -0,0 +1,162 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
.. _init-manager:
Selecting an Initialization Manager
***********************************
By default, the Yocto Project uses :wikipedia:`SysVinit <Init#SysV-style>` as
the initialization manager. There is also support for BusyBox init, a simpler
implementation, as well as support for :wikipedia:`systemd <Systemd>`, which
is a full replacement for init with parallel starting of services, reduced
shell overhead, increased security and resource limits for services, and other
features that are used by many distributions.
Within the system, SysVinit and BusyBox init treat system components as
services. These services are maintained as shell scripts stored in the
``/etc/init.d/`` directory.
SysVinit is more elaborate than BusyBox init and organizes services in
different run levels. This organization is maintained by putting links
to the services in the ``/etc/rcN.d/`` directories, where `N/` is one
of the following options: "S", "0", "1", "2", "3", "4", "5", or "6".
.. note::
Each runlevel has a dependency on the previous runlevel. This
dependency allows the services to work properly.
Both SysVinit and BusyBox init are configured through the ``/etc/inittab``
file, with a very similar syntax, though of course BusyBox init features
are more limited.
In comparison, systemd treats components as units. Using units is a
broader concept as compared to using a service. A unit includes several
different types of entities. ``Service`` is one of the types of entities.
The runlevel concept in SysVinit corresponds to the concept of a target
in systemd, where target is also a type of supported unit.
In systems with SysVinit or BusyBox init, services load sequentially (i.e. one
by one) during init and parallelization is not supported. With systemd, services
start in parallel. This method can have an impact on the startup performance
of a given service, though systemd will also provide more services by default,
therefore increasing the total system boot time. systemd also substantially
increases system size because of its multiple components and the extra
dependencies it pulls.
On the contrary, BusyBox init is the simplest and the lightest solution and
also comes with BusyBox mdev as device manager, a lighter replacement to
:wikipedia:`udev <Udev>`, which SysVinit and systemd both use.
The ":ref:`device-manager`" chapter has more details about device managers.
Using SysVinit with udev
=========================
SysVinit with the udev device manager corresponds to the
default setting in Poky. This corresponds to setting::
INIT_MANAGER = "sysvinit"
Using BusyBox init with BusyBox mdev
====================================
BusyBox init with BusyBox mdev is the simplest and lightest solution
for small root filesystems. All you need is BusyBox, which most systems
have anyway::
INIT_MANAGER = "mdev-busybox"
Using systemd
=============
The last option is to use systemd together with the udev device
manager. This is the most powerful and versatile solution, especially
for more complex systems::
INIT_MANAGER = "systemd"
This will enable systemd and remove sysvinit components from the image.
See :yocto_git:`meta/conf/distro/include/init-manager-systemd.inc
</poky/tree/meta/conf/distro/include/init-manager-systemd.inc>` for exact
details on what this does.
Controling systemd from the target command line
-----------------------------------------------
Here is a quick reference for controling systemd from the command line on the
target. Instead of opening and sometimes modifying files, most interaction
happens through the ``systemctl`` and ``journalctl`` commands:
- ``systemctl status``: show the status of all services
- ``systemctl status <service>``: show the status of one service
- ``systemctl [start|stop] <service>``: start or stop a service
- ``systemctl [enable|disable] <service>``: enable or disable a service at boot time
- ``systemctl list-units``: list all available units
- ``journalctl -a``: show all logs for all services
- ``journalctl -f``: show only the last log entries, and keep printing updates as they arrive
- ``journalctl -u``: show only logs from a particular service
Using systemd-journald without a traditional syslog daemon
----------------------------------------------------------
Counter-intuitively, ``systemd-journald`` is not a syslog runtime or provider,
and the proper way to use ``systemd-journald`` as your sole logging mechanism is to
effectively disable syslog entirely by setting these variables in your distribution
configuration file::
VIRTUAL-RUNTIME_syslog = ""
VIRTUAL-RUNTIME_base-utils-syslog = ""
Doing so will prevent ``rsyslog`` / ``busybox-syslog`` from being pulled in by
default, leaving only ``systemd-journald``.
Summary
-------
The Yocto Project supports three different initialization managers, offering
increasing levels of complexity and functionality:
.. list-table::
:widths: 40 20 20 20
:header-rows: 1
* -
- BusyBox init
- SysVinit
- systemd
* - Size
- Small
- Small
- Big [#footnote-systemd-size]_
* - Complexity
- Small
- Medium
- High
* - Support for boot profiles
- No
- Yes ("runlevels")
- Yes ("targets")
* - Services defined as
- Shell scripts
- Shell scripts
- Description files
* - Starting services in parallel
- No
- No
- Yes
* - Setting service resource limits
- No
- No
- Yes
* - Support service isolation
- No
- No
- Yes
* - Integrated logging
- No
- No
- Yes
.. [#footnote-systemd-size] Using systemd increases the ``core-image-minimal``
image size by 160\% for ``qemux86-64`` on Mickledore (4.2), compared to SysVinit.
poky/documentation/dev-manual/intro.rst
+59
View File
@@ -0,0 +1,59 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
******************************************
The Yocto Project Development Tasks Manual
******************************************
Welcome
=======
Welcome to the Yocto Project Development Tasks Manual. This manual
provides relevant procedures necessary for developing in the Yocto
Project environment (i.e. developing embedded Linux images and
user-space applications that run on targeted devices). This manual groups
related procedures into higher-level sections. Procedures can consist of
high-level steps or low-level steps depending on the topic.
This manual provides the following:
- Procedures that help you get going with the Yocto Project; for
example, procedures that show you how to set up a build host and work
with the Yocto Project source repositories.
- Procedures that show you how to submit changes to the Yocto Project.
Changes can be improvements, new features, or bug fixes.
- Procedures related to "everyday" tasks you perform while developing
images and applications using the Yocto Project, such as
creating a new layer, customizing an image, writing a new recipe,
and so forth.
This manual does not provide the following:
- Redundant step-by-step instructions: For example, the
:doc:`/sdk-manual/index` manual contains detailed
instructions on how to install an SDK, which is used to develop
applications for target hardware.
- Reference or conceptual material: This type of material resides in an
appropriate reference manual. As an example, system variables are
documented in the :doc:`/ref-manual/index`.
- Detailed public information not specific to the Yocto Project: For
example, exhaustive information on how to use the Git version
control system is better covered with Internet searches and official Git
documentation than through the Yocto Project documentation.
Other Information
=================
Because this manual presents information for many different topics,
supplemental information is recommended for full comprehension. For
introductory information on the Yocto Project, see the
:yocto_home:`Yocto Project Website <>`. If you want to build an image with no
knowledge of Yocto Project as a way of quickly testing it out, see the
:doc:`/brief-yoctoprojectqs/index` document.
For a comprehensive list of links and other documentation, see the
":ref:`ref-manual/resources:links and related documentation`"
section in the Yocto Project Reference Manual.
poky/documentation/dev-manual/layers.rst
+905
View File
@@ -0,0 +1,905 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Understanding and Creating Layers
*********************************
The OpenEmbedded build system supports organizing
:term:`Metadata` into multiple layers.
Layers allow you to isolate different types of customizations from each
other. For introductory information on the Yocto Project Layer Model,
see the
":ref:`overview-manual/yp-intro:the yocto project layer model`"
section in the Yocto Project Overview and Concepts Manual.
Creating Your Own Layer
=======================
.. note::
It is very easy to create your own layers to use with the OpenEmbedded
build system, as the Yocto Project ships with tools that speed up creating
layers. This section describes the steps you perform by hand to create
layers so that you can better understand them. For information about the
layer-creation tools, see the
":ref:`bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script`"
section in the Yocto Project Board Support Package (BSP) Developer's
Guide and the ":ref:`dev-manual/layers:creating a general layer using the \`\`bitbake-layers\`\` script`"
section further down in this manual.
Follow these general steps to create your layer without using tools:
#. *Check Existing Layers:* Before creating a new layer, you should be
sure someone has not already created a layer containing the Metadata
you need. You can see the :oe_layerindex:`OpenEmbedded Metadata Index <>`
for a list of layers from the OpenEmbedded community that can be used in
the Yocto Project. You could find a layer that is identical or close
to what you need.
#. *Create a Directory:* Create the directory for your layer. When you
create the layer, be sure to create the directory in an area not
associated with the Yocto Project :term:`Source Directory`
(e.g. the cloned ``poky`` repository).
While not strictly required, prepend the name of the directory with
the string "meta-". For example::
meta-mylayer
meta-GUI_xyz
meta-mymachine
With rare exceptions, a layer's name follows this form::
meta-root_name
Following this layer naming convention can save
you trouble later when tools, components, or variables "assume" your
layer name begins with "meta-". A notable example is in configuration
files as shown in the following step where layer names without the
"meta-" string are appended to several variables used in the
configuration.
#. *Create a Layer Configuration File:* Inside your new layer folder,
you need to create a ``conf/layer.conf`` file. It is easiest to take
an existing layer configuration file and copy that to your layer's
``conf`` directory and then modify the file as needed.
The ``meta-yocto-bsp/conf/layer.conf`` file in the Yocto Project
:yocto_git:`Source Repositories </poky/tree/meta-yocto-bsp/conf>`
demonstrates the required syntax. For your layer, you need to replace
"yoctobsp" with a unique identifier for your layer (e.g. "machinexyz"
for a layer named "meta-machinexyz")::
# We have a conf and classes directory, add to BBPATH
BBPATH .= ":${LAYERDIR}"
# We have recipes-* directories, add to BBFILES
BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
${LAYERDIR}/recipes-*/*/*.bbappend"
BBFILE_COLLECTIONS += "yoctobsp"
BBFILE_PATTERN_yoctobsp = "^${LAYERDIR}/"
BBFILE_PRIORITY_yoctobsp = "5"
LAYERVERSION_yoctobsp = "4"
LAYERSERIES_COMPAT_yoctobsp = "dunfell"
Following is an explanation of the layer configuration file:
- :term:`BBPATH`: Adds the layer's
root directory to BitBake's search path. Through the use of the
:term:`BBPATH` variable, BitBake locates class files (``.bbclass``),
configuration files, and files that are included with ``include``
and ``require`` statements. For these cases, BitBake uses the
first file that matches the name found in :term:`BBPATH`. This is
similar to the way the ``PATH`` variable is used for binaries. It
is recommended, therefore, that you use unique class and
configuration filenames in your custom layer.
- :term:`BBFILES`: Defines the
location for all recipes in the layer.
- :term:`BBFILE_COLLECTIONS`:
Establishes the current layer through a unique identifier that is
used throughout the OpenEmbedded build system to refer to the
layer. In this example, the identifier "yoctobsp" is the
representation for the container layer named "meta-yocto-bsp".
- :term:`BBFILE_PATTERN`:
Expands immediately during parsing to provide the directory of the
layer.
- :term:`BBFILE_PRIORITY`:
Establishes a priority to use for recipes in the layer when the
OpenEmbedded build finds recipes of the same name in different
layers.
- :term:`LAYERVERSION`:
Establishes a version number for the layer. You can use this
version number to specify this exact version of the layer as a
dependency when using the
:term:`LAYERDEPENDS`
variable.
- :term:`LAYERDEPENDS`:
Lists all layers on which this layer depends (if any).
- :term:`LAYERSERIES_COMPAT`:
Lists the :yocto_wiki:`Yocto Project </Releases>`
releases for which the current version is compatible. This
variable is a good way to indicate if your particular layer is
current.
#. *Add Content:* Depending on the type of layer, add the content. If
the layer adds support for a machine, add the machine configuration
in a ``conf/machine/`` file within the layer. If the layer adds
distro policy, add the distro configuration in a ``conf/distro/``
file within the layer. If the layer introduces new recipes, put the
recipes you need in ``recipes-*`` subdirectories within the layer.
.. note::
For an explanation of layer hierarchy that is compliant with the
Yocto Project, see the ":ref:`bsp-guide/bsp:example filesystem layout`"
section in the Yocto Project Board Support Package (BSP) Developer's Guide.
#. *Optionally Test for Compatibility:* If you want permission to use
the Yocto Project Compatibility logo with your layer or application
that uses your layer, perform the steps to apply for compatibility.
See the
":ref:`dev-manual/layers:making sure your layer is compatible with yocto project`"
section for more information.
Following Best Practices When Creating Layers
=============================================
To create layers that are easier to maintain and that will not impact
builds for other machines, you should consider the information in the
following list:
- *Avoid "Overlaying" Entire Recipes from Other Layers in Your
Configuration:* In other words, do not copy an entire recipe into
your layer and then modify it. Rather, use an append file
(``.bbappend``) to override only those parts of the original recipe
you need to modify.
- *Avoid Duplicating Include Files:* Use append files (``.bbappend``)
for each recipe that uses an include file. Or, if you are introducing
a new recipe that requires the included file, use the path relative
to the original layer directory to refer to the file. For example,
use ``require recipes-core/``\ `package`\ ``/``\ `file`\ ``.inc`` instead
of ``require`` `file`\ ``.inc``. If you're finding you have to overlay
the include file, it could indicate a deficiency in the include file
in the layer to which it originally belongs. If this is the case, you
should try to address that deficiency instead of overlaying the
include file. For example, you could address this by getting the
maintainer of the include file to add a variable or variables to make
it easy to override the parts needing to be overridden.
- *Structure Your Layers:* Proper use of overrides within append files
and placement of machine-specific files within your layer can ensure
that a build is not using the wrong Metadata and negatively impacting
a build for a different machine. Following are some examples:
- *Modify Variables to Support a Different Machine:* Suppose you
have a layer named ``meta-one`` that adds support for building
machine "one". To do so, you use an append file named
``base-files.bbappend`` and create a dependency on "foo" by
altering the :term:`DEPENDS`
variable::
DEPENDS = "foo"
The dependency is created during any
build that includes the layer ``meta-one``. However, you might not
want this dependency for all machines. For example, suppose you
are building for machine "two" but your ``bblayers.conf`` file has
the ``meta-one`` layer included. During the build, the
``base-files`` for machine "two" will also have the dependency on
``foo``.
To make sure your changes apply only when building machine "one",
use a machine override with the :term:`DEPENDS` statement::
DEPENDS:one = "foo"
You should follow the same strategy when using ``:append``
and ``:prepend`` operations::
DEPENDS:append:one = " foo"
DEPENDS:prepend:one = "foo "
As an actual example, here's a
snippet from the generic kernel include file ``linux-yocto.inc``,
wherein the kernel compile and link options are adjusted in the
case of a subset of the supported architectures::
DEPENDS:append:aarch64 = " libgcc"
KERNEL_CC:append:aarch64 = " ${TOOLCHAIN_OPTIONS}"
KERNEL_LD:append:aarch64 = " ${TOOLCHAIN_OPTIONS}"
DEPENDS:append:nios2 = " libgcc"
KERNEL_CC:append:nios2 = " ${TOOLCHAIN_OPTIONS}"
KERNEL_LD:append:nios2 = " ${TOOLCHAIN_OPTIONS}"
DEPENDS:append:arc = " libgcc"
KERNEL_CC:append:arc = " ${TOOLCHAIN_OPTIONS}"
KERNEL_LD:append:arc = " ${TOOLCHAIN_OPTIONS}"
KERNEL_FEATURES:append:qemuall=" features/debug/printk.scc"
- *Place Machine-Specific Files in Machine-Specific Locations:* When
you have a base recipe, such as ``base-files.bb``, that contains a
:term:`SRC_URI` statement to a
file, you can use an append file to cause the build to use your
own version of the file. For example, an append file in your layer
at ``meta-one/recipes-core/base-files/base-files.bbappend`` could
extend :term:`FILESPATH` using :term:`FILESEXTRAPATHS` as follows::
FILESEXTRAPATHS:prepend := "${THISDIR}/${BPN}:"
The build for machine "one" will pick up your machine-specific file as
long as you have the file in
``meta-one/recipes-core/base-files/base-files/``. However, if you
are building for a different machine and the ``bblayers.conf``
file includes the ``meta-one`` layer and the location of your
machine-specific file is the first location where that file is
found according to :term:`FILESPATH`, builds for all machines will
also use that machine-specific file.
You can make sure that a machine-specific file is used for a
particular machine by putting the file in a subdirectory specific
to the machine. For example, rather than placing the file in
``meta-one/recipes-core/base-files/base-files/`` as shown above,
put it in ``meta-one/recipes-core/base-files/base-files/one/``.
Not only does this make sure the file is used only when building
for machine "one", but the build process locates the file more
quickly.
In summary, you need to place all files referenced from
:term:`SRC_URI` in a machine-specific subdirectory within the layer in
order to restrict those files to machine-specific builds.
- *Perform Steps to Apply for Yocto Project Compatibility:* If you want
permission to use the Yocto Project Compatibility logo with your
layer or application that uses your layer, perform the steps to apply
for compatibility. See the
":ref:`dev-manual/layers:making sure your layer is compatible with yocto project`"
section for more information.
- *Follow the Layer Naming Convention:* Store custom layers in a Git
repository that use the ``meta-layer_name`` format.
- *Group Your Layers Locally:* Clone your repository alongside other
cloned ``meta`` directories from the :term:`Source Directory`.
Making Sure Your Layer is Compatible With Yocto Project
=======================================================
When you create a layer used with the Yocto Project, it is advantageous
to make sure that the layer interacts well with existing Yocto Project
layers (i.e. the layer is compatible with the Yocto Project). Ensuring
compatibility makes the layer easy to be consumed by others in the Yocto
Project community and could allow you permission to use the Yocto
Project Compatible Logo.
.. note::
Only Yocto Project member organizations are permitted to use the
Yocto Project Compatible Logo. The logo is not available for general
use. For information on how to become a Yocto Project member
organization, see the :yocto_home:`Yocto Project Website <>`.
The Yocto Project Compatibility Program consists of a layer application
process that requests permission to use the Yocto Project Compatibility
Logo for your layer and application. The process consists of two parts:
#. Successfully passing a script (``yocto-check-layer``) that when run
against your layer, tests it against constraints based on experiences
of how layers have worked in the real world and where pitfalls have
been found. Getting a "PASS" result from the script is required for
successful compatibility registration.
#. Completion of an application acceptance form, which you can find at
:yocto_home:`/webform/yocto-project-compatible-registration`.
To be granted permission to use the logo, you need to satisfy the
following:
- Be able to check the box indicating that you got a "PASS" when
running the script against your layer.
- Answer "Yes" to the questions on the form or have an acceptable
explanation for any questions answered "No".
- Be a Yocto Project Member Organization.
The remainder of this section presents information on the registration
form and on the ``yocto-check-layer`` script.
Yocto Project Compatible Program Application
--------------------------------------------
Use the form to apply for your layer's approval. Upon successful
application, you can use the Yocto Project Compatibility Logo with your
layer and the application that uses your layer.
To access the form, use this link:
:yocto_home:`/webform/yocto-project-compatible-registration`.
Follow the instructions on the form to complete your application.
The application consists of the following sections:
- *Contact Information:* Provide your contact information as the fields
require. Along with your information, provide the released versions
of the Yocto Project for which your layer is compatible.
- *Acceptance Criteria:* Provide "Yes" or "No" answers for each of the
items in the checklist. There is space at the bottom of the form for
any explanations for items for which you answered "No".
- *Recommendations:* Provide answers for the questions regarding Linux
kernel use and build success.
``yocto-check-layer`` Script
----------------------------
The ``yocto-check-layer`` script provides you a way to assess how
compatible your layer is with the Yocto Project. You should run this
script prior to using the form to apply for compatibility as described
in the previous section. You need to achieve a "PASS" result in order to
have your application form successfully processed.
The script divides tests into three areas: COMMON, BSP, and DISTRO. For
example, given a distribution layer (DISTRO), the layer must pass both
the COMMON and DISTRO related tests. Furthermore, if your layer is a BSP
layer, the layer must pass the COMMON and BSP set of tests.
To execute the script, enter the following commands from your build
directory::
$ source oe-init-build-env
$ yocto-check-layer your_layer_directory
Be sure to provide the actual directory for your
layer as part of the command.
Entering the command causes the script to determine the type of layer
and then to execute a set of specific tests against the layer. The
following list overviews the test:
- ``common.test_readme``: Tests if a ``README`` file exists in the
layer and the file is not empty.
- ``common.test_parse``: Tests to make sure that BitBake can parse the
files without error (i.e. ``bitbake -p``).
- ``common.test_show_environment``: Tests that the global or per-recipe
environment is in order without errors (i.e. ``bitbake -e``).
- ``common.test_world``: Verifies that ``bitbake world`` works.
- ``common.test_signatures``: Tests to be sure that BSP and DISTRO
layers do not come with recipes that change signatures.
- ``common.test_layerseries_compat``: Verifies layer compatibility is
set properly.
- ``bsp.test_bsp_defines_machines``: Tests if a BSP layer has machine
configurations.
- ``bsp.test_bsp_no_set_machine``: Tests to ensure a BSP layer does not
set the machine when the layer is added.
- ``bsp.test_machine_world``: Verifies that ``bitbake world`` works
regardless of which machine is selected.
- ``bsp.test_machine_signatures``: Verifies that building for a
particular machine affects only the signature of tasks specific to
that machine.
- ``distro.test_distro_defines_distros``: Tests if a DISTRO layer has
distro configurations.
- ``distro.test_distro_no_set_distros``: Tests to ensure a DISTRO layer
does not set the distribution when the layer is added.
Enabling Your Layer
===================
Before the OpenEmbedded build system can use your new layer, you need to
enable it. To enable your layer, simply add your layer's path to the
:term:`BBLAYERS` variable in your ``conf/bblayers.conf`` file, which is
found in the :term:`Build Directory`. The following example shows how to
enable your new ``meta-mylayer`` layer (note how your new layer exists
outside of the official ``poky`` repository which you would have checked
out earlier)::
# POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
POKY_BBLAYERS_CONF_VERSION = "2"
BBPATH = "${TOPDIR}"
BBFILES ?= ""
BBLAYERS ?= " \
/home/user/poky/meta \
/home/user/poky/meta-poky \
/home/user/poky/meta-yocto-bsp \
/home/user/mystuff/meta-mylayer \
"
BitBake parses each ``conf/layer.conf`` file from the top down as
specified in the :term:`BBLAYERS` variable within the ``conf/bblayers.conf``
file. During the processing of each ``conf/layer.conf`` file, BitBake
adds the recipes, classes and configurations contained within the
particular layer to the source directory.
Appending Other Layers Metadata With Your Layer
===============================================
A recipe that appends Metadata to another recipe is called a BitBake
append file. A BitBake append file uses the ``.bbappend`` file type
suffix, while the corresponding recipe to which Metadata is being
appended uses the ``.bb`` file type suffix.
You can use a ``.bbappend`` file in your layer to make additions or
changes to the content of another layer's recipe without having to copy
the other layer's recipe into your layer. Your ``.bbappend`` file
resides in your layer, while the main ``.bb`` recipe file to which you
are appending Metadata resides in a different layer.
Being able to append information to an existing recipe not only avoids
duplication, but also automatically applies recipe changes from a
different layer into your layer. If you were copying recipes, you would
have to manually merge changes as they occur.
When you create an append file, you must use the same root name as the
corresponding recipe file. For example, the append file
``someapp_3.1.bbappend`` must apply to ``someapp_3.1.bb``. This
means the original recipe and append filenames are version
number-specific. If the corresponding recipe is renamed to update to a
newer version, you must also rename and possibly update the
corresponding ``.bbappend`` as well. During the build process, BitBake
displays an error on starting if it detects a ``.bbappend`` file that
does not have a corresponding recipe with a matching name. See the
:term:`BB_DANGLINGAPPENDS_WARNONLY`
variable for information on how to handle this error.
Overlaying a File Using Your Layer
----------------------------------
As an example, consider the main formfactor recipe and a corresponding
formfactor append file both from the :term:`Source Directory`.
Here is the main
formfactor recipe, which is named ``formfactor_0.0.bb`` and located in
the "meta" layer at ``meta/recipes-bsp/formfactor``::
SUMMARY = "Device formfactor information"
DESCRIPTION = "A formfactor configuration file provides information about the \
target hardware for which the image is being built and information that the \
build system cannot obtain from other sources such as the kernel."
SECTION = "base"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
PR = "r45"
SRC_URI = "file://config file://machconfig"
S = "${WORKDIR}"
PACKAGE_ARCH = "${MACHINE_ARCH}"
INHIBIT_DEFAULT_DEPS = "1"
do_install() {
# Install file only if it has contents
install -d ${D}${sysconfdir}/formfactor/
install -m 0644 ${S}/config ${D}${sysconfdir}/formfactor/
if [ -s "${S}/machconfig" ]; then
install -m 0644 ${S}/machconfig ${D}${sysconfdir}/formfactor/
fi
}
In the main recipe, note the :term:`SRC_URI`
variable, which tells the OpenEmbedded build system where to find files
during the build.
Following is the append file, which is named ``formfactor_0.0.bbappend``
and is from the Raspberry Pi BSP Layer named ``meta-raspberrypi``. The
file is in the layer at ``recipes-bsp/formfactor``::
FILESEXTRAPATHS:prepend := "${THISDIR}/${PN}:"
By default, the build system uses the
:term:`FILESPATH` variable to
locate files. This append file extends the locations by setting the
:term:`FILESEXTRAPATHS`
variable. Setting this variable in the ``.bbappend`` file is the most
reliable and recommended method for adding directories to the search
path used by the build system to find files.
The statement in this example extends the directories to include
``${``\ :term:`THISDIR`\ ``}/${``\ :term:`PN`\ ``}``,
which resolves to a directory named ``formfactor`` in the same directory
in which the append file resides (i.e.
``meta-raspberrypi/recipes-bsp/formfactor``. This implies that you must
have the supporting directory structure set up that will contain any
files or patches you will be including from the layer.
Using the immediate expansion assignment operator ``:=`` is important
because of the reference to :term:`THISDIR`. The trailing colon character is
important as it ensures that items in the list remain colon-separated.
.. note::
BitBake automatically defines the :term:`THISDIR` variable. You should
never set this variable yourself. Using ":prepend" as part of the
:term:`FILESEXTRAPATHS` ensures your path will be searched prior to other
paths in the final list.
Also, not all append files add extra files. Many append files simply
allow to add build options (e.g. ``systemd``). For these cases, your
append file would not even use the :term:`FILESEXTRAPATHS` statement.
The end result of this ``.bbappend`` file is that on a Raspberry Pi, where
``rpi`` will exist in the list of :term:`OVERRIDES`, the file
``meta-raspberrypi/recipes-bsp/formfactor/formfactor/rpi/machconfig`` will be
used during :ref:`ref-tasks-fetch` and the test for a non-zero file size in
:ref:`ref-tasks-install` will return true, and the file will be installed.
Installing Additional Files Using Your Layer
--------------------------------------------
As another example, consider the main ``xserver-xf86-config`` recipe and a
corresponding ``xserver-xf86-config`` append file both from the :term:`Source
Directory`. Here is the main ``xserver-xf86-config`` recipe, which is named
``xserver-xf86-config_0.1.bb`` and located in the "meta" layer at
``meta/recipes-graphics/xorg-xserver``::
SUMMARY = "X.Org X server configuration file"
HOMEPAGE = "http://www.x.org"
SECTION = "x11/base"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
PR = "r33"
SRC_URI = "file://xorg.conf"
S = "${WORKDIR}"
CONFFILES:${PN} = "${sysconfdir}/X11/xorg.conf"
PACKAGE_ARCH = "${MACHINE_ARCH}"
ALLOW_EMPTY:${PN} = "1"
do_install () {
if test -s ${WORKDIR}/xorg.conf; then
install -d ${D}/${sysconfdir}/X11
install -m 0644 ${WORKDIR}/xorg.conf ${D}/${sysconfdir}/X11/
fi
}
Following is the append file, which is named ``xserver-xf86-config_%.bbappend``
and is from the Raspberry Pi BSP Layer named ``meta-raspberrypi``. The
file is in the layer at ``recipes-graphics/xorg-xserver``::
FILESEXTRAPATHS:prepend := "${THISDIR}/${PN}:"
SRC_URI:append:rpi = " \
file://xorg.conf.d/98-pitft.conf \
file://xorg.conf.d/99-calibration.conf \
"
do_install:append:rpi () {
PITFT="${@bb.utils.contains("MACHINE_FEATURES", "pitft", "1", "0", d)}"
if [ "${PITFT}" = "1" ]; then
install -d ${D}/${sysconfdir}/X11/xorg.conf.d/
install -m 0644 ${WORKDIR}/xorg.conf.d/98-pitft.conf ${D}/${sysconfdir}/X11/xorg.conf.d/
install -m 0644 ${WORKDIR}/xorg.conf.d/99-calibration.conf ${D}/${sysconfdir}/X11/xorg.conf.d/
fi
}
FILES:${PN}:append:rpi = " ${sysconfdir}/X11/xorg.conf.d/*"
Building off of the previous example, we once again are setting the
:term:`FILESEXTRAPATHS` variable. In this case we are also using
:term:`SRC_URI` to list additional source files to use when ``rpi`` is found in
the list of :term:`OVERRIDES`. The :ref:`ref-tasks-install` task will then perform a
check for an additional :term:`MACHINE_FEATURES` that if set will cause these
additional files to be installed. These additional files are listed in
:term:`FILES` so that they will be packaged.
Prioritizing Your Layer
=======================
Each layer is assigned a priority value. Priority values control which
layer takes precedence if there are recipe files with the same name in
multiple layers. For these cases, the recipe file from the layer with a
higher priority number takes precedence. Priority values also affect the
order in which multiple ``.bbappend`` files for the same recipe are
applied. You can either specify the priority manually, or allow the
build system to calculate it based on the layer's dependencies.
To specify the layer's priority manually, use the
:term:`BBFILE_PRIORITY`
variable and append the layer's root name::
BBFILE_PRIORITY_mylayer = "1"
.. note::
It is possible for a recipe with a lower version number
:term:`PV` in a layer that has a higher
priority to take precedence.
Also, the layer priority does not currently affect the precedence
order of ``.conf`` or ``.bbclass`` files. Future versions of BitBake
might address this.
Managing Layers
===============
You can use the BitBake layer management tool ``bitbake-layers`` to
provide a view into the structure of recipes across a multi-layer
project. Being able to generate output that reports on configured layers
with their paths and priorities and on ``.bbappend`` files and their
applicable recipes can help to reveal potential problems.
For help on the BitBake layer management tool, use the following
command::
$ bitbake-layers --help
The following list describes the available commands:
- ``help:`` Displays general help or help on a specified command.
- ``show-layers:`` Shows the current configured layers.
- ``show-overlayed:`` Lists overlayed recipes. A recipe is overlayed
when a recipe with the same name exists in another layer that has a
higher layer priority.
- ``show-recipes:`` Lists available recipes and the layers that
provide them.
- ``show-appends:`` Lists ``.bbappend`` files and the recipe files to
which they apply.
- ``show-cross-depends:`` Lists dependency relationships between
recipes that cross layer boundaries.
- ``add-layer:`` Adds a layer to ``bblayers.conf``.
- ``remove-layer:`` Removes a layer from ``bblayers.conf``
- ``flatten:`` Flattens the layer configuration into a separate
output directory. Flattening your layer configuration builds a
"flattened" directory that contains the contents of all layers, with
any overlayed recipes removed and any ``.bbappend`` files appended to
the corresponding recipes. You might have to perform some manual
cleanup of the flattened layer as follows:
- Non-recipe files (such as patches) are overwritten. The flatten
command shows a warning for these files.
- Anything beyond the normal layer setup has been added to the
``layer.conf`` file. Only the lowest priority layer's
``layer.conf`` is used.
- Overridden and appended items from ``.bbappend`` files need to be
cleaned up. The contents of each ``.bbappend`` end up in the
flattened recipe. However, if there are appended or changed
variable values, you need to tidy these up yourself. Consider the
following example. Here, the ``bitbake-layers`` command adds the
line ``#### bbappended ...`` so that you know where the following
lines originate::
...
DESCRIPTION = "A useful utility"
...
EXTRA_OECONF = "--enable-something"
...
#### bbappended from meta-anotherlayer ####
DESCRIPTION = "Customized utility"
EXTRA_OECONF += "--enable-somethingelse"
Ideally, you would tidy up these utilities as follows::
...
DESCRIPTION = "Customized utility"
...
EXTRA_OECONF = "--enable-something --enable-somethingelse"
...
- ``layerindex-fetch``: Fetches a layer from a layer index, along
with its dependent layers, and adds the layers to the
``conf/bblayers.conf`` file.
- ``layerindex-show-depends``: Finds layer dependencies from the
layer index.
- ``save-build-conf``: Saves the currently active build configuration
(``conf/local.conf``, ``conf/bblayers.conf``) as a template into a layer.
This template can later be used for setting up builds via :term:``TEMPLATECONF``.
For information about saving and using configuration templates, see
":ref:`dev-manual/custom-template-configuration-directory:creating a custom template configuration directory`".
- ``create-layer``: Creates a basic layer.
- ``create-layers-setup``: Writes out a configuration file and/or a script that
can replicate the directory structure and revisions of the layers in a current build.
For more information, see ":ref:`dev-manual/layers:saving and restoring the layers setup`".
Creating a General Layer Using the ``bitbake-layers`` Script
============================================================
The ``bitbake-layers`` script with the ``create-layer`` subcommand
simplifies creating a new general layer.
.. note::
- For information on BSP layers, see the ":ref:`bsp-guide/bsp:bsp layers`"
section in the Yocto
Project Board Specific (BSP) Developer's Guide.
- In order to use a layer with the OpenEmbedded build system, you
need to add the layer to your ``bblayers.conf`` configuration
file. See the ":ref:`dev-manual/layers:adding a layer using the \`\`bitbake-layers\`\` script`"
section for more information.
The default mode of the script's operation with this subcommand is to
create a layer with the following:
- A layer priority of 6.
- A ``conf`` subdirectory that contains a ``layer.conf`` file.
- A ``recipes-example`` subdirectory that contains a further
subdirectory named ``example``, which contains an ``example.bb``
recipe file.
- A ``COPYING.MIT``, which is the license statement for the layer. The
script assumes you want to use the MIT license, which is typical for
most layers, for the contents of the layer itself.
- A ``README`` file, which is a file describing the contents of your
new layer.
In its simplest form, you can use the following command form to create a
layer. The command creates a layer whose name corresponds to
"your_layer_name" in the current directory::
$ bitbake-layers create-layer your_layer_name
As an example, the following command creates a layer named ``meta-scottrif``
in your home directory::
$ cd /usr/home
$ bitbake-layers create-layer meta-scottrif
NOTE: Starting bitbake server...
Add your new layer with 'bitbake-layers add-layer meta-scottrif'
If you want to set the priority of the layer to other than the default
value of "6", you can either use the ``--priority`` option or you
can edit the
:term:`BBFILE_PRIORITY` value
in the ``conf/layer.conf`` after the script creates it. Furthermore, if
you want to give the example recipe file some name other than the
default, you can use the ``--example-recipe-name`` option.
The easiest way to see how the ``bitbake-layers create-layer`` command
works is to experiment with the script. You can also read the usage
information by entering the following::
$ bitbake-layers create-layer --help
NOTE: Starting bitbake server...
usage: bitbake-layers create-layer [-h] [--priority PRIORITY]
[--example-recipe-name EXAMPLERECIPE]
layerdir
Create a basic layer
positional arguments:
layerdir Layer directory to create
optional arguments:
-h, --help show this help message and exit
--priority PRIORITY, -p PRIORITY
Layer directory to create
--example-recipe-name EXAMPLERECIPE, -e EXAMPLERECIPE
Filename of the example recipe
Adding a Layer Using the ``bitbake-layers`` Script
==================================================
Once you create your general layer, you must add it to your
``bblayers.conf`` file. Adding the layer to this configuration file
makes the OpenEmbedded build system aware of your layer so that it can
search it for metadata.
Add your layer by using the ``bitbake-layers add-layer`` command::
$ bitbake-layers add-layer your_layer_name
Here is an example that adds a
layer named ``meta-scottrif`` to the configuration file. Following the
command that adds the layer is another ``bitbake-layers`` command that
shows the layers that are in your ``bblayers.conf`` file::
$ bitbake-layers add-layer meta-scottrif
NOTE: Starting bitbake server...
Parsing recipes: 100% |##########################################################| Time: 0:00:49
Parsing of 1441 .bb files complete (0 cached, 1441 parsed). 2055 targets, 56 skipped, 0 masked, 0 errors.
$ bitbake-layers show-layers
NOTE: Starting bitbake server...
layer path priority
==========================================================================
meta /home/scottrif/poky/meta 5
meta-poky /home/scottrif/poky/meta-poky 5
meta-yocto-bsp /home/scottrif/poky/meta-yocto-bsp 5
workspace /home/scottrif/poky/build/workspace 99
meta-scottrif /home/scottrif/poky/build/meta-scottrif 6
Adding the layer to this file
enables the build system to locate the layer during the build.
.. note::
During a build, the OpenEmbedded build system looks in the layers
from the top of the list down to the bottom in that order.
Saving and restoring the layers setup
=====================================
Once you have a working build with the correct set of layers, it is beneficial
to capture the layer setup --- what they are, which repositories they come from
and which SCM revisions they're at --- into a configuration file, so that this
setup can be easily replicated later, perhaps on a different machine. Here's
how to do this::
$ bitbake-layers create-layers-setup /srv/work/alex/meta-alex/
NOTE: Starting bitbake server...
NOTE: Created /srv/work/alex/meta-alex/setup-layers.json
NOTE: Created /srv/work/alex/meta-alex/setup-layers
The tool needs a single argument which tells where to place the output, consisting
of a json formatted layer configuration, and a ``setup-layers`` script that can use that configuration
to restore the layers in a different location, or on a different host machine. The argument
can point to a custom layer (which is then deemed a "bootstrap" layer that needs to be
checked out first), or into a completely independent location.
The replication of the layers is performed by running the ``setup-layers`` script provided
above:
#. Clone the bootstrap layer or some other repository to obtain
the json config and the setup script that can use it.
#. Run the script directly with no options::
alex@Zen2:/srv/work/alex/my-build$ meta-alex/setup-layers
Note: not checking out source meta-alex, use --force-bootstraplayer-checkout to override.
Setting up source meta-intel, revision 15.0-hardknott-3.3-310-g0a96edae, branch master
Running 'git init -q /srv/work/alex/my-build/meta-intel'
Running 'git remote remove origin > /dev/null 2>&1; git remote add origin git://git.yoctoproject.org/meta-intel' in /srv/work/alex/my-build/meta-intel
Running 'git fetch -q origin || true' in /srv/work/alex/my-build/meta-intel
Running 'git checkout -q 0a96edae609a3f48befac36af82cf1eed6786b4a' in /srv/work/alex/my-build/meta-intel
Setting up source poky, revision 4.1_M1-372-g55483d28f2, branch akanavin/setup-layers
Running 'git init -q /srv/work/alex/my-build/poky'
Running 'git remote remove origin > /dev/null 2>&1; git remote add origin git://git.yoctoproject.org/poky' in /srv/work/alex/my-build/poky
Running 'git fetch -q origin || true' in /srv/work/alex/my-build/poky
Running 'git remote remove poky-contrib > /dev/null 2>&1; git remote add poky-contrib ssh://git@push.yoctoproject.org/poky-contrib' in /srv/work/alex/my-build/poky
Running 'git fetch -q poky-contrib || true' in /srv/work/alex/my-build/poky
Running 'git checkout -q 11db0390b02acac1324e0f827beb0e2e3d0d1d63' in /srv/work/alex/my-build/poky
.. note::
This will work to update an existing checkout as well.
.. note::
The script is self-sufficient and requires only python3
and git on the build machine.
.. note::
Both the ``create-layers-setup`` and the ``setup-layers`` provided several additional options
that customize their behavior - you are welcome to study them via ``--help`` command line parameter.
poky/documentation/dev-manual/libraries.rst
+267
View File
@@ -0,0 +1,267 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Working With Libraries
**********************
Libraries are an integral part of your system. This section describes
some common practices you might find helpful when working with libraries
to build your system:
- :ref:`How to include static library files
<dev-manual/libraries:including static library files>`
- :ref:`How to use the Multilib feature to combine multiple versions of
library files into a single image
<dev-manual/libraries:combining multiple versions of library files into one image>`
- :ref:`How to install multiple versions of the same library in parallel on
the same system
<dev-manual/libraries:installing multiple versions of the same library>`
Including Static Library Files
==============================
If you are building a library and the library offers static linking, you
can control which static library files (``*.a`` files) get included in
the built library.
The :term:`PACKAGES` and
:term:`FILES:* <FILES>` variables in the
``meta/conf/bitbake.conf`` configuration file define how files installed
by the :ref:`ref-tasks-install` task are packaged. By default, the :term:`PACKAGES`
variable includes ``${PN}-staticdev``, which represents all static
library files.
.. note::
Some previously released versions of the Yocto Project defined the
static library files through ``${PN}-dev``.
Following is part of the BitBake configuration file, where you can see
how the static library files are defined::
PACKAGE_BEFORE_PN ?= ""
PACKAGES = "${PN}-src ${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale ${PACKAGE_BEFORE_PN} ${PN}"
PACKAGES_DYNAMIC = "^${PN}-locale-.*"
FILES = ""
FILES:${PN} = "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} \
${sysconfdir} ${sharedstatedir} ${localstatedir} \
${base_bindir}/* ${base_sbindir}/* \
${base_libdir}/*${SOLIBS} \
${base_prefix}/lib/udev ${prefix}/lib/udev \
${base_libdir}/udev ${libdir}/udev \
${datadir}/${BPN} ${libdir}/${BPN}/* \
${datadir}/pixmaps ${datadir}/applications \
${datadir}/idl ${datadir}/omf ${datadir}/sounds \
${libdir}/bonobo/servers"
FILES:${PN}-bin = "${bindir}/* ${sbindir}/*"
FILES:${PN}-doc = "${docdir} ${mandir} ${infodir} ${datadir}/gtk-doc \
${datadir}/gnome/help"
SECTION:${PN}-doc = "doc"
FILES_SOLIBSDEV ?= "${base_libdir}/lib*${SOLIBSDEV} ${libdir}/lib*${SOLIBSDEV}"
FILES:${PN}-dev = "${includedir} ${FILES_SOLIBSDEV} ${libdir}/*.la \
${libdir}/*.o ${libdir}/pkgconfig ${datadir}/pkgconfig \
${datadir}/aclocal ${base_libdir}/*.o \
${libdir}/${BPN}/*.la ${base_libdir}/*.la \
${libdir}/cmake ${datadir}/cmake"
SECTION:${PN}-dev = "devel"
ALLOW_EMPTY:${PN}-dev = "1"
RDEPENDS:${PN}-dev = "${PN} (= ${EXTENDPKGV})"
FILES:${PN}-staticdev = "${libdir}/*.a ${base_libdir}/*.a ${libdir}/${BPN}/*.a"
SECTION:${PN}-staticdev = "devel"
RDEPENDS:${PN}-staticdev = "${PN}-dev (= ${EXTENDPKGV})"
Combining Multiple Versions of Library Files into One Image
===========================================================
The build system offers the ability to build libraries with different
target optimizations or architecture formats and combine these together
into one system image. You can link different binaries in the image
against the different libraries as needed for specific use cases. This
feature is called "Multilib".
An example would be where you have most of a system compiled in 32-bit
mode using 32-bit libraries, but you have something large, like a
database engine, that needs to be a 64-bit application and uses 64-bit
libraries. Multilib allows you to get the best of both 32-bit and 64-bit
libraries.
While the Multilib feature is most commonly used for 32 and 64-bit
differences, the approach the build system uses facilitates different
target optimizations. You could compile some binaries to use one set of
libraries and other binaries to use a different set of libraries. The
libraries could differ in architecture, compiler options, or other
optimizations.
There are several examples in the ``meta-skeleton`` layer found in the
:term:`Source Directory`:
- :oe_git:`conf/multilib-example.conf </openembedded-core/tree/meta-skeleton/conf/multilib-example.conf>`
configuration file.
- :oe_git:`conf/multilib-example2.conf </openembedded-core/tree/meta-skeleton/conf/multilib-example2.conf>`
configuration file.
- :oe_git:`recipes-multilib/images/core-image-multilib-example.bb </openembedded-core/tree/meta-skeleton/recipes-multilib/images/core-image-multilib-example.bb>`
recipe
Preparing to Use Multilib
-------------------------
User-specific requirements drive the Multilib feature. Consequently,
there is no one "out-of-the-box" configuration that would
meet your needs.
In order to enable Multilib, you first need to ensure your recipe is
extended to support multiple libraries. Many standard recipes are
already extended and support multiple libraries. You can check in the
``meta/conf/multilib.conf`` configuration file in the
:term:`Source Directory` to see how this is
done using the
:term:`BBCLASSEXTEND` variable.
Eventually, all recipes will be covered and this list will not be
needed.
For the most part, the :ref:`Multilib <ref-classes-multilib*>`
class extension works automatically to
extend the package name from ``${PN}`` to ``${MLPREFIX}${PN}``, where
:term:`MLPREFIX` is the particular multilib (e.g. "lib32-" or "lib64-").
Standard variables such as
:term:`DEPENDS`,
:term:`RDEPENDS`,
:term:`RPROVIDES`,
:term:`RRECOMMENDS`,
:term:`PACKAGES`, and
:term:`PACKAGES_DYNAMIC` are
automatically extended by the system. If you are extending any manual
code in the recipe, you can use the ``${MLPREFIX}`` variable to ensure
those names are extended correctly.
Using Multilib
--------------
After you have set up the recipes, you need to define the actual
combination of multiple libraries you want to build. You accomplish this
through your ``local.conf`` configuration file in the
:term:`Build Directory`. An example configuration would be as follows::
MACHINE = "qemux86-64"
require conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE:virtclass-multilib-lib32 = "x86"
IMAGE_INSTALL:append = " lib32-glib-2.0"
This example enables an additional library named
``lib32`` alongside the normal target packages. When combining these
"lib32" alternatives, the example uses "x86" for tuning. For information
on this particular tuning, see
``meta/conf/machine/include/ia32/arch-ia32.inc``.
The example then includes ``lib32-glib-2.0`` in all the images, which
illustrates one method of including a multiple library dependency. You
can use a normal image build to include this dependency, for example::
$ bitbake core-image-sato
You can also build Multilib packages
specifically with a command like this::
$ bitbake lib32-glib-2.0
Additional Implementation Details
---------------------------------
There are generic implementation details as well as details that are specific to
package management systems. Following are implementation details
that exist regardless of the package management system:
- The typical convention used for the class extension code as used by
Multilib assumes that all package names specified in
:term:`PACKAGES` that contain
``${PN}`` have ``${PN}`` at the start of the name. When that
convention is not followed and ``${PN}`` appears at the middle or the
end of a name, problems occur.
- The :term:`TARGET_VENDOR`
value under Multilib will be extended to "-vendormlmultilib" (e.g.
"-pokymllib32" for a "lib32" Multilib with Poky). The reason for this
slightly unwieldy contraction is that any "-" characters in the
vendor string presently break Autoconf's ``config.sub``, and other
separators are problematic for different reasons.
Here are the implementation details for the RPM Package Management System:
- A unique architecture is defined for the Multilib packages, along
with creating a unique deploy folder under ``tmp/deploy/rpm`` in the
:term:`Build Directory`. For example, consider ``lib32`` in a
``qemux86-64`` image. The possible architectures in the system are "all",
"qemux86_64", "lib32:qemux86_64", and "lib32:x86".
- The ``${MLPREFIX}`` variable is stripped from ``${PN}`` during RPM
packaging. The naming for a normal RPM package and a Multilib RPM
package in a ``qemux86-64`` system resolves to something similar to
``bash-4.1-r2.x86_64.rpm`` and ``bash-4.1.r2.lib32_x86.rpm``,
respectively.
- When installing a Multilib image, the RPM backend first installs the
base image and then installs the Multilib libraries.
- The build system relies on RPM to resolve the identical files in the
two (or more) Multilib packages.
Here are the implementation details for the IPK Package Management System:
- The ``${MLPREFIX}`` is not stripped from ``${PN}`` during IPK
packaging. The naming for a normal RPM package and a Multilib IPK
package in a ``qemux86-64`` system resolves to something like
``bash_4.1-r2.x86_64.ipk`` and ``lib32-bash_4.1-rw:x86.ipk``,
respectively.
- The IPK deploy folder is not modified with ``${MLPREFIX}`` because
packages with and without the Multilib feature can exist in the same
folder due to the ``${PN}`` differences.
- IPK defines a sanity check for Multilib installation using certain
rules for file comparison, overridden, etc.
Installing Multiple Versions of the Same Library
================================================
There are be situations where you need to install and use multiple versions
of the same library on the same system at the same time. This
almost always happens when a library API changes and you have
multiple pieces of software that depend on the separate versions of the
library. To accommodate these situations, you can install multiple
versions of the same library in parallel on the same system.
The process is straightforward as long as the libraries use proper
versioning. With properly versioned libraries, all you need to do to
individually specify the libraries is create separate, appropriately
named recipes where the :term:`PN` part of
the name includes a portion that differentiates each library version
(e.g. the major part of the version number). Thus, instead of having a
single recipe that loads one version of a library (e.g. ``clutter``),
you provide multiple recipes that result in different versions of the
libraries you want. As an example, the following two recipes would allow
the two separate versions of the ``clutter`` library to co-exist on the
same system:
.. code-block:: none
clutter-1.6_1.6.20.bb
clutter-1.8_1.8.4.bb
Additionally, if
you have other recipes that depend on a given library, you need to use
the :term:`DEPENDS` variable to
create the dependency. Continuing with the same example, if you want to
have a recipe depend on the 1.8 version of the ``clutter`` library, use
the following in your recipe::
DEPENDS = "clutter-1.8"
poky/documentation/dev-manual/licenses.rst
+522
View File
@@ -0,0 +1,522 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Working With Licenses
*********************
As mentioned in the ":ref:`overview-manual/development-environment:licensing`"
section in the Yocto Project Overview and Concepts Manual, open source
projects are open to the public and they consequently have different
licensing structures in place. This section describes the mechanism by
which the :term:`OpenEmbedded Build System`
tracks changes to
licensing text and covers how to maintain open source license compliance
during your project's lifecycle. The section also describes how to
enable commercially licensed recipes, which by default are disabled.
Tracking License Changes
========================
The license of an upstream project might change in the future. In order
to prevent these changes going unnoticed, the
:term:`LIC_FILES_CHKSUM`
variable tracks changes to the license text. The checksums are validated
at the end of the configure step, and if the checksums do not match, the
build will fail.
Specifying the ``LIC_FILES_CHKSUM`` Variable
--------------------------------------------
The :term:`LIC_FILES_CHKSUM` variable contains checksums of the license text
in the source code for the recipe. Following is an example of how to
specify :term:`LIC_FILES_CHKSUM`::
LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \
file://licfile1.txt;beginline=5;endline=29;md5=yyyy \
file://licfile2.txt;endline=50;md5=zzzz \
..."
.. note::
- When using "beginline" and "endline", realize that line numbering
begins with one and not zero. Also, the included lines are
inclusive (i.e. lines five through and including 29 in the
previous example for ``licfile1.txt``).
- When a license check fails, the selected license text is included
as part of the QA message. Using this output, you can determine
the exact start and finish for the needed license text.
The build system uses the :term:`S`
variable as the default directory when searching files listed in
:term:`LIC_FILES_CHKSUM`. The previous example employs the default
directory.
Consider this next example::
LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\
md5=bb14ed3c4cda583abc85401304b5cd4e"
LIC_FILES_CHKSUM = "file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6"
The first line locates a file in ``${S}/src/ls.c`` and isolates lines
five through 16 as license text. The second line refers to a file in
:term:`WORKDIR`.
Note that :term:`LIC_FILES_CHKSUM` variable is mandatory for all recipes,
unless the :term:`LICENSE` variable is set to "CLOSED".
Explanation of Syntax
---------------------
As mentioned in the previous section, the :term:`LIC_FILES_CHKSUM` variable
lists all the important files that contain the license text for the
source code. It is possible to specify a checksum for an entire file, or
a specific section of a file (specified by beginning and ending line
numbers with the "beginline" and "endline" parameters, respectively).
The latter is useful for source files with a license notice header,
README documents, and so forth. If you do not use the "beginline"
parameter, then it is assumed that the text begins on the first line of
the file. Similarly, if you do not use the "endline" parameter, it is
assumed that the license text ends with the last line of the file.
The "md5" parameter stores the md5 checksum of the license text. If the
license text changes in any way as compared to this parameter then a
mismatch occurs. This mismatch triggers a build failure and notifies the
developer. Notification allows the developer to review and address the
license text changes. Also note that if a mismatch occurs during the
build, the correct md5 checksum is placed in the build log and can be
easily copied to the recipe.
There is no limit to how many files you can specify using the
:term:`LIC_FILES_CHKSUM` variable. Generally, however, every project
requires a few specifications for license tracking. Many projects have a
"COPYING" file that stores the license information for all the source
code files. This practice allows you to just track the "COPYING" file as
long as it is kept up to date.
.. note::
- If you specify an empty or invalid "md5" parameter,
:term:`BitBake` returns an md5
mis-match error and displays the correct "md5" parameter value
during the build. The correct parameter is also captured in the
build log.
- If the whole file contains only license text, you do not need to
use the "beginline" and "endline" parameters.
Enabling Commercially Licensed Recipes
======================================
By default, the OpenEmbedded build system disables components that have
commercial or other special licensing requirements. Such requirements
are defined on a recipe-by-recipe basis through the
:term:`LICENSE_FLAGS` variable
definition in the affected recipe. For instance, the
``poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly`` recipe
contains the following statement::
LICENSE_FLAGS = "commercial"
Here is a
slightly more complicated example that contains both an explicit recipe
name and version (after variable expansion)::
LICENSE_FLAGS = "license_${PN}_${PV}"
In order for a component restricted by a
:term:`LICENSE_FLAGS` definition to be enabled and included in an image, it
needs to have a matching entry in the global
:term:`LICENSE_FLAGS_ACCEPTED`
variable, which is a variable typically defined in your ``local.conf``
file. For example, to enable the
``poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly`` package, you
could add either the string "commercial_gst-plugins-ugly" or the more
general string "commercial" to :term:`LICENSE_FLAGS_ACCEPTED`. See the
":ref:`dev-manual/licenses:license flag matching`" section for a full
explanation of how :term:`LICENSE_FLAGS` matching works. Here is the
example::
LICENSE_FLAGS_ACCEPTED = "commercial_gst-plugins-ugly"
Likewise, to additionally enable the package built from the recipe
containing ``LICENSE_FLAGS = "license_${PN}_${PV}"``, and assuming that
the actual recipe name was ``emgd_1.10.bb``, the following string would
enable that package as well as the original ``gst-plugins-ugly``
package::
LICENSE_FLAGS_ACCEPTED = "commercial_gst-plugins-ugly license_emgd_1.10"
As a convenience, you do not need to specify the
complete license string for every package. You can use
an abbreviated form, which consists of just the first portion or
portions of the license string before the initial underscore character
or characters. A partial string will match any license that contains the
given string as the first portion of its license. For example, the
following value will also match both of the packages
previously mentioned as well as any other packages that have licenses
starting with "commercial" or "license"::
LICENSE_FLAGS_ACCEPTED = "commercial license"
License Flag Matching
---------------------
License flag matching allows you to control what recipes the
OpenEmbedded build system includes in the build. Fundamentally, the
build system attempts to match :term:`LICENSE_FLAGS` strings found in
recipes against strings found in :term:`LICENSE_FLAGS_ACCEPTED`.
A match causes the build system to include a recipe in the
build, while failure to find a match causes the build system to exclude
a recipe.
In general, license flag matching is simple. However, understanding some
concepts will help you correctly and effectively use matching.
Before a flag defined by a particular recipe is tested against the
entries of :term:`LICENSE_FLAGS_ACCEPTED`, the expanded
string ``_${PN}`` is appended to the flag. This expansion makes each
:term:`LICENSE_FLAGS` value recipe-specific. After expansion, the
string is then matched against the entries. Thus, specifying
``LICENSE_FLAGS = "commercial"`` in recipe "foo", for example, results
in the string ``"commercial_foo"``. And, to create a match, that string
must appear among the entries of :term:`LICENSE_FLAGS_ACCEPTED`.
Judicious use of the :term:`LICENSE_FLAGS` strings and the contents of the
:term:`LICENSE_FLAGS_ACCEPTED` variable allows you a lot of flexibility for
including or excluding recipes based on licensing. For example, you can
broaden the matching capabilities by using license flags string subsets
in :term:`LICENSE_FLAGS_ACCEPTED`.
.. note::
When using a string subset, be sure to use the part of the expanded
string that precedes the appended underscore character (e.g.
``usethispart_1.3``, ``usethispart_1.4``, and so forth).
For example, simply specifying the string "commercial" in the
:term:`LICENSE_FLAGS_ACCEPTED` variable matches any expanded
:term:`LICENSE_FLAGS` definition that starts with the string
"commercial" such as "commercial_foo" and "commercial_bar", which
are the strings the build system automatically generates for
hypothetical recipes named "foo" and "bar" assuming those recipes simply
specify the following::
LICENSE_FLAGS = "commercial"
Thus, you can choose to exhaustively enumerate each license flag in the
list and allow only specific recipes into the image, or you can use a
string subset that causes a broader range of matches to allow a range of
recipes into the image.
This scheme works even if the :term:`LICENSE_FLAGS` string already has
``_${PN}`` appended. For example, the build system turns the license
flag "commercial_1.2_foo" into "commercial_1.2_foo_foo" and would match
both the general "commercial" and the specific "commercial_1.2_foo"
strings found in the :term:`LICENSE_FLAGS_ACCEPTED` variable, as expected.
Here are some other scenarios:
- You can specify a versioned string in the recipe such as
"commercial_foo_1.2" in a "foo" recipe. The build system expands this
string to "commercial_foo_1.2_foo". Combine this license flag with a
:term:`LICENSE_FLAGS_ACCEPTED` variable that has the string
"commercial" and you match the flag along with any other flag that
starts with the string "commercial".
- Under the same circumstances, you can add "commercial_foo" in the
:term:`LICENSE_FLAGS_ACCEPTED` variable and the build system not only
matches "commercial_foo_1.2" but also matches any license flag with
the string "commercial_foo", regardless of the version.
- You can be very specific and use both the package and version parts
in the :term:`LICENSE_FLAGS_ACCEPTED` list (e.g.
"commercial_foo_1.2") to specifically match a versioned recipe.
Other Variables Related to Commercial Licenses
----------------------------------------------
There are other helpful variables related to commercial license handling,
defined in the
``poky/meta/conf/distro/include/default-distrovars.inc`` file::
COMMERCIAL_AUDIO_PLUGINS ?= ""
COMMERCIAL_VIDEO_PLUGINS ?= ""
If you want to enable these components, you can do so by making sure you have
statements similar to the following in your ``local.conf`` configuration file::
COMMERCIAL_AUDIO_PLUGINS = "gst-plugins-ugly-mad \
gst-plugins-ugly-mpegaudioparse"
COMMERCIAL_VIDEO_PLUGINS = "gst-plugins-ugly-mpeg2dec \
gst-plugins-ugly-mpegstream gst-plugins-bad-mpegvideoparse"
LICENSE_FLAGS_ACCEPTED = "commercial_gst-plugins-ugly commercial_gst-plugins-bad commercial_qmmp"
Of course, you could also create a matching list for those components using the
more general "commercial" string in the :term:`LICENSE_FLAGS_ACCEPTED` variable,
but that would also enable all the other packages with :term:`LICENSE_FLAGS`
containing "commercial", which you may or may not want::
LICENSE_FLAGS_ACCEPTED = "commercial"
Specifying audio and video plugins as part of the
:term:`COMMERCIAL_AUDIO_PLUGINS` and :term:`COMMERCIAL_VIDEO_PLUGINS` statements
(along with :term:`LICENSE_FLAGS_ACCEPTED`) includes the plugins or
components into built images, thus adding support for media formats or
components.
.. note::
GStreamer "ugly" and "bad" plugins are actually available through
open source licenses. However, the "ugly" ones can be subject to software
patents in some countries, making it necessary to pay licensing fees
to distribute them. The "bad" ones are just deemed unreliable by the
GStreamer community and should therefore be used with care.
Maintaining Open Source License Compliance During Your Product's Lifecycle
==========================================================================
One of the concerns for a development organization using open source
software is how to maintain compliance with various open source
licensing during the lifecycle of the product. While this section does
not provide legal advice or comprehensively cover all scenarios, it does
present methods that you can use to assist you in meeting the compliance
requirements during a software release.
With hundreds of different open source licenses that the Yocto Project
tracks, it is difficult to know the requirements of each and every
license. However, the requirements of the major FLOSS licenses can begin
to be covered by assuming that there are three main areas of concern:
- Source code must be provided.
- License text for the software must be provided.
- Compilation scripts and modifications to the source code must be
provided.
There are other requirements beyond the scope of these three and the
methods described in this section (e.g. the mechanism through which
source code is distributed).
As different organizations have different methods of complying with open
source licensing, this section is not meant to imply that there is only
one single way to meet your compliance obligations, but rather to
describe one method of achieving compliance. The remainder of this
section describes methods supported to meet the previously mentioned
three requirements. Once you take steps to meet these requirements, and
prior to releasing images, sources, and the build system, you should
audit all artifacts to ensure completeness.
.. note::
The Yocto Project generates a license manifest during image creation
that is located in ``${DEPLOY_DIR}/licenses/``\ `image_name`\ ``-``\ `datestamp`
to assist with any audits.
Providing the Source Code
-------------------------
Compliance activities should begin before you generate the final image.
The first thing you should look at is the requirement that tops the list
for most compliance groups --- providing the source. The Yocto Project has
a few ways of meeting this requirement.
One of the easiest ways to meet this requirement is to provide the
entire :term:`DL_DIR` used by the
build. This method, however, has a few issues. The most obvious is the
size of the directory since it includes all sources used in the build
and not just the source used in the released image. It will include
toolchain source, and other artifacts, which you would not generally
release. However, the more serious issue for most companies is
accidental release of proprietary software. The Yocto Project provides
an :ref:`ref-classes-archiver` class to help avoid some of these concerns.
Before you employ :term:`DL_DIR` or the :ref:`ref-classes-archiver` class, you
need to decide how you choose to provide source. The source
:ref:`ref-classes-archiver` class can generate tarballs and SRPMs and can
create them with various levels of compliance in mind.
One way of doing this (but certainly not the only way) is to release
just the source as a tarball. You can do this by adding the following to
the ``local.conf`` file found in the :term:`Build Directory`::
INHERIT += "archiver"
ARCHIVER_MODE[src] = "original"
During the creation of your
image, the source from all recipes that deploy packages to the image is
placed within subdirectories of ``DEPLOY_DIR/sources`` based on the
:term:`LICENSE` for each recipe.
Releasing the entire directory enables you to comply with requirements
concerning providing the unmodified source. It is important to note that
the size of the directory can get large.
A way to help mitigate the size issue is to only release tarballs for
licenses that require the release of source. Let us assume you are only
concerned with GPL code as identified by running the following script:
.. code-block:: shell
# Script to archive a subset of packages matching specific license(s)
# Source and license files are copied into sub folders of package folder
# Must be run from build folder
#!/bin/bash
src_release_dir="source-release"
mkdir -p $src_release_dir
for a in tmp/deploy/sources/*; do
for d in $a/*; do
# Get package name from path
p=`basename $d`
p=${p%-*}
p=${p%-*}
# Only archive GPL packages (update *GPL* regex for your license check)
numfiles=`ls tmp/deploy/licenses/$p/*GPL* 2> /dev/null | wc -l`
if [ $numfiles -ge 1 ]; then
echo Archiving $p
mkdir -p $src_release_dir/$p/source
cp $d/* $src_release_dir/$p/source 2> /dev/null
mkdir -p $src_release_dir/$p/license
cp tmp/deploy/licenses/$p/* $src_release_dir/$p/license 2> /dev/null
fi
done
done
At this point, you
could create a tarball from the ``gpl_source_release`` directory and
provide that to the end user. This method would be a step toward
achieving compliance with section 3a of GPLv2 and with section 6 of
GPLv3.
Providing License Text
----------------------
One requirement that is often overlooked is inclusion of license text.
This requirement also needs to be dealt with prior to generating the
final image. Some licenses require the license text to accompany the
binary. You can achieve this by adding the following to your
``local.conf`` file::
COPY_LIC_MANIFEST = "1"
COPY_LIC_DIRS = "1"
LICENSE_CREATE_PACKAGE = "1"
Adding these statements to the
configuration file ensures that the licenses collected during package
generation are included on your image.
.. note::
Setting all three variables to "1" results in the image having two
copies of the same license file. One copy resides in
``/usr/share/common-licenses`` and the other resides in
``/usr/share/license``.
The reason for this behavior is because
:term:`COPY_LIC_DIRS` and
:term:`COPY_LIC_MANIFEST`
add a copy of the license when the image is built but do not offer a
path for adding licenses for newly installed packages to an image.
:term:`LICENSE_CREATE_PACKAGE`
adds a separate package and an upgrade path for adding licenses to an
image.
As the source :ref:`ref-classes-archiver` class has already archived the
original unmodified source that contains the license files, you would have
already met the requirements for inclusion of the license information
with source as defined by the GPL and other open source licenses.
Providing Compilation Scripts and Source Code Modifications
-----------------------------------------------------------
At this point, we have addressed all we need to prior to generating the
image. The next two requirements are addressed during the final
packaging of the release.
By releasing the version of the OpenEmbedded build system and the layers
used during the build, you will be providing both compilation scripts
and the source code modifications in one step.
If the deployment team has a :ref:`overview-manual/concepts:bsp layer`
and a distro layer, and those
those layers are used to patch, compile, package, or modify (in any way)
any open source software included in your released images, you might be
required to release those layers under section 3 of GPLv2 or section 1
of GPLv3. One way of doing that is with a clean checkout of the version
of the Yocto Project and layers used during your build. Here is an
example:
.. code-block:: shell
# We built using the dunfell branch of the poky repo
$ git clone -b dunfell git://git.yoctoproject.org/poky
$ cd poky
# We built using the release_branch for our layers
$ git clone -b release_branch git://git.mycompany.com/meta-my-bsp-layer
$ git clone -b release_branch git://git.mycompany.com/meta-my-software-layer
# clean up the .git repos
$ find . -name ".git" -type d -exec rm -rf {} \;
One thing a development organization might want to consider for end-user
convenience is to modify
``meta-poky/conf/templates/default/bblayers.conf.sample`` to ensure that when
the end user utilizes the released build system to build an image, the
development organization's layers are included in the ``bblayers.conf`` file
automatically::
# POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
POKY_BBLAYERS_CONF_VERSION = "2"
BBPATH = "${TOPDIR}"
BBFILES ?= ""
BBLAYERS ?= " \
##OEROOT##/meta \
##OEROOT##/meta-poky \
##OEROOT##/meta-yocto-bsp \
##OEROOT##/meta-mylayer \
"
Creating and
providing an archive of the :term:`Metadata`
layers (recipes, configuration files, and so forth) enables you to meet
your requirements to include the scripts to control compilation as well
as any modifications to the original source.
Compliance Limitations with Executables Built from Static Libraries
-------------------------------------------------------------------
When package A is added to an image via the :term:`RDEPENDS` or :term:`RRECOMMENDS`
mechanisms as well as explicitly included in the image recipe with
:term:`IMAGE_INSTALL`, and depends on a static linked library recipe B
(``DEPENDS += "B"``), package B will neither appear in the generated license
manifest nor in the generated source tarballs. This occurs as the
:ref:`ref-classes-license` and :ref:`ref-classes-archiver` classes assume that
only packages included via :term:`RDEPENDS` or :term:`RRECOMMENDS`
end up in the image.
As a result, potential obligations regarding license compliance for package B
may not be met.
The Yocto Project doesn't enable static libraries by default, in part because
of this issue. Before a solution to this limitation is found, you need to
keep in mind that if your root filesystem is built from static libraries,
you will need to manually ensure that your deliveries are compliant
with the licenses of these libraries.
Copying Non Standard Licenses
=============================
Some packages, such as the linux-firmware package, have many licenses
that are not in any way common. You can avoid adding a lot of these
types of common license files, which are only applicable to a specific
package, by using the
:term:`NO_GENERIC_LICENSE`
variable. Using this variable also avoids QA errors when you use a
non-common, non-CLOSED license in a recipe.
Here is an example that uses the ``LICENSE.Abilis.txt`` file as
the license from the fetched source::
NO_GENERIC_LICENSE[Firmware-Abilis] = "LICENSE.Abilis.txt"
poky/documentation/dev-manual/new-machine.rst
+118
View File
@@ -0,0 +1,118 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Adding a New Machine
********************
Adding a new machine to the Yocto Project is a straightforward process.
This section describes how to add machines that are similar to those
that the Yocto Project already supports.
.. note::
Although well within the capabilities of the Yocto Project, adding a
totally new architecture might require changes to ``gcc``/``glibc``
and to the site information, which is beyond the scope of this
manual.
For a complete example that shows how to add a new machine, see the
":ref:`bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script`"
section in the Yocto Project Board Support Package (BSP) Developer's
Guide.
Adding the Machine Configuration File
=====================================
To add a new machine, you need to add a new machine configuration file
to the layer's ``conf/machine`` directory. This configuration file
provides details about the device you are adding.
The OpenEmbedded build system uses the root name of the machine
configuration file to reference the new machine. For example, given a
machine configuration file named ``crownbay.conf``, the build system
recognizes the machine as "crownbay".
The most important variables you must set in your machine configuration
file or include from a lower-level configuration file are as follows:
- :term:`TARGET_ARCH` (e.g. "arm")
- ``PREFERRED_PROVIDER_virtual/kernel``
- :term:`MACHINE_FEATURES` (e.g. "apm screen wifi")
You might also need these variables:
- :term:`SERIAL_CONSOLES` (e.g. "115200;ttyS0 115200;ttyS1")
- :term:`KERNEL_IMAGETYPE` (e.g. "zImage")
- :term:`IMAGE_FSTYPES` (e.g. "tar.gz jffs2")
You can find full details on these variables in the reference section.
You can leverage existing machine ``.conf`` files from
``meta-yocto-bsp/conf/machine/``.
Adding a Kernel for the Machine
===============================
The OpenEmbedded build system needs to be able to build a kernel for the
machine. You need to either create a new kernel recipe for this machine,
or extend an existing kernel recipe. You can find several kernel recipe
examples in the Source Directory at ``meta/recipes-kernel/linux`` that
you can use as references.
If you are creating a new kernel recipe, normal recipe-writing rules
apply for setting up a :term:`SRC_URI`. Thus, you need to specify any
necessary patches and set :term:`S` to point at the source code. You need to
create a :ref:`ref-tasks-configure` task that configures the unpacked kernel with
a ``defconfig`` file. You can do this by using a ``make defconfig``
command or, more commonly, by copying in a suitable ``defconfig`` file
and then running ``make oldconfig``. By making use of ``inherit kernel``
and potentially some of the ``linux-*.inc`` files, most other
functionality is centralized and the defaults of the class normally work
well.
If you are extending an existing kernel recipe, it is usually a matter
of adding a suitable ``defconfig`` file. The file needs to be added into
a location similar to ``defconfig`` files used for other machines in a
given kernel recipe. A possible way to do this is by listing the file in
the :term:`SRC_URI` and adding the machine to the expression in
:term:`COMPATIBLE_MACHINE`::
COMPATIBLE_MACHINE = '(qemux86|qemumips)'
For more information on ``defconfig`` files, see the
":ref:`kernel-dev/common:changing the configuration`"
section in the Yocto Project Linux Kernel Development Manual.
Adding a Formfactor Configuration File
======================================
A formfactor configuration file provides information about the target
hardware for which the image is being built and information that the
build system cannot obtain from other sources such as the kernel. Some
examples of information contained in a formfactor configuration file
include framebuffer orientation, whether or not the system has a
keyboard, the positioning of the keyboard in relation to the screen, and
the screen resolution.
The build system uses reasonable defaults in most cases. However, if
customization is necessary, you need to create a ``machconfig`` file in
the ``meta/recipes-bsp/formfactor/files`` directory. This directory
contains directories for specific machines such as ``qemuarm`` and
``qemux86``. For information about the settings available and the
defaults, see the ``meta/recipes-bsp/formfactor/files/config`` file
found in the same area.
Following is an example for "qemuarm" machine::
HAVE_TOUCHSCREEN=1
HAVE_KEYBOARD=1
DISPLAY_CAN_ROTATE=0
DISPLAY_ORIENTATION=0
#DISPLAY_WIDTH_PIXELS=640
#DISPLAY_HEIGHT_PIXELS=480
#DISPLAY_BPP=16
DISPLAY_DPI=150
DISPLAY_SUBPIXEL_ORDER=vrgb
poky/documentation/dev-manual/new-recipe.rst
+1680
View File
File diff suppressed because it is too large Load Diff
poky/documentation/dev-manual/packages.rst
+1238
View File
File diff suppressed because it is too large Load Diff
poky/documentation/dev-manual/prebuilt-libraries.rst
+209
View File
@@ -0,0 +1,209 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Working with Pre-Built Libraries
********************************
Introduction
============
Some library vendors do not release source code for their software but do
release pre-built binaries. When shared libraries are built, they should
be versioned (see `this article
<https://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html>`__
for some background), but sometimes this is not done.
To summarize, a versioned library must meet two conditions:
#. The filename must have the version appended, for example: ``libfoo.so.1.2.3``.
#. The library must have the ELF tag ``SONAME`` set to the major version
of the library, for example: ``libfoo.so.1``. You can check this by
running ``readelf -d filename | grep SONAME``.
This section shows how to deal with both versioned and unversioned
pre-built libraries.
Versioned Libraries
===================
In this example we work with pre-built libraries for the FT4222H USB I/O chip.
Libraries are built for several target architecture variants and packaged in
an archive as follows::
├── build-arm-hisiv300
│   └── libft4222.so.1.4.4.44
├── build-arm-v5-sf
│   └── libft4222.so.1.4.4.44
├── build-arm-v6-hf
│   └── libft4222.so.1.4.4.44
├── build-arm-v7-hf
│   └── libft4222.so.1.4.4.44
├── build-arm-v8
│   └── libft4222.so.1.4.4.44
├── build-i386
│   └── libft4222.so.1.4.4.44
├── build-i486
│   └── libft4222.so.1.4.4.44
├── build-mips-eglibc-hf
│   └── libft4222.so.1.4.4.44
├── build-pentium
│   └── libft4222.so.1.4.4.44
├── build-x86_64
│   └── libft4222.so.1.4.4.44
├── examples
│   ├── get-version.c
│   ├── i2cm.c
│   ├── spim.c
│   └── spis.c
├── ftd2xx.h
├── install4222.sh
├── libft4222.h
├── ReadMe.txt
└── WinTypes.h
To write a recipe to use such a library in your system:
- The vendor will probably have a proprietary licence, so set
:term:`LICENSE_FLAGS` in your recipe.
- The vendor provides a tarball containing libraries so set :term:`SRC_URI`
appropriately.
- Set :term:`COMPATIBLE_HOST` so that the recipe cannot be used with an
unsupported architecture. In the following example, we only support the 32
and 64 bit variants of the ``x86`` architecture.
- As the vendor provides versioned libraries, we can use ``oe_soinstall``
from :ref:`ref-classes-utils` to install the shared library and create
symbolic links. If the vendor does not do this, we need to follow the
non-versioned library guidelines in the next section.
- As the vendor likely used :term:`LDFLAGS` different from those in your Yocto
Project build, disable the corresponding checks by adding ``ldflags``
to :term:`INSANE_SKIP`.
- The vendor will typically ship release builds without debugging symbols.
Avoid errors by preventing the packaging task from stripping out the symbols
and adding them to a separate debug package. This is done by setting the
``INHIBIT_`` flags shown below.
The complete recipe would look like this::
SUMMARY = "FTDI FT4222H Library"
SECTION = "libs"
LICENSE_FLAGS = "ftdi"
LICENSE = "CLOSED"
COMPATIBLE_HOST = "(i.86|x86_64).*-linux"
# Sources available in a .tgz file in .zip archive
# at https://ftdichip.com/wp-content/uploads/2021/01/libft4222-linux-1.4.4.44.zip
# Found on https://ftdichip.com/software-examples/ft4222h-software-examples/
# Since dealing with this particular type of archive is out of topic here,
# we use a local link.
SRC_URI = "file://libft4222-linux-${PV}.tgz"
S = "${WORKDIR}"
ARCH_DIR:x86-64 = "build-x86_64"
ARCH_DIR:i586 = "build-i386"
ARCH_DIR:i686 = "build-i386"
INSANE_SKIP:${PN} = "ldflags"
INHIBIT_PACKAGE_STRIP = "1"
INHIBIT_SYSROOT_STRIP = "1"
INHIBIT_PACKAGE_DEBUG_SPLIT = "1"
do_install () {
install -m 0755 -d ${D}${libdir}
oe_soinstall ${S}/${ARCH_DIR}/libft4222.so.${PV} ${D}${libdir}
install -d ${D}${includedir}
install -m 0755 ${S}/*.h ${D}${includedir}
}
If the precompiled binaries are not statically linked and have dependencies on
other libraries, then by adding those libraries to :term:`DEPENDS`, the linking
can be examined and the appropriate :term:`RDEPENDS` automatically added.
Non-Versioned Libraries
=======================
Some Background
---------------
Libraries in Linux systems are generally versioned so that it is possible
to have multiple versions of the same library installed, which eases upgrades
and support for older software. For example, suppose that in a versioned
library, an actual library is called ``libfoo.so.1.2``, a symbolic link named
``libfoo.so.1`` points to ``libfoo.so.1.2``, and a symbolic link named
``libfoo.so`` points to ``libfoo.so.1.2``. Given these conditions, when you
link a binary against a library, you typically provide the unversioned file
name (i.e. ``-lfoo`` to the linker). However, the linker follows the symbolic
link and actually links against the versioned filename. The unversioned symbolic
link is only used at development time. Consequently, the library is packaged
along with the headers in the development package ``${PN}-dev`` along with the
actual library and versioned symbolic links in ``${PN}``. Because versioned
libraries are far more common than unversioned libraries, the default packaging
rules assume versioned libraries.
Yocto Library Packaging Overview
--------------------------------
It follows that packaging an unversioned library requires a bit of work in the
recipe. By default, ``libfoo.so`` gets packaged into ``${PN}-dev``, which
triggers a QA warning that a non-symlink library is in a ``-dev`` package,
and binaries in the same recipe link to the library in ``${PN}-dev``,
which triggers more QA warnings. To solve this problem, you need to package the
unversioned library into ``${PN}`` where it belongs. The following are the abridged
default :term:`FILES` variables in ``bitbake.conf``::
SOLIBS = ".so.*"
SOLIBSDEV = ".so"
FILES:${PN} = "... ${libdir}/lib*${SOLIBS} ..."
FILES_SOLIBSDEV ?= "... ${libdir}/lib*${SOLIBSDEV} ..."
FILES:${PN}-dev = "... ${FILES_SOLIBSDEV} ..."
:term:`SOLIBS` defines a pattern that matches real shared object libraries.
:term:`SOLIBSDEV` matches the development form (unversioned symlink). These two
variables are then used in ``FILES:${PN}`` and ``FILES:${PN}-dev``, which puts
the real libraries into ``${PN}`` and the unversioned symbolic link into ``${PN}-dev``.
To package unversioned libraries, you need to modify the variables in the recipe
as follows::
SOLIBS = ".so"
FILES_SOLIBSDEV = ""
The modifications cause the ``.so`` file to be the real library
and unset :term:`FILES_SOLIBSDEV` so that no libraries get packaged into
``${PN}-dev``. The changes are required because unless :term:`PACKAGES` is changed,
``${PN}-dev`` collects files before `${PN}`. ``${PN}-dev`` must not collect any of
the files you want in ``${PN}``.
Finally, loadable modules, essentially unversioned libraries that are linked
at runtime using ``dlopen()`` instead of at build time, should generally be
installed in a private directory. However, if they are installed in ``${libdir}``,
then the modules can be treated as unversioned libraries.
Example
-------
The example below installs an unversioned x86-64 pre-built library named
``libfoo.so``. The :term:`COMPATIBLE_HOST` variable limits recipes to the
x86-64 architecture while the :term:`INSANE_SKIP`, :term:`INHIBIT_PACKAGE_STRIP`
and :term:`INHIBIT_SYSROOT_STRIP` variables are all set as in the above
versioned library example. The "magic" is setting the :term:`SOLIBS` and
:term:`FILES_SOLIBSDEV` variables as explained above::
SUMMARY = "libfoo sample recipe"
SECTION = "libs"
LICENSE = "CLOSED"
SRC_URI = "file://libfoo.so"
COMPATIBLE_HOST = "x86_64.*-linux"
INSANE_SKIP:${PN} = "ldflags"
INHIBIT_PACKAGE_STRIP = "1"
INHIBIT_SYSROOT_STRIP = "1"
SOLIBS = ".so"
FILES_SOLIBSDEV = ""
do_install () {
install -d ${D}${libdir}
install -m 0755 ${WORKDIR}/libfoo.so ${D}${libdir}
}
poky/documentation/dev-manual/python-development-shell.rst
+50
View File
@@ -0,0 +1,50 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Using a Python Development Shell
********************************
Similar to working within a development shell as described in the
previous section, you can also spawn and work within an interactive
Python development shell. When debugging certain commands or even when
just editing packages, ``pydevshell`` can be a useful tool. When you
invoke the ``pydevshell`` task, all tasks up to and including
:ref:`ref-tasks-patch` are run for the
specified target. Then a new terminal is opened. Additionally, key
Python objects and code are available in the same way they are to
BitBake tasks, in particular, the data store 'd'. So, commands such as
the following are useful when exploring the data store and running
functions::
pydevshell> d.getVar("STAGING_DIR")
'/media/build1/poky/build/tmp/sysroots'
pydevshell> d.getVar("STAGING_DIR", False)
'${TMPDIR}/sysroots'
pydevshell> d.setVar("FOO", "bar")
pydevshell> d.getVar("FOO")
'bar'
pydevshell> d.delVar("FOO")
pydevshell> d.getVar("FOO")
pydevshell> bb.build.exec_func("do_unpack", d)
pydevshell>
See the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:functions you can call from within python`"
section in the BitBake User Manual for details about available functions.
The commands execute just as if the OpenEmbedded build
system were executing them. Consequently, working this way can be
helpful when debugging a build or preparing software to be used with the
OpenEmbedded build system.
Following is an example that uses ``pydevshell`` on a target named
``matchbox-desktop``::
$ bitbake matchbox-desktop -c pydevshell
This command spawns a terminal and places you in an interactive Python
interpreter within the OpenEmbedded build environment. The
:term:`OE_TERMINAL` variable
controls what type of shell is opened.
When you are finished using ``pydevshell``, you can exit the shell
either by using Ctrl+d or closing the terminal window.
poky/documentation/dev-manual/qemu.rst
+471
View File
@@ -0,0 +1,471 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
*******************************
Using the Quick EMUlator (QEMU)
*******************************
The Yocto Project uses an implementation of the Quick EMUlator (QEMU)
Open Source project as part of the Yocto Project development "tool set".
This chapter provides both procedures that show you how to use the Quick
EMUlator (QEMU) and other QEMU information helpful for development
purposes.
Overview
========
Within the context of the Yocto Project, QEMU is an emulator and
virtualization machine that allows you to run a complete image you have
built using the Yocto Project as just another task on your build system.
QEMU is useful for running and testing images and applications on
supported Yocto Project architectures without having actual hardware.
Among other things, the Yocto Project uses QEMU to run automated Quality
Assurance (QA) tests on final images shipped with each release.
.. note::
This implementation is not the same as QEMU in general.
This section provides a brief reference for the Yocto Project
implementation of QEMU.
For official information and documentation on QEMU in general, see the
following references:
- `QEMU Website <https://wiki.qemu.org/Main_Page>`__\ *:* The official
website for the QEMU Open Source project.
- `Documentation <https://wiki.qemu.org/Manual>`__\ *:* The QEMU user
manual.
Running QEMU
============
To use QEMU, you need to have QEMU installed and initialized as well as
have the proper artifacts (i.e. image files and root filesystems)
available. Follow these general steps to run QEMU:
#. *Install QEMU:* QEMU is made available with the Yocto Project a
number of ways. One method is to install a Software Development Kit
(SDK). See ":ref:`sdk-manual/intro:the qemu emulator`" section in the
Yocto Project Application Development and the Extensible Software
Development Kit (eSDK) manual for information on how to install QEMU.
#. *Setting Up the Environment:* How you set up the QEMU environment
depends on how you installed QEMU:
- If you cloned the ``poky`` repository or you downloaded and
unpacked a Yocto Project release tarball, you can source the build
environment script (i.e. :ref:`structure-core-script`)::
$ cd poky
$ source oe-init-build-env
- If you installed a cross-toolchain, you can run the script that
initializes the toolchain. For example, the following commands run
the initialization script from the default ``poky_sdk`` directory::
. poky_sdk/environment-setup-core2-64-poky-linux
#. *Ensure the Artifacts are in Place:* You need to be sure you have a
pre-built kernel that will boot in QEMU. You also need the target
root filesystem for your target machine's architecture:
- If you have previously built an image for QEMU (e.g. ``qemux86``,
``qemuarm``, and so forth), then the artifacts are in place in
your :term:`Build Directory`.
- If you have not built an image, you can go to the
:yocto_dl:`machines/qemu </releases/yocto/yocto-&DISTRO;/machines/qemu/>` area and download a
pre-built image that matches your architecture and can be run on
QEMU.
See the ":ref:`sdk-manual/appendix-obtain:extracting the root filesystem`"
section in the Yocto Project Application Development and the
Extensible Software Development Kit (eSDK) manual for information on
how to extract a root filesystem.
#. *Run QEMU:* The basic ``runqemu`` command syntax is as follows::
$ runqemu [option ] [...]
Based on what you provide on the command
line, ``runqemu`` does a good job of figuring out what you are trying
to do. For example, by default, QEMU looks for the most recently
built image according to the timestamp when it needs to look for an
image. Minimally, through the use of options, you must provide either
a machine name, a virtual machine image (``*wic.vmdk``), or a kernel
image (``*.bin``).
Here are some additional examples to help illustrate further QEMU:
- This example starts QEMU with MACHINE set to "qemux86-64".
Assuming a standard :term:`Build Directory`, ``runqemu``
automatically finds the ``bzImage-qemux86-64.bin`` image file and
the ``core-image-minimal-qemux86-64-20200218002850.rootfs.ext4``
(assuming the current build created a ``core-image-minimal``
image)::
$ runqemu qemux86-64
.. note::
When more than one image with the same name exists, QEMU finds
and uses the most recently built image according to the
timestamp.
- This example produces the exact same results as the previous
example. This command, however, specifically provides the image
and root filesystem type::
$ runqemu qemux86-64 core-image-minimal ext4
- This example specifies to boot an :term:`Initramfs` image and to
enable audio in QEMU. For this case, ``runqemu`` sets the internal
variable ``FSTYPE`` to ``cpio.gz``. Also, for audio to be enabled,
an appropriate driver must be installed (see the ``audio`` option
in :ref:`dev-manual/qemu:\`\`runqemu\`\` command-line options`
for more information)::
$ runqemu qemux86-64 ramfs audio
- This example does not provide enough information for QEMU to
launch. While the command does provide a root filesystem type, it
must also minimally provide a `MACHINE`, `KERNEL`, or `VM` option::
$ runqemu ext4
- This example specifies to boot a virtual machine image
(``.wic.vmdk`` file). From the ``.wic.vmdk``, ``runqemu``
determines the QEMU architecture (`MACHINE`) to be "qemux86-64" and
the root filesystem type to be "vmdk"::
$ runqemu /home/scott-lenovo/vm/core-image-minimal-qemux86-64.wic.vmdk
Switching Between Consoles
==========================
When booting or running QEMU, you can switch between supported consoles
by using Ctrl+Alt+number. For example, Ctrl+Alt+3 switches you to the
serial console as long as that console is enabled. Being able to switch
consoles is helpful, for example, if the main QEMU console breaks for
some reason.
.. note::
Usually, "2" gets you to the main console and "3" gets you to the
serial console.
Removing the Splash Screen
==========================
You can remove the splash screen when QEMU is booting by using Alt+left.
Removing the splash screen allows you to see what is happening in the
background.
Disabling the Cursor Grab
=========================
The default QEMU integration captures the cursor within the main window.
It does this since standard mouse devices only provide relative input
and not absolute coordinates. You then have to break out of the grab
using the "Ctrl+Alt" key combination. However, the Yocto Project's
integration of QEMU enables the wacom USB touch pad driver by default to
allow input of absolute coordinates. This default means that the mouse
can enter and leave the main window without the grab taking effect
leading to a better user experience.
Running Under a Network File System (NFS) Server
================================================
One method for running QEMU is to run it on an NFS server. This is
useful when you need to access the same file system from both the build
and the emulated system at the same time. It is also worth noting that
the system does not need root privileges to run. It uses a user space
NFS server to avoid that. Follow these steps to set up for running QEMU
using an NFS server.
#. *Extract a Root Filesystem:* Once you are able to run QEMU in your
environment, you can use the ``runqemu-extract-sdk`` script, which is
located in the ``scripts`` directory along with the ``runqemu``
script.
The ``runqemu-extract-sdk`` takes a root filesystem tarball and
extracts it into a location that you specify. Here is an example that
takes a file system and extracts it to a directory named
``test-nfs``:
.. code-block:: none
runqemu-extract-sdk ./tmp/deploy/images/qemux86-64/core-image-sato-qemux86-64.tar.bz2 test-nfs
#. *Start QEMU:* Once you have extracted the file system, you can run
``runqemu`` normally with the additional location of the file system.
You can then also make changes to the files within ``./test-nfs`` and
see those changes appear in the image in real time. Here is an
example using the ``qemux86`` image:
.. code-block:: none
runqemu qemux86-64 ./test-nfs
.. note::
Should you need to start, stop, or restart the NFS share, you can use
the following commands:
- To start the NFS share::
runqemu-export-rootfs start file-system-location
- To stop the NFS share::
runqemu-export-rootfs stop file-system-location
- To restart the NFS share::
runqemu-export-rootfs restart file-system-location
QEMU CPU Compatibility Under KVM
================================
By default, the QEMU build compiles for and targets 64-bit and x86 Intel
Core2 Duo processors and 32-bit x86 Intel Pentium II processors. QEMU
builds for and targets these CPU types because they display a broad
range of CPU feature compatibility with many commonly used CPUs.
Despite this broad range of compatibility, the CPUs could support a
feature that your host CPU does not support. Although this situation is
not a problem when QEMU uses software emulation of the feature, it can
be a problem when QEMU is running with KVM enabled. Specifically,
software compiled with a certain CPU feature crashes when run on a CPU
under KVM that does not support that feature. To work around this
problem, you can override QEMU's runtime CPU setting by changing the
``QB_CPU_KVM`` variable in ``qemuboot.conf`` in the :term:`Build Directory`
``deploy/image`` directory. This setting specifies a ``-cpu`` option passed
into QEMU in the ``runqemu`` script. Running ``qemu -cpu help`` returns a
list of available supported CPU types.
QEMU Performance
================
Using QEMU to emulate your hardware can result in speed issues depending
on the target and host architecture mix. For example, using the
``qemux86`` image in the emulator on an Intel-based 32-bit (x86) host
machine is fast because the target and host architectures match. On the
other hand, using the ``qemuarm`` image on the same Intel-based host can
be slower. But, you still achieve faithful emulation of ARM-specific
issues.
To speed things up, the QEMU images support using ``distcc`` to call a
cross-compiler outside the emulated system. If you used ``runqemu`` to
start QEMU, and the ``distccd`` application is present on the host
system, any BitBake cross-compiling toolchain available from the build
system is automatically used from within QEMU simply by calling
``distcc``. You can accomplish this by defining the cross-compiler
variable (e.g. ``export CC="distcc"``). Alternatively, if you are using
a suitable SDK image or the appropriate stand-alone toolchain is
present, the toolchain is also automatically used.
.. note::
There are several mechanisms to connect to the system running
on the QEMU emulator:
- QEMU provides a framebuffer interface that makes standard consoles
available.
- Generally, headless embedded devices have a serial port. If so,
you can configure the operating system of the running image to use
that port to run a console. The connection uses standard IP
networking.
- SSH servers are available in some QEMU images. The ``core-image-sato``
QEMU image has a Dropbear secure shell (SSH) server that runs with
the root password disabled. The ``core-image-full-cmdline`` and
``core-image-lsb`` QEMU images have OpenSSH instead of Dropbear.
Including these SSH servers allow you to use standard ``ssh`` and
``scp`` commands. The ``core-image-minimal`` QEMU image, however,
contains no SSH server.
- You can use a provided, user-space NFS server to boot the QEMU
session using a local copy of the root filesystem on the host. In
order to make this connection, you must extract a root filesystem
tarball by using the ``runqemu-extract-sdk`` command. After
running the command, you must then point the ``runqemu`` script to
the extracted directory instead of a root filesystem image file.
See the
":ref:`dev-manual/qemu:running under a network file system (nfs) server`"
section for more information.
QEMU Command-Line Syntax
========================
The basic ``runqemu`` command syntax is as follows::
$ runqemu [option ] [...]
Based on what you provide on the command line, ``runqemu`` does a
good job of figuring out what you are trying to do. For example, by
default, QEMU looks for the most recently built image according to the
timestamp when it needs to look for an image. Minimally, through the use
of options, you must provide either a machine name, a virtual machine
image (``*wic.vmdk``), or a kernel image (``*.bin``).
Following is the command-line help output for the ``runqemu`` command::
$ runqemu --help
Usage: you can run this script with any valid combination
of the following environment variables (in any order):
KERNEL - the kernel image file to use
ROOTFS - the rootfs image file or nfsroot directory to use
MACHINE - the machine name (optional, autodetected from KERNEL filename if unspecified)
Simplified QEMU command-line options can be passed with:
nographic - disable video console
serial - enable a serial console on /dev/ttyS0
slirp - enable user networking, no root privileges required
kvm - enable KVM when running x86/x86_64 (VT-capable CPU required)
kvm-vhost - enable KVM with vhost when running x86/x86_64 (VT-capable CPU required)
publicvnc - enable a VNC server open to all hosts
audio - enable audio
[*/]ovmf* - OVMF firmware file or base name for booting with UEFI
tcpserial=<port> - specify tcp serial port number
biosdir=<dir> - specify custom bios dir
biosfilename=<filename> - specify bios filename
qemuparams=<xyz> - specify custom parameters to QEMU
bootparams=<xyz> - specify custom kernel parameters during boot
help, -h, --help: print this text
Examples:
runqemu
runqemu qemuarm
runqemu tmp/deploy/images/qemuarm
runqemu tmp/deploy/images/qemux86/<qemuboot.conf>
runqemu qemux86-64 core-image-sato ext4
runqemu qemux86-64 wic-image-minimal wic
runqemu path/to/bzImage-qemux86.bin path/to/nfsrootdir/ serial
runqemu qemux86 iso/hddimg/wic.vmdk/wic.qcow2/wic.vdi/ramfs/cpio.gz...
runqemu qemux86 qemuparams="-m 256"
runqemu qemux86 bootparams="psplash=false"
runqemu path/to/<image>-<machine>.wic
runqemu path/to/<image>-<machine>.wic.vmdk
``runqemu`` Command-Line Options
================================
Following is a description of ``runqemu`` options you can provide on the
command line:
.. note::
If you do provide some "illegal" option combination or perhaps you do
not provide enough in the way of options, ``runqemu``
provides appropriate error messaging to help you correct the problem.
- `QEMUARCH`: The QEMU machine architecture, which must be "qemuarm",
"qemuarm64", "qemumips", "qemumips64", "qemuppc", "qemux86", or
"qemux86-64".
- `VM`: The virtual machine image, which must be a ``.wic.vmdk``
file. Use this option when you want to boot a ``.wic.vmdk`` image.
The image filename you provide must contain one of the following
strings: "qemux86-64", "qemux86", "qemuarm", "qemumips64",
"qemumips", "qemuppc", or "qemush4".
- `ROOTFS`: A root filesystem that has one of the following filetype
extensions: "ext2", "ext3", "ext4", "jffs2", "nfs", or "btrfs". If
the filename you provide for this option uses "nfs", it must provide
an explicit root filesystem path.
- `KERNEL`: A kernel image, which is a ``.bin`` file. When you provide a
``.bin`` file, ``runqemu`` detects it and assumes the file is a
kernel image.
- `MACHINE`: The architecture of the QEMU machine, which must be one of
the following: "qemux86", "qemux86-64", "qemuarm", "qemuarm64",
"qemumips", "qemumips64", or "qemuppc". The MACHINE and QEMUARCH
options are basically identical. If you do not provide a MACHINE
option, ``runqemu`` tries to determine it based on other options.
- ``ramfs``: Indicates you are booting an :term:`Initramfs`
image, which means the ``FSTYPE`` is ``cpio.gz``.
- ``iso``: Indicates you are booting an ISO image, which means the
``FSTYPE`` is ``.iso``.
- ``nographic``: Disables the video console, which sets the console to
"ttys0". This option is useful when you have logged into a server and
you do not want to disable forwarding from the X Window System (X11)
to your workstation or laptop.
- ``serial``: Enables a serial console on ``/dev/ttyS0``.
- ``biosdir``: Establishes a custom directory for BIOS, VGA BIOS and
keymaps.
- ``biosfilename``: Establishes a custom BIOS name.
- ``qemuparams=\"xyz\"``: Specifies custom QEMU parameters. Use this
option to pass options other than the simple "kvm" and "serial"
options.
- ``bootparams=\"xyz\"``: Specifies custom boot parameters for the
kernel.
- ``audio``: Enables audio in QEMU. The MACHINE option must be either
"qemux86" or "qemux86-64" in order for audio to be enabled.
Additionally, the ``snd_intel8x0`` or ``snd_ens1370`` driver must be
installed in linux guest.
- ``slirp``: Enables "slirp" networking, which is a different way of
networking that does not need root access but also is not as easy to
use or comprehensive as the default.
Using ``slirp`` by default will forward the guest machine's
22 and 23 TCP ports to host machine's 2222 and 2323 ports
(or the next free ports). Specific forwarding rules can be configured
by setting ``QB_SLIRP_OPT`` as environment variable or in ``qemuboot.conf``
in the :term:`Build Directory` ``deploy/image`` directory.
Examples::
QB_SLIRP_OPT="-netdev user,id=net0,hostfwd=tcp::8080-:80"
QB_SLIRP_OPT="-netdev user,id=net0,hostfwd=tcp::8080-:80,hostfwd=tcp::2222-:22"
The first example forwards TCP port 80 from the emulated system to
port 8080 (or the next free port) on the host system,
allowing access to an http server running in QEMU from
``http://<host ip>:8080/``.
The second example does the same, but also forwards TCP port 22 on the
guest system to 2222 (or the next free port) on the host system,
allowing ssh access to the emulated system using
``ssh -P 2222 <user>@<host ip>``.
Keep in mind that proper configuration of firewall software is required.
- ``kvm``: Enables KVM when running "qemux86" or "qemux86-64" QEMU
architectures. For KVM to work, all the following conditions must be
met:
- Your MACHINE must be either qemux86" or "qemux86-64".
- Your build host has to have the KVM modules installed, which are
``/dev/kvm``.
- The build host ``/dev/kvm`` directory has to be both writable and
readable.
- ``kvm-vhost``: Enables KVM with VHOST support when running "qemux86"
or "qemux86-64" QEMU architectures. For KVM with VHOST to work, the
following conditions must be met:
- ``kvm`` option conditions defined above must be met.
- Your build host has to have virtio net device, which are
``/dev/vhost-net``.
- The build host ``/dev/vhost-net`` directory has to be either
readable or writable and "slirp-enabled".
- ``publicvnc``: Enables a VNC server open to all hosts.
poky/documentation/dev-manual/quilt.rst
+89
View File
@@ -0,0 +1,89 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Using Quilt in Your Workflow
****************************
`Quilt <https://savannah.nongnu.org/projects/quilt>`__ is a powerful tool
that allows you to capture source code changes without having a clean
source tree. This section outlines the typical workflow you can use to
modify source code, test changes, and then preserve the changes in the
form of a patch all using Quilt.
.. note::
With regard to preserving changes to source files, if you clean a
recipe or have :ref:`ref-classes-rm-work` enabled, the
:ref:`devtool workflow <sdk-manual/extensible:using \`\`devtool\`\` in your sdk workflow>`
as described in the Yocto Project Application Development and the
Extensible Software Development Kit (eSDK) manual is a safer
development flow than the flow that uses Quilt.
Follow these general steps:
#. *Find the Source Code:* Temporary source code used by the
OpenEmbedded build system is kept in the :term:`Build Directory`. See the
":ref:`dev-manual/temporary-source-code:finding temporary source code`" section to
learn how to locate the directory that has the temporary source code for a
particular package.
#. *Change Your Working Directory:* You need to be in the directory that
has the temporary source code. That directory is defined by the
:term:`S` variable.
#. *Create a New Patch:* Before modifying source code, you need to
create a new patch. To create a new patch file, use ``quilt new`` as
below::
$ quilt new my_changes.patch
#. *Notify Quilt and Add Files:* After creating the patch, you need to
notify Quilt about the files you plan to edit. You notify Quilt by
adding the files to the patch you just created::
$ quilt add file1.c file2.c file3.c
#. *Edit the Files:* Make your changes in the source code to the files
you added to the patch.
#. *Test Your Changes:* Once you have modified the source code, the
easiest way to test your changes is by calling the :ref:`ref-tasks-compile`
task as shown in the following example::
$ bitbake -c compile -f package
The ``-f`` or ``--force`` option forces the specified task to
execute. If you find problems with your code, you can just keep
editing and re-testing iteratively until things work as expected.
.. note::
All the modifications you make to the temporary source code disappear
once you run the :ref:`ref-tasks-clean` or :ref:`ref-tasks-cleanall`
tasks using BitBake (i.e. ``bitbake -c clean package`` and
``bitbake -c cleanall package``). Modifications will also disappear if
you use the :ref:`ref-classes-rm-work` feature as described in
the ":ref:`dev-manual/disk-space:conserving disk space during builds`"
section.
#. *Generate the Patch:* Once your changes work as expected, you need to
use Quilt to generate the final patch that contains all your
modifications::
$ quilt refresh
At this point, the
``my_changes.patch`` file has all your edits made to the ``file1.c``,
``file2.c``, and ``file3.c`` files.
You can find the resulting patch file in the ``patches/``
subdirectory of the source (:term:`S`) directory.
#. *Copy the Patch File:* For simplicity, copy the patch file into a
directory named ``files``, which you can create in the same directory
that holds the recipe (``.bb``) file or the append (``.bbappend``)
file. Placing the patch here guarantees that the OpenEmbedded build
system will find the patch. Next, add the patch into the :term:`SRC_URI`
of the recipe. Here is an example::
SRC_URI += "file://my_changes.patch"
poky/documentation/dev-manual/read-only-rootfs.rst
+89
View File
@@ -0,0 +1,89 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Creating a Read-Only Root Filesystem
************************************
Suppose, for security reasons, you need to disable your target device's
root filesystem's write permissions (i.e. you need a read-only root
filesystem). Or, perhaps you are running the device's operating system
from a read-only storage device. For either case, you can customize your
image for that behavior.
.. note::
Supporting a read-only root filesystem requires that the system and
applications do not try to write to the root filesystem. You must
configure all parts of the target system to write elsewhere, or to
gracefully fail in the event of attempting to write to the root
filesystem.
Creating the Root Filesystem
============================
To create the read-only root filesystem, simply add the
"read-only-rootfs" feature to your image, normally in one of two ways.
The first way is to add the "read-only-rootfs" image feature in the
image's recipe file via the :term:`IMAGE_FEATURES` variable::
IMAGE_FEATURES += "read-only-rootfs"
As an alternative, you can add the same feature
from within your :term:`Build Directory`'s ``local.conf`` file with the
associated :term:`EXTRA_IMAGE_FEATURES` variable, as in::
EXTRA_IMAGE_FEATURES = "read-only-rootfs"
For more information on how to use these variables, see the
":ref:`dev-manual/customizing-images:Customizing Images Using Custom \`\`IMAGE_FEATURES\`\` and \`\`EXTRA_IMAGE_FEATURES\`\``"
section. For information on the variables, see
:term:`IMAGE_FEATURES` and
:term:`EXTRA_IMAGE_FEATURES`.
Post-Installation Scripts and Read-Only Root Filesystem
=======================================================
It is very important that you make sure all post-Installation
(``pkg_postinst``) scripts for packages that are installed into the
image can be run at the time when the root filesystem is created during
the build on the host system. These scripts cannot attempt to run during
the first boot on the target device. With the "read-only-rootfs" feature
enabled, the build system makes sure that all post-installation scripts
succeed at file system creation time. If any of these scripts
still need to be run after the root filesystem is created, the build
immediately fails. These build-time checks ensure that the build fails
rather than the target device fails later during its initial boot
operation.
Most of the common post-installation scripts generated by the build
system for the out-of-the-box Yocto Project are engineered so that they
can run during root filesystem creation (e.g. post-installation scripts
for caching fonts). However, if you create and add custom scripts, you
need to be sure they can be run during this file system creation.
Here are some common problems that prevent post-installation scripts
from running during root filesystem creation:
- *Not using $D in front of absolute paths:* The build system defines
``$``\ :term:`D` when the root
filesystem is created. Furthermore, ``$D`` is blank when the script
is run on the target device. This implies two purposes for ``$D``:
ensuring paths are valid in both the host and target environments,
and checking to determine which environment is being used as a method
for taking appropriate actions.
- *Attempting to run processes that are specific to or dependent on the
target architecture:* You can work around these attempts by using
native tools, which run on the host system, to accomplish the same
tasks, or by alternatively running the processes under QEMU, which
has the ``qemu_run_binary`` function. For more information, see the
:ref:`ref-classes-qemu` class.
Areas With Write Access
=======================
With the "read-only-rootfs" feature enabled, any attempt by the target
to write to the root filesystem at runtime fails. Consequently, you must
make sure that you configure processes and applications that attempt
these types of writes do so to directories with write access (e.g.
``/tmp`` or ``/var/run``).
poky/documentation/dev-manual/runtime-testing.rst
+598
View File
@@ -0,0 +1,598 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Performing Automated Runtime Testing
************************************
The OpenEmbedded build system makes available a series of automated
tests for images to verify runtime functionality. You can run these
tests on either QEMU or actual target hardware. Tests are written in
Python making use of the ``unittest`` module, and the majority of them
run commands on the target system over SSH. This section describes how
you set up the environment to use these tests, run available tests, and
write and add your own tests.
For information on the test and QA infrastructure available within the
Yocto Project, see the ":ref:`ref-manual/release-process:testing and quality assurance`"
section in the Yocto Project Reference Manual.
Enabling Tests
==============
Depending on whether you are planning to run tests using QEMU or on the
hardware, you have to take different steps to enable the tests. See the
following subsections for information on how to enable both types of
tests.
Enabling Runtime Tests on QEMU
------------------------------
In order to run tests, you need to do the following:
- *Set up to avoid interaction with sudo for networking:* To
accomplish this, you must do one of the following:
- Add ``NOPASSWD`` for your user in ``/etc/sudoers`` either for all
commands or just for ``runqemu-ifup``. You must provide the full
path as that can change if you are using multiple clones of the
source repository.
.. note::
On some distributions, you also need to comment out "Defaults
requiretty" in ``/etc/sudoers``.
- Manually configure a tap interface for your system.
- Run as root the script in ``scripts/runqemu-gen-tapdevs``, which
should generate a list of tap devices. This is the option
typically chosen for Autobuilder-type environments.
.. note::
- Be sure to use an absolute path when calling this script
with sudo.
- The package recipe ``qemu-helper-native`` is required to run
this script. Build the package using the following command::
$ bitbake qemu-helper-native
- *Set the DISPLAY variable:* You need to set this variable so that
you have an X server available (e.g. start ``vncserver`` for a
headless machine).
- *Be sure your host's firewall accepts incoming connections from
192.168.7.0/24:* Some of the tests (in particular DNF tests) start an
HTTP server on a random high number port, which is used to serve
files to the target. The DNF module serves
``${WORKDIR}/oe-rootfs-repo`` so it can run DNF channel commands.
That means your host's firewall must accept incoming connections from
192.168.7.0/24, which is the default IP range used for tap devices by
``runqemu``.
- *Be sure your host has the correct packages installed:* Depending
your host's distribution, you need to have the following packages
installed:
- Ubuntu and Debian: ``sysstat`` and ``iproute2``
- openSUSE: ``sysstat`` and ``iproute2``
- Fedora: ``sysstat`` and ``iproute``
- CentOS: ``sysstat`` and ``iproute``
Once you start running the tests, the following happens:
#. A copy of the root filesystem is written to ``${WORKDIR}/testimage``.
#. The image is booted under QEMU using the standard ``runqemu`` script.
#. A default timeout of 500 seconds occurs to allow for the boot process
to reach the login prompt. You can change the timeout period by
setting
:term:`TEST_QEMUBOOT_TIMEOUT`
in the ``local.conf`` file.
#. Once the boot process is reached and the login prompt appears, the
tests run. The full boot log is written to
``${WORKDIR}/testimage/qemu_boot_log``.
#. Each test module loads in the order found in :term:`TEST_SUITES`. You can
find the full output of the commands run over SSH in
``${WORKDIR}/testimgage/ssh_target_log``.
#. If no failures occur, the task running the tests ends successfully.
You can find the output from the ``unittest`` in the task log at
``${WORKDIR}/temp/log.do_testimage``.
Enabling Runtime Tests on Hardware
----------------------------------
The OpenEmbedded build system can run tests on real hardware, and for
certain devices it can also deploy the image to be tested onto the
device beforehand.
For automated deployment, a "controller image" is installed onto the
hardware once as part of setup. Then, each time tests are to be run, the
following occurs:
#. The controller image is booted into and used to write the image to be
tested to a second partition.
#. The device is then rebooted using an external script that you need to
provide.
#. The device boots into the image to be tested.
When running tests (independent of whether the image has been deployed
automatically or not), the device is expected to be connected to a
network on a pre-determined IP address. You can either use static IP
addresses written into the image, or set the image to use DHCP and have
your DHCP server on the test network assign a known IP address based on
the MAC address of the device.
In order to run tests on hardware, you need to set :term:`TEST_TARGET` to an
appropriate value. For QEMU, you do not have to change anything, the
default value is "qemu". For running tests on hardware, the following
options are available:
- *"simpleremote":* Choose "simpleremote" if you are going to run tests
on a target system that is already running the image to be tested and
is available on the network. You can use "simpleremote" in
conjunction with either real hardware or an image running within a
separately started QEMU or any other virtual machine manager.
- *"SystemdbootTarget":* Choose "SystemdbootTarget" if your hardware is
an EFI-based machine with ``systemd-boot`` as bootloader and
``core-image-testmaster`` (or something similar) is installed. Also,
your hardware under test must be in a DHCP-enabled network that gives
it the same IP address for each reboot.
If you choose "SystemdbootTarget", there are additional requirements
and considerations. See the
":ref:`dev-manual/runtime-testing:selecting systemdboottarget`" section, which
follows, for more information.
- *"BeagleBoneTarget":* Choose "BeagleBoneTarget" if you are deploying
images and running tests on the BeagleBone "Black" or original
"White" hardware. For information on how to use these tests, see the
comments at the top of the BeagleBoneTarget
``meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py`` file.
- *"EdgeRouterTarget":* Choose "EdgeRouterTarget" if you are deploying
images and running tests on the Ubiquiti Networks EdgeRouter Lite.
For information on how to use these tests, see the comments at the
top of the EdgeRouterTarget
``meta-yocto-bsp/lib/oeqa/controllers/edgeroutertarget.py`` file.
- *"GrubTarget":* Choose "GrubTarget" if you are deploying images and running
tests on any generic PC that boots using GRUB. For information on how
to use these tests, see the comments at the top of the GrubTarget
``meta-yocto-bsp/lib/oeqa/controllers/grubtarget.py`` file.
- *"your-target":* Create your own custom target if you want to run
tests when you are deploying images and running tests on a custom
machine within your BSP layer. To do this, you need to add a Python
unit that defines the target class under ``lib/oeqa/controllers/``
within your layer. You must also provide an empty ``__init__.py``.
For examples, see files in ``meta-yocto-bsp/lib/oeqa/controllers/``.
Selecting SystemdbootTarget
---------------------------
If you did not set :term:`TEST_TARGET` to "SystemdbootTarget", then you do
not need any information in this section. You can skip down to the
":ref:`dev-manual/runtime-testing:running tests`" section.
If you did set :term:`TEST_TARGET` to "SystemdbootTarget", you also need to
perform a one-time setup of your controller image by doing the following:
#. *Set EFI_PROVIDER:* Be sure that :term:`EFI_PROVIDER` is as follows::
EFI_PROVIDER = "systemd-boot"
#. *Build the controller image:* Build the ``core-image-testmaster`` image.
The ``core-image-testmaster`` recipe is provided as an example for a
"controller" image and you can customize the image recipe as you would
any other recipe.
Here are the image recipe requirements:
- Inherits ``core-image`` so that kernel modules are installed.
- Installs normal linux utilities not BusyBox ones (e.g. ``bash``,
``coreutils``, ``tar``, ``gzip``, and ``kmod``).
- Uses a custom :term:`Initramfs` image with a custom
installer. A normal image that you can install usually creates a
single root filesystem partition. This image uses another installer that
creates a specific partition layout. Not all Board Support
Packages (BSPs) can use an installer. For such cases, you need to
manually create the following partition layout on the target:
- First partition mounted under ``/boot``, labeled "boot".
- The main root filesystem partition where this image gets installed,
which is mounted under ``/``.
- Another partition labeled "testrootfs" where test images get
deployed.
#. *Install image:* Install the image that you just built on the target
system.
The final thing you need to do when setting :term:`TEST_TARGET` to
"SystemdbootTarget" is to set up the test image:
#. *Set up your local.conf file:* Make sure you have the following
statements in your ``local.conf`` file::
IMAGE_FSTYPES += "tar.gz"
INHERIT += "testimage"
TEST_TARGET = "SystemdbootTarget"
TEST_TARGET_IP = "192.168.2.3"
#. *Build your test image:* Use BitBake to build the image::
$ bitbake core-image-sato
Power Control
-------------
For most hardware targets other than "simpleremote", you can control
power:
- You can use :term:`TEST_POWERCONTROL_CMD` together with
:term:`TEST_POWERCONTROL_EXTRA_ARGS` as a command that runs on the host
and does power cycling. The test code passes one argument to that
command: off, on or cycle (off then on). Here is an example that
could appear in your ``local.conf`` file::
TEST_POWERCONTROL_CMD = "powercontrol.exp test 10.11.12.1 nuc1"
In this example, the expect
script does the following:
.. code-block:: shell
ssh test@10.11.12.1 "pyctl nuc1 arg"
It then runs a Python script that controls power for a label called
``nuc1``.
.. note::
You need to customize :term:`TEST_POWERCONTROL_CMD` and
:term:`TEST_POWERCONTROL_EXTRA_ARGS` for your own setup. The one requirement
is that it accepts "on", "off", and "cycle" as the last argument.
- When no command is defined, it connects to the device over SSH and
uses the classic reboot command to reboot the device. Classic reboot
is fine as long as the machine actually reboots (i.e. the SSH test
has not failed). It is useful for scenarios where you have a simple
setup, typically with a single board, and where some manual
interaction is okay from time to time.
If you have no hardware to automatically perform power control but still
wish to experiment with automated hardware testing, you can use the
``dialog-power-control`` script that shows a dialog prompting you to perform
the required power action. This script requires either KDialog or Zenity
to be installed. To use this script, set the
:term:`TEST_POWERCONTROL_CMD`
variable as follows::
TEST_POWERCONTROL_CMD = "${COREBASE}/scripts/contrib/dialog-power-control"
Serial Console Connection
-------------------------
For test target classes requiring a serial console to interact with the
bootloader (e.g. BeagleBoneTarget, EdgeRouterTarget, and GrubTarget),
you need to specify a command to use to connect to the serial console of
the target machine by using the
:term:`TEST_SERIALCONTROL_CMD`
variable and optionally the
:term:`TEST_SERIALCONTROL_EXTRA_ARGS`
variable.
These cases could be a serial terminal program if the machine is
connected to a local serial port, or a ``telnet`` or ``ssh`` command
connecting to a remote console server. Regardless of the case, the
command simply needs to connect to the serial console and forward that
connection to standard input and output as any normal terminal program
does. For example, to use the picocom terminal program on serial device
``/dev/ttyUSB0`` at 115200bps, you would set the variable as follows::
TEST_SERIALCONTROL_CMD = "picocom /dev/ttyUSB0 -b 115200"
For local
devices where the serial port device disappears when the device reboots,
an additional "serdevtry" wrapper script is provided. To use this
wrapper, simply prefix the terminal command with
``${COREBASE}/scripts/contrib/serdevtry``::
TEST_SERIALCONTROL_CMD = "${COREBASE}/scripts/contrib/serdevtry picocom -b 115200 /dev/ttyUSB0"
Running Tests
=============
You can start the tests automatically or manually:
- *Automatically running tests:* To run the tests automatically after the
OpenEmbedded build system successfully creates an image, first set the
:term:`TESTIMAGE_AUTO` variable to "1" in your ``local.conf`` file in the
:term:`Build Directory`::
TESTIMAGE_AUTO = "1"
Next, build your image. If the image successfully builds, the
tests run::
bitbake core-image-sato
- *Manually running tests:* To manually run the tests, first globally
inherit the :ref:`ref-classes-testimage` class by editing your
``local.conf`` file::
INHERIT += "testimage"
Next, use BitBake to run the tests::
bitbake -c testimage image
All test files reside in ``meta/lib/oeqa/runtime/cases`` in the
:term:`Source Directory`. A test name maps
directly to a Python module. Each test module may contain a number of
individual tests. Tests are usually grouped together by the area tested
(e.g tests for systemd reside in ``meta/lib/oeqa/runtime/cases/systemd.py``).
You can add tests to any layer provided you place them in the proper
area and you extend :term:`BBPATH` in
the ``local.conf`` file as normal. Be sure that tests reside in
``layer/lib/oeqa/runtime/cases``.
.. note::
Be sure that module names do not collide with module names used in
the default set of test modules in ``meta/lib/oeqa/runtime/cases``.
You can change the set of tests run by appending or overriding
:term:`TEST_SUITES` variable in
``local.conf``. Each name in :term:`TEST_SUITES` represents a required test
for the image. Test modules named within :term:`TEST_SUITES` cannot be
skipped even if a test is not suitable for an image (e.g. running the
RPM tests on an image without ``rpm``). Appending "auto" to
:term:`TEST_SUITES` causes the build system to try to run all tests that are
suitable for the image (i.e. each test module may elect to skip itself).
The order you list tests in :term:`TEST_SUITES` is important and influences
test dependencies. Consequently, tests that depend on other tests should
be added after the test on which they depend. For example, since the
``ssh`` test depends on the ``ping`` test, "ssh" needs to come after
"ping" in the list. The test class provides no re-ordering or dependency
handling.
.. note::
Each module can have multiple classes with multiple test methods.
And, Python ``unittest`` rules apply.
Here are some things to keep in mind when running tests:
- The default tests for the image are defined as::
DEFAULT_TEST_SUITES:pn-image = "ping ssh df connman syslog xorg scp vnc date rpm dnf dmesg"
- Add your own test to the list of the by using the following::
TEST_SUITES:append = " mytest"
- Run a specific list of tests as follows::
TEST_SUITES = "test1 test2 test3"
Remember, order is important. Be sure to place a test that is
dependent on another test later in the order.
Exporting Tests
===============
You can export tests so that they can run independently of the build
system. Exporting tests is required if you want to be able to hand the
test execution off to a scheduler. You can only export tests that are
defined in :term:`TEST_SUITES`.
If your image is already built, make sure the following are set in your
``local.conf`` file::
INHERIT += "testexport"
TEST_TARGET_IP = "IP-address-for-the-test-target"
TEST_SERVER_IP = "IP-address-for-the-test-server"
You can then export the tests with the
following BitBake command form::
$ bitbake image -c testexport
Exporting the tests places them in the :term:`Build Directory` in
``tmp/testexport/``\ image, which is controlled by the :term:`TEST_EXPORT_DIR`
variable.
You can now run the tests outside of the build environment::
$ cd tmp/testexport/image
$ ./runexported.py testdata.json
Here is a complete example that shows IP addresses and uses the
``core-image-sato`` image::
INHERIT += "testexport"
TEST_TARGET_IP = "192.168.7.2"
TEST_SERVER_IP = "192.168.7.1"
Use BitBake to export the tests::
$ bitbake core-image-sato -c testexport
Run the tests outside of
the build environment using the following::
$ cd tmp/testexport/core-image-sato
$ ./runexported.py testdata.json
Writing New Tests
=================
As mentioned previously, all new test files need to be in the proper
place for the build system to find them. New tests for additional
functionality outside of the core should be added to the layer that adds
the functionality, in ``layer/lib/oeqa/runtime/cases`` (as long as
:term:`BBPATH` is extended in the
layer's ``layer.conf`` file as normal). Just remember the following:
- Filenames need to map directly to test (module) names.
- Do not use module names that collide with existing core tests.
- Minimally, an empty ``__init__.py`` file must be present in the runtime
directory.
To create a new test, start by copying an existing module (e.g.
``syslog.py`` or ``gcc.py`` are good ones to use). Test modules can use
code from ``meta/lib/oeqa/utils``, which are helper classes.
.. note::
Structure shell commands such that you rely on them and they return a
single code for success. Be aware that sometimes you will need to
parse the output. See the ``df.py`` and ``date.py`` modules for examples.
You will notice that all test classes inherit ``oeRuntimeTest``, which
is found in ``meta/lib/oetest.py``. This base class offers some helper
attributes, which are described in the following sections:
Class Methods
-------------
Class methods are as follows:
- *hasPackage(pkg):* Returns "True" if ``pkg`` is in the installed
package list of the image, which is based on the manifest file that
is generated during the :ref:`ref-tasks-rootfs` task.
- *hasFeature(feature):* Returns "True" if the feature is in
:term:`IMAGE_FEATURES` or
:term:`DISTRO_FEATURES`.
Class Attributes
----------------
Class attributes are as follows:
- *pscmd:* Equals "ps -ef" if ``procps`` is installed in the image.
Otherwise, ``pscmd`` equals "ps" (busybox).
- *tc:* The called test context, which gives access to the
following attributes:
- *d:* The BitBake datastore, which allows you to use stuff such
as ``oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager")``.
- *testslist and testsrequired:* Used internally. The tests
do not need these.
- *filesdir:* The absolute path to
``meta/lib/oeqa/runtime/files``, which contains helper files for
tests meant for copying on the target such as small files written
in C for compilation.
- *target:* The target controller object used to deploy and
start an image on a particular target (e.g. Qemu, SimpleRemote,
and SystemdbootTarget). Tests usually use the following:
- *ip:* The target's IP address.
- *server_ip:* The host's IP address, which is usually used
by the DNF test suite.
- *run(cmd, timeout=None):* The single, most used method.
This command is a wrapper for: ``ssh root@host "cmd"``. The
command returns a tuple: (status, output), which are what their
names imply - the return code of "cmd" and whatever output it
produces. The optional timeout argument represents the number
of seconds the test should wait for "cmd" to return. If the
argument is "None", the test uses the default instance's
timeout period, which is 300 seconds. If the argument is "0",
the test runs until the command returns.
- *copy_to(localpath, remotepath):*
``scp localpath root@ip:remotepath``.
- *copy_from(remotepath, localpath):*
``scp root@host:remotepath localpath``.
Instance Attributes
-------------------
There is a single instance attribute, which is ``target``. The ``target``
instance attribute is identical to the class attribute of the same name,
which is described in the previous section. This attribute exists as
both an instance and class attribute so tests can use
``self.target.run(cmd)`` in instance methods instead of
``oeRuntimeTest.tc.target.run(cmd)``.
Installing Packages in the DUT Without the Package Manager
==========================================================
When a test requires a package built by BitBake, it is possible to
install that package. Installing the package does not require a package
manager be installed in the device under test (DUT). It does, however,
require an SSH connection and the target must be using the
``sshcontrol`` class.
.. note::
This method uses ``scp`` to copy files from the host to the target, which
causes permissions and special attributes to be lost.
A JSON file is used to define the packages needed by a test. This file
must be in the same path as the file used to define the tests.
Furthermore, the filename must map directly to the test module name with
a ``.json`` extension.
The JSON file must include an object with the test name as keys of an
object or an array. This object (or array of objects) uses the following
data:
- "pkg" --- a mandatory string that is the name of the package to be
installed.
- "rm" --- an optional boolean, which defaults to "false", that specifies
to remove the package after the test.
- "extract" --- an optional boolean, which defaults to "false", that
specifies if the package must be extracted from the package format.
When set to "true", the package is not automatically installed into
the DUT.
Following is an example JSON file that handles test "foo" installing
package "bar" and test "foobar" installing packages "foo" and "bar".
Once the test is complete, the packages are removed from the DUT::
{
"foo": {
"pkg": "bar"
},
"foobar": [
{
"pkg": "foo",
"rm": true
},
{
"pkg": "bar",
"rm": true
}
]
}
poky/documentation/dev-manual/sbom.rst
+75
View File
@@ -0,0 +1,75 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Creating a Software Bill of Materials
*************************************
Once you are able to build an image for your project, once the licenses for
each software component are all identified (see
":ref:`dev-manual/licenses:working with licenses`") and once vulnerability
fixes are applied (see ":ref:`dev-manual/vulnerabilities:checking
for vulnerabilities`"), the OpenEmbedded build system can generate
a description of all the components you used, their licenses, their dependencies,
their sources, the changes that were applied to them and the known
vulnerabilities that were fixed.
This description is generated in the form of a *Software Bill of Materials*
(:term:`SBOM`), using the :term:`SPDX` standard.
When you release software, this is the most standard way to provide information
about the Software Supply Chain of your software image and SDK. The
:term:`SBOM` tooling is often used to ensure open source license compliance by
providing the license texts used in the product which legal departments and end
users can read in standardized format.
:term:`SBOM` information is also critical to performing vulnerability exposure
assessments, as all the components used in the Software Supply Chain are listed.
The OpenEmbedded build system doesn't generate such information by default.
To make this happen, you must inherit the
:ref:`ref-classes-create-spdx` class from a configuration file::
INHERIT += "create-spdx"
You then get :term:`SPDX` output in JSON format as an
``IMAGE-MACHINE.spdx.json`` file in ``tmp/deploy/images/MACHINE/`` inside the
:term:`Build Directory`.
This is a toplevel file accompanied by an ``IMAGE-MACHINE.spdx.index.json``
containing an index of JSON :term:`SPDX` files for individual recipes, together
with an ``IMAGE-MACHINE.spdx.tar.zst`` compressed archive containing all such
files.
The :ref:`ref-classes-create-spdx` class offers options to include
more information in the output :term:`SPDX` data, such as making the generated
files more human readable (:term:`SPDX_PRETTY`), adding compressed archives of
the files in the generated target packages (:term:`SPDX_ARCHIVE_PACKAGED`),
adding a description of the source files used to generate host tools and target
packages (:term:`SPDX_INCLUDE_SOURCES`) and adding archives of these source
files themselves (:term:`SPDX_ARCHIVE_SOURCES`).
Though the toplevel :term:`SPDX` output is available in
``tmp/deploy/images/MACHINE/`` inside the :term:`Build Directory`, ancillary
generated files are available in ``tmp/deploy/spdx/MACHINE`` too, such as:
- The individual :term:`SPDX` JSON files in the ``IMAGE-MACHINE.spdx.tar.zst``
archive.
- Compressed archives of the files in the generated target packages,
in ``packages/packagename.tar.zst`` (when :term:`SPDX_ARCHIVE_PACKAGED`
is set).
- Compressed archives of the source files used to build the host tools
and the target packages in ``recipes/recipe-packagename.tar.zst``
(when :term:`SPDX_ARCHIVE_SOURCES` is set). Those are needed to fulfill
"source code access" license requirements.
See also the :term:`SPDX_CUSTOM_ANNOTATION_VARS` variable which allows
to associate custom notes to a recipe.
See the `tools page <https://spdx.dev/resources/tools/>`__ on the :term:`SPDX`
project website for a list of tools to consume and transform the :term:`SPDX`
data generated by the OpenEmbedded build system.
See also Joshua Watt's
`Automated SBoM generation with OpenEmbedded and the Yocto Project <https://youtu.be/Q5UQUM6zxVU>`__
presentation at FOSDEM 2023.
poky/documentation/dev-manual/securing-images.rst
+156
View File
@@ -0,0 +1,156 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Making Images More Secure
*************************
Security is of increasing concern for embedded devices. Consider the
issues and problems discussed in just this sampling of work found across
the Internet:
- *"*\ `Security Risks of Embedded
Systems <https://www.schneier.com/blog/archives/2014/01/security_risks_9.html>`__\ *"*
by Bruce Schneier
- *"*\ `Internet Census
2012 <http://census2012.sourceforge.net/paper.html>`__\ *"* by Carna
Botnet
- *"*\ `Security Issues for Embedded
Devices <https://elinux.org/images/6/6f/Security-issues.pdf>`__\ *"*
by Jake Edge
When securing your image is of concern, there are steps, tools, and
variables that you can consider to help you reach the security goals you
need for your particular device. Not all situations are identical when
it comes to making an image secure. Consequently, this section provides
some guidance and suggestions for consideration when you want to make
your image more secure.
.. note::
Because the security requirements and risks are different for every
type of device, this section cannot provide a complete reference on
securing your custom OS. It is strongly recommended that you also
consult other sources of information on embedded Linux system
hardening and on security.
General Considerations
======================
There are general considerations that help you create more secure images.
You should consider the following suggestions to make your device
more secure:
- Scan additional code you are adding to the system (e.g. application
code) by using static analysis tools. Look for buffer overflows and
other potential security problems.
- Pay particular attention to the security for any web-based
administration interface.
Web interfaces typically need to perform administrative functions and
tend to need to run with elevated privileges. Thus, the consequences
resulting from the interface's security becoming compromised can be
serious. Look for common web vulnerabilities such as
cross-site-scripting (XSS), unvalidated inputs, and so forth.
As with system passwords, the default credentials for accessing a
web-based interface should not be the same across all devices. This
is particularly true if the interface is enabled by default as it can
be assumed that many end-users will not change the credentials.
- Ensure you can update the software on the device to mitigate
vulnerabilities discovered in the future. This consideration
especially applies when your device is network-enabled.
- Regularly scan and apply fixes for CVE security issues affecting
all software components in the product, see ":ref:`dev-manual/vulnerabilities:checking for vulnerabilities`".
- Regularly update your version of Poky and OE-Core from their upstream
developers, e.g. to apply updates and security fixes from stable
and :term:`LTS` branches.
- Ensure you remove or disable debugging functionality before producing
the final image. For information on how to do this, see the
":ref:`dev-manual/securing-images:considerations specific to the openembedded build system`"
section.
- Ensure you have no network services listening that are not needed.
- Remove any software from the image that is not needed.
- Enable hardware support for secure boot functionality when your
device supports this functionality.
Security Flags
==============
The Yocto Project has security flags that you can enable that help make
your build output more secure. The security flags are in the
``meta/conf/distro/include/security_flags.inc`` file in your
:term:`Source Directory` (e.g. ``poky``).
.. note::
Depending on the recipe, certain security flags are enabled and
disabled by default.
Use the following line in your ``local.conf`` file or in your custom
distribution configuration file to enable the security compiler and
linker flags for your build::
require conf/distro/include/security_flags.inc
Considerations Specific to the OpenEmbedded Build System
========================================================
You can take some steps that are specific to the OpenEmbedded build
system to make your images more secure:
- Ensure "debug-tweaks" is not one of your selected
:term:`IMAGE_FEATURES`.
When creating a new project, the default is to provide you with an
initial ``local.conf`` file that enables this feature using the
:term:`EXTRA_IMAGE_FEATURES`
variable with the line::
EXTRA_IMAGE_FEATURES = "debug-tweaks"
To disable that feature, simply comment out that line in your
``local.conf`` file, or make sure :term:`IMAGE_FEATURES` does not contain
"debug-tweaks" before producing your final image. Among other things,
leaving this in place sets the root password as blank, which makes
logging in for debugging or inspection easy during development but
also means anyone can easily log in during production.
- It is possible to set a root password for the image and also to set
passwords for any extra users you might add (e.g. administrative or
service type users). When you set up passwords for multiple images or
users, you should not duplicate passwords.
To set up passwords, use the :ref:`ref-classes-extrausers` class, which
is the preferred method. For an example on how to set up both root and
user passwords, see the ":ref:`ref-classes-extrausers`" section.
.. note::
When adding extra user accounts or setting a root password, be
cautious about setting the same password on every device. If you
do this, and the password you have set is exposed, then every
device is now potentially compromised. If you need this access but
want to ensure security, consider setting a different, random
password for each device. Typically, you do this as a separate
step after you deploy the image onto the device.
- Consider enabling a Mandatory Access Control (MAC) framework such as
SMACK or SELinux and tuning it appropriately for your device's usage.
You can find more information in the
:yocto_git:`meta-selinux </meta-selinux/>` layer.
Tools for Hardening Your Image
==============================
The Yocto Project provides tools for making your image more secure. You
can find these tools in the ``meta-security`` layer of the
:yocto_git:`Yocto Project Source Repositories <>`.
poky/documentation/dev-manual/speeding-up-build.rst
+109
View File
@@ -0,0 +1,109 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Speeding Up a Build
*******************
Build time can be an issue. By default, the build system uses simple
controls to try and maximize build efficiency. In general, the default
settings for all the following variables result in the most efficient
build times when dealing with single socket systems (i.e. a single CPU).
If you have multiple CPUs, you might try increasing the default values
to gain more speed. See the descriptions in the glossary for each
variable for more information:
- :term:`BB_NUMBER_THREADS`:
The maximum number of threads BitBake simultaneously executes.
- :term:`BB_NUMBER_PARSE_THREADS`:
The number of threads BitBake uses during parsing.
- :term:`PARALLEL_MAKE`: Extra
options passed to the ``make`` command during the
:ref:`ref-tasks-compile` task in
order to specify parallel compilation on the local build host.
- :term:`PARALLEL_MAKEINST`:
Extra options passed to the ``make`` command during the
:ref:`ref-tasks-install` task in
order to specify parallel installation on the local build host.
As mentioned, these variables all scale to the number of processor cores
available on the build system. For single socket systems, this
auto-scaling ensures that the build system fundamentally takes advantage
of potential parallel operations during the build based on the build
machine's capabilities.
Following are additional factors that can affect build speed:
- File system type: The file system type that the build is being
performed on can also influence performance. Using ``ext4`` is
recommended as compared to ``ext2`` and ``ext3`` due to ``ext4``
improved features such as extents.
- Disabling the updating of access time using ``noatime``: The
``noatime`` mount option prevents the build system from updating file
and directory access times.
- Setting a longer commit: Using the "commit=" mount option increases
the interval in seconds between disk cache writes. Changing this
interval from the five second default to something longer increases
the risk of data loss but decreases the need to write to the disk,
thus increasing the build performance.
- Choosing the packaging backend: Of the available packaging backends,
IPK is the fastest. Additionally, selecting a singular packaging
backend also helps.
- Using ``tmpfs`` for :term:`TMPDIR`
as a temporary file system: While this can help speed up the build,
the benefits are limited due to the compiler using ``-pipe``. The
build system goes to some lengths to avoid ``sync()`` calls into the
file system on the principle that if there was a significant failure,
the :term:`Build Directory` contents could easily be rebuilt.
- Inheriting the :ref:`ref-classes-rm-work` class:
Inheriting this class has shown to speed up builds due to
significantly lower amounts of data stored in the data cache as well
as on disk. Inheriting this class also makes cleanup of
:term:`TMPDIR` faster, at the
expense of being easily able to dive into the source code. File
system maintainers have recommended that the fastest way to clean up
large numbers of files is to reformat partitions rather than delete
files due to the linear nature of partitions. This, of course,
assumes you structure the disk partitions and file systems in a way
that this is practical.
Aside from the previous list, you should keep some trade offs in mind
that can help you speed up the build:
- Remove items from
:term:`DISTRO_FEATURES`
that you might not need.
- Exclude debug symbols and other debug information: If you do not need
these symbols and other debug information, disabling the ``*-dbg``
package generation can speed up the build. You can disable this
generation by setting the
:term:`INHIBIT_PACKAGE_DEBUG_SPLIT`
variable to "1".
- Disable static library generation for recipes derived from
``autoconf`` or ``libtool``: Following is an example showing how to
disable static libraries and still provide an override to handle
exceptions::
STATICLIBCONF = "--disable-static"
STATICLIBCONF:sqlite3-native = ""
EXTRA_OECONF += "${STATICLIBCONF}"
.. note::
- Some recipes need static libraries in order to work correctly
(e.g. ``pseudo-native`` needs ``sqlite3-native``). Overrides,
as in the previous example, account for these kinds of
exceptions.
- Some packages have packaging code that assumes the presence of
the static libraries. If so, you might need to exclude them as
well.
poky/documentation/dev-manual/start.rst
+858
View File
@@ -0,0 +1,858 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
***********************************
Setting Up to Use the Yocto Project
***********************************
This chapter provides guidance on how to prepare to use the Yocto
Project. You can learn about creating a team environment to develop
using the Yocto Project, how to set up a :ref:`build
host <dev-manual/start:preparing the build host>`, how to locate
Yocto Project source repositories, and how to create local Git
repositories.
Creating a Team Development Environment
=======================================
It might not be immediately clear how you can use the Yocto Project in a
team development environment, or how to scale it for a large team of
developers. You can adapt the Yocto Project to many different use cases
and scenarios; however, this flexibility could cause difficulties if you
are trying to create a working setup that scales effectively.
To help you understand how to set up this type of environment, this
section presents a procedure that gives you information that can help
you get the results you want. The procedure is high-level and presents
some of the project's most successful experiences, practices, solutions,
and available technologies that have proved to work well in the past;
however, keep in mind, the procedure here is simply a starting point.
You can build off these steps and customize the procedure to fit any
particular working environment and set of practices.
#. *Determine Who is Going to be Developing:* You first need to
understand who is going to be doing anything related to the Yocto
Project and determine their roles. Making this determination is
essential to completing subsequent steps, which are to get your
equipment together and set up your development environment's
hardware topology.
Here are possible roles:
- *Application Developer:* This type of developer does application
level work on top of an existing software stack.
- *Core System Developer:* This type of developer works on the
contents of the operating system image itself.
- *Build Engineer:* This type of developer manages Autobuilders and
releases. Depending on the specifics of the environment, not all
situations might need a Build Engineer.
- *Test Engineer:* This type of developer creates and manages
automated tests that are used to ensure all application and core
system development meets desired quality standards.
#. *Gather the Hardware:* Based on the size and make-up of the team,
get the hardware together. Ideally, any development, build, or test
engineer uses a system that runs a supported Linux distribution.
These systems, in general, should be high performance (e.g. dual,
six-core Xeons with 24 Gbytes of RAM and plenty of disk space). You
can help ensure efficiency by having any machines used for testing
or that run Autobuilders be as high performance as possible.
.. note::
Given sufficient processing power, you might also consider
building Yocto Project development containers to be run under
Docker, which is described later.
#. *Understand the Hardware Topology of the Environment:* Once you
understand the hardware involved and the make-up of the team, you
can understand the hardware topology of the development environment.
You can get a visual idea of the machines and their roles across the
development environment.
#. *Use Git as Your Source Control Manager (SCM):* Keeping your
:term:`Metadata` (i.e. recipes,
configuration files, classes, and so forth) and any software you are
developing under the control of an SCM system that is compatible
with the OpenEmbedded build system is advisable. Of all of the SCMs
supported by BitBake, the Yocto Project team strongly recommends using
:ref:`overview-manual/development-environment:git`.
Git is a distributed system
that is easy to back up, allows you to work remotely, and then
connects back to the infrastructure.
.. note::
For information about BitBake, see the
:doc:`bitbake:index`.
It is relatively easy to set up Git services and create
infrastructure like :yocto_git:`/`, which is based on
server software called ``gitolite`` with ``cgit`` being used to
generate the web interface that lets you view the repositories. The
``gitolite`` software identifies users using SSH keys and allows
branch-based access controls to repositories that you can control as
little or as much as necessary.
.. note::
The setup of these services is beyond the scope of this manual.
However, here are sites describing how to perform setup:
- `Gitolite <https://gitolite.com>`__: Information for
``gitolite``.
- `Interfaces, frontends, and
tools <https://git.wiki.kernel.org/index.php/Interfaces,_frontends,_and_tools>`__:
Documentation on how to create interfaces and frontends for
Git.
#. *Set up the Application Development Machines:* As mentioned earlier,
application developers are creating applications on top of existing
software stacks. Following are some best practices for setting up
machines used for application development:
- Use a pre-built toolchain that contains the software stack
itself. Then, develop the application code on top of the stack.
This method works well for small numbers of relatively isolated
applications.
- Keep your cross-development toolchains updated. You can do this
through provisioning either as new toolchain downloads or as
updates through a package update mechanism using ``opkg`` to
provide updates to an existing toolchain. The exact mechanics of
how and when to do this depend on local policy.
- Use multiple toolchains installed locally into different
locations to allow development across versions.
#. *Set up the Core Development Machines:* As mentioned earlier, core
developers work on the contents of the operating system itself.
Following are some best practices for setting up machines used for
developing images:
- Have the :term:`OpenEmbedded Build System` available on
the developer workstations so developers can run their own builds
and directly rebuild the software stack.
- Keep the core system unchanged as much as possible and do your
work in layers on top of the core system. Doing so gives you a
greater level of portability when upgrading to new versions of
the core system or Board Support Packages (BSPs).
- Share layers amongst the developers of a particular project and
contain the policy configuration that defines the project.
#. *Set up an Autobuilder:* Autobuilders are often the core of the
development environment. It is here that changes from individual
developers are brought together and centrally tested. Based on this
automated build and test environment, subsequent decisions about
releases can be made. Autobuilders also allow for "continuous
integration" style testing of software components and regression
identification and tracking.
See ":yocto_ab:`Yocto Project Autobuilder <>`" for more
information and links to buildbot. The Yocto Project team has found
this implementation works well in this role. A public example of
this is the Yocto Project Autobuilders, which the Yocto Project team
uses to test the overall health of the project.
The features of this system are:
- Highlights when commits break the build.
- Populates an :ref:`sstate
cache <overview-manual/concepts:shared state cache>` from which
developers can pull rather than requiring local builds.
- Allows commit hook triggers, which trigger builds when commits
are made.
- Allows triggering of automated image booting and testing under
the QuickEMUlator (QEMU).
- Supports incremental build testing and from-scratch builds.
- Shares output that allows developer testing and historical
regression investigation.
- Creates output that can be used for releases.
- Allows scheduling of builds so that resources can be used
efficiently.
#. *Set up Test Machines:* Use a small number of shared, high
performance systems for testing purposes. Developers can use these
systems for wider, more extensive testing while they continue to
develop locally using their primary development system.
#. *Document Policies and Change Flow:* The Yocto Project uses a
hierarchical structure and a pull model. There are scripts to create and
send pull requests (i.e. ``create-pull-request`` and
``send-pull-request``). This model is in line with other open source
projects where maintainers are responsible for specific areas of the
project and a single maintainer handles the final "top-of-tree"
merges.
.. note::
You can also use a more collective push model. The ``gitolite``
software supports both the push and pull models quite easily.
As with any development environment, it is important to document the
policy used as well as any main project guidelines so they are
understood by everyone. It is also a good idea to have
well-structured commit messages, which are usually a part of a
project's guidelines. Good commit messages are essential when
looking back in time and trying to understand why changes were made.
If you discover that changes are needed to the core layer of the
project, it is worth sharing those with the community as soon as
possible. Chances are if you have discovered the need for changes,
someone else in the community needs them also.
#. *Development Environment Summary:* Aside from the previous steps,
here are best practices within the Yocto Project development
environment:
- Use :ref:`overview-manual/development-environment:git` as the source control
system.
- Maintain your Metadata in layers that make sense for your
situation. See the ":ref:`overview-manual/yp-intro:the yocto project layer model`"
section in the Yocto Project Overview and Concepts Manual and the
":ref:`dev-manual/layers:understanding and creating layers`"
section for more information on layers.
- Separate the project's Metadata and code by using separate Git
repositories. See the ":ref:`overview-manual/development-environment:yocto project source repositories`"
section in the Yocto Project Overview and Concepts Manual for
information on these repositories. See the
":ref:`dev-manual/start:locating yocto project source files`"
section for information on how to set up local Git repositories
for related upstream Yocto Project Git repositories.
- Set up the directory for the shared state cache
(:term:`SSTATE_DIR`) where
it makes sense. For example, set up the sstate cache on a system
used by developers in the same organization and share the same
source directories on their machines.
- Set up an Autobuilder and have it populate the sstate cache and
source directories.
- The Yocto Project community encourages you to send patches to the
project to fix bugs or add features. If you do submit patches,
follow the project commit guidelines for writing good commit
messages. See the
":ref:`dev-manual/changes:submitting a change to the yocto project`"
section.
- Send changes to the core sooner than later as others are likely
to run into the same issues. For some guidance on mailing lists
to use, see the list in the
":ref:`dev-manual/changes:submitting a change to the yocto project`"
section. For a description
of the available mailing lists, see the ":ref:`resources-mailinglist`" section in
the Yocto Project Reference Manual.
Preparing the Build Host
========================
This section provides procedures to set up a system to be used as your
:term:`Build Host` for
development using the Yocto Project. Your build host can be a native
Linux machine (recommended), it can be a machine (Linux, Mac, or
Windows) that uses `CROPS <https://github.com/crops/poky-container>`__,
which leverages `Docker Containers <https://www.docker.com/>`__ or it
can be a Windows machine capable of running version 2 of Windows Subsystem
For Linux (WSL 2).
.. note::
The Yocto Project is not compatible with version 1 of
:wikipedia:`Windows Subsystem for Linux <Windows_Subsystem_for_Linux>`.
It is compatible but neither officially supported nor validated with
WSL 2. If you still decide to use WSL please upgrade to
`WSL 2 <https://learn.microsoft.com/en-us/windows/wsl/install>`__.
Once your build host is set up to use the Yocto Project, further steps
are necessary depending on what you want to accomplish. See the
following references for information on how to prepare for Board Support
Package (BSP) development and kernel development:
- *BSP Development:* See the ":ref:`bsp-guide/bsp:preparing your build host to work with bsp layers`"
section in the Yocto Project Board Support Package (BSP) Developer's
Guide.
- *Kernel Development:* See the ":ref:`kernel-dev/common:preparing the build host to work on the kernel`"
section in the Yocto Project Linux Kernel Development Manual.
Setting Up a Native Linux Host
------------------------------
Follow these steps to prepare a native Linux machine as your Yocto
Project Build Host:
#. *Use a Supported Linux Distribution:* You should have a reasonably
current Linux-based host system. You will have the best results with
a recent release of Fedora, openSUSE, Debian, Ubuntu, RHEL or CentOS
as these releases are frequently tested against the Yocto Project and
officially supported. For a list of the distributions under
validation and their status, see the ":ref:`Supported Linux
Distributions <system-requirements-supported-distros>`"
section in the Yocto Project Reference Manual and the wiki page at
:yocto_wiki:`Distribution Support </Distribution_Support>`.
#. *Have Enough Free Memory:* Your system should have at least 50 Gbytes
of free disk space for building images.
#. *Meet Minimal Version Requirements:* The OpenEmbedded build system
should be able to run on any modern distribution that has the
following versions for Git, tar, Python, gcc and make.
- Git &MIN_GIT_VERSION; or greater
- tar &MIN_TAR_VERSION; or greater
- Python &MIN_PYTHON_VERSION; or greater.
- gcc &MIN_GCC_VERSION; or greater.
- GNU make &MIN_MAKE_VERSION; or greater
If your build host does not meet any of these listed version
requirements, you can take steps to prepare the system so that you
can still use the Yocto Project. See the
":ref:`ref-manual/system-requirements:required git, tar, python, make and gcc versions`"
section in the Yocto Project Reference Manual for information.
#. *Install Development Host Packages:* Required development host
packages vary depending on your build host and what you want to do
with the Yocto Project. Collectively, the number of required packages
is large if you want to be able to cover all cases.
For lists of required packages for all scenarios, see the
":ref:`ref-manual/system-requirements:required packages for the build host`"
section in the Yocto Project Reference Manual.
Once you have completed the previous steps, you are ready to continue
using a given development path on your native Linux machine. If you are
going to use BitBake, see the
":ref:`dev-manual/start:cloning the \`\`poky\`\` repository`"
section. If you are going
to use the Extensible SDK, see the ":doc:`/sdk-manual/extensible`" Chapter in the Yocto
Project Application Development and the Extensible Software Development
Kit (eSDK) manual. If you want to work on the kernel, see the :doc:`/kernel-dev/index`. If you are going to use
Toaster, see the ":doc:`/toaster-manual/setup-and-use`"
section in the Toaster User Manual.
Setting Up to Use CROss PlatformS (CROPS)
-----------------------------------------
With `CROPS <https://github.com/crops/poky-container>`__, which
leverages `Docker Containers <https://www.docker.com/>`__, you can
create a Yocto Project development environment that is operating system
agnostic. You can set up a container in which you can develop using the
Yocto Project on a Windows, Mac, or Linux machine.
Follow these general steps to prepare a Windows, Mac, or Linux machine
as your Yocto Project build host:
#. *Determine What Your Build Host Needs:*
`Docker <https://www.docker.com/what-docker>`__ is a software
container platform that you need to install on the build host.
Depending on your build host, you might have to install different
software to support Docker containers. Go to the Docker installation
page and read about the platform requirements in "`Supported
Platforms <https://docs.docker.com/engine/install/#supported-platforms>`__"
your build host needs to run containers.
#. *Choose What To Install:* Depending on whether or not your build host
meets system requirements, you need to install "Docker CE Stable" or
the "Docker Toolbox". Most situations call for Docker CE. However, if
you have a build host that does not meet requirements (e.g.
Pre-Windows 10 or Windows 10 "Home" version), you must install Docker
Toolbox instead.
#. *Go to the Install Site for Your Platform:* Click the link for the
Docker edition associated with your build host's native software. For
example, if your build host is running Microsoft Windows Version 10
and you want the Docker CE Stable edition, click that link under
"Supported Platforms".
#. *Install the Software:* Once you have understood all the
pre-requisites, you can download and install the appropriate
software. Follow the instructions for your specific machine and the
type of the software you need to install:
- Install `Docker Desktop on
Windows <https://docs.docker.com/docker-for-windows/install/#install-docker-desktop-on-windows>`__
for Windows build hosts that meet requirements.
- Install `Docker Desktop on
MacOs <https://docs.docker.com/docker-for-mac/install/#install-and-run-docker-desktop-on-mac>`__
for Mac build hosts that meet requirements.
- Install `Docker Engine on
CentOS <https://docs.docker.com/engine/install/centos/>`__
for Linux build hosts running the CentOS distribution.
- Install `Docker Engine on
Debian <https://docs.docker.com/engine/install/debian/>`__
for Linux build hosts running the Debian distribution.
- Install `Docker Engine for
Fedora <https://docs.docker.com/engine/install/fedora/>`__
for Linux build hosts running the Fedora distribution.
- Install `Docker Engine for
Ubuntu <https://docs.docker.com/engine/install/ubuntu/>`__
for Linux build hosts running the Ubuntu distribution.
#. *Optionally Orient Yourself With Docker:* If you are unfamiliar with
Docker and the container concept, you can learn more here -
https://docs.docker.com/get-started/.
#. *Launch Docker or Docker Toolbox:* You should be able to launch
Docker or the Docker Toolbox and have a terminal shell on your
development host.
#. *Set Up the Containers to Use the Yocto Project:* Go to
https://github.com/crops/docker-win-mac-docs/wiki and follow
the directions for your particular build host (i.e. Linux, Mac, or
Windows).
Once you complete the setup instructions for your machine, you have
the Poky, Extensible SDK, and Toaster containers available. You can
click those links from the page and learn more about using each of
those containers.
Once you have a container set up, everything is in place to develop just
as if you were running on a native Linux machine. If you are going to
use the Poky container, see the
":ref:`dev-manual/start:cloning the \`\`poky\`\` repository`"
section. If you are going to use the Extensible SDK container, see the
":doc:`/sdk-manual/extensible`" Chapter in the Yocto
Project Application Development and the Extensible Software Development
Kit (eSDK) manual. If you are going to use the Toaster container, see
the ":doc:`/toaster-manual/setup-and-use`"
section in the Toaster User Manual.
Setting Up to Use Windows Subsystem For Linux (WSL 2)
-----------------------------------------------------
With `Windows Subsystem for Linux (WSL 2)
<https://learn.microsoft.com/en-us/windows/wsl/>`__,
you can create a Yocto Project development environment that allows you
to build on Windows. You can set up a Linux distribution inside Windows
in which you can develop using the Yocto Project.
Follow these general steps to prepare a Windows machine using WSL 2 as
your Yocto Project build host:
#. *Make sure your Windows machine is capable of running WSL 2:*
While all Windows 11 and Windows Server 2022 builds support WSL 2,
the first versions of Windows 10 and Windows Server 2019 didn't.
Check the minimum build numbers for `Windows 10
<https://learn.microsoft.com/en-us/windows/wsl/install-manual#step-2---check-requirements-for-running-wsl-2>`__
and for `Windows Server 2019
<https://learn.microsoft.com/en-us/windows/wsl/install-on-server>`__.
To check which build version you are running, you may open a command
prompt on Windows and execute the command "ver"::
C:\Users\myuser> ver
Microsoft Windows [Version 10.0.19041.153]
#. *Install the Linux distribution of your choice inside WSL 2:*
Once you know your version of Windows supports WSL 2, you can
install the distribution of your choice from the Microsoft Store.
Open the Microsoft Store and search for Linux. While there are
several Linux distributions available, the assumption is that your
pick will be one of the distributions supported by the Yocto Project
as stated on the instructions for using a native Linux host. After
making your selection, simply click "Get" to download and install the
distribution.
#. *Check which Linux distribution WSL 2 is using:* Open a Windows
PowerShell and run::
C:\WINDOWS\system32> wsl -l -v
NAME STATE VERSION
*Ubuntu Running 2
Note that WSL 2 supports running as many different Linux distributions
as you want to install.
#. *Optionally Get Familiar with WSL:* You can learn more on
https://docs.microsoft.com/en-us/windows/wsl/wsl2-about.
#. *Launch your WSL Distibution:* From the Windows start menu simply
launch your WSL distribution just like any other application.
#. *Optimize your WSL 2 storage often:* Due to the way storage is
handled on WSL 2, the storage space used by the underlying Linux
distribution is not reflected immediately, and since BitBake heavily
uses storage, after several builds, you may be unaware you are
running out of space. As WSL 2 uses a VHDX file for storage, this issue
can be easily avoided by regularly optimizing this file in a manual way:
1. *Find the location of your VHDX file:*
First you need to find the distro app package directory, to achieve this
open a Windows Powershell as Administrator and run::
C:\WINDOWS\system32> Get-AppxPackage -Name "*Ubuntu*" | Select PackageFamilyName
PackageFamilyName
-----------------
CanonicalGroupLimited.UbuntuonWindows_79abcdefgh
You should now
replace the PackageFamilyName and your user on the following path
to find your VHDX file::
ls C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\
Mode LastWriteTime Length Name
-a---- 3/14/2020 9:52 PM 57418973184 ext4.vhdx
Your VHDX file path is:
``C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\ext4.vhdx``
2a. *Optimize your VHDX file using Windows Powershell:*
To use the ``optimize-vhd`` cmdlet below, first install the Hyper-V
option on Windows. Then, open a Windows Powershell as Administrator to
optimize your VHDX file, shutting down WSL first::
C:\WINDOWS\system32> wsl --shutdown
C:\WINDOWS\system32> optimize-vhd -Path C:\Users\myuser\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\ext4.vhdx -Mode full
A progress bar should be shown while optimizing the
VHDX file, and storage should now be reflected correctly on the
Windows Explorer.
2b. *Optimize your VHDX file using DiskPart:*
The ``optimize-vhd`` cmdlet noted in step 2a above is provided by
Hyper-V. Not all SKUs of Windows can install Hyper-V. As an alternative,
use the DiskPart tool. To start, open a Windows command prompt as
Administrator to optimize your VHDX file, shutting down WSL first::
C:\WINDOWS\system32> wsl --shutdown
C:\WINDOWS\system32> diskpart
DISKPART> select vdisk file="<path_to_VHDX_file>"
DISKPART> attach vdisk readonly
DISKPART> compact vdisk
DISKPART> exit
.. note::
The current implementation of WSL 2 does not have out-of-the-box
access to external devices such as those connected through a USB
port, but it automatically mounts your ``C:`` drive on ``/mnt/c/``
(and others), which you can use to share deploy artifacts to be later
flashed on hardware through Windows, but your :term:`Build Directory`
should not reside inside this mountpoint.
Once you have WSL 2 set up, everything is in place to develop just as if
you were running on a native Linux machine. If you are going to use the
Extensible SDK container, see the ":doc:`/sdk-manual/extensible`" Chapter in the Yocto
Project Application Development and the Extensible Software Development
Kit (eSDK) manual. If you are going to use the Toaster container, see
the ":doc:`/toaster-manual/setup-and-use`"
section in the Toaster User Manual.
Locating Yocto Project Source Files
===================================
This section shows you how to locate, fetch and configure the source
files you'll need to work with the Yocto Project.
.. note::
- For concepts and introductory information about Git as it is used
in the Yocto Project, see the ":ref:`overview-manual/development-environment:git`"
section in the Yocto Project Overview and Concepts Manual.
- For concepts on Yocto Project source repositories, see the
":ref:`overview-manual/development-environment:yocto project source repositories`"
section in the Yocto Project Overview and Concepts Manual."
Accessing Source Repositories
-----------------------------
Working from a copy of the upstream :ref:`dev-manual/start:accessing source repositories` is the
preferred method for obtaining and using a Yocto Project release. You
can view the Yocto Project Source Repositories at
:yocto_git:`/`. In particular, you can find the ``poky``
repository at :yocto_git:`/poky`.
Use the following procedure to locate the latest upstream copy of the
``poky`` Git repository:
#. *Access Repositories:* Open a browser and go to
:yocto_git:`/` to access the GUI-based interface into the
Yocto Project source repositories.
#. *Select the Repository:* Click on the repository in which you are
interested (e.g. ``poky``).
#. *Find the URL Used to Clone the Repository:* At the bottom of the
page, note the URL used to clone that repository
(e.g. :yocto_git:`/poky`).
.. note::
For information on cloning a repository, see the
":ref:`dev-manual/start:cloning the \`\`poky\`\` repository`" section.
Accessing Source Archives
-------------------------
The Yocto Project also provides source archives of its releases, which
are available on :yocto_dl:`/releases/yocto/`. Then, choose the subdirectory
containing the release you wish to use, for example
:yocto_dl:`yocto-&DISTRO; </releases/yocto/yocto-&DISTRO;/>`.
You will find there source archives of individual components (if you wish
to use them individually), and of the corresponding Poky release bundling
a selection of these components.
.. note::
The recommended method for accessing Yocto Project components is to
use Git to clone the upstream repository and work from within that
locally cloned repository.
Using the Downloads Page
------------------------
The :yocto_home:`Yocto Project Website <>` uses a "DOWNLOADS" page
from which you can locate and download tarballs of any Yocto Project
release. Rather than Git repositories, these files represent snapshot
tarballs similar to the tarballs located in the Index of Releases
described in the ":ref:`dev-manual/start:accessing source archives`" section.
#. *Go to the Yocto Project Website:* Open The
:yocto_home:`Yocto Project Website <>` in your browser.
#. *Get to the Downloads Area:* Select the "DOWNLOADS" item from the
pull-down "SOFTWARE" tab menu near the top of the page.
#. *Select a Yocto Project Release:* Use the menu next to "RELEASE" to
display and choose a recent or past supported Yocto Project release
(e.g. &DISTRO_NAME_NO_CAP;, &DISTRO_NAME_NO_CAP_MINUS_ONE;, and so forth).
.. note::
For a "map" of Yocto Project releases to version numbers, see the
:yocto_wiki:`Releases </Releases>` wiki page.
You can use the "RELEASE ARCHIVE" link to reveal a menu of all Yocto
Project releases.
#. *Download Tools or Board Support Packages (BSPs):* From the
"DOWNLOADS" page, you can download tools or BSPs as well. Just scroll
down the page and look for what you need.
Cloning and Checking Out Branches
=================================
To use the Yocto Project for development, you need a release locally
installed on your development system. This locally installed set of
files is referred to as the :term:`Source Directory`
in the Yocto Project documentation.
The preferred method of creating your Source Directory is by using
:ref:`overview-manual/development-environment:git` to clone a local copy of the upstream
``poky`` repository. Working from a cloned copy of the upstream
repository allows you to contribute back into the Yocto Project or to
simply work with the latest software on a development branch. Because
Git maintains and creates an upstream repository with a complete history
of changes and you are working with a local clone of that repository,
you have access to all the Yocto Project development branches and tag
names used in the upstream repository.
Cloning the ``poky`` Repository
-------------------------------
Follow these steps to create a local version of the upstream
:term:`Poky` Git repository.
#. *Set Your Directory:* Change your working directory to where you want
to create your local copy of ``poky``.
#. *Clone the Repository:* The following example command clones the
``poky`` repository and uses the default name "poky" for your local
repository::
$ git clone git://git.yoctoproject.org/poky
Cloning into 'poky'...
remote: Counting objects: 432160, done.
remote: Compressing objects: 100% (102056/102056), done.
remote: Total 432160 (delta 323116), reused 432037 (delta 323000)
Receiving objects: 100% (432160/432160), 153.81 MiB | 8.54 MiB/s, done.
Resolving deltas: 100% (323116/323116), done.
Checking connectivity... done.
Unless you
specify a specific development branch or tag name, Git clones the
"master" branch, which results in a snapshot of the latest
development changes for "master". For information on how to check out
a specific development branch or on how to check out a local branch
based on a tag name, see the
":ref:`dev-manual/start:checking out by branch in poky`" and
":ref:`dev-manual/start:checking out by tag in poky`" sections, respectively.
Once the local repository is created, you can change to that
directory and check its status. The ``master`` branch is checked out
by default::
$ cd poky
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean
$ git branch
* master
Your local repository of poky is identical to the
upstream poky repository at the time from which it was cloned. As you
work with the local branch, you can periodically use the
``git pull --rebase`` command to be sure you are up-to-date
with the upstream branch.
Checking Out by Branch in Poky
------------------------------
When you clone the upstream poky repository, you have access to all its
development branches. Each development branch in a repository is unique
as it forks off the "master" branch. To see and use the files of a
particular development branch locally, you need to know the branch name
and then specifically check out that development branch.
.. note::
Checking out an active development branch by branch name gives you a
snapshot of that particular branch at the time you check it out.
Further development on top of the branch that occurs after check it
out can occur.
#. *Switch to the Poky Directory:* If you have a local poky Git
repository, switch to that directory. If you do not have the local
copy of poky, see the
":ref:`dev-manual/start:cloning the \`\`poky\`\` repository`"
section.
#. *Determine Existing Branch Names:*
::
$ git branch -a
* master
remotes/origin/1.1_M1
remotes/origin/1.1_M2
remotes/origin/1.1_M3
remotes/origin/1.1_M4
remotes/origin/1.2_M1
remotes/origin/1.2_M2
remotes/origin/1.2_M3
. . .
remotes/origin/thud
remotes/origin/thud-next
remotes/origin/warrior
remotes/origin/warrior-next
remotes/origin/zeus
remotes/origin/zeus-next
... and so on ...
#. *Check out the Branch:* Check out the development branch in which you
want to work. For example, to access the files for the Yocto Project
&DISTRO; Release (&DISTRO_NAME;), use the following command::
$ git checkout -b &DISTRO_NAME_NO_CAP; origin/&DISTRO_NAME_NO_CAP;
Branch &DISTRO_NAME_NO_CAP; set up to track remote branch &DISTRO_NAME_NO_CAP; from origin.
Switched to a new branch '&DISTRO_NAME_NO_CAP;'
The previous command checks out the "&DISTRO_NAME_NO_CAP;" development
branch and reports that the branch is tracking the upstream
"origin/&DISTRO_NAME_NO_CAP;" branch.
The following command displays the branches that are now part of your
local poky repository. The asterisk character indicates the branch
that is currently checked out for work::
$ git branch
master
* &DISTRO_NAME_NO_CAP;
Checking Out by Tag in Poky
---------------------------
Similar to branches, the upstream repository uses tags to mark specific
commits associated with significant points in a development branch (i.e.
a release point or stage of a release). You might want to set up a local
branch based on one of those points in the repository. The process is
similar to checking out by branch name except you use tag names.
.. note::
Checking out a branch based on a tag gives you a stable set of files
not affected by development on the branch above the tag.
#. *Switch to the Poky Directory:* If you have a local poky Git
repository, switch to that directory. If you do not have the local
copy of poky, see the
":ref:`dev-manual/start:cloning the \`\`poky\`\` repository`"
section.
#. *Fetch the Tag Names:* To checkout the branch based on a tag name,
you need to fetch the upstream tags into your local repository::
$ git fetch --tags
$
#. *List the Tag Names:* You can list the tag names now::
$ git tag
1.1_M1.final
1.1_M1.rc1
1.1_M1.rc2
1.1_M2.final
1.1_M2.rc1
.
.
.
yocto-2.5
yocto-2.5.1
yocto-2.5.2
yocto-2.5.3
yocto-2.6
yocto-2.6.1
yocto-2.6.2
yocto-2.7
yocto_1.5_M5.rc8
#. *Check out the Branch:*
::
$ git checkout tags/yocto-&DISTRO; -b my_yocto_&DISTRO;
Switched to a new branch 'my_yocto_&DISTRO;'
$ git branch
master
* my_yocto_&DISTRO;
The previous command creates and
checks out a local branch named "my_yocto_&DISTRO;", which is based on
the commit in the upstream poky repository that has the same tag. In
this example, the files you have available locally as a result of the
``checkout`` command are a snapshot of the "&DISTRO_NAME_NO_CAP;"
development branch at the point where Yocto Project &DISTRO; was
released.
poky/documentation/dev-manual/temporary-source-code.rst
+66
View File
@@ -0,0 +1,66 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Finding Temporary Source Code
*****************************
You might find it helpful during development to modify the temporary
source code used by recipes to build packages. For example, suppose you
are developing a patch and you need to experiment a bit to figure out
your solution. After you have initially built the package, you can
iteratively tweak the source code, which is located in the
:term:`Build Directory`, and then you can force a re-compile and quickly
test your altered code. Once you settle on a solution, you can then preserve
your changes in the form of patches.
During a build, the unpacked temporary source code used by recipes to
build packages is available in the :term:`Build Directory` as defined by the
:term:`S` variable. Below is the default value for the :term:`S` variable as
defined in the ``meta/conf/bitbake.conf`` configuration file in the
:term:`Source Directory`::
S = "${WORKDIR}/${BP}"
You should be aware that many recipes override the
:term:`S` variable. For example, recipes that fetch their source from Git
usually set :term:`S` to ``${WORKDIR}/git``.
.. note::
The :term:`BP` represents the base recipe name, which consists of the name
and version::
BP = "${BPN}-${PV}"
The path to the work directory for the recipe
(:term:`WORKDIR`) is defined as
follows::
${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR}
The actual directory depends on several things:
- :term:`TMPDIR`: The top-level build
output directory.
- :term:`MULTIMACH_TARGET_SYS`:
The target system identifier.
- :term:`PN`: The recipe name.
- :term:`EXTENDPE`: The epoch --- if
:term:`PE` is not specified, which is
usually the case for most recipes, then :term:`EXTENDPE` is blank.
- :term:`PV`: The recipe version.
- :term:`PR`: The recipe revision.
As an example, assume a Source Directory top-level folder named
``poky``, a default :term:`Build Directory` at ``poky/build``, and a
``qemux86-poky-linux`` machine target system. Furthermore, suppose your
recipe is named ``foo_1.3.0.bb``. In this case, the work directory the
build system uses to build the package would be as follows::
poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0
poky/documentation/dev-manual/upgrading-recipes.rst
+397
View File
@@ -0,0 +1,397 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Upgrading Recipes
*****************
Over time, upstream developers publish new versions for software built
by layer recipes. It is recommended to keep recipes up-to-date with
upstream version releases.
While there are several methods to upgrade a recipe, you might
consider checking on the upgrade status of a recipe first. You can do so
using the ``devtool check-upgrade-status`` command. See the
":ref:`devtool-checking-on-the-upgrade-status-of-a-recipe`"
section in the Yocto Project Reference Manual for more information.
The remainder of this section describes three ways you can upgrade a
recipe. You can use the Automated Upgrade Helper (AUH) to set up
automatic version upgrades. Alternatively, you can use
``devtool upgrade`` to set up semi-automatic version upgrades. Finally,
you can manually upgrade a recipe by editing the recipe itself.
Using the Auto Upgrade Helper (AUH)
===================================
The AUH utility works in conjunction with the OpenEmbedded build system
in order to automatically generate upgrades for recipes based on new
versions being published upstream. Use AUH when you want to create a
service that performs the upgrades automatically and optionally sends
you an email with the results.
AUH allows you to update several recipes with a single use. You can also
optionally perform build and integration tests using images with the
results saved to your hard drive and emails of results optionally sent
to recipe maintainers. Finally, AUH creates Git commits with appropriate
commit messages in the layer's tree for the changes made to recipes.
.. note::
In some conditions, you should not use AUH to upgrade recipes
and should instead use either ``devtool upgrade`` or upgrade your
recipes manually:
- When AUH cannot complete the upgrade sequence. This situation
usually results because custom patches carried by the recipe
cannot be automatically rebased to the new version. In this case,
``devtool upgrade`` allows you to manually resolve conflicts.
- When for any reason you want fuller control over the upgrade
process. For example, when you want special arrangements for
testing.
The following steps describe how to set up the AUH utility:
#. *Be Sure the Development Host is Set Up:* You need to be sure that
your development host is set up to use the Yocto Project. For
information on how to set up your host, see the
":ref:`dev-manual/start:Preparing the Build Host`" section.
#. *Make Sure Git is Configured:* The AUH utility requires Git to be
configured because AUH uses Git to save upgrades. Thus, you must have
Git user and email configured. The following command shows your
configurations::
$ git config --list
If you do not have the user and
email configured, you can use the following commands to do so::
$ git config --global user.name some_name
$ git config --global user.email username@domain.com
#. *Clone the AUH Repository:* To use AUH, you must clone the repository
onto your development host. The following command uses Git to create
a local copy of the repository on your system::
$ git clone git://git.yoctoproject.org/auto-upgrade-helper
Cloning into 'auto-upgrade-helper'... remote: Counting objects: 768, done.
remote: Compressing objects: 100% (300/300), done.
remote: Total 768 (delta 499), reused 703 (delta 434)
Receiving objects: 100% (768/768), 191.47 KiB | 98.00 KiB/s, done.
Resolving deltas: 100% (499/499), done.
Checking connectivity... done.
AUH is not part of the :term:`OpenEmbedded-Core (OE-Core)` or
:term:`Poky` repositories.
#. *Create a Dedicated Build Directory:* Run the :ref:`structure-core-script`
script to create a fresh :term:`Build Directory` that you use exclusively
for running the AUH utility::
$ cd poky
$ source oe-init-build-env your_AUH_build_directory
Re-using an existing :term:`Build Directory` and its configurations is not
recommended as existing settings could cause AUH to fail or behave
undesirably.
#. *Make Configurations in Your Local Configuration File:* Several
settings are needed in the ``local.conf`` file in the build
directory you just created for AUH. Make these following
configurations:
- If you want to enable :ref:`Build
History <dev-manual/build-quality:maintaining build output quality>`,
which is optional, you need the following lines in the
``conf/local.conf`` file::
INHERIT =+ "buildhistory"
BUILDHISTORY_COMMIT = "1"
With this configuration and a successful
upgrade, a build history "diff" file appears in the
``upgrade-helper/work/recipe/buildhistory-diff.txt`` file found in
your :term:`Build Directory`.
- If you want to enable testing through the :ref:`ref-classes-testimage`
class, which is optional, you need to have the following set in
your ``conf/local.conf`` file::
INHERIT += "testimage"
.. note::
If your distro does not enable by default ptest, which Poky
does, you need the following in your ``local.conf`` file::
DISTRO_FEATURES:append = " ptest"
#. *Optionally Start a vncserver:* If you are running in a server
without an X11 session, you need to start a vncserver::
$ vncserver :1
$ export DISPLAY=:1
#. *Create and Edit an AUH Configuration File:* You need to have the
``upgrade-helper/upgrade-helper.conf`` configuration file in your
:term:`Build Directory`. You can find a sample configuration file in the
:yocto_git:`AUH source repository </auto-upgrade-helper/tree/>`.
Read through the sample file and make configurations as needed. For
example, if you enabled build history in your ``local.conf`` as
described earlier, you must enable it in ``upgrade-helper.conf``.
Also, if you are using the default ``maintainers.inc`` file supplied
with Poky and located in ``meta-yocto`` and you do not set a
"maintainers_whitelist" or "global_maintainer_override" in the
``upgrade-helper.conf`` configuration, and you specify "-e all" on
the AUH command-line, the utility automatically sends out emails to
all the default maintainers. Please avoid this.
This next set of examples describes how to use the AUH:
- *Upgrading a Specific Recipe:* To upgrade a specific recipe, use the
following form::
$ upgrade-helper.py recipe_name
For example, this command upgrades the ``xmodmap`` recipe::
$ upgrade-helper.py xmodmap
- *Upgrading a Specific Recipe to a Particular Version:* To upgrade a
specific recipe to a particular version, use the following form::
$ upgrade-helper.py recipe_name -t version
For example, this command upgrades the ``xmodmap`` recipe to version 1.2.3::
$ upgrade-helper.py xmodmap -t 1.2.3
- *Upgrading all Recipes to the Latest Versions and Suppressing Email
Notifications:* To upgrade all recipes to their most recent versions
and suppress the email notifications, use the following command::
$ upgrade-helper.py all
- *Upgrading all Recipes to the Latest Versions and Send Email
Notifications:* To upgrade all recipes to their most recent versions
and send email messages to maintainers for each attempted recipe as
well as a status email, use the following command::
$ upgrade-helper.py -e all
Once you have run the AUH utility, you can find the results in the AUH
:term:`Build Directory`::
${BUILDDIR}/upgrade-helper/timestamp
The AUH utility
also creates recipe update commits from successful upgrade attempts in
the layer tree.
You can easily set up to run the AUH utility on a regular basis by using
a cron job. See the
:yocto_git:`weeklyjob.sh </auto-upgrade-helper/tree/weeklyjob.sh>`
file distributed with the utility for an example.
Using ``devtool upgrade``
=========================
As mentioned earlier, an alternative method for upgrading recipes to
newer versions is to use
:doc:`devtool upgrade </ref-manual/devtool-reference>`.
You can read about ``devtool upgrade`` in general in the
":ref:`sdk-manual/extensible:use \`\`devtool upgrade\`\` to create a version of the recipe that supports a newer version of the software`"
section in the Yocto Project Application Development and the Extensible
Software Development Kit (eSDK) Manual.
To see all the command-line options available with ``devtool upgrade``,
use the following help command::
$ devtool upgrade -h
If you want to find out what version a recipe is currently at upstream
without any attempt to upgrade your local version of the recipe, you can
use the following command::
$ devtool latest-version recipe_name
As mentioned in the previous section describing AUH, ``devtool upgrade``
works in a less-automated manner than AUH. Specifically,
``devtool upgrade`` only works on a single recipe that you name on the
command line, cannot perform build and integration testing using images,
and does not automatically generate commits for changes in the source
tree. Despite all these "limitations", ``devtool upgrade`` updates the
recipe file to the new upstream version and attempts to rebase custom
patches contained by the recipe as needed.
.. note::
AUH uses much of ``devtool upgrade`` behind the scenes making AUH somewhat
of a "wrapper" application for ``devtool upgrade``.
A typical scenario involves having used Git to clone an upstream
repository that you use during build operations. Because you have built the
recipe in the past, the layer is likely added to your
configuration already. If for some reason, the layer is not added, you
could add it easily using the
":ref:`bitbake-layers <bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script>`"
script. For example, suppose you use the ``nano.bb`` recipe from the
``meta-oe`` layer in the ``meta-openembedded`` repository. For this
example, assume that the layer has been cloned into following area::
/home/scottrif/meta-openembedded
The following command from your :term:`Build Directory` adds the layer to
your build configuration (i.e. ``${BUILDDIR}/conf/bblayers.conf``)::
$ bitbake-layers add-layer /home/scottrif/meta-openembedded/meta-oe
NOTE: Starting bitbake server...
Parsing recipes: 100% |##########################################| Time: 0:00:55
Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors.
Removing 12 recipes from the x86_64 sysroot: 100% |##############| Time: 0:00:00
Removing 1 recipes from the x86_64_i586 sysroot: 100% |##########| Time: 0:00:00
Removing 5 recipes from the i586 sysroot: 100% |#################| Time: 0:00:00
Removing 5 recipes from the qemux86 sysroot: 100% |##############| Time: 0:00:00
For this example, assume that the ``nano.bb`` recipe that
is upstream has a 2.9.3 version number. However, the version in the
local repository is 2.7.4. The following command from your build
directory automatically upgrades the recipe for you::
$ devtool upgrade nano -V 2.9.3
NOTE: Starting bitbake server...
NOTE: Creating workspace layer in /home/scottrif/poky/build/workspace
Parsing recipes: 100% |##########################################| Time: 0:00:46
Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors.
NOTE: Extracting current version source...
NOTE: Resolving any missing task queue dependencies
.
.
.
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 74 tasks of which 72 didn't need to be rerun and all succeeded.
Adding changed files: 100% |#####################################| Time: 0:00:00
NOTE: Upgraded source extracted to /home/scottrif/poky/build/workspace/sources/nano
NOTE: New recipe is /home/scottrif/poky/build/workspace/recipes/nano/nano_2.9.3.bb
.. note::
Using the ``-V`` option is not necessary. Omitting the version number causes
``devtool upgrade`` to upgrade the recipe to the most recent version.
Continuing with this example, you can use ``devtool build`` to build the
newly upgraded recipe::
$ devtool build nano
NOTE: Starting bitbake server...
Loading cache: 100% |################################################################################################| Time: 0:00:01
Loaded 2040 entries from dependency cache.
Parsing recipes: 100% |##############################################################################################| Time: 0:00:00
Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
.
.
.
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: nano: compiling from external source tree /home/scottrif/poky/build/workspace/sources/nano
NOTE: Tasks Summary: Attempted 520 tasks of which 304 didn't need to be rerun and all succeeded.
Within the ``devtool upgrade`` workflow, you can
deploy and test your rebuilt software. For this example,
however, running ``devtool finish`` cleans up the workspace once the
source in your workspace is clean. This usually means using Git to stage
and submit commits for the changes generated by the upgrade process.
Once the tree is clean, you can clean things up in this example with the
following command from the ``${BUILDDIR}/workspace/sources/nano``
directory::
$ devtool finish nano meta-oe
NOTE: Starting bitbake server...
Loading cache: 100% |################################################################################################| Time: 0:00:00
Loaded 2040 entries from dependency cache.
Parsing recipes: 100% |##############################################################################################| Time: 0:00:01
Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors.
NOTE: Adding new patch 0001-nano.bb-Stuff-I-changed-when-upgrading-nano.bb.patch
NOTE: Updating recipe nano_2.9.3.bb
NOTE: Removing file /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano/nano_2.7.4.bb
NOTE: Moving recipe file to /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano
NOTE: Leaving source tree /home/scottrif/poky/build/workspace/sources/nano as-is; if you no longer need it then please delete it manually
Using the ``devtool finish`` command cleans up the workspace and creates a patch
file based on your commits. The tool puts all patch files back into the
source directory in a sub-directory named ``nano`` in this case.
Manually Upgrading a Recipe
===========================
If for some reason you choose not to upgrade recipes using
:ref:`dev-manual/upgrading-recipes:Using the Auto Upgrade Helper (AUH)` or
by :ref:`dev-manual/upgrading-recipes:Using \`\`devtool upgrade\`\``,
you can manually edit the recipe files to upgrade the versions.
.. note::
Manually updating multiple recipes scales poorly and involves many
steps. The recommendation to upgrade recipe versions is through AUH
or ``devtool upgrade``, both of which automate some steps and provide
guidance for others needed for the manual process.
To manually upgrade recipe versions, follow these general steps:
#. *Change the Version:* Rename the recipe such that the version (i.e.
the :term:`PV` part of the recipe name)
changes appropriately. If the version is not part of the recipe name,
change the value as it is set for :term:`PV` within the recipe itself.
#. *Update* :term:`SRCREV` *if Needed*: If the source code your recipe builds
is fetched from Git or some other version control system, update
:term:`SRCREV` to point to the
commit hash that matches the new version.
#. *Build the Software:* Try to build the recipe using BitBake. Typical
build failures include the following:
- License statements were updated for the new version. For this
case, you need to review any changes to the license and update the
values of :term:`LICENSE` and
:term:`LIC_FILES_CHKSUM`
as needed.
.. note::
License changes are often inconsequential. For example, the
license text's copyright year might have changed.
- Custom patches carried by the older version of the recipe might
fail to apply to the new version. For these cases, you need to
review the failures. Patches might not be necessary for the new
version of the software if the upgraded version has fixed those
issues. If a patch is necessary and failing, you need to rebase it
into the new version.
#. *Optionally Attempt to Build for Several Architectures:* Once you
successfully build the new software for a given architecture, you
could test the build for other architectures by changing the
:term:`MACHINE` variable and
rebuilding the software. This optional step is especially important
if the recipe is to be released publicly.
#. *Check the Upstream Change Log or Release Notes:* Checking both these
reveals if there are new features that could break
backwards-compatibility. If so, you need to take steps to mitigate or
eliminate that situation.
#. *Optionally Create a Bootable Image and Test:* If you want, you can
test the new software by booting it onto actual hardware.
#. *Create a Commit with the Change in the Layer Repository:* After all
builds work and any testing is successful, you can create commits for
any changes in the layer holding your upgraded recipe.
poky/documentation/dev-manual/vulnerabilities.rst
+214
View File
@@ -0,0 +1,214 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Checking for Vulnerabilities
****************************
Vulnerabilities in Poky and OE-Core
===================================
The Yocto Project has an infrastructure to track and address unfixed
known security vulnerabilities, as tracked by the public
:wikipedia:`Common Vulnerabilities and Exposures (CVE) <Common_Vulnerabilities_and_Exposures>`
database.
The Yocto Project maintains a `list of known vulnerabilities
<https://autobuilder.yocto.io/pub/non-release/patchmetrics/>`__
for packages in Poky and OE-Core, tracking the evolution of the number of
unpatched CVEs and the status of patches. Such information is available for
the current development version and for each supported release.
Security is a process, not a product, and thus at any time, a number of security
issues may be impacting Poky and OE-Core. It is up to the maintainers, users,
contributors and anyone interested in the issues to investigate and possibly fix them by
updating software components to newer versions or by applying patches to address them.
It is recommended to work with Poky and OE-Core upstream maintainers and submit
patches to fix them, see ":ref:`dev-manual/changes:submitting a change to the yocto project`" for details.
Vulnerability check at build time
=================================
To enable a check for CVE security vulnerabilities using
:ref:`ref-classes-cve-check` in the specific image or target you are building,
add the following setting to your configuration::
INHERIT += "cve-check"
The CVE database contains some old incomplete entries which have been
deemed not to impact Poky or OE-Core. These CVE entries can be excluded from the
check using build configuration::
include conf/distro/include/cve-extra-exclusions.inc
With this CVE check enabled, BitBake build will try to map each compiled software component
recipe name and version information to the CVE database and generate recipe and
image specific reports. These reports will contain:
- metadata about the software component like names and versions
- metadata about the CVE issue such as description and NVD link
- for each software component, a list of CVEs which are possibly impacting this version
- status of each CVE: ``Patched``, ``Unpatched`` or ``Ignored``
The status ``Patched`` means that a patch file to address the security issue has been
applied. ``Unpatched`` status means that no patches to address the issue have been
applied and that the issue needs to be investigated. ``Ignored`` means that after
analysis, it has been deemed to ignore the issue as it for example affects
the software component on a different operating system platform.
After a build with CVE check enabled, reports for each compiled source recipe will be
found in ``build/tmp/deploy/cve``.
For example the CVE check report for the ``flex-native`` recipe looks like::
$ cat poky/build/tmp/deploy/cve/flex-native
LAYER: meta
PACKAGE NAME: flex-native
PACKAGE VERSION: 2.6.4
CVE: CVE-2016-6354
CVE STATUS: Patched
CVE SUMMARY: Heap-based buffer overflow in the yy_get_next_buffer function in Flex before 2.6.1 might allow context-dependent attackers to cause a denial of service or possibly execute arbitrary code via vectors involving num_to_read.
CVSS v2 BASE SCORE: 7.5
CVSS v3 BASE SCORE: 9.8
VECTOR: NETWORK
MORE INFORMATION: https://nvd.nist.gov/vuln/detail/CVE-2016-6354
LAYER: meta
PACKAGE NAME: flex-native
PACKAGE VERSION: 2.6.4
CVE: CVE-2019-6293
CVE STATUS: Ignored
CVE SUMMARY: An issue was discovered in the function mark_beginning_as_normal in nfa.c in flex 2.6.4. There is a stack exhaustion problem caused by the mark_beginning_as_normal function making recursive calls to itself in certain scenarios involving lots of '*' characters. Remote attackers could leverage this vulnerability to cause a denial-of-service.
CVSS v2 BASE SCORE: 4.3
CVSS v3 BASE SCORE: 5.5
VECTOR: NETWORK
MORE INFORMATION: https://nvd.nist.gov/vuln/detail/CVE-2019-6293
For images, a summary of all recipes included in the image and their CVEs is also
generated in textual and JSON formats. These ``.cve`` and ``.json`` reports can be found
in the ``tmp/deploy/images`` directory for each compiled image.
At build time CVE check will also throw warnings about ``Unpatched`` CVEs::
WARNING: flex-2.6.4-r0 do_cve_check: Found unpatched CVE (CVE-2019-6293), for more information check /poky/build/tmp/work/core2-64-poky-linux/flex/2.6.4-r0/temp/cve.log
WARNING: libarchive-3.5.1-r0 do_cve_check: Found unpatched CVE (CVE-2021-36976), for more information check /poky/build/tmp/work/core2-64-poky-linux/libarchive/3.5.1-r0/temp/cve.log
It is also possible to check the CVE status of individual packages as follows::
bitbake -c cve_check flex libarchive
Fixing CVE product name and version mappings
============================================
By default, :ref:`ref-classes-cve-check` uses the recipe name :term:`BPN` as CVE
product name when querying the CVE database. If this mapping contains false positives, e.g.
some reported CVEs are not for the software component in question, or false negatives like
some CVEs are not found to impact the recipe when they should, then the problems can be
in the recipe name to CVE product mapping. These mapping issues can be fixed by setting
the :term:`CVE_PRODUCT` variable inside the recipe. This defines the name of the software component in the
upstream `NIST CVE database <https://nvd.nist.gov/>`__.
The variable supports using vendor and product names like this::
CVE_PRODUCT = "flex_project:flex"
In this example the vendor name used in the CVE database is ``flex_project`` and the
product is ``flex``. With this setting the ``flex`` recipe only maps to this specific
product and not products from other vendors with same name ``flex``.
Similarly, when the recipe version :term:`PV` is not compatible with software versions used by
the upstream software component releases and the CVE database, these can be fixed using
the :term:`CVE_VERSION` variable.
Note that if the CVE entries in the NVD database contain bugs or have missing or incomplete
information, it is recommended to fix the information there directly instead of working
around the issues possibly for a long time in Poky and OE-Core side recipes. Feedback to
NVD about CVE entries can be provided through the `NVD contact form <https://nvd.nist.gov/info/contact-form>`__.
Fixing vulnerabilities in recipes
=================================
If a CVE security issue impacts a software component, it can be fixed by updating to a newer
version of the software component or by applying a patch. For Poky and OE-Core master branches, updating
to a newer software component release with fixes is the best option, but patches can be applied
if releases are not yet available.
For stable branches, it is preferred to apply patches for the issues. For some software
components minor version updates can also be applied if they are backwards compatible.
Here is an example of fixing CVE security issues with patch files,
an example from the :oe_layerindex:`ffmpeg recipe</layerindex/recipe/47350>`::
SRC_URI = "https://www.ffmpeg.org/releases/${BP}.tar.xz \
file://0001-libavutil-include-assembly-with-full-path-from-sourc.patch \
file://fix-CVE-2020-20446.patch \
file://fix-CVE-2020-20453.patch \
file://fix-CVE-2020-22015.patch \
file://fix-CVE-2020-22021.patch \
file://fix-CVE-2020-22033-CVE-2020-22019.patch \
file://fix-CVE-2021-33815.patch \
A good practice is to include the CVE identifier in both the patch file name
and inside the patch file commit message using the format::
CVE: CVE-2020-22033
CVE checker will then capture this information and change the CVE status to ``Patched``
in the generated reports.
If analysis shows that the CVE issue does not impact the recipe due to configuration, platform,
version or other reasons, the CVE can be marked as ``Ignored`` using the :term:`CVE_CHECK_IGNORE` variable.
As mentioned previously, if data in the CVE database is wrong, it is recommend to fix those
issues in the CVE database directly.
Recipes can be completely skipped by CVE check by including the recipe name in
the :term:`CVE_CHECK_SKIP_RECIPE` variable.
Implementation details
======================
Here's what the :ref:`ref-classes-cve-check` class does to find unpatched CVE IDs.
First the code goes through each patch file provided by a recipe. If a valid CVE ID
is found in the name of the file, the corresponding CVE is considered as patched.
Don't forget that if multiple CVE IDs are found in the filename, only the last
one is considered. Then, the code looks for ``CVE: CVE-ID`` lines in the patch
file. The found CVE IDs are also considered as patched.
Then, the code looks up all the CVE IDs in the NIST database for all the
products defined in :term:`CVE_PRODUCT`. Then, for each found CVE:
- If the package name (:term:`PN`) is part of
:term:`CVE_CHECK_SKIP_RECIPE`, it is considered as ``Patched``.
- If the CVE ID is part of :term:`CVE_CHECK_IGNORE`, it is
set as ``Ignored``.
- If the CVE ID is part of the patched CVE for the recipe, it is
already considered as ``Patched``.
- Otherwise, the code checks whether the recipe version (:term:`PV`)
is within the range of versions impacted by the CVE. If so, the CVE
is considered as ``Unpatched``.
The CVE database is stored in :term:`DL_DIR` and can be inspected using
``sqlite3`` command as follows::
sqlite3 downloads/CVE_CHECK/nvdcve_1.1.db .dump | grep CVE-2021-37462
When analyzing CVEs, it is recommended to:
- study the latest information in `CVE database <https://nvd.nist.gov/vuln/search>`__.
- check how upstream developers of the software component addressed the issue, e.g.
what patch was applied, which upstream release contains the fix.
- check what other Linux distributions like `Debian <https://security-tracker.debian.org/tracker/>`__
did to analyze and address the issue.
- follow security notices from other Linux distributions.
- follow public `open source security mailing lists <https://oss-security.openwall.org/wiki/mailing-lists>`__ for
discussions and advance notifications of CVE bugs and software releases with fixes.
poky/documentation/dev-manual/wayland.rst
+90
View File
@@ -0,0 +1,90 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Using Wayland and Weston
************************
:wikipedia:`Wayland <Wayland_(display_server_protocol)>`
is a computer display server protocol that provides a method for
compositing window managers to communicate directly with applications
and video hardware and expects them to communicate with input hardware
using other libraries. Using Wayland with supporting targets can result
in better control over graphics frame rendering than an application
might otherwise achieve.
The Yocto Project provides the Wayland protocol libraries and the
reference :wikipedia:`Weston <Wayland_(display_server_protocol)#Weston>`
compositor as part of its release. You can find the integrated packages
in the ``meta`` layer of the :term:`Source Directory`.
Specifically, you
can find the recipes that build both Wayland and Weston at
``meta/recipes-graphics/wayland``.
You can build both the Wayland and Weston packages for use only with targets
that accept the :wikipedia:`Mesa 3D and Direct Rendering Infrastructure
<Mesa_(computer_graphics)>`, which is also known as Mesa DRI. This implies that
you cannot build and use the packages if your target uses, for example, the
Intel Embedded Media and Graphics Driver (Intel EMGD) that overrides Mesa DRI.
.. note::
Due to lack of EGL support, Weston 1.0.3 will not run directly on the
emulated QEMU hardware. However, this version of Weston will run
under X emulation without issues.
This section describes what you need to do to implement Wayland and use
the Weston compositor when building an image for a supporting target.
Enabling Wayland in an Image
============================
To enable Wayland, you need to enable it to be built and enable it to be
included (installed) in the image.
Building Wayland
----------------
To cause Mesa to build the ``wayland-egl`` platform and Weston to build
Wayland with Kernel Mode Setting
(`KMS <https://wiki.archlinux.org/index.php/Kernel_Mode_Setting>`__)
support, include the "wayland" flag in the
:term:`DISTRO_FEATURES`
statement in your ``local.conf`` file::
DISTRO_FEATURES:append = " wayland"
.. note::
If X11 has been enabled elsewhere, Weston will build Wayland with X11
support
Installing Wayland and Weston
-----------------------------
To install the Wayland feature into an image, you must include the
following
:term:`CORE_IMAGE_EXTRA_INSTALL`
statement in your ``local.conf`` file::
CORE_IMAGE_EXTRA_INSTALL += "wayland weston"
Running Weston
==============
To run Weston inside X11, enabling it as described earlier and building
a Sato image is sufficient. If you are running your image under Sato, a
Weston Launcher appears in the "Utility" category.
Alternatively, you can run Weston through the command-line interpretor
(CLI), which is better suited for development work. To run Weston under
the CLI, you need to do the following after your image is built:
#. Run these commands to export ``XDG_RUNTIME_DIR``::
mkdir -p /tmp/$USER-weston
chmod 0700 /tmp/$USER-weston
export XDG_RUNTIME_DIR=/tmp/$USER-weston
#. Launch Weston in the shell::
weston
poky/documentation/dev-manual/wic.rst
+729
View File
@@ -0,0 +1,729 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Creating Partitioned Images Using Wic
*************************************
Creating an image for a particular hardware target using the
OpenEmbedded build system does not necessarily mean you can boot that
image as is on your device. Physical devices accept and boot images in
various ways depending on the specifics of the device. Usually,
information about the hardware can tell you what image format the device
requires. Should your device require multiple partitions on an SD card,
flash, or an HDD, you can use the OpenEmbedded Image Creator, Wic, to
create the properly partitioned image.
The ``wic`` command generates partitioned images from existing
OpenEmbedded build artifacts. Image generation is driven by partitioning
commands contained in an OpenEmbedded kickstart file (``.wks``)
specified either directly on the command line or as one of a selection
of canned kickstart files as shown with the ``wic list images`` command
in the
":ref:`dev-manual/wic:generate an image using an existing kickstart file`"
section. When you apply the command to a given set of build artifacts, the
result is an image or set of images that can be directly written onto media and
used on a particular system.
.. note::
For a kickstart file reference, see the
":ref:`ref-manual/kickstart:openembedded kickstart (\`\`.wks\`\`) reference`"
Chapter in the Yocto Project Reference Manual.
The ``wic`` command and the infrastructure it is based on is by
definition incomplete. The purpose of the command is to allow the
generation of customized images, and as such, was designed to be
completely extensible through a plugin interface. See the
":ref:`dev-manual/wic:using the wic plugin interface`" section
for information on these plugins.
This section provides some background information on Wic, describes what
you need to have in place to run the tool, provides instruction on how
to use the Wic utility, provides information on using the Wic plugins
interface, and provides several examples that show how to use Wic.
Background
==========
This section provides some background on the Wic utility. While none of
this information is required to use Wic, you might find it interesting.
- The name "Wic" is derived from OpenEmbedded Image Creator (oeic). The
"oe" diphthong in "oeic" was promoted to the letter "w", because
"oeic" is both difficult to remember and to pronounce.
- Wic is loosely based on the Meego Image Creator (``mic``) framework.
The Wic implementation has been heavily modified to make direct use
of OpenEmbedded build artifacts instead of package installation and
configuration, which are already incorporated within the OpenEmbedded
artifacts.
- Wic is a completely independent standalone utility that initially
provides easier-to-use and more flexible replacements for an existing
functionality in OE-Core's :ref:`ref-classes-image-live`
class. The difference between Wic and those examples is that with Wic
the functionality of those scripts is implemented by a
general-purpose partitioning language, which is based on Redhat
kickstart syntax.
Requirements
============
In order to use the Wic utility with the OpenEmbedded Build system, your
system needs to meet the following requirements:
- The Linux distribution on your development host must support the
Yocto Project. See the ":ref:`system-requirements-supported-distros`"
section in the Yocto Project Reference Manual for the list of
distributions that support the Yocto Project.
- The standard system utilities, such as ``cp``, must be installed on
your development host system.
- You must have sourced the build environment setup script (i.e.
:ref:`structure-core-script`) found in the :term:`Build Directory`.
- You need to have the build artifacts already available, which
typically means that you must have already created an image using the
OpenEmbedded build system (e.g. ``core-image-minimal``). While it
might seem redundant to generate an image in order to create an image
using Wic, the current version of Wic requires the artifacts in the
form generated by the OpenEmbedded build system.
- You must build several native tools, which are built to run on the
build system::
$ bitbake parted-native dosfstools-native mtools-native
- Include "wic" as part of the
:term:`IMAGE_FSTYPES`
variable.
- Include the name of the :ref:`wic kickstart file <openembedded-kickstart-wks-reference>`
as part of the :term:`WKS_FILE` variable. If multiple candidate files can
be provided by different layers, specify all the possible names through the
:term:`WKS_FILES` variable instead.
Getting Help
============
You can get general help for the ``wic`` command by entering the ``wic``
command by itself or by entering the command with a help argument as
follows::
$ wic -h
$ wic --help
$ wic help
Currently, Wic supports seven commands: ``cp``, ``create``, ``help``,
``list``, ``ls``, ``rm``, and ``write``. You can get help for all these
commands except "help" by using the following form::
$ wic help command
For example, the following command returns help for the ``write``
command::
$ wic help write
Wic supports help for three topics: ``overview``, ``plugins``, and
``kickstart``. You can get help for any topic using the following form::
$ wic help topic
For example, the following returns overview help for Wic::
$ wic help overview
There is one additional level of help for Wic. You can get help on
individual images through the ``list`` command. You can use the ``list``
command to return the available Wic images as follows::
$ wic list images
genericx86 Create an EFI disk image for genericx86*
edgerouter Create SD card image for Edgerouter
beaglebone-yocto Create SD card image for Beaglebone
qemux86-directdisk Create a qemu machine 'pcbios' direct disk image
systemd-bootdisk Create an EFI disk image with systemd-boot
mkhybridiso Create a hybrid ISO image
mkefidisk Create an EFI disk image
sdimage-bootpart Create SD card image with a boot partition
directdisk-multi-rootfs Create multi rootfs image using rootfs plugin
directdisk Create a 'pcbios' direct disk image
directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config
qemuriscv Create qcow2 image for RISC-V QEMU machines
directdisk-gpt Create a 'pcbios' direct disk image
efi-bootdisk
Once you know the list of available
Wic images, you can use ``help`` with the command to get help on a
particular image. For example, the following command returns help on the
"beaglebone-yocto" image::
$ wic list beaglebone-yocto help
Creates a partitioned SD card image for Beaglebone.
Boot files are located in the first vfat partition.
Operational Modes
=================
You can use Wic in two different modes, depending on how much control
you need for specifying the OpenEmbedded build artifacts that are used
for creating the image: Raw and Cooked:
- *Raw Mode:* You explicitly specify build artifacts through Wic
command-line arguments.
- *Cooked Mode:* The current
:term:`MACHINE` setting and image
name are used to automatically locate and provide the build
artifacts. You just supply a kickstart file and the name of the image
from which to use artifacts.
Regardless of the mode you use, you need to have the build artifacts
ready and available.
Raw Mode
--------
Running Wic in raw mode allows you to specify all the partitions through
the ``wic`` command line. The primary use for raw mode is if you have
built your kernel outside of the Yocto Project :term:`Build Directory`.
In other words, you can point to arbitrary kernel, root filesystem locations,
and so forth. Contrast this behavior with cooked mode where Wic looks in the
:term:`Build Directory` (e.g. ``tmp/deploy/images/``\ machine).
The general form of the ``wic`` command in raw mode is::
$ wic create wks_file options ...
Where:
wks_file:
An OpenEmbedded kickstart file. You can provide
your own custom file or use a file from a set of
existing files as described by further options.
optional arguments:
-h, --help show this help message and exit
-o OUTDIR, --outdir OUTDIR
name of directory to create image in
-e IMAGE_NAME, --image-name IMAGE_NAME
name of the image to use the artifacts from e.g. core-
image-sato
-r ROOTFS_DIR, --rootfs-dir ROOTFS_DIR
path to the /rootfs dir to use as the .wks rootfs
source
-b BOOTIMG_DIR, --bootimg-dir BOOTIMG_DIR
path to the dir containing the boot artifacts (e.g.
/EFI or /syslinux dirs) to use as the .wks bootimg
source
-k KERNEL_DIR, --kernel-dir KERNEL_DIR
path to the dir containing the kernel to use in the
.wks bootimg
-n NATIVE_SYSROOT, --native-sysroot NATIVE_SYSROOT
path to the native sysroot containing the tools to use
to build the image
-s, --skip-build-check
skip the build check
-f, --build-rootfs build rootfs
-c {gzip,bzip2,xz}, --compress-with {gzip,bzip2,xz}
compress image with specified compressor
-m, --bmap generate .bmap
--no-fstab-update Do not change fstab file.
-v VARS_DIR, --vars VARS_DIR
directory with <image>.env files that store bitbake
variables
-D, --debug output debug information
.. note::
You do not need root privileges to run Wic. In fact, you should not
run as root when using the utility.
Cooked Mode
-----------
Running Wic in cooked mode leverages off artifacts in the
:term:`Build Directory`. In other words, you do not have to specify kernel or
root filesystem locations as part of the command. All you need to provide is
a kickstart file and the name of the image from which to use artifacts
by using the "-e" option. Wic looks in the :term:`Build Directory` (e.g.
``tmp/deploy/images/``\ machine) for artifacts.
The general form of the ``wic`` command using Cooked Mode is as follows::
$ wic create wks_file -e IMAGE_NAME
Where:
wks_file:
An OpenEmbedded kickstart file. You can provide
your own custom file or use a file from a set of
existing files provided with the Yocto Project
release.
required argument:
-e IMAGE_NAME, --image-name IMAGE_NAME
name of the image to use the artifacts from e.g. core-
image-sato
Using an Existing Kickstart File
================================
If you do not want to create your own kickstart file, you can use an
existing file provided by the Wic installation. As shipped, kickstart
files can be found in the :ref:`overview-manual/development-environment:yocto project source repositories` in the
following two locations::
poky/meta-yocto-bsp/wic
poky/scripts/lib/wic/canned-wks
Use the following command to list the available kickstart files::
$ wic list images
genericx86 Create an EFI disk image for genericx86*
beaglebone-yocto Create SD card image for Beaglebone
edgerouter Create SD card image for Edgerouter
qemux86-directdisk Create a QEMU machine 'pcbios' direct disk image
directdisk-gpt Create a 'pcbios' direct disk image
mkefidisk Create an EFI disk image
directdisk Create a 'pcbios' direct disk image
systemd-bootdisk Create an EFI disk image with systemd-boot
mkhybridiso Create a hybrid ISO image
sdimage-bootpart Create SD card image with a boot partition
directdisk-multi-rootfs Create multi rootfs image using rootfs plugin
directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config
When you use an existing file, you
do not have to use the ``.wks`` extension. Here is an example in Raw
Mode that uses the ``directdisk`` file::
$ wic create directdisk -r rootfs_dir -b bootimg_dir \
-k kernel_dir -n native_sysroot
Here are the actual partition language commands used in the
``genericx86.wks`` file to generate an image::
# short-description: Create an EFI disk image for genericx86*
# long-description: Creates a partitioned EFI disk image for genericx86* machines
part /boot --source bootimg-efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos --active --align 1024
part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid
part swap --ondisk sda --size 44 --label swap1 --fstype=swap
bootloader --ptable gpt --timeout=5 --append="rootfstype=ext4 console=ttyS0,115200 console=tty0"
Using the Wic Plugin Interface
==============================
You can extend and specialize Wic functionality by using Wic plugins.
This section explains the Wic plugin interface.
.. note::
Wic plugins consist of "source" and "imager" plugins. Imager plugins
are beyond the scope of this section.
Source plugins provide a mechanism to customize partition content during
the Wic image generation process. You can use source plugins to map
values that you specify using ``--source`` commands in kickstart files
(i.e. ``*.wks``) to a plugin implementation used to populate a given
partition.
.. note::
If you use plugins that have build-time dependencies (e.g. native
tools, bootloaders, and so forth) when building a Wic image, you need
to specify those dependencies using the :term:`WKS_FILE_DEPENDS`
variable.
Source plugins are subclasses defined in plugin files. As shipped, the
Yocto Project provides several plugin files. You can see the source
plugin files that ship with the Yocto Project
:yocto_git:`here </poky/tree/scripts/lib/wic/plugins/source>`.
Each of these plugin files contains source plugins that are designed to
populate a specific Wic image partition.
Source plugins are subclasses of the ``SourcePlugin`` class, which is
defined in the ``poky/scripts/lib/wic/pluginbase.py`` file. For example,
the ``BootimgEFIPlugin`` source plugin found in the ``bootimg-efi.py``
file is a subclass of the ``SourcePlugin`` class, which is found in the
``pluginbase.py`` file.
You can also implement source plugins in a layer outside of the Source
Repositories (external layer). To do so, be sure that your plugin files
are located in a directory whose path is
``scripts/lib/wic/plugins/source/`` within your external layer. When the
plugin files are located there, the source plugins they contain are made
available to Wic.
When the Wic implementation needs to invoke a partition-specific
implementation, it looks for the plugin with the same name as the
``--source`` parameter used in the kickstart file given to that
partition. For example, if the partition is set up using the following
command in a kickstart file::
part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024
The methods defined as class
members of the matching source plugin (i.e. ``bootimg-pcbios``) in the
``bootimg-pcbios.py`` plugin file are used.
To be more concrete, here is the corresponding plugin definition from
the ``bootimg-pcbios.py`` file for the previous command along with an
example method called by the Wic implementation when it needs to prepare
a partition using an implementation-specific function::
.
.
.
class BootimgPcbiosPlugin(SourcePlugin):
"""
Create MBR boot partition and install syslinux on it.
"""
name = 'bootimg-pcbios'
.
.
.
@classmethod
def do_prepare_partition(cls, part, source_params, creator, cr_workdir,
oe_builddir, bootimg_dir, kernel_dir,
rootfs_dir, native_sysroot):
"""
Called to do the actual content population for a partition i.e. it
'prepares' the partition to be incorporated into the image.
In this case, prepare content for legacy bios boot partition.
"""
.
.
.
If a
subclass (plugin) itself does not implement a particular function, Wic
locates and uses the default version in the superclass. It is for this
reason that all source plugins are derived from the ``SourcePlugin``
class.
The ``SourcePlugin`` class defined in the ``pluginbase.py`` file defines
a set of methods that source plugins can implement or override. Any
plugins (subclass of ``SourcePlugin``) that do not implement a
particular method inherit the implementation of the method from the
``SourcePlugin`` class. For more information, see the ``SourcePlugin``
class in the ``pluginbase.py`` file for details:
The following list describes the methods implemented in the
``SourcePlugin`` class:
- ``do_prepare_partition()``: Called to populate a partition with
actual content. In other words, the method prepares the final
partition image that is incorporated into the disk image.
- ``do_configure_partition()``: Called before
``do_prepare_partition()`` to create custom configuration files for a
partition (e.g. syslinux or grub configuration files).
- ``do_install_disk()``: Called after all partitions have been
prepared and assembled into a disk image. This method provides a hook
to allow finalization of a disk image (e.g. writing an MBR).
- ``do_stage_partition()``: Special content-staging hook called
before ``do_prepare_partition()``. This method is normally empty.
Typically, a partition just uses the passed-in parameters (e.g. the
unmodified value of ``bootimg_dir``). However, in some cases, things
might need to be more tailored. As an example, certain files might
additionally need to be taken from ``bootimg_dir + /boot``. This hook
allows those files to be staged in a customized fashion.
.. note::
``get_bitbake_var()`` allows you to access non-standard variables that
you might want to use for this behavior.
You can extend the source plugin mechanism. To add more hooks, create
more source plugin methods within ``SourcePlugin`` and the corresponding
derived subclasses. The code that calls the plugin methods uses the
``plugin.get_source_plugin_methods()`` function to find the method or
methods needed by the call. Retrieval of those methods is accomplished
by filling up a dict with keys that contain the method names of
interest. On success, these will be filled in with the actual methods.
See the Wic implementation for examples and details.
Wic Examples
============
This section provides several examples that show how to use the Wic
utility. All the examples assume the list of requirements in the
":ref:`dev-manual/wic:requirements`" section have been met. The
examples assume the previously generated image is
``core-image-minimal``.
Generate an Image using an Existing Kickstart File
--------------------------------------------------
This example runs in Cooked Mode and uses the ``mkefidisk`` kickstart
file::
$ wic create mkefidisk -e core-image-minimal
INFO: Building wic-tools...
.
.
.
INFO: The new image(s) can be found here:
./mkefidisk-201804191017-sda.direct
The following build artifacts were used to create the image(s):
ROOTFS_DIR: /home/stephano/yocto/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs
BOOTIMG_DIR: /home/stephano/yocto/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
KERNEL_DIR: /home/stephano/yocto/build/tmp-glibc/deploy/images/qemux86
NATIVE_SYSROOT: /home/stephano/yocto/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native
INFO: The image(s) were created using OE kickstart file:
/home/stephano/yocto/openembedded-core/scripts/lib/wic/canned-wks/mkefidisk.wks
The previous example shows the easiest way to create an image by running
in cooked mode and supplying a kickstart file and the "-e" option to
point to the existing build artifacts. Your ``local.conf`` file needs to
have the :term:`MACHINE` variable set
to the machine you are using, which is "qemux86" in this example.
Once the image builds, the output provides image location, artifact use,
and kickstart file information.
.. note::
You should always verify the details provided in the output to make
sure that the image was indeed created exactly as expected.
Continuing with the example, you can now write the image from the
:term:`Build Directory` onto a USB stick, or whatever media for which you
built your image, and boot from the media. You can write the image by using
``bmaptool`` or ``dd``::
$ oe-run-native bmap-tools-native bmaptool copy mkefidisk-201804191017-sda.direct /dev/sdX
or ::
$ sudo dd if=mkefidisk-201804191017-sda.direct of=/dev/sdX
.. note::
For more information on how to use the ``bmaptool``
to flash a device with an image, see the
":ref:`dev-manual/bmaptool:flashing images using \`\`bmaptool\`\``"
section.
Using a Modified Kickstart File
-------------------------------
Because partitioned image creation is driven by the kickstart file, it
is easy to affect image creation by changing the parameters in the file.
This next example demonstrates that through modification of the
``directdisk-gpt`` kickstart file.
As mentioned earlier, you can use the command ``wic list images`` to
show the list of existing kickstart files. The directory in which the
``directdisk-gpt.wks`` file resides is
``scripts/lib/image/canned-wks/``, which is located in the
:term:`Source Directory` (e.g. ``poky``).
Because available files reside in this directory, you can create and add
your own custom files to the directory. Subsequent use of the
``wic list images`` command would then include your kickstart files.
In this example, the existing ``directdisk-gpt`` file already does most
of what is needed. However, for the hardware in this example, the image
will need to boot from ``sdb`` instead of ``sda``, which is what the
``directdisk-gpt`` kickstart file uses.
The example begins by making a copy of the ``directdisk-gpt.wks`` file
in the ``scripts/lib/image/canned-wks`` directory and then by changing
the lines that specify the target disk from which to boot::
$ cp /home/stephano/yocto/poky/scripts/lib/wic/canned-wks/directdisk-gpt.wks \
/home/stephano/yocto/poky/scripts/lib/wic/canned-wks/directdisksdb-gpt.wks
Next, the example modifies the ``directdisksdb-gpt.wks`` file and
changes all instances of "``--ondisk sda``" to "``--ondisk sdb``". The
example changes the following two lines and leaves the remaining lines
untouched::
part /boot --source bootimg-pcbios --ondisk sdb --label boot --active --align 1024
part / --source rootfs --ondisk sdb --fstype=ext4 --label platform --align 1024 --use-uuid
Once the lines are changed, the
example generates the ``directdisksdb-gpt`` image. The command points
the process at the ``core-image-minimal`` artifacts for the Next Unit of
Computing (nuc) :term:`MACHINE` the
``local.conf``::
$ wic create directdisksdb-gpt -e core-image-minimal
INFO: Building wic-tools...
.
.
.
Initialising tasks: 100% |#######################################| Time: 0:00:01
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 1161 tasks of which 1157 didn't need to be rerun and all succeeded.
INFO: Creating image(s)...
INFO: The new image(s) can be found here:
./directdisksdb-gpt-201710090938-sdb.direct
The following build artifacts were used to create the image(s):
ROOTFS_DIR: /home/stephano/yocto/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs
BOOTIMG_DIR: /home/stephano/yocto/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
KERNEL_DIR: /home/stephano/yocto/build/tmp-glibc/deploy/images/qemux86
NATIVE_SYSROOT: /home/stephano/yocto/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native
INFO: The image(s) were created using OE kickstart file:
/home/stephano/yocto/poky/scripts/lib/wic/canned-wks/directdisksdb-gpt.wks
Continuing with the example, you can now directly ``dd`` the image to a
USB stick, or whatever media for which you built your image, and boot
the resulting media::
$ sudo dd if=directdisksdb-gpt-201710090938-sdb.direct of=/dev/sdb
140966+0 records in
140966+0 records out
72174592 bytes (72 MB, 69 MiB) copied, 78.0282 s, 925 kB/s
$ sudo eject /dev/sdb
Using a Modified Kickstart File and Running in Raw Mode
-------------------------------------------------------
This next example manually specifies each build artifact (runs in Raw
Mode) and uses a modified kickstart file. The example also uses the
``-o`` option to cause Wic to create the output somewhere other than the
default output directory, which is the current directory::
$ wic create test.wks -o /home/stephano/testwic \
--rootfs-dir /home/stephano/yocto/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/rootfs \
--bootimg-dir /home/stephano/yocto/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share \
--kernel-dir /home/stephano/yocto/build/tmp/deploy/images/qemux86 \
--native-sysroot /home/stephano/yocto/build/tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot-native
INFO: Creating image(s)...
INFO: The new image(s) can be found here:
/home/stephano/testwic/test-201710091445-sdb.direct
The following build artifacts were used to create the image(s):
ROOTFS_DIR: /home/stephano/yocto/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs
BOOTIMG_DIR: /home/stephano/yocto/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share
KERNEL_DIR: /home/stephano/yocto/build/tmp-glibc/deploy/images/qemux86
NATIVE_SYSROOT: /home/stephano/yocto/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native
INFO: The image(s) were created using OE kickstart file:
test.wks
For this example,
:term:`MACHINE` did not have to be
specified in the ``local.conf`` file since the artifact is manually
specified.
Using Wic to Manipulate an Image
--------------------------------
Wic image manipulation allows you to shorten turnaround time during
image development. For example, you can use Wic to delete the kernel
partition of a Wic image and then insert a newly built kernel. This
saves you time from having to rebuild the entire image each time you
modify the kernel.
.. note::
In order to use Wic to manipulate a Wic image as in this example,
your development machine must have the ``mtools`` package installed.
The following example examines the contents of the Wic image, deletes
the existing kernel, and then inserts a new kernel:
#. *List the Partitions:* Use the ``wic ls`` command to list all the
partitions in the Wic image::
$ wic ls tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic
Num Start End Size Fstype
1 1048576 25041919 23993344 fat16
2 25165824 72157183 46991360 ext4
The previous output shows two partitions in the
``core-image-minimal-qemux86.wic`` image.
#. *Examine a Particular Partition:* Use the ``wic ls`` command again
but in a different form to examine a particular partition.
.. note::
You can get command usage on any Wic command using the following
form::
$ wic help command
For example, the following command shows you the various ways to
use the
wic ls
command::
$ wic help ls
The following command shows what is in partition one::
$ wic ls tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1
Volume in drive : is boot
Volume Serial Number is E894-1809
Directory for ::/
libcom32 c32 186500 2017-10-09 16:06
libutil c32 24148 2017-10-09 16:06
syslinux cfg 220 2017-10-09 16:06
vesamenu c32 27104 2017-10-09 16:06
vmlinuz 6904608 2017-10-09 16:06
5 files 7 142 580 bytes
16 582 656 bytes free
The previous output shows five files, with the
``vmlinuz`` being the kernel.
.. note::
If you see the following error, you need to update or create a
``~/.mtoolsrc`` file and be sure to have the line "mtools_skip_check=1"
in the file. Then, run the Wic command again::
ERROR: _exec_cmd: /usr/bin/mdir -i /tmp/wic-parttfokuwra ::/ returned '1' instead of 0
output: Total number of sectors (47824) not a multiple of sectors per track (32)!
Add mtools_skip_check=1 to your .mtoolsrc file to skip this test
#. *Remove the Old Kernel:* Use the ``wic rm`` command to remove the
``vmlinuz`` file (kernel)::
$ wic rm tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1/vmlinuz
#. *Add In the New Kernel:* Use the ``wic cp`` command to add the
updated kernel to the Wic image. Depending on how you built your
kernel, it could be in different places. If you used ``devtool`` and
an SDK to build your kernel, it resides in the ``tmp/work`` directory
of the extensible SDK. If you used ``make`` to build the kernel, the
kernel will be in the ``workspace/sources`` area.
The following example assumes ``devtool`` was used to build the
kernel::
$ wic cp poky_sdk/tmp/work/qemux86-poky-linux/linux-yocto/4.12.12+git999-r0/linux-yocto-4.12.12+git999/arch/x86/boot/bzImage \
poky/build/tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1/vmlinuz
Once the new kernel is added back into the image, you can use the
``dd`` command or :ref:`bmaptool
<dev-manual/bmaptool:flashing images using \`\`bmaptool\`\`>`
to flash your wic image onto an SD card or USB stick and test your
target.
.. note::
Using ``bmaptool`` is generally 10 to 20 times faster than using ``dd``.
poky/documentation/dev-manual/x32-psabi.rst
+54
View File
@@ -0,0 +1,54 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Using x32 psABI
***************
x32 processor-specific Application Binary Interface (`x32
psABI <https://software.intel.com/en-us/node/628948>`__) is a native
32-bit processor-specific ABI for Intel 64 (x86-64) architectures. An
ABI defines the calling conventions between functions in a processing
environment. The interface determines what registers are used and what
the sizes are for various C data types.
Some processing environments prefer using 32-bit applications even when
running on Intel 64-bit platforms. Consider the i386 psABI, which is a
very old 32-bit ABI for Intel 64-bit platforms. The i386 psABI does not
provide efficient use and access of the Intel 64-bit processor
resources, leaving the system underutilized. Now consider the x86_64
psABI. This ABI is newer and uses 64-bits for data sizes and program
pointers. The extra bits increase the footprint size of the programs,
libraries, and also increases the memory and file system size
requirements. Executing under the x32 psABI enables user programs to
utilize CPU and system resources more efficiently while keeping the
memory footprint of the applications low. Extra bits are used for
registers but not for addressing mechanisms.
The Yocto Project supports the final specifications of x32 psABI as
follows:
- You can create packages and images in x32 psABI format on x86_64
architecture targets.
- You can successfully build recipes with the x32 toolchain.
- You can create and boot ``core-image-minimal`` and
``core-image-sato`` images.
- There is RPM Package Manager (RPM) support for x32 binaries.
- There is support for large images.
To use the x32 psABI, you need to edit your ``conf/local.conf``
configuration file as follows::
MACHINE = "qemux86-64"
DEFAULTTUNE = "x86-64-x32"
baselib = "${@d.getVar('BASE_LIB:tune-' + (d.getVar('DEFAULTTUNE') \
or 'INVALID')) or 'lib'}"
Once you have set
up your configuration file, use BitBake to build an image that supports
the x32 psABI. Here is an example::
$ bitbake core-image-sato
poky/documentation/figures/yp-how-it-works-new-diagram.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 214 KiB

poky/documentation/genindex.rst
+5
View File
@@ -0,0 +1,5 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
=====
Index
=====
poky/documentation/index.rst
+52
View File
@@ -0,0 +1,52 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
.. The Yocto Project documentation master file, created by
sphinx-quickstart on Mon Apr 13 09:38:33 2020.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to the Yocto Project Documentation
==========================================
|
.. toctree::
:maxdepth: 1
:caption: Introduction and Overview
Quick Build <brief-yoctoprojectqs/index>
what-i-wish-id-known
transitioning-to-a-custom-environment
Yocto Project Software Overview <https://www.yoctoproject.org/software-overview/>
Tips and Tricks Wiki <https://wiki.yoctoproject.org/wiki/TipsAndTricks>
.. toctree::
:maxdepth: 1
:caption: Manuals
Overview and Concepts Manual <overview-manual/index>
Reference Manual <ref-manual/index>
Board Support Package (BSP) Developer's guide <bsp-guide/index>
Development Tasks Manual <dev-manual/index>
Linux Kernel Development Manual <kernel-dev/index>
Profile and Tracing Manual <profile-manual/index>
Application Development and the Extensible SDK (eSDK) <sdk-manual/index>
Toaster Manual <toaster-manual/index>
Test Environment Manual <test-manual/index>
bitbake
.. toctree::
:maxdepth: 1
:caption: Release Manuals
:hidden:
Release Information <migration-guides/index>
releases
.. toctree::
:maxdepth: 1
:caption: Documentation Index
:hidden:
genindex
poky/documentation/kernel-dev/advanced.rst
+913
View File
@@ -0,0 +1,913 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
*******************************************************
Working with Advanced Metadata (``yocto-kernel-cache``)
*******************************************************
Overview
========
In addition to supporting configuration fragments and patches, the Yocto
Project kernel tools also support rich
:term:`Metadata` that you can use to define
complex policies and Board Support Package (BSP) support. The purpose of
the Metadata and the tools that manage it is to help you manage the
complexity of the configuration and sources used to support multiple
BSPs and Linux kernel types.
Kernel Metadata exists in many places. One area in the
:ref:`overview-manual/development-environment:yocto project source repositories`
is the ``yocto-kernel-cache`` Git repository. You can find this repository
grouped under the "Yocto Linux Kernel" heading in the
:yocto_git:`Yocto Project Source Repositories <>`.
Kernel development tools ("kern-tools") are also available in the Yocto Project
Source Repositories under the "Yocto Linux Kernel" heading in the
``yocto-kernel-tools`` Git repository. The recipe that builds these
tools is ``meta/recipes-kernel/kern-tools/kern-tools-native_git.bb`` in
the :term:`Source Directory` (e.g.
``poky``).
Using Kernel Metadata in a Recipe
=================================
As mentioned in the introduction, the Yocto Project contains kernel
Metadata, which is located in the ``yocto-kernel-cache`` Git repository.
This Metadata defines Board Support Packages (BSPs) that correspond to
definitions in linux-yocto recipes for corresponding BSPs. A BSP
consists of an aggregation of kernel policy and enabled
hardware-specific features. The BSP can be influenced from within the
linux-yocto recipe.
.. note::
A Linux kernel recipe that contains kernel Metadata (e.g. inherits
from the ``linux-yocto.inc`` file) is said to be a "linux-yocto style" recipe.
Every linux-yocto style recipe must define the
:term:`KMACHINE` variable. This
variable is typically set to the same value as the :term:`MACHINE` variable,
which is used by :term:`BitBake`.
However, in some cases, the variable might instead refer to the
underlying platform of the :term:`MACHINE`.
Multiple BSPs can reuse the same :term:`KMACHINE` name if they are built
using the same BSP description. Multiple Corei7-based BSPs could share
the same "intel-corei7-64" value for :term:`KMACHINE`. It is important to
realize that :term:`KMACHINE` is just for kernel mapping, while :term:`MACHINE`
is the machine type within a BSP Layer. Even with this distinction,
however, these two variables can hold the same value. See the
":ref:`kernel-dev/advanced:bsp descriptions`" section for more information.
Every linux-yocto style recipe must also indicate the Linux kernel
source repository branch used to build the Linux kernel. The
:term:`KBRANCH` variable must be set
to indicate the branch.
.. note::
You can use the :term:`KBRANCH` value to define an alternate branch typically
with a machine override as shown here from the ``meta-yocto-bsp`` layer::
KBRANCH:edgerouter = "standard/edgerouter"
The linux-yocto style recipes can optionally define the following
variables:
- :term:`KERNEL_FEATURES`
- :term:`LINUX_KERNEL_TYPE`
:term:`LINUX_KERNEL_TYPE`
defines the kernel type to be used in assembling the configuration. If
you do not specify a :term:`LINUX_KERNEL_TYPE`, it defaults to "standard".
Together with :term:`KMACHINE`, :term:`LINUX_KERNEL_TYPE` defines the search
arguments used by the kernel tools to find the appropriate description
within the kernel Metadata with which to build out the sources and
configuration. The linux-yocto recipes define "standard", "tiny", and
"preempt-rt" kernel types. See the ":ref:`kernel-dev/advanced:kernel types`"
section for more information on kernel types.
During the build, the kern-tools search for the BSP description file
that most closely matches the :term:`KMACHINE` and :term:`LINUX_KERNEL_TYPE`
variables passed in from the recipe. The tools use the first BSP
description they find that matches both variables. If the tools cannot find
a match, they issue a warning.
The tools first search for the :term:`KMACHINE` and then for the
:term:`LINUX_KERNEL_TYPE`. If the tools cannot find a partial match, they
will use the sources from the :term:`KBRANCH` and any configuration
specified in the :term:`SRC_URI`.
You can use the
:term:`KERNEL_FEATURES`
variable to include features (configuration fragments, patches, or both)
that are not already included by the :term:`KMACHINE` and
:term:`LINUX_KERNEL_TYPE` variable combination. For example, to include a
feature specified as "features/netfilter/netfilter.scc", specify::
KERNEL_FEATURES += "features/netfilter/netfilter.scc"
To include a
feature called "cfg/sound.scc" just for the ``qemux86`` machine,
specify::
KERNEL_FEATURES:append:qemux86 = " cfg/sound.scc"
The value of
the entries in :term:`KERNEL_FEATURES` are dependent on their location
within the kernel Metadata itself. The examples here are taken from the
``yocto-kernel-cache`` repository. Each branch of this repository
contains "features" and "cfg" subdirectories at the top-level. For more
information, see the ":ref:`kernel-dev/advanced:kernel metadata syntax`"
section.
Kernel Metadata Syntax
======================
The kernel Metadata consists of three primary types of files: ``scc``
[1]_ description files, configuration fragments, and patches. The
``scc`` files define variables and include or otherwise reference any of
the three file types. The description files are used to aggregate all
types of kernel Metadata into what ultimately describes the sources and
the configuration required to build a Linux kernel tailored to a
specific machine.
The ``scc`` description files are used to define two fundamental types
of kernel Metadata:
- Features
- Board Support Packages (BSPs)
Features aggregate sources in the form of patches and configuration
fragments into a modular reusable unit. You can use features to
implement conceptually separate kernel Metadata descriptions such as
pure configuration fragments, simple patches, complex features, and
kernel types. :ref:`kernel-dev/advanced:kernel types` define general kernel
features and policy to be reused in the BSPs.
BSPs define hardware-specific features and aggregate them with kernel
types to form the final description of what will be assembled and built.
While the kernel Metadata syntax does not enforce any logical separation
of configuration fragments, patches, features or kernel types, best
practices dictate a logical separation of these types of Metadata. The
following Metadata file hierarchy is recommended::
base/
bsp/
cfg/
features/
ktypes/
patches/
The ``bsp`` directory contains the :ref:`kernel-dev/advanced:bsp descriptions`.
The remaining directories all contain "features". Separating ``bsp`` from the
rest of the structure aids conceptualizing intended usage.
Use these guidelines to help place your ``scc`` description files within
the structure:
- If your file contains only configuration fragments, place the file in
the ``cfg`` directory.
- If your file contains only source-code fixes, place the file in the
``patches`` directory.
- If your file encapsulates a major feature, often combining sources
and configurations, place the file in ``features`` directory.
- If your file aggregates non-hardware configuration and patches in
order to define a base kernel policy or major kernel type to be
reused across multiple BSPs, place the file in ``ktypes`` directory.
These distinctions can easily become blurred --- especially as out-of-tree
features slowly merge upstream over time. Also, remember that how the
description files are placed is a purely logical organization and has no
impact on the functionality of the kernel Metadata. There is no impact
because all of ``cfg``, ``features``, ``patches``, and ``ktypes``,
contain "features" as far as the kernel tools are concerned.
Paths used in kernel Metadata files are relative to base, which is
either
:term:`FILESEXTRAPATHS` if
you are creating Metadata in
:ref:`recipe-space <kernel-dev/advanced:recipe-space metadata>`,
or the top level of
:yocto_git:`yocto-kernel-cache </yocto-kernel-cache/tree/>`
if you are creating
:ref:`kernel-dev/advanced:metadata outside the recipe-space`.
.. [1]
``scc`` stands for Series Configuration Control, but the naming has
less significance in the current implementation of the tooling than
it had in the past. Consider ``scc`` files to be description files.
Configuration
-------------
The simplest unit of kernel Metadata is the configuration-only feature.
This feature consists of one or more Linux kernel configuration
parameters in a configuration fragment file (``.cfg``) and a ``.scc``
file that describes the fragment.
As an example, consider the Symmetric Multi-Processing (SMP) fragment
used with the ``linux-yocto-4.12`` kernel as defined outside of the
recipe space (i.e. ``yocto-kernel-cache``). This Metadata consists of
two files: ``smp.scc`` and ``smp.cfg``. You can find these files in the
``cfg`` directory of the ``yocto-4.12`` branch in the
``yocto-kernel-cache`` Git repository::
cfg/smp.scc:
define KFEATURE_DESCRIPTION "Enable SMP for 32 bit builds"
define KFEATURE_COMPATIBILITY all
kconf hardware smp.cfg
cfg/smp.cfg:
CONFIG_SMP=y
CONFIG_SCHED_SMT=y
# Increase default NR_CPUS from 8 to 64 so that platform with
# more than 8 processors can be all activated at boot time
CONFIG_NR_CPUS=64
# The following is needed when setting NR_CPUS to something
# greater than 8 on x86 architectures, it should be automatically
# disregarded by Kconfig when using a different arch
CONFIG_X86_BIGSMP=y
You can find general information on configuration
fragment files in the ":ref:`kernel-dev/common:creating configuration fragments`" section.
Within the ``smp.scc`` file, the
:term:`KFEATURE_DESCRIPTION`
statement provides a short description of the fragment. Higher level
kernel tools use this description.
Also within the ``smp.scc`` file, the ``kconf`` command includes the
actual configuration fragment in an ``.scc`` file, and the "hardware"
keyword identifies the fragment as being hardware enabling, as opposed
to general policy, which would use the "non-hardware" keyword. The
distinction is made for the benefit of the configuration validation
tools, which warn you if a hardware fragment overrides a policy set by a
non-hardware fragment.
.. note::
The description file can include multiple ``kconf`` statements, one per
fragment.
As described in the
":ref:`kernel-dev/common:validating configuration`" section, you can
use the following BitBake command to audit your configuration::
$ bitbake linux-yocto -c kernel_configcheck -f
Patches
-------
Patch descriptions are very similar to configuration fragment
descriptions, which are described in the previous section. However,
instead of a ``.cfg`` file, these descriptions work with source patches
(i.e. ``.patch`` files).
A typical patch includes a description file and the patch itself. As an
example, consider the build patches used with the ``linux-yocto-4.12``
kernel as defined outside of the recipe space (i.e.
``yocto-kernel-cache``). This Metadata consists of several files:
``build.scc`` and a set of ``*.patch`` files. You can find these files
in the ``patches/build`` directory of the ``yocto-4.12`` branch in the
``yocto-kernel-cache`` Git repository.
The following listings show the ``build.scc`` file and part of the
``modpost-mask-trivial-warnings.patch`` file::
patches/build/build.scc:
patch arm-serialize-build-targets.patch
patch powerpc-serialize-image-targets.patch
patch kbuild-exclude-meta-directory-from-distclean-processi.patch
# applied by kgit
# patch kbuild-add-meta-files-to-the-ignore-li.patch
patch modpost-mask-trivial-warnings.patch
patch menuconfig-check-lxdiaglog.sh-Allow-specification-of.patch
patches/build/modpost-mask-trivial-warnings.patch:
From bd48931bc142bdd104668f3a062a1f22600aae61 Mon Sep 17 00:00:00 2001
From: Paul Gortmaker <paul.gortmaker@windriver.com>
Date: Sun, 25 Jan 2009 17:58:09 -0500
Subject: [PATCH] modpost: mask trivial warnings
Newer HOSTCC will complain about various stdio fcns because
.
.
.
char *dump_write = NULL, *files_source = NULL;
int opt;
--
2.10.1
generated by cgit v0.10.2 at 2017-09-28 15:23:23 (GMT)
The description file can
include multiple patch statements where each statement handles a single
patch. In the example ``build.scc`` file, there are five patch statements
for the five patches in the directory.
You can create a typical ``.patch`` file using ``diff -Nurp`` or
``git format-patch`` commands. For information on how to create patches,
see the ":ref:`kernel-dev/common:using \`\`devtool\`\` to patch the kernel`"
and ":ref:`kernel-dev/common:using traditional kernel development to patch the kernel`"
sections.
Features
--------
Features are complex kernel Metadata types that consist of configuration
fragments, patches, and possibly other feature description files. As an
example, consider the following generic listing::
features/myfeature.scc
define KFEATURE_DESCRIPTION "Enable myfeature"
patch 0001-myfeature-core.patch
patch 0002-myfeature-interface.patch
include cfg/myfeature_dependency.scc
kconf non-hardware myfeature.cfg
This example shows how the ``patch`` and ``kconf`` commands are used as well
as how an additional feature description file is included with the
``include`` command.
Typically, features are less granular than configuration fragments and
are more likely than configuration fragments and patches to be the types
of things you want to specify in the :term:`KERNEL_FEATURES` variable of the
Linux kernel recipe. See the
":ref:`kernel-dev/advanced:using kernel metadata in a recipe`" section earlier
in the manual.
Kernel Types
------------
A kernel type defines a high-level kernel policy by aggregating non-hardware
configuration fragments with patches you want to use when building a Linux
kernel of a specific type (e.g. a real-time kernel). Syntactically, kernel
types are no different than features as described in the
":ref:`kernel-dev/advanced:features`" section. The :term:`LINUX_KERNEL_TYPE`
variable in the kernel recipe selects the kernel type. For example, in the
``linux-yocto_4.12.bb`` kernel recipe found in ``poky/meta/recipes-kernel/linux``, a
:ref:`require <bitbake-user-manual/bitbake-user-manual-metadata:\`\`require\`\` directive>`
directive includes the ``poky/meta/recipes-kernel/linux/linux-yocto.inc`` file,
which has the following statement that defines the default kernel type::
LINUX_KERNEL_TYPE ??= "standard"
Another example would be the real-time kernel (i.e.
``linux-yocto-rt_4.12.bb``). This kernel recipe directly sets the kernel
type as follows::
LINUX_KERNEL_TYPE = "preempt-rt"
.. note::
You can find kernel recipes in the ``meta/recipes-kernel/linux`` directory
of the :ref:`overview-manual/development-environment:yocto project source repositories`
(e.g. ``poky/meta/recipes-kernel/linux/linux-yocto_4.12.bb``). See the
":ref:`kernel-dev/advanced:using kernel metadata in a recipe`"
section for more information.
Three kernel types ("standard", "tiny", and "preempt-rt") are supported
for Linux Yocto kernels:
- "standard": Includes the generic Linux kernel policy of the Yocto
Project linux-yocto kernel recipes. This policy includes, among other
things, which file systems, networking options, core kernel features,
and debugging and tracing options are supported.
- "preempt-rt": Applies the ``PREEMPT_RT`` patches and the
configuration options required to build a real-time Linux kernel.
This kernel type inherits from the "standard" kernel type.
- "tiny": Defines a bare minimum configuration meant to serve as a base
for very small Linux kernels. The "tiny" kernel type is independent
from the "standard" configuration. Although the "tiny" kernel type
does not currently include any source changes, it might in the
future.
For any given kernel type, the Metadata is defined by the ``.scc`` (e.g.
``standard.scc``). Here is a partial listing for the ``standard.scc``
file, which is found in the ``ktypes/standard`` directory of the
``yocto-kernel-cache`` Git repository::
# Include this kernel type fragment to get the standard features and
# configuration values.
# Note: if only the features are desired, but not the configuration
# then this should be included as:
# include ktypes/standard/standard.scc nocfg
# if no chained configuration is desired, include it as:
# include ktypes/standard/standard.scc nocfg inherit
include ktypes/base/base.scc
branch standard
kconf non-hardware standard.cfg
include features/kgdb/kgdb.scc
.
.
.
include cfg/net/ip6_nf.scc
include cfg/net/bridge.scc
include cfg/systemd.scc
include features/rfkill/rfkill.scc
As with any ``.scc`` file, a kernel type definition can aggregate other
``.scc`` files with ``include`` commands. These definitions can also
directly pull in configuration fragments and patches with the ``kconf``
and ``patch`` commands, respectively.
.. note::
It is not strictly necessary to create a kernel type ``.scc``
file. The Board Support Package (BSP) file can implicitly define the
kernel type using a ``define`` :term:`KTYPE` ``myktype`` line. See the
":ref:`kernel-dev/advanced:bsp descriptions`" section for more
information.
BSP Descriptions
----------------
BSP descriptions (i.e. ``*.scc`` files) combine kernel types with
hardware-specific features. The hardware-specific Metadata is typically
defined independently in the BSP layer, and then aggregated with each
supported kernel type.
.. note::
For BSPs supported by the Yocto Project, the BSP description files
are located in the ``bsp`` directory of the ``yocto-kernel-cache``
repository organized under the "Yocto Linux Kernel" heading in the
:yocto_git:`Yocto Project Source Repositories <>`.
This section overviews the BSP description structure, the aggregation
concepts, and presents a detailed example using a BSP supported by the
Yocto Project (i.e. BeagleBone Board). For complete information on BSP
layer file hierarchy, see the :doc:`/bsp-guide/index`.
Description Overview
~~~~~~~~~~~~~~~~~~~~
For simplicity, consider the following root BSP layer description files
for the BeagleBone board. These files employ both a structure and naming
convention for consistency. The naming convention for the file is as
follows::
bsp_root_name-kernel_type.scc
Here are some example root layer
BSP filenames for the BeagleBone Board BSP, which is supported by the
Yocto Project::
beaglebone-standard.scc
beaglebone-preempt-rt.scc
Each file uses the root name (i.e "beaglebone") BSP name followed by the
kernel type.
Examine the ``beaglebone-standard.scc`` file::
define KMACHINE beaglebone
define KTYPE standard
define KARCH arm
include ktypes/standard/standard.scc
branch beaglebone
include beaglebone.scc
# default policy for standard kernels
include features/latencytop/latencytop.scc
include features/profiling/profiling.scc
Every top-level BSP description file
should define the :term:`KMACHINE`,
:term:`KTYPE`, and
:term:`KARCH` variables. These
variables allow the OpenEmbedded build system to identify the
description as meeting the criteria set by the recipe being built. This
example supports the "beaglebone" machine for the "standard" kernel and
the "arm" architecture.
Be aware that there is no hard link between the :term:`KTYPE` variable and a kernel
type description file. Thus, if you do not have the
kernel type defined in your kernel Metadata as it is here, you only need
to ensure that the
:term:`LINUX_KERNEL_TYPE`
variable in the kernel recipe and the :term:`KTYPE` variable in the BSP
description file match.
To separate your kernel policy from your hardware configuration, you
include a kernel type (``ktype``), such as "standard". In the previous
example, this is done using the following::
include ktypes/standard/standard.scc
This file aggregates all the configuration
fragments, patches, and features that make up your standard kernel
policy. See the ":ref:`kernel-dev/advanced:kernel types`" section for more
information.
To aggregate common configurations and features specific to the kernel
for `mybsp`, use the following::
include mybsp.scc
You can see that in the BeagleBone example with the following::
include beaglebone.scc
For information on how to break a complete ``.config`` file into the various
configuration fragments, see the ":ref:`kernel-dev/common:creating configuration fragments`" section.
Finally, if you have any configurations specific to the hardware that
are not in a ``*.scc`` file, you can include them as follows::
kconf hardware mybsp-extra.cfg
The BeagleBone example does not include these
types of configurations. However, the Malta 32-bit board does
("mti-malta32"). Here is the ``mti-malta32-le-standard.scc`` file::
define KMACHINE mti-malta32-le
define KMACHINE qemumipsel
define KTYPE standard
define KARCH mips
include ktypes/standard/standard.scc
branch mti-malta32
include mti-malta32.scc
kconf hardware mti-malta32-le.cfg
Example
~~~~~~~
Many real-world examples are more complex. Like any other ``.scc`` file,
BSP descriptions can aggregate features. Consider the Minnow BSP
definition given the ``linux-yocto-4.4`` branch of the
``yocto-kernel-cache`` (i.e. ``yocto-kernel-cache/bsp/minnow/minnow.scc``)::
include cfg/x86.scc
include features/eg20t/eg20t.scc
include cfg/dmaengine.scc
include features/power/intel.scc
include cfg/efi.scc
include features/usb/ehci-hcd.scc
include features/usb/ohci-hcd.scc
include features/usb/usb-gadgets.scc
include features/usb/touchscreen-composite.scc
include cfg/timer/hpet.scc
include features/leds/leds.scc
include features/spi/spidev.scc
include features/i2c/i2cdev.scc
include features/mei/mei-txe.scc
# Earlyprintk and port debug requires 8250
kconf hardware cfg/8250.cfg
kconf hardware minnow.cfg
kconf hardware minnow-dev.cfg
.. note::
Although the Minnow Board BSP is unused, the Metadata remains and is
being used here just as an example.
The ``minnow.scc`` description file includes a hardware configuration
fragment (``minnow.cfg``) specific to the Minnow BSP as well as several
more general configuration fragments and features enabling hardware
found on the machine. This ``minnow.scc`` description file is then
included in each of the three "minnow" description files for the
supported kernel types (i.e. "standard", "preempt-rt", and "tiny").
Consider the "minnow" description for the "standard" kernel type (i.e.
``minnow-standard.scc``)::
define KMACHINE minnow
define KTYPE standard
define KARCH i386
include ktypes/standard
include minnow.scc
# Extra minnow configs above the minimal defined in minnow.scc
include cfg/efi-ext.scc
include features/media/media-all.scc
include features/sound/snd_hda_intel.scc
# The following should really be in standard.scc
# USB live-image support
include cfg/usb-mass-storage.scc
include cfg/boot-live.scc
# Basic profiling
include features/latencytop/latencytop.scc
include features/profiling/profiling.scc
# Requested drivers that don't have an existing scc
kconf hardware minnow-drivers-extra.cfg
The ``include`` command midway through the file includes the ``minnow.scc`` description
that defines all enabled hardware for the BSP that is common to all
kernel types. Using this command significantly reduces duplication.
Now consider the "minnow" description for the "tiny" kernel type (i.e.
``minnow-tiny.scc``)::
define KMACHINE minnow
define KTYPE tiny
define KARCH i386
include ktypes/tiny
include minnow.scc
As you might expect,
the "tiny" description includes quite a bit less. In fact, it includes
only the minimal policy defined by the "tiny" kernel type and the
hardware-specific configuration required for booting the machine along
with the most basic functionality of the system as defined in the base
"minnow" description file.
Notice again the three critical variables:
:term:`KMACHINE`,
:term:`KTYPE`, and
:term:`KARCH`. Of these variables, only
:term:`KTYPE` has changed to specify the "tiny" kernel type.
Kernel Metadata Location
========================
Kernel Metadata always exists outside of the kernel tree either defined
in a kernel recipe (recipe-space) or outside of the recipe. Where you
choose to define the Metadata depends on what you want to do and how you
intend to work. Regardless of where you define the kernel Metadata, the
syntax used applies equally.
If you are unfamiliar with the Linux kernel and only wish to apply a
configuration and possibly a couple of patches provided to you by
others, the recipe-space method is recommended. This method is also a
good approach if you are working with Linux kernel sources you do not
control or if you just do not want to maintain a Linux kernel Git
repository on your own. For partial information on how you can define
kernel Metadata in the recipe-space, see the
":ref:`kernel-dev/common:modifying an existing recipe`" section.
Conversely, if you are actively developing a kernel and are already
maintaining a Linux kernel Git repository of your own, you might find it
more convenient to work with kernel Metadata kept outside the
recipe-space. Working with Metadata in this area can make iterative
development of the Linux kernel more efficient outside of the BitBake
environment.
Recipe-Space Metadata
---------------------
When stored in recipe-space, the kernel Metadata files reside in a
directory hierarchy below :term:`FILESEXTRAPATHS`. For
a linux-yocto recipe or for a Linux kernel recipe derived by copying
:oe_git:`meta-skeleton/recipes-kernel/linux/linux-yocto-custom.bb
</openembedded-core/tree/meta-skeleton/recipes-kernel/linux/linux-yocto-custom.bb>`
into your layer and modifying it, :term:`FILESEXTRAPATHS` is typically set to
``${``\ :term:`THISDIR`\ ``}/${``\ :term:`PN`\ ``}``.
See the ":ref:`kernel-dev/common:modifying an existing recipe`"
section for more information.
Here is an example that shows a trivial tree of kernel Metadata stored
in recipe-space within a BSP layer::
meta-my_bsp_layer/
`-- recipes-kernel
`-- linux
`-- linux-yocto
|-- bsp-standard.scc
|-- bsp.cfg
`-- standard.cfg
When the Metadata is stored in recipe-space, you must take steps to
ensure BitBake has the necessary information to decide what files to
fetch and when they need to be fetched again. It is only necessary to
specify the ``.scc`` files on the
:term:`SRC_URI`. BitBake parses them
and fetches any files referenced in the ``.scc`` files by the
``include``, ``patch``, or ``kconf`` commands. Because of this, it is
necessary to bump the recipe :term:`PR`
value when changing the content of files not explicitly listed in the
:term:`SRC_URI`.
If the BSP description is in recipe space, you cannot simply list the
``*.scc`` in the :term:`SRC_URI` statement. You need to use the following
form from your kernel append file::
SRC_URI:append:myplatform = " \
file://myplatform;type=kmeta;destsuffix=myplatform \
"
Metadata Outside the Recipe-Space
---------------------------------
When stored outside of the recipe-space, the kernel Metadata files
reside in a separate repository. The OpenEmbedded build system adds the
Metadata to the build as a "type=kmeta" repository through the
:term:`SRC_URI` variable. As an
example, consider the following :term:`SRC_URI` statement from the
``linux-yocto_5.15.bb`` kernel recipe::
SRC_URI = "git://git.yoctoproject.org/linux-yocto.git;name=machine;branch=${KBRANCH};protocol=https \
git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-5.15;destsuffix=${KMETA};protocol=https"
``${KMETA}``, in this context, is simply used to name the directory into
which the Git fetcher places the Metadata. This behavior is no different
than any multi-repository :term:`SRC_URI` statement used in a recipe (e.g.
see the previous section).
You can keep kernel Metadata in a "kernel-cache", which is a directory
containing configuration fragments. As with any Metadata kept outside
the recipe-space, you simply need to use the :term:`SRC_URI` statement with
the "type=kmeta" attribute. Doing so makes the kernel Metadata available
during the configuration phase.
If you modify the Metadata, you must not forget to update the :term:`SRCREV`
statements in the kernel's recipe. In particular, you need to update the
``SRCREV_meta`` variable to match the commit in the ``KMETA`` branch you
wish to use. Changing the data in these branches and not updating the
:term:`SRCREV` statements to match will cause the build to fetch an older
commit.
Organizing Your Source
======================
Many recipes based on the ``linux-yocto-custom.bb`` recipe use Linux
kernel sources that have only a single branch. This type of
repository structure is fine for linear development supporting a single
machine and architecture. However, if you work with multiple boards and
architectures, a kernel source repository with multiple branches is more
efficient. For example, suppose you need a series of patches for one
board to boot. Sometimes, these patches are works-in-progress or
fundamentally wrong, yet they are still necessary for specific boards.
In these situations, you most likely do not want to include these
patches in every kernel you build (i.e. have the patches as part of the
default branch). It is situations like these that give rise to
multiple branches used within a Linux kernel sources Git repository.
Here are repository organization strategies maximizing source reuse,
removing redundancy, and logically ordering your changes. This section
presents strategies for the following cases:
- Encapsulating patches in a feature description and only including the
patches in the BSP descriptions of the applicable boards.
- Creating a machine branch in your kernel source repository and
applying the patches on that branch only.
- Creating a feature branch in your kernel source repository and
merging that branch into your BSP when needed.
The approach you take is entirely up to you and depends on what works
best for your development model.
Encapsulating Patches
---------------------
If you are reusing patches from an external tree and are not working on
the patches, you might find the encapsulated feature to be appropriate.
Given this scenario, you do not need to create any branches in the
source repository. Rather, you just take the static patches you need and
encapsulate them within a feature description. Once you have the feature
description, you simply include that into the BSP description as
described in the ":ref:`kernel-dev/advanced:bsp descriptions`" section.
You can find information on how to create patches and BSP descriptions
in the ":ref:`kernel-dev/advanced:patches`" and
":ref:`kernel-dev/advanced:bsp descriptions`" sections.
Machine Branches
----------------
When you have multiple machines and architectures to support, or you are
actively working on board support, it is more efficient to create
branches in the repository based on individual machines. Having machine
branches allows common source to remain in the development branch with any
features specific to a machine stored in the appropriate machine branch.
This organization method frees you from continually reintegrating your
patches into a feature.
Once you have a new branch, you can set up your kernel Metadata to use
the branch a couple different ways. In the recipe, you can specify the
new branch as the :term:`KBRANCH` to use for the board as follows::
KBRANCH = "mynewbranch"
Another method is to use the ``branch`` command in the BSP
description::
mybsp.scc:
define KMACHINE mybsp
define KTYPE standard
define KARCH i386
include standard.scc
branch mynewbranch
include mybsp-hw.scc
If you find yourself with numerous branches, you might consider using a
hierarchical branching system similar to what the Yocto Linux Kernel Git
repositories use::
common/kernel_type/machine
If you had two kernel types, "standard" and "small" for instance, three
machines, and common as ``mydir``, the branches in your Git repository
might look like this::
mydir/base
mydir/standard/base
mydir/standard/machine_a
mydir/standard/machine_b
mydir/standard/machine_c
mydir/small/base
mydir/small/machine_a
This organization can help clarify the branch relationships. In this
case, ``mydir/standard/machine_a`` includes everything in ``mydir/base``
and ``mydir/standard/base``. The "standard" and "small" branches add
sources specific to those kernel types that for whatever reason are not
appropriate for the other branches.
.. note::
The "base" branches are an artifact of the way Git manages its data
internally on the filesystem: Git will not allow you to use
``mydir/standard`` and ``mydir/standard/machine_a`` because it would have to
create a file and a directory named "standard".
Feature Branches
----------------
When you are actively developing new features, it can be more efficient
to work with that feature as a branch, rather than as a set of patches
that have to be regularly updated. The Yocto Project Linux kernel tools
provide for this with the ``git merge`` command.
To merge a feature branch into a BSP, insert the ``git merge`` command
after any ``branch`` commands::
mybsp.scc:
define KMACHINE mybsp
define KTYPE standard
define KARCH i386
include standard.scc
branch mynewbranch
git merge myfeature
include mybsp-hw.scc
SCC Description File Reference
==============================
This section provides a brief reference for the commands you can use
within an SCC description file (``.scc``):
- ``branch [ref]``: Creates a new branch relative to the current branch
(typically ``${KTYPE}``) using the currently checked-out branch, or
"ref" if specified.
- ``define``: Defines variables, such as
:term:`KMACHINE`,
:term:`KTYPE`,
:term:`KARCH`, and
:term:`KFEATURE_DESCRIPTION`.
- ``include SCC_FILE``: Includes an SCC file in the current file. The
file is parsed as if you had inserted it inline.
- ``kconf [hardware|non-hardware] CFG_FILE``: Queues a configuration
fragment for merging into the final Linux ``.config`` file.
- ``git merge GIT_BRANCH``: Merges the feature branch into the current
branch.
- ``patch PATCH_FILE``: Applies the patch to the current Git branch.
poky/documentation/kernel-dev/common.rst
+1876
View File
File diff suppressed because it is too large Load Diff
poky/documentation/kernel-dev/concepts-appx.rst
+420
View File
@@ -0,0 +1,420 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
************************
Advanced Kernel Concepts
************************
Yocto Project Kernel Development and Maintenance
================================================
Kernels available through the Yocto Project (Yocto Linux kernels), like
other kernels, are based off the Linux kernel releases from
https://www.kernel.org. At the beginning of a major Linux kernel
development cycle, the Yocto Project team chooses a Linux kernel based
on factors such as release timing, the anticipated release timing of
final upstream ``kernel.org`` versions, and Yocto Project feature
requirements. Typically, the Linux kernel chosen is in the final stages
of development by the Linux community. In other words, the Linux kernel
is in the release candidate or "rc" phase and has yet to reach final
release. But, by being in the final stages of external development, the
team knows that the ``kernel.org`` final release will clearly be within
the early stages of the Yocto Project development window.
This balance allows the Yocto Project team to deliver the most
up-to-date Yocto Linux kernel possible, while still ensuring that the
team has a stable official release for the baseline Linux kernel
version.
As implied earlier, the ultimate source for Yocto Linux kernels are
released kernels from ``kernel.org``. In addition to a foundational
kernel from ``kernel.org``, the available Yocto Linux kernels contain a
mix of important new mainline developments, non-mainline developments
(when no alternative exists), Board Support Package (BSP) developments,
and custom features. These additions result in a commercially released
Yocto Project Linux kernel that caters to specific embedded designer
needs for targeted hardware.
You can find a web interface to the Yocto Linux kernels in the
:ref:`overview-manual/development-environment:yocto project source repositories`
at :yocto_git:`/`. If you look at the interface, you will see to
the left a grouping of Git repositories titled "Yocto Linux Kernel".
Within this group, you will find several Linux Yocto kernels developed
and included with Yocto Project releases:
- *linux-yocto-4.1:* The stable Yocto Project kernel to use with
the Yocto Project Release 2.0. This kernel is based on the Linux 4.1
released kernel.
- *linux-yocto-4.4:* The stable Yocto Project kernel to use with
the Yocto Project Release 2.1. This kernel is based on the Linux 4.4
released kernel.
- *linux-yocto-4.6:* A temporary kernel that is not tied to any
Yocto Project release.
- *linux-yocto-4.8:* The stable yocto Project kernel to use with
the Yocto Project Release 2.2.
- *linux-yocto-4.9:* The stable Yocto Project kernel to use with
the Yocto Project Release 2.3. This kernel is based on the Linux 4.9
released kernel.
- *linux-yocto-4.10:* The default stable Yocto Project kernel to
use with the Yocto Project Release 2.3. This kernel is based on the
Linux 4.10 released kernel.
- *linux-yocto-4.12:* The default stable Yocto Project kernel to
use with the Yocto Project Release 2.4. This kernel is based on the
Linux 4.12 released kernel.
- *yocto-kernel-cache:* The ``linux-yocto-cache`` contains patches
and configurations for the linux-yocto kernel tree. This repository
is useful when working on the linux-yocto kernel. For more
information on this "Advanced Kernel Metadata", see the
":doc:`/kernel-dev/advanced`" Chapter.
- *linux-yocto-dev:* A development kernel based on the latest
upstream release candidate available.
.. note::
Long Term Support Initiative (LTSI) for Yocto Linux kernels is as
follows:
- For Yocto Project releases 1.7, 1.8, and 2.0, the LTSI kernel is
``linux-yocto-3.14``.
- For Yocto Project releases 2.1, 2.2, and 2.3, the LTSI kernel is
``linux-yocto-4.1``.
- For Yocto Project release 2.4, the LTSI kernel is
``linux-yocto-4.9``
- ``linux-yocto-4.4`` is an LTS kernel.
Once a Yocto Linux kernel is officially released, the Yocto Project team
goes into their next development cycle, or upward revision (uprev)
cycle, while still continuing maintenance on the released kernel. It is
important to note that the most sustainable and stable way to include
feature development upstream is through a kernel uprev process.
Back-porting hundreds of individual fixes and minor features from
various kernel versions is not sustainable and can easily compromise
quality.
During the uprev cycle, the Yocto Project team uses an ongoing analysis
of Linux kernel development, BSP support, and release timing to select
the best possible ``kernel.org`` Linux kernel version on which to base
subsequent Yocto Linux kernel development. The team continually monitors
Linux community kernel development to look for significant features of
interest. The team does consider back-porting large features if they
have a significant advantage. User or community demand can also trigger
a back-port or creation of new functionality in the Yocto Project
baseline kernel during the uprev cycle.
Generally speaking, every new Linux kernel both adds features and
introduces new bugs. These consequences are the basic properties of
upstream Linux kernel development and are managed by the Yocto Project
team's Yocto Linux kernel development strategy. It is the Yocto Project
team's policy to not back-port minor features to the released Yocto
Linux kernel. They only consider back-porting significant technological
jumps --- and, that is done after a complete gap analysis. The reason
for this policy is that back-porting any small to medium sized change
from an evolving Linux kernel can easily create mismatches,
incompatibilities and very subtle errors.
The policies described in this section result in both a stable and a
cutting edge Yocto Linux kernel that mixes forward ports of existing
Linux kernel features and significant and critical new functionality.
Forward porting Linux kernel functionality into the Yocto Linux kernels
available through the Yocto Project can be thought of as a "micro
uprev". The many "micro uprevs" produce a Yocto Linux kernel version
with a mix of important new mainline, non-mainline, BSP developments and
feature integrations. This Yocto Linux kernel gives insight into new
features and allows focused amounts of testing to be done on the kernel,
which prevents surprises when selecting the next major uprev. The
quality of these cutting edge Yocto Linux kernels is evolving and the
kernels are used in leading edge feature and BSP development.
Yocto Linux Kernel Architecture and Branching Strategies
========================================================
As mentioned earlier, a key goal of the Yocto Project is to present the
developer with a kernel that has a clear and continuous history that is
visible to the user. The architecture and mechanisms, in particular the
branching strategies, used achieve that goal in a manner similar to
upstream Linux kernel development in ``kernel.org``.
You can think of a Yocto Linux kernel as consisting of a baseline Linux
kernel with added features logically structured on top of the baseline.
The features are tagged and organized by way of a branching strategy
implemented by the Yocto Project team using the Source Code Manager
(SCM) Git.
.. note::
- Git is the obvious SCM for meeting the Yocto Linux kernel
organizational and structural goals described in this section. Not
only is Git the SCM for Linux kernel development in ``kernel.org``
but, Git continues to grow in popularity and supports many
different work flows, front-ends and management techniques.
- You can find documentation on Git at https://git-scm.com/doc. You can
also get an introduction to Git as it applies to the Yocto Project in the
":ref:`overview-manual/development-environment:git`" section in the Yocto Project
Overview and Concepts Manual. The latter reference provides an
overview of Git and presents a minimal set of Git commands that
allows you to be functional using Git. You can use as much, or as
little, of what Git has to offer to accomplish what you need for
your project. You do not have to be a "Git Expert" in order to use
it with the Yocto Project.
Using Git's tagging and branching features, the Yocto Project team
creates kernel branches at points where functionality is no longer
shared and thus, needs to be isolated. For example, board-specific
incompatibilities would require different functionality and would
require a branch to separate the features. Likewise, for specific kernel
features, the same branching strategy is used.
This "tree-like" architecture results in a structure that has features
organized to be specific for particular functionality, single kernel
types, or a subset of kernel types. Thus, the user has the ability to
see the added features and the commits that make up those features. In
addition to being able to see added features, the user can also view the
history of what made up the baseline Linux kernel.
Another consequence of this strategy results in not having to store the
same feature twice internally in the tree. Rather, the kernel team
stores the unique differences required to apply the feature onto the
kernel type in question.
.. note::
The Yocto Project team strives to place features in the tree such
that features can be shared by all boards and kernel types where
possible. However, during development cycles or when large features
are merged, the team cannot always follow this practice. In those
cases, the team uses isolated branches to merge features.
BSP-specific code additions are handled in a similar manner to
kernel-specific additions. Some BSPs only make sense given certain
kernel types. So, for these types, the team creates branches off the end
of that kernel type for all of the BSPs that are supported on that
kernel type. From the perspective of the tools that create the BSP
branch, the BSP is really no different than a feature. Consequently, the
same branching strategy applies to BSPs as it does to kernel features.
So again, rather than store the BSP twice, the team only stores the
unique differences for the BSP across the supported multiple kernels.
While this strategy can result in a tree with a significant number of
branches, it is important to realize that from the developer's point of
view, there is a linear path that travels from the baseline
``kernel.org``, through a select group of features and ends with their
BSP-specific commits. In other words, the divisions of the kernel are
transparent and are not relevant to the developer on a day-to-day basis.
From the developer's perspective, this path is the development branch.
The developer does not need to be aware of the existence of
any other branches at all. Of course, it can make sense to have these
branches in the tree, should a person decide to explore them. For
example, a comparison between two BSPs at either the commit level or at
the line-by-line code ``diff`` level is now a trivial operation.
The following illustration shows the conceptual Yocto Linux kernel.
.. image:: figures/kernel-architecture-overview.png
:width: 100%
In the illustration, the "Kernel.org Branch Point" marks the specific
spot (or Linux kernel release) from which the Yocto Linux kernel is
created. From this point forward in the tree, features and differences
are organized and tagged.
The "Yocto Project Baseline Kernel" contains functionality that is
common to every kernel type and BSP that is organized further along in
the tree. Placing these common features in the tree this way means
features do not have to be duplicated along individual branches of the
tree structure.
From the "Yocto Project Baseline Kernel", branch points represent
specific functionality for individual Board Support Packages (BSPs) as
well as real-time kernels. The illustration represents this through
three BSP-specific branches and a real-time kernel branch. Each branch
represents some unique functionality for the BSP or for a real-time
Yocto Linux kernel.
In this example structure, the "Real-time (rt) Kernel" branch has common
features for all real-time Yocto Linux kernels and contains more
branches for individual BSP-specific real-time kernels. The illustration
shows three branches as an example. Each branch points the way to
specific, unique features for a respective real-time kernel as they
apply to a given BSP.
The resulting tree structure presents a clear path of markers (or
branches) to the developer that, for all practical purposes, is the
Yocto Linux kernel needed for any given set of requirements.
.. note::
Keep in mind the figure does not take into account all the supported
Yocto Linux kernels, but rather shows a single generic kernel just
for conceptual purposes. Also keep in mind that this structure
represents the
:ref:`overview-manual/development-environment:yocto project source repositories`
that are either pulled from during the build or established on the
host development system prior to the build by either cloning a
particular kernel's Git repository or by downloading and unpacking a
tarball.
Working with the kernel as a structured tree follows recognized
community best practices. In particular, the kernel as shipped with the
product, should be considered an "upstream source" and viewed as a
series of historical and documented modifications (commits). These
modifications represent the development and stabilization done by the
Yocto Project kernel development team.
Because commits only change at significant release points in the product
life cycle, developers can work on a branch created from the last
relevant commit in the shipped Yocto Project Linux kernel. As mentioned
previously, the structure is transparent to the developer because the
kernel tree is left in this state after cloning and building the kernel.
Kernel Build File Hierarchy
===========================
Upstream storage of all the available kernel source code is one thing,
while representing and using the code on your host development system is
another. Conceptually, you can think of the kernel source repositories
as all the source files necessary for all the supported Yocto Linux
kernels. As a developer, you are just interested in the source files for
the kernel on which you are working. And, furthermore, you need them
available on your host system.
Kernel source code is available on your host system several different
ways:
- *Files Accessed While using devtool:* ``devtool``, which is
available with the Yocto Project, is the preferred method by which to
modify the kernel. See the ":ref:`kernel-dev/intro:kernel modification workflow`" section.
- *Cloned Repository:* If you are working in the kernel all the time,
you probably would want to set up your own local Git repository of
the Yocto Linux kernel tree. For information on how to clone a Yocto
Linux kernel Git repository, see the
":ref:`kernel-dev/common:preparing the build host to work on the kernel`"
section.
- *Temporary Source Files from a Build:* If you just need to make some
patches to the kernel using a traditional BitBake workflow (i.e. not
using the ``devtool``), you can access temporary kernel source files
that were extracted and used during a kernel build.
The temporary kernel source files resulting from a build using BitBake
have a particular hierarchy. When you build the kernel on your
development system, all files needed for the build are taken from the
source repositories pointed to by the
:term:`SRC_URI` variable and gathered
in a temporary work area where they are subsequently used to create the
unique kernel. Thus, in a sense, the process constructs a local source
tree specific to your kernel from which to generate the new kernel
image.
The following figure shows the temporary file structure created on your
host system when you build the kernel using BitBake. This
:term:`Build Directory` contains all the source files used during the build.
.. image:: figures/kernel-overview-2-generic.png
:align: center
:width: 70%
Again, for additional information on the Yocto Project kernel's
architecture and its branching strategy, see the
":ref:`kernel-dev/concepts-appx:yocto linux kernel architecture and branching strategies`"
section. You can also reference the
":ref:`kernel-dev/common:using \`\`devtool\`\` to patch the kernel`"
and
":ref:`kernel-dev/common:using traditional kernel development to patch the kernel`"
sections for detailed example that modifies the kernel.
Determining Hardware and Non-Hardware Features for the Kernel Configuration Audit Phase
=======================================================================================
This section describes part of the kernel configuration audit phase that
most developers can ignore. For general information on kernel
configuration including ``menuconfig``, ``defconfig`` files, and
configuration fragments, see the
":ref:`kernel-dev/common:configuring the kernel`" section.
During this part of the audit phase, the contents of the final
``.config`` file are compared against the fragments specified by the
system. These fragments can be system fragments, distro fragments, or
user-specified configuration elements. Regardless of their origin, the
OpenEmbedded build system warns the user if a specific option is not
included in the final kernel configuration.
By default, in order to not overwhelm the user with configuration
warnings, the system only reports missing "hardware" options as they
could result in a boot failure or indicate that important hardware is
not available.
To determine whether or not a given option is "hardware" or
"non-hardware", the kernel Metadata in ``yocto-kernel-cache`` contains
files that classify individual or groups of options as either hardware
or non-hardware. To better show this, consider a situation where the
``yocto-kernel-cache`` contains the following files::
yocto-kernel-cache/features/drm-psb/hardware.cfg
yocto-kernel-cache/features/kgdb/hardware.cfg
yocto-kernel-cache/ktypes/base/hardware.cfg
yocto-kernel-cache/bsp/mti-malta32/hardware.cfg
yocto-kernel-cache/bsp/qemu-ppc32/hardware.cfg
yocto-kernel-cache/bsp/qemuarma9/hardware.cfg
yocto-kernel-cache/bsp/mti-malta64/hardware.cfg
yocto-kernel-cache/bsp/arm-versatile-926ejs/hardware.cfg
yocto-kernel-cache/bsp/common-pc/hardware.cfg
yocto-kernel-cache/bsp/common-pc-64/hardware.cfg
yocto-kernel-cache/features/rfkill/non-hardware.cfg
yocto-kernel-cache/ktypes/base/non-hardware.cfg
yocto-kernel-cache/features/aufs/non-hardware.kcf
yocto-kernel-cache/features/ocf/non-hardware.kcf
yocto-kernel-cache/ktypes/base/non-hardware.kcf
yocto-kernel-cache/ktypes/base/hardware.kcf
yocto-kernel-cache/bsp/qemu-ppc32/hardware.kcf
Here are explanations for the various files:
- ``hardware.kcf``: Specifies a list of kernel Kconfig files that
contain hardware options only.
- ``non-hardware.kcf``: Specifies a list of kernel Kconfig files that
contain non-hardware options only.
- ``hardware.cfg``: Specifies a list of kernel ``CONFIG_`` options that
are hardware, regardless of whether or not they are within a Kconfig
file specified by a hardware or non-hardware Kconfig file (i.e.
``hardware.kcf`` or ``non-hardware.kcf``).
- ``non-hardware.cfg``: Specifies a list of kernel ``CONFIG_`` options
that are not hardware, regardless of whether or not they are within a
Kconfig file specified by a hardware or non-hardware Kconfig file
(i.e. ``hardware.kcf`` or ``non-hardware.kcf``).
Here is a specific example using the
``kernel-cache/bsp/mti-malta32/hardware.cfg``::
CONFIG_SERIAL_8250
CONFIG_SERIAL_8250_CONSOLE
CONFIG_SERIAL_8250_NR_UARTS
CONFIG_SERIAL_8250_PCI
CONFIG_SERIAL_CORE
CONFIG_SERIAL_CORE_CONSOLE
CONFIG_VGA_ARB
The kernel configuration audit automatically detects
these files (hence the names must be exactly the ones discussed here),
and uses them as inputs when generating warnings about the final
``.config`` file.
A user-specified kernel Metadata repository, or recipe space feature,
can use these same files to classify options that are found within its
``.cfg`` files as hardware or non-hardware, to prevent the OpenEmbedded
build system from producing an error or warning when an option is not in
the final ``.config`` file.
poky/documentation/kernel-dev/faq.rst
+76
View File
@@ -0,0 +1,76 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
**********************
Kernel Development FAQ
**********************
Common Questions and Solutions
==============================
Here are some solutions for common questions.
How do I use my own Linux kernel ``.config`` file?
--------------------------------------------------
Refer to the
":ref:`kernel-dev/common:changing the configuration`"
section for information.
How do I create configuration fragments?
----------------------------------------
A: Refer to the
":ref:`kernel-dev/common:creating configuration fragments`"
section for information.
How do I use my own Linux kernel sources?
-----------------------------------------
Refer to the
":ref:`kernel-dev/common:working with your own sources`"
section for information.
How do I install/not-install the kernel image on the root filesystem?
---------------------------------------------------------------------
The kernel image (e.g. ``vmlinuz``) is provided by the
``kernel-image`` package. Image recipes depend on ``kernel-base``. To
specify whether or not the kernel image is installed in the generated
root filesystem, override ``RRECOMMENDS:${KERNEL_PACKAGE_NAME}-base`` to include or not
include "kernel-image". See the
":ref:`dev-manual/layers:appending other layers metadata with your layer`"
section in the
Yocto Project Development Tasks Manual for information on how to use an
append file to override metadata.
How do I install a specific kernel module?
------------------------------------------
Linux kernel modules are packaged individually. To ensure a
specific kernel module is included in an image, include it in the
appropriate machine :term:`RRECOMMENDS` variable.
These other variables are useful for installing specific modules:
- :term:`MACHINE_ESSENTIAL_EXTRA_RDEPENDS`
- :term:`MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS`
- :term:`MACHINE_EXTRA_RDEPENDS`
- :term:`MACHINE_EXTRA_RRECOMMENDS`
For example, set the following in the ``qemux86.conf`` file to include
the ``ab123`` kernel modules with images built for the ``qemux86``
machine::
MACHINE_EXTRA_RRECOMMENDS += "kernel-module-ab123"
For more information, see the
":ref:`kernel-dev/common:incorporating out-of-tree modules`" section.
How do I change the Linux kernel command line?
----------------------------------------------
The Linux kernel command line is
typically specified in the machine config using the :term:`APPEND` variable.
For example, you can add some helpful debug information doing the
following::
APPEND += "printk.time=y initcall_debug debug"
poky/documentation/kernel-dev/figures/kernel-architecture-overview.png
Executable
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 40 KiB

poky/documentation/kernel-dev/figures/kernel-dev-flow.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 52 KiB

poky/documentation/kernel-dev/figures/kernel-dev-title.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 13 KiB

poky/documentation/kernel-dev/figures/kernel-overview-2-generic.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 48 KiB

poky/documentation/kernel-dev/index.rst
+20
View File
@@ -0,0 +1,20 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
=============================================
Yocto Project Linux Kernel Development Manual
=============================================
|
.. toctree::
:caption: Table of Contents
:numbered:
intro
common
advanced
concepts-appx
maint-appx
faq
.. include:: /boilerplate.rst
poky/documentation/kernel-dev/intro.rst
+178
View File
@@ -0,0 +1,178 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
************
Introduction
************
Overview
========
Regardless of how you intend to make use of the Yocto Project, chances
are you will work with the Linux kernel. This manual describes how to
set up your build host to support kernel development, introduces the
kernel development process, provides background information on the Yocto
Linux kernel :term:`Metadata`, describes
common tasks you can perform using the kernel tools, shows you how to
use the kernel Metadata needed to work with the kernel inside the Yocto
Project, and provides insight into how the Yocto Project team develops
and maintains Yocto Linux kernel Git repositories and Metadata.
Each Yocto Project release has a set of Yocto Linux kernel recipes,
whose Git repositories you can view in the Yocto
:yocto_git:`Source Repositories <>` under the "Yocto Linux Kernel"
heading. New recipes for the release track the latest Linux kernel
upstream developments from https://www.kernel.org and introduce
newly-supported platforms. Previous recipes in the release are refreshed
and supported for at least one additional Yocto Project release. As they
align, these previous releases are updated to include the latest from
the Long Term Support Initiative (LTSI) project. You can learn more
about Yocto Linux kernels and LTSI in the
":ref:`kernel-dev/concepts-appx:yocto project kernel development and maintenance`" section.
Also included is a Yocto Linux kernel development recipe
(``linux-yocto-dev.bb``) should you want to work with the very latest in
upstream Yocto Linux kernel development and kernel Metadata development.
.. note::
For more on Yocto Linux kernels, see the
":ref:`kernel-dev/concepts-appx:yocto project kernel development and maintenance`"
section.
The Yocto Project also provides a powerful set of kernel tools for
managing Yocto Linux kernel sources and configuration data. You can use
these tools to make a single configuration change, apply multiple
patches, or work with your own kernel sources.
In particular, the kernel tools allow you to generate configuration
fragments that specify only what you must, and nothing more.
Configuration fragments only need to contain the highest level visible
``CONFIG`` options as presented by the Yocto Linux kernel ``menuconfig``
system. Contrast this against a complete Yocto Linux kernel ``.config``
file, which includes all the automatically selected ``CONFIG`` options.
This efficiency reduces your maintenance effort and allows you to
further separate your configuration in ways that make sense for your
project. A common split separates policy and hardware. For example, all
your kernels might support the ``proc`` and ``sys`` filesystems, but
only specific boards require sound, USB, or specific drivers. Specifying
these configurations individually allows you to aggregate them together
as needed, but maintains them in only one place. Similar logic applies
to separating source changes.
If you do not maintain your own kernel sources and need to make only
minimal changes to the sources, the released recipes provide a vetted
base upon which to layer your changes. Doing so allows you to benefit
from the continual kernel integration and testing performed during
development of the Yocto Project.
If, instead, you have a very specific Linux kernel source tree and are
unable to align with one of the official Yocto Linux kernel recipes,
you have a way to use the Yocto Project Linux kernel tools with your
own kernel sources.
The remainder of this manual provides instructions for completing
specific Linux kernel development tasks. These instructions assume you
are comfortable working with :oe_wiki:`BitBake </Bitbake>` recipes and basic
open-source development tools. Understanding these concepts will
facilitate the process of working with the kernel recipes. If you find
you need some additional background, please be sure to review and
understand the following documentation:
- :doc:`/brief-yoctoprojectqs/index` document.
- :doc:`/overview-manual/index`.
- :ref:`devtool
workflow <sdk-manual/extensible:using \`\`devtool\`\` in your sdk workflow>`
as described in the Yocto Project Application Development and the
Extensible Software Development Kit (eSDK) manual.
- The ":ref:`dev-manual/layers:understanding and creating layers`"
section in the Yocto Project Development Tasks Manual.
- The ":ref:`kernel-dev/intro:kernel modification workflow`" section.
Kernel Modification Workflow
============================
Kernel modification involves changing the Yocto Project kernel, which
could involve changing configuration options as well as adding new
kernel recipes. Configuration changes can be added in the form of
configuration fragments, while recipe modification comes through the
kernel's ``recipes-kernel`` area in a kernel layer you create.
This section presents a high-level overview of the Yocto Project kernel
modification workflow. The illustration and accompanying list provide
general information and references for further information.
.. image:: figures/kernel-dev-flow.png
:width: 100%
#. *Set up Your Host Development System to Support Development Using the
Yocto Project*: See the ":doc:`/dev-manual/start`" section in
the Yocto Project Development Tasks Manual for options on how to get
a build host ready to use the Yocto Project.
#. *Set Up Your Host Development System for Kernel Development:* It is
recommended that you use ``devtool`` for kernel
development. Alternatively, you can use traditional kernel
development methods with the Yocto Project. Either way, there are
steps you need to take to get the development environment ready.
Using ``devtool`` requires that you have a clean build
of the image. For
more information, see the
":ref:`kernel-dev/common:getting ready to develop using \`\`devtool\`\``"
section.
Using traditional kernel development requires that you have the
kernel source available in an isolated local Git repository. For more
information, see the
":ref:`kernel-dev/common:getting ready for traditional kernel development`"
section.
#. *Make Changes to the Kernel Source Code if applicable:* Modifying the
kernel does not always mean directly changing source files. However,
if you have to do this, you make the changes to the files in the
Yocto's :term:`Build Directory` if you are using ``devtool``. For more
information, see the
":ref:`kernel-dev/common:using \`\`devtool\`\` to patch the kernel`"
section.
If you are using traditional kernel development, you edit the source
files in the kernel's local Git repository. For more information, see the
":ref:`kernel-dev/common:using traditional kernel development to patch the kernel`"
section.
#. *Make Kernel Configuration Changes if Applicable:* If your situation
calls for changing the kernel's configuration, you can use
:ref:`menuconfig <kernel-dev/common:using \`\`menuconfig\`\`>`,
which allows you to
interactively develop and test the configuration changes you are
making to the kernel. Saving changes you make with ``menuconfig``
updates the kernel's ``.config`` file.
.. note::
Try to resist the temptation to directly edit an existing ``.config``
file, which is found in the :term:`Build Directory` among the source code
used for the build. Doing so, can produce unexpected results when
the OpenEmbedded build system regenerates the configuration file.
Once you are satisfied with the configuration changes made using
``menuconfig`` and you have saved them, you can directly compare the
resulting ``.config`` file against an existing original and gather
those changes into a
:ref:`configuration fragment file <kernel-dev/common:creating configuration fragments>` to be
referenced from within the kernel's ``.bbappend`` file.
Additionally, if you are working in a BSP layer and need to modify
the BSP's kernel's configuration, you can use ``menuconfig``.
#. *Rebuild the Kernel Image With Your Changes:* Rebuilding the kernel
image applies your changes. Depending on your target hardware, you
can verify your changes on actual hardware or perhaps QEMU.
The remainder of this developer's guide covers common tasks typically
used during kernel development, advanced Metadata usage, and Yocto Linux
kernel maintenance concepts.
poky/documentation/kernel-dev/maint-appx.rst
+233
View File
@@ -0,0 +1,233 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
******************
Kernel Maintenance
******************
Tree Construction
=================
This section describes construction of the Yocto Project kernel source
repositories as accomplished by the Yocto Project team to create Yocto
Linux kernel repositories. These kernel repositories are found under the
heading "Yocto Linux Kernel" at :yocto_git:`/` and
are shipped as part of a Yocto Project release. The team creates these
repositories by compiling and executing the set of feature descriptions
for every BSP and feature in the product. Those feature descriptions
list all necessary patches, configurations, branches, tags, and feature
divisions found in a Yocto Linux kernel. Thus, the Yocto Project Linux
kernel repository (or tree) and accompanying Metadata in the
``yocto-kernel-cache`` are built.
The existence of these repositories allow you to access and clone a
particular Yocto Project Linux kernel repository and use it to build
images based on their configurations and features.
You can find the files used to describe all the valid features and BSPs
in the Yocto Project Linux kernel in any clone of the Yocto Project
Linux kernel source repository and ``yocto-kernel-cache`` Git trees. For
example, the following commands clone the Yocto Project baseline Linux
kernel that branches off ``linux.org`` version 4.12 and the
``yocto-kernel-cache``, which contains stores of kernel Metadata::
$ git clone git://git.yoctoproject.org/linux-yocto-4.12
$ git clone git://git.yoctoproject.org/linux-kernel-cache
For more information on
how to set up a local Git repository of the Yocto Project Linux kernel
files, see the
":ref:`kernel-dev/common:preparing the build host to work on the kernel`"
section.
Once you have cloned the kernel Git repository and the cache of Metadata
on your local machine, you can discover the branches that are available
in the repository using the following Git command::
$ git branch -a
Checking out a branch allows you to work with a particular Yocto Linux
kernel. For example, the following commands check out the
"standard/beagleboard" branch of the Yocto Linux kernel repository and
the "yocto-4.12" branch of the ``yocto-kernel-cache`` repository::
$ cd ~/linux-yocto-4.12
$ git checkout -b my-kernel-4.12 remotes/origin/standard/beagleboard
$ cd ~/linux-kernel-cache
$ git checkout -b my-4.12-metadata remotes/origin/yocto-4.12
.. note::
Branches in the ``yocto-kernel-cache`` repository correspond to Yocto Linux
kernel versions (e.g. "yocto-4.12", "yocto-4.10", "yocto-4.9", and so forth).
Once you have checked out and switched to appropriate branches, you can
see a snapshot of all the kernel source files used to build that
particular Yocto Linux kernel for a particular board.
To see the features and configurations for a particular Yocto Linux
kernel, you need to examine the ``yocto-kernel-cache`` Git repository.
As mentioned, branches in the ``yocto-kernel-cache`` repository
correspond to Yocto Linux kernel versions (e.g. ``yocto-4.12``).
Branches contain descriptions in the form of ``.scc`` and ``.cfg``
files.
You should realize, however, that browsing your local
``yocto-kernel-cache`` repository for feature descriptions and patches
is not an effective way to determine what is in a particular kernel
branch. Instead, you should use Git directly to discover the changes in
a branch. Using Git is an efficient and flexible way to inspect changes
to the kernel.
.. note::
Ground up reconstruction of the complete kernel tree is an action
only taken by the Yocto Project team during an active development
cycle. When you create a clone of the kernel Git repository, you are
simply making it efficiently available for building and development.
The following steps describe what happens when the Yocto Project Team
constructs the Yocto Project kernel source Git repository (or tree)
found at :yocto_git:`/` given the introduction of a new
top-level kernel feature or BSP. The following actions effectively
provide the Metadata and create the tree that includes the new feature,
patch, or BSP:
#. *Pass Feature to the OpenEmbedded Build System:* A top-level kernel
feature is passed to the kernel build subsystem. Normally, this
feature is a BSP for a particular kernel type.
#. *Locate Feature:* The file that describes the top-level feature is
located by searching these system directories:
- The in-tree kernel-cache directories, which are located in the
:yocto_git:`yocto-kernel-cache </yocto-kernel-cache/tree/bsp>`
repository organized under the "Yocto Linux Kernel" heading in the
:yocto_git:`Yocto Project Source Repositories <>`.
- Areas pointed to by :term:`SRC_URI` statements found in kernel recipes.
For a typical build, the target of the search is a feature
description in an ``.scc`` file whose name follows this format (e.g.
``beaglebone-standard.scc`` and ``beaglebone-preempt-rt.scc``)::
bsp_root_name-kernel_type.scc
#. *Expand Feature:* Once located, the feature description is either
expanded into a simple script of actions, or into an existing
equivalent script that is already part of the shipped kernel.
#. *Append Extra Features:* Extra features are appended to the top-level
feature description. These features can come from the
:term:`KERNEL_FEATURES`
variable in recipes.
#. *Locate, Expand, and Append Each Feature:* Each extra feature is
located, expanded and appended to the script as described in step
three.
#. *Execute the Script:* The script is executed to produce files
``.scc`` and ``.cfg`` files in appropriate directories of the
``yocto-kernel-cache`` repository. These files are descriptions of
all the branches, tags, patches and configurations that need to be
applied to the base Git repository to completely create the source
(build) branch for the new BSP or feature.
#. *Clone Base Repository:* The base repository is cloned, and the
actions listed in the ``yocto-kernel-cache`` directories are applied
to the tree.
#. *Perform Cleanup:* The Git repositories are left with the desired
branches checked out and any required branching, patching and tagging
has been performed.
The kernel tree and cache are ready for developer consumption to be
locally cloned, configured, and built into a Yocto Project kernel
specific to some target hardware.
.. note::
- The generated ``yocto-kernel-cache`` repository adds to the kernel
as shipped with the Yocto Project release. Any add-ons and
configuration data are applied to the end of an existing branch.
The full repository generation that is found in the official Yocto
Project kernel repositories at :yocto_git:`/` is the
combination of all supported boards and configurations.
- The technique the Yocto Project team uses is flexible and allows
for seamless blending of an immutable history with additional
patches specific to a deployment. Any additions to the kernel
become an integrated part of the branches.
- The full kernel tree that you see on :yocto_git:`/` is
generated through repeating the above steps for all valid BSPs.
The end result is a branched, clean history tree that makes up the
kernel for a given release. You can see the script (``kgit-scc``)
responsible for this in the
:yocto_git:`yocto-kernel-tools </yocto-kernel-tools/tree/tools>`
repository.
- The steps used to construct the full kernel tree are the same
steps that BitBake uses when it builds a kernel image.
Build Strategy
==============
Once you have cloned a Yocto Linux kernel repository and the cache
repository (``yocto-kernel-cache``) onto your development system, you
can consider the compilation phase of kernel development, which is
building a kernel image. Some prerequisites are validated by
the build process before compilation starts:
- The :term:`SRC_URI` points to the
kernel Git repository.
- A BSP build branch with Metadata exists in the ``yocto-kernel-cache``
repository. The branch is based on the Yocto Linux kernel version and
has configurations and features grouped under the
``yocto-kernel-cache/bsp`` directory. For example, features and
configurations for the BeagleBone Board assuming a
``linux-yocto_4.12`` kernel reside in the following area of the
``yocto-kernel-cache`` repository: yocto-kernel-cache/bsp/beaglebone
.. note::
In the previous example, the "yocto-4.12" branch is checked out in
the ``yocto-kernel-cache`` repository.
The OpenEmbedded build system makes sure these conditions are satisfied before
attempting compilation. Other means, however, do exist, such as
bootstrapping a BSP.
Before building a kernel, the build process verifies the tree and
configures the kernel by processing all of the configuration "fragments"
specified by feature descriptions in the ``.scc`` files. As the features
are compiled, associated kernel configuration fragments are noted and
recorded in the series of directories in their compilation order. The
fragments are migrated, pre-processed and passed to the Linux Kernel
Configuration subsystem (``lkc``) as raw input in the form of a
``.config`` file. The ``lkc`` uses its own internal dependency
constraints to do the final processing of that information and generates
the final ``.config`` file that is used during compilation.
Using the board's architecture and other relevant values from the
board's template, kernel compilation is started and a kernel image is
produced.
The other thing that you notice once you configure a kernel is that the
build process generates a build tree that is separate from your kernel's
local Git source repository tree. This build tree has a name that uses
the following form, where ``${MACHINE}`` is the metadata name of the
machine (BSP) and "kernel_type" is one of the Yocto Project supported
kernel types (e.g. "standard")::
linux-${MACHINE}-kernel_type-build
The existing support in the ``kernel.org`` tree achieves this default
functionality.
This behavior means that all the generated files for a particular
machine or BSP are now in the build tree directory. The files include
the final ``.config`` file, all the ``.o`` files, the ``.a`` files, and
so forth. Since each machine or BSP has its own separate
:term:`Build Directory` in its own separate branch of the Git repository,
you can easily switch between different builds.
poky/documentation/migration-guides/index.rst
+39
View File
@@ -0,0 +1,39 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
====================
Release Information
====================
|
Each document in this chapter provides release notes and information about how
to move to one release of the Yocto Project from the previous one.
.. toctree::
migration-general
release-4.3
release-4.2
release-4.1
release-4.0
release-3.4
migration-3.3
migration-3.2
migration-3.1
migration-3.0
migration-2.7
migration-2.6
migration-2.5
migration-2.4
migration-2.3
migration-2.2
migration-2.1
migration-2.0
migration-1.8
migration-1.7
migration-1.6
migration-1.5
migration-1.4
migration-1.3
.. include:: /boilerplate.rst
poky/documentation/migration-guides/migration-1.3.rst
+193
View File
@@ -0,0 +1,193 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 1.3 (danny)
===================
This section provides migration information for moving to the Yocto
Project 1.3 Release (codename "danny") from the prior release.
.. _1.3-local-configuration:
Local Configuration
-------------------
Differences include changes for
:term:`SSTATE_MIRRORS` and ``bblayers.conf``.
.. _migration-1.3-sstate-mirrors:
SSTATE_MIRRORS
~~~~~~~~~~~~~~
The shared state cache (sstate-cache), as pointed to by
:term:`SSTATE_DIR`, by default now has two-character
subdirectories to prevent issues arising from too many files in the same
directory. Also, native sstate-cache packages, which are built to run on
the host system, will go into a subdirectory named using the distro ID
string. If you copy the newly structured sstate-cache to a mirror
location (either local or remote) and then point to it in
:term:`SSTATE_MIRRORS`, you need to append "PATH"
to the end of the mirror URL so that the path used by BitBake before the
mirror substitution is appended to the path used to access the mirror.
Here is an example::
SSTATE_MIRRORS = "file://.* http://someserver.tld/share/sstate/PATH"
.. _migration-1.3-bblayers-conf:
bblayers.conf
~~~~~~~~~~~~~
The ``meta-yocto`` layer consists of two parts that correspond to the
Poky reference distribution and the reference hardware Board Support
Packages (BSPs), respectively: ``meta-yocto`` and ``meta-yocto-bsp``.
When running BitBake for the first time after upgrading, your
``conf/bblayers.conf`` file will be updated to handle this change and
you will be asked to re-run or restart for the changes to take effect.
.. _1.3-recipes:
Recipes
-------
Differences include changes for the following:
.. _migration-1.3-python-function-whitespace:
Python Function Whitespace
~~~~~~~~~~~~~~~~~~~~~~~~~~
All Python functions must now use four spaces for indentation.
Previously, an inconsistent mix of spaces and tabs existed, which made
extending these functions using ``_append`` or ``_prepend`` complicated
given that Python treats whitespace as syntactically significant. If you
are defining or extending any Python functions (e.g.
``populate_packages``, :ref:`ref-tasks-unpack`, :ref:`ref-tasks-patch` and so forth) in
custom recipes or classes, you need to ensure you are using consistent
four-space indentation.
.. _migration-1.3-proto=-in-src-uri:
proto= in SRC_URI
~~~~~~~~~~~~~~~~~
Any use of ``proto=`` in :term:`SRC_URI` needs to be
changed to ``protocol=``. In particular, this applies to the following
URIs:
- ``svn://``
- ``bzr://``
- ``hg://``
- ``osc://``
Other URIs were already using ``protocol=``. This change improves
consistency.
.. _migration-1.3-nativesdk:
nativesdk
~~~~~~~~~
The suffix ``nativesdk`` is now implemented as a prefix, which simplifies a lot
of the packaging code for :ref:`ref-classes-nativesdk` recipes. All custom
:ref:`ref-classes-nativesdk` recipes, which are relocatable packages that are
native to :term:`SDK_ARCH`, and any references need to be updated to use
``nativesdk-*`` instead of ``*-nativesdk``.
.. _migration-1.3-task-recipes:
Task Recipes
~~~~~~~~~~~~
"Task" recipes are now known as "Package groups" and have been renamed
from ``task-*.bb`` to ``packagegroup-*.bb``. Existing references to the
previous ``task-*`` names should work in most cases as there is an
automatic upgrade path for most packages. However, you should update
references in your own recipes and configurations as they could be
removed in future releases. You should also rename any custom ``task-*``
recipes to ``packagegroup-*``, and change them to inherit
:ref:`ref-classes-packagegroup` instead of ``task``, as well
as taking the opportunity to remove anything now handled by
:ref:`ref-classes-packagegroup`, such as providing ``-dev`` and ``-dbg``
packages, setting :term:`LIC_FILES_CHKSUM`, and so forth. See the
:ref:`ref-classes-packagegroup` section for further details.
.. _migration-1.3-image-features:
IMAGE_FEATURES
~~~~~~~~~~~~~~
Image recipes that previously included ``apps-console-core`` in
:term:`IMAGE_FEATURES` should now include ``splash``
instead to enable the boot-up splash screen. Retaining
``apps-console-core`` will still include the splash screen but generates a
warning. The ``apps-x11-core`` and ``apps-x11-games`` :term:`IMAGE_FEATURES`
features have been removed.
.. _migration-1.3-removed-recipes:
Removed Recipes
~~~~~~~~~~~~~~~
The following recipes have been removed. For most of them, it is
unlikely that you would have any references to them in your own
:term:`Metadata`. However, you should check your metadata
against this list to be sure:
- ``libx11-trim``: Replaced by ``libx11``, which has a negligible
size difference with modern Xorg.
- ``xserver-xorg-lite``: Use ``xserver-xorg``, which has a negligible
size difference when DRI and GLX modules are not installed.
- ``xserver-kdrive``: Effectively unmaintained for many years.
- ``mesa-xlib``: No longer serves any purpose.
- ``galago``: Replaced by telepathy.
- ``gail``: Functionality was integrated into GTK+ 2.13.
- ``eggdbus``: No longer needed.
- ``gcc-*-intermediate``: The build has been restructured to avoid
the need for this step.
- ``libgsmd``: Unmaintained for many years. Functionality now
provided by ``ofono`` instead.
- *contacts, dates, tasks, eds-tools*: Largely unmaintained PIM
application suite. It has been moved to ``meta-gnome`` in
``meta-openembedded``.
In addition to the previously listed changes, the ``meta-demoapps``
directory has also been removed because the recipes in it were not being
maintained and many had become obsolete or broken. Additionally, these
recipes were not parsed in the default configuration. Many of these
recipes are already provided in an updated and maintained form within
the OpenEmbedded community layers such as ``meta-oe`` and
``meta-gnome``. For the remainder, you can now find them in the
``meta-extras`` repository, which is in the
:yocto_git:`Source Repositories <>` at
:yocto_git:`/meta-extras/`.
.. _1.3-linux-kernel-naming:
Linux Kernel Naming
-------------------
The naming scheme for kernel output binaries has been changed to now
include :term:`PE` as part of the filename::
KERNEL_IMAGE_BASE_NAME ?= "${KERNEL_IMAGETYPE}-${PE}-${PV}-${PR}-${MACHINE}-${DATETIME}"
Because the :term:`PE` variable is not set by default, these binary files
could result with names that include two dash characters. Here is an
example::
bzImage--3.10.9+git0+cd502a8814_7144bcc4b8-r0-qemux86-64-20130830085431.bin
poky/documentation/migration-guides/migration-1.4.rst
+237
View File
@@ -0,0 +1,237 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 1.4 (dylan)
===================
This section provides migration information for moving to the Yocto
Project 1.4 Release (codename "dylan") from the prior release.
.. _migration-1.4-bitbake:
BitBake
-------
Differences include the following:
- *Comment Continuation:* If a comment ends with a line continuation
(\\) character, then the next line must also be a comment. Any
instance where this is not the case, now triggers a warning. You must
either remove the continuation character, or be sure the next line is
a comment.
- *Package Name Overrides:* The runtime package specific variables
:term:`RDEPENDS`,
:term:`RRECOMMENDS`,
:term:`RSUGGESTS`,
:term:`RPROVIDES`,
:term:`RCONFLICTS`,
:term:`RREPLACES`, :term:`FILES`,
:term:`ALLOW_EMPTY`, and the pre, post, install,
and uninstall script functions ``pkg_preinst``, ``pkg_postinst``,
``pkg_prerm``, and ``pkg_postrm`` should always have a package name
override. For example, use ``RDEPENDS_${PN}`` for the main package
instead of :term:`RDEPENDS`. BitBake uses more strict checks when it
parses recipes.
.. _migration-1.4-build-behavior:
Build Behavior
--------------
Differences include the following:
- *Shared State Code:* The shared state code has been optimized to
avoid running unnecessary tasks. For example, the following no longer
populates the target sysroot since that is not necessary::
$ bitbake -c rootfs some-image
Instead, the system just needs to extract the
output package contents, re-create the packages, and construct the
root filesystem. This change is unlikely to cause any problems unless
you have missing declared dependencies.
- *Scanning Directory Names:* When scanning for files in
:term:`SRC_URI`, the build system now uses
:term:`FILESOVERRIDES` instead of
:term:`OVERRIDES` for the directory names. In
general, the values previously in :term:`OVERRIDES` are now in
:term:`FILESOVERRIDES` as well. However, if you relied upon an additional
value you previously added to :term:`OVERRIDES`, you might now need to
add it to :term:`FILESOVERRIDES` unless you are already adding it through
the :term:`MACHINEOVERRIDES` or
:term:`DISTROOVERRIDES` variables, as
appropriate. For more related changes, see the
":ref:`migration-guides/migration-1.4:variables`" section.
.. _migration-1.4-proxies-and-fetching-source:
Proxies and Fetching Source
---------------------------
A new ``oe-git-proxy`` script has been added to replace previous methods
of handling proxies and fetching source from Git. See the
``meta-yocto/conf/site.conf.sample`` file for information on how to use
this script.
.. _migration-1.4-custom-interfaces-file-netbase-change:
Custom Interfaces File (netbase change)
---------------------------------------
If you have created your own custom ``etc/network/interfaces`` file by
creating an append file for the ``netbase`` recipe, you now need to
create an append file for the ``init-ifupdown`` recipe instead, which
you can find in the :term:`Source Directory` at
``meta/recipes-core/init-ifupdown``. For information on how to use
append files, see the
":ref:`dev-manual/layers:appending other layers metadata with your layer`"
section in the Yocto Project Development Tasks Manual.
.. _migration-1.4-remote-debugging:
Remote Debugging
----------------
Support for remote debugging with the Eclipse IDE is now separated into
an image feature (``eclipse-debug``) that corresponds to the
``packagegroup-core-eclipse-debug`` package group. Previously, the
debugging feature was included through the ``tools-debug`` image
feature, which corresponds to the ``packagegroup-core-tools-debug``
package group.
.. _migration-1.4-variables:
Variables
---------
The following variables have changed:
- :term:`SANITY_TESTED_DISTROS`: This variable now uses a distribution
ID, which is composed of the host distributor ID followed by the
release. Previously,
:term:`SANITY_TESTED_DISTROS` was
composed of the description field. For example, "Ubuntu 12.10"
becomes "Ubuntu-12.10". You do not need to worry about this change if
you are not specifically setting this variable, or if you are
specifically setting it to "".
- :term:`SRC_URI`: The ``${``\ :term:`PN`\ ``}``,
``${``\ :term:`PF`\ ``}``,
``${``\ :term:`P`\ ``}``, and ``FILE_DIRNAME`` directories
have been dropped from the default value of the
:term:`FILESPATH` variable, which is used as the
search path for finding files referred to in
:term:`SRC_URI`. If you have a recipe that relied upon
these directories, which would be unusual, then you will need to add
the appropriate paths within the recipe or, alternatively, rearrange
the files. The most common locations are still covered by ``${``\ :term:`BP`\ ``}``,
``${``\ :term:`BPN`\ ``}``, and "files", which all remain in the default value of
:term:`FILESPATH`.
.. _migration-target-package-management-with-rpm:
Target Package Management with RPM
----------------------------------
If runtime package management is enabled and the RPM backend is
selected, Smart is now installed for package download, dependency
resolution, and upgrades instead of Zypper. For more information on how
to use Smart, run the following command on the target::
smart --help
.. _migration-1.4-recipes-moved:
Recipes Moved
-------------
The following recipes were moved from their previous locations because
they are no longer used by anything in the OpenEmbedded-Core:
- ``clutter-box2d``: Now resides in the ``meta-oe`` layer.
- ``evolution-data-server``: Now resides in the ``meta-gnome`` layer.
- ``gthumb``: Now resides in the ``meta-gnome`` layer.
- ``gtkhtml2``: Now resides in the ``meta-oe`` layer.
- ``gupnp``: Now resides in the ``meta-multimedia`` layer.
- ``gypsy``: Now resides in the ``meta-oe`` layer.
- ``libcanberra``: Now resides in the ``meta-gnome`` layer.
- ``libgdata``: Now resides in the ``meta-gnome`` layer.
- ``libmusicbrainz``: Now resides in the ``meta-multimedia`` layer.
- ``metacity``: Now resides in the ``meta-gnome`` layer.
- ``polkit``: Now resides in the ``meta-oe`` layer.
- ``zeroconf``: Now resides in the ``meta-networking`` layer.
.. _migration-1.4-removals-and-renames:
Removals and Renames
--------------------
The following list shows what has been removed or renamed:
- ``evieext``: Removed because it has been removed from ``xserver``
since 2008.
- *Gtk+ DirectFB:* Removed support because upstream Gtk+ no longer
supports it as of version 2.18.
- ``libxfontcache / xfontcacheproto``: Removed because they were
removed from the Xorg server in 2008.
- ``libxp / libxprintapputil / libxprintutil / printproto``: Removed
because the XPrint server was removed from Xorg in 2008.
- ``libxtrap / xtrapproto``: Removed because their functionality was
broken upstream.
- *linux-yocto 3.0 kernel:* Removed with linux-yocto 3.8 kernel being
added. The linux-yocto 3.2 and linux-yocto 3.4 kernels remain as part
of the release.
- ``lsbsetup``: Removed with functionality now provided by
``lsbtest``.
- ``matchbox-stroke``: Removed because it was never more than a
proof-of-concept.
- ``matchbox-wm-2 / matchbox-theme-sato-2``: Removed because they are
not maintained. However, ``matchbox-wm`` and ``matchbox-theme-sato``
are still provided.
- ``mesa-dri``: Renamed to ``mesa``.
- ``mesa-xlib``: Removed because it was no longer useful.
- ``mutter``: Removed because nothing ever uses it and the recipe is
very old.
- ``orinoco-conf``: Removed because it has become obsolete.
- ``update-modules``: Removed because it is no longer used. The
kernel module ``postinstall`` and ``postrm`` scripts can now do the
same task without the use of this script.
- ``web``: Removed because it is not maintained. Superseded by
``web-webkit``.
- ``xf86bigfontproto``: Removed because upstream it has been disabled
by default since 2007. Nothing uses ``xf86bigfontproto``.
- ``xf86rushproto``: Removed because its dependency in ``xserver``
was spurious and it was removed in 2005.
- ``zypper / libzypp / sat-solver``: Removed and been functionally
replaced with Smart (``python-smartpm``) when RPM packaging is used
and package management is enabled on the target.
poky/documentation/migration-guides/migration-1.5.rst
+355
View File
@@ -0,0 +1,355 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 1.5 (dora)
==================
This section provides migration information for moving to the Yocto
Project 1.5 Release (codename "dora") from the prior release.
.. _migration-1.5-host-dependency-changes:
Host Dependency Changes
-----------------------
The OpenEmbedded build system now has some additional requirements on
the host system:
- Python 2.7.3+
- Tar 1.24+
- Git 1.7.8+
- Patched version of Make if you are using 3.82. Most distributions
that provide Make 3.82 use the patched version.
If the Linux distribution you are using on your build host does not
provide packages for these, you can install and use the Buildtools
tarball, which provides an SDK-like environment containing them.
For more information on this requirement, see the
":ref:`system-requirements-buildtools`" section.
.. _migration-1.5-atom-pc-bsp:
``atom-pc`` Board Support Package (BSP)
---------------------------------------
The ``atom-pc`` hardware reference BSP has been replaced by a
``genericx86`` BSP. This BSP is not necessarily guaranteed to work on
all x86 hardware, but it will run on a wider range of systems than the
``atom-pc`` did.
.. note::
Additionally, a ``genericx86-64`` BSP has been added for 64-bit Atom
systems.
.. _migration-1.5-bitbake:
BitBake
-------
The following changes have been made that relate to BitBake:
- BitBake now supports a ``_remove`` operator. The addition of this
operator means you will have to rename any items in recipe space
(functions, variables) whose names currently contain ``_remove_`` or
end with ``_remove`` to avoid unexpected behavior.
- BitBake's global method pool has been removed. This method is not
particularly useful and led to clashes between recipes containing
functions that had the same name.
- The "none" server backend has been removed. The "process" server
backend has been serving well as the default for a long time now.
- The ``bitbake-runtask`` script has been removed.
- ``${``\ :term:`P`\ ``}`` and
``${``\ :term:`PF`\ ``}`` are no longer added to
:term:`PROVIDES` by default in ``bitbake.conf``.
These version-specific :term:`PROVIDES` items were seldom used.
Attempting to use them could result in two versions being built
simultaneously rather than just one version due to the way BitBake
resolves dependencies.
.. _migration-1.5-qa-warnings:
QA Warnings
-----------
The following changes have been made to the package QA checks:
- If you have customized :term:`ERROR_QA` or
:term:`WARN_QA` values in your configuration, check
that they contain all of the issues that you wish to be reported.
Previous Yocto Project versions contained a bug that meant that any
item not mentioned in :term:`ERROR_QA` or :term:`WARN_QA` would be treated as
a warning. Consequently, several important items were not already in
the default value of :term:`WARN_QA`. All of the possible QA checks are
now documented in the ":ref:`ref-classes-insane`" section.
- An additional QA check has been added to check if
``/usr/share/info/dir`` is being installed. Your recipe should delete
this file within :ref:`ref-tasks-install` if "make
install" is installing it.
- If you are using the :ref:`ref-classes-buildhistory` class, the check for the
package version going backwards is now controlled using a standard QA check.
Thus, if you have customized your :term:`ERROR_QA` or :term:`WARN_QA` values
and still wish to have this check performed, you should add
"version-going-backwards" to your value for one or the other
variables depending on how you wish it to be handled. See the
documented QA checks in the ":ref:`ref-classes-insane`" section.
.. _migration-1.5-directory-layout-changes:
Directory Layout Changes
------------------------
The following directory changes exist:
- Output SDK installer files are now named to include the image name
and tuning architecture through the :term:`SDK_NAME`
variable.
- Images and related files are now installed into a directory that is
specific to the machine, instead of a parent directory containing
output files for multiple machines. The
:term:`DEPLOY_DIR_IMAGE` variable continues
to point to the directory containing images for the current
:term:`MACHINE` and should be used anywhere there is a
need to refer to this directory. The ``runqemu`` script now uses this
variable to find images and kernel binaries and will use BitBake to
determine the directory. Alternatively, you can set the
:term:`DEPLOY_DIR_IMAGE` variable in the external environment.
- When buildhistory is enabled, its output is now written under the
:term:`Build Directory` rather than :term:`TMPDIR`. Doing so makes
it easier to delete :term:`TMPDIR` and preserve the build history.
Additionally, data for produced SDKs is now split by :term:`IMAGE_NAME`.
- When :ref:`ref-classes-buildhistory` is enabled, its output
is now written under the :term:`Build Directory` rather than :term:`TMPDIR`.
Doing so makes it easier to delete :term:`TMPDIR` and preserve the build
history. Additionally, data for produced SDKs is now split by :term:`IMAGE_NAME`.
- The ``pkgdata`` directory produced as part of the packaging process
has been collapsed into a single machine-specific directory. This
directory is located under ``sysroots`` and uses a machine-specific
name (i.e. ``tmp/sysroots/machine/pkgdata``).
.. _migration-1.5-shortened-git-srcrev-values:
Shortened Git ``SRCREV`` Values
-------------------------------
BitBake will now shorten revisions from Git repositories from the normal
40 characters down to 10 characters within :term:`SRCPV`
for improved usability in path and filenames. This change should be
safe within contexts where these revisions are used because the chances
of spatially close collisions is very low. Distant collisions are not a
major issue in the way the values are used.
.. _migration-1.5-image-features:
``IMAGE_FEATURES``
------------------
The following changes have been made that relate to
:term:`IMAGE_FEATURES`:
- The value of :term:`IMAGE_FEATURES` is now validated to ensure invalid
feature items are not added. Some users mistakenly add package names
to this variable instead of using
:term:`IMAGE_INSTALL` in order to have the
package added to the image, which does not work. This change is
intended to catch those kinds of situations. Valid :term:`IMAGE_FEATURES`
are drawn from ``PACKAGE_GROUP`` definitions,
:term:`COMPLEMENTARY_GLOB` and a new
"validitems" varflag on :term:`IMAGE_FEATURES`. The "validitems" varflag
change allows additional features to be added if they are not
provided using the previous two mechanisms.
- The previously deprecated "apps-console-core" :term:`IMAGE_FEATURES` item
is no longer supported. Add "splash" to :term:`IMAGE_FEATURES` if you
wish to have the splash screen enabled, since this is all that
apps-console-core was doing.
.. _migration-1.5-run:
``/run``
--------
The ``/run`` directory from the Filesystem Hierarchy Standard 3.0 has
been introduced. You can find some of the implications for this change
:oe_git:`here </openembedded-core/commit/?id=0e326280a15b0f2c4ef2ef4ec441f63f55b75873>`.
The change also means that recipes that install files to ``/var/run``
must be changed. You can find a guide on how to make these changes
`here <https://www.mail-archive.com/openembedded-devel@lists.openembedded.org/msg31649.html>`__.
.. _migration-1.5-removal-of-package-manager-database-within-image-recipes:
Removal of Package Manager Database Within Image Recipes
--------------------------------------------------------
The image ``core-image-minimal`` no longer adds
``remove_packaging_data_files`` to
:term:`ROOTFS_POSTPROCESS_COMMAND`.
This addition is now handled automatically when "package-management" is
not in :term:`IMAGE_FEATURES`. If you have custom
image recipes that make this addition, you should remove the lines, as
they are not needed and might interfere with correct operation of
postinstall scripts.
.. _migration-1.5-images-now-rebuild-only-on-changes-instead-of-every-time:
Images Now Rebuild Only on Changes Instead of Every Time
--------------------------------------------------------
The :ref:`ref-tasks-rootfs` and other related image
construction tasks are no longer marked as "nostamp". Consequently, they
will only be re-executed when their inputs have changed. Previous
versions of the OpenEmbedded build system always rebuilt the image when
requested rather when necessary.
.. _migration-1.5-task-recipes:
Task Recipes
------------
The previously deprecated ``task.bbclass`` has now been dropped. For
recipes that previously inherited from this class, you should rename
them from ``task-*`` to ``packagegroup-*`` and inherit
:ref:`ref-classes-packagegroup` instead.
For more information, see the ":ref:`ref-classes-packagegroup`" section.
.. _migration-1.5-busybox:
BusyBox
-------
By default, we now split BusyBox into two binaries: one that is suid
root for those components that need it, and another for the rest of the
components. Splitting BusyBox allows for optimization that eliminates
the ``tinylogin`` recipe as recommended by upstream. You can disable
this split by setting
:term:`BUSYBOX_SPLIT_SUID` to "0".
.. _migration-1.5-automated-image-testing:
Automated Image Testing
-----------------------
A new automated image testing framework has been added through the
:ref:`ref-classes-testimage` classes. This
framework replaces the older ``imagetest-qemu`` framework.
You can learn more about performing automated image tests in the
":ref:`dev-manual/runtime-testing:performing automated runtime testing`"
section in the Yocto Project Development Tasks Manual.
.. _migration-1.5-build-history:
Build History
-------------
Following are changes to Build History:
- Installed package sizes: ``installed-package-sizes.txt`` for an image
now records the size of the files installed by each package instead
of the size of each compressed package archive file.
- The dependency graphs (``depends*.dot``) now use the actual package
names instead of replacing dashes, dots and plus signs with
underscores.
- The ``buildhistory-diff`` and ``buildhistory-collect-srcrevs``
utilities have improved command-line handling. Use the ``--help``
option for each utility for more information on the new syntax.
For more information on Build History, see the
":ref:`dev-manual/build-quality:maintaining build output quality`"
section in the Yocto Project Development Tasks Manual.
.. _migration-1.5-udev:
``udev``
--------
Following are changes to ``udev``:
- ``udev`` no longer brings in ``udev-extraconf`` automatically through
:term:`RRECOMMENDS`, since this was originally
intended to be optional. If you need the extra rules, then add
``udev-extraconf`` to your image.
- ``udev`` no longer brings in ``pciutils-ids`` or ``usbutils-ids``
through :term:`RRECOMMENDS`. These are not needed by ``udev`` itself and
removing them saves around 350KB.
.. _migration-1.5-removed-renamed-recipes:
Removed and Renamed Recipes
---------------------------
- The ``linux-yocto`` 3.2 kernel has been removed.
- ``libtool-nativesdk`` has been renamed to ``nativesdk-libtool``.
- ``tinylogin`` has been removed. It has been replaced by a suid
portion of Busybox. See the ":ref:`migration-1.5-busybox`"
section for more information.
- ``external-python-tarball`` has been renamed to
``buildtools-tarball``.
- ``web-webkit`` has been removed. It has been functionally replaced by
``midori``.
- ``imake`` has been removed. It is no longer needed by any other
recipe.
- ``transfig-native`` has been removed. It is no longer needed by any
other recipe.
- ``anjuta-remote-run`` has been removed. Anjuta IDE integration has
not been officially supported for several releases.
.. _migration-1.5-other-changes:
Other Changes
-------------
Following is a list of short entries describing other changes:
- ``run-postinsts``: Make this generic.
- ``base-files``: Remove the unnecessary ``media/``\ xxx directories.
- ``alsa-state``: Provide an empty ``asound.conf`` by default.
- ``classes/image``: Ensure
:term:`BAD_RECOMMENDATIONS` supports
pre-renamed package names.
- ``classes/rootfs_rpm``: Implement :term:`BAD_RECOMMENDATIONS` for RPM.
- ``systemd``: Remove ``systemd_unitdir`` if ``systemd`` is not in
:term:`DISTRO_FEATURES`.
- ``systemd``: Remove ``init.d`` dir if ``systemd`` unit file is
present and ``sysvinit`` is not a distro feature.
- ``libpam``: Deny all services for the ``OTHER`` entries.
- :ref:`ref-classes-image`: Move ``runtime_mapping_rename`` to avoid conflict
with ``multilib``. See :yocto_bugs:`YOCTO #4993 </show_bug.cgi?id=4993>`
in Bugzilla for more information.
- ``linux-dtb``: Use kernel build system to generate the ``dtb`` files.
- ``kern-tools``: Switch from guilt to new ``kgit-s2q`` tool.
poky/documentation/migration-guides/migration-1.6.rst
+414
View File
@@ -0,0 +1,414 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 1.6 (daisy)
===================
This section provides migration information for moving to the Yocto
Project 1.6 Release (codename "daisy") from the prior release.
.. _migration-1.6-archiver-class:
``archiver`` Class
------------------
The :ref:`ref-classes-archiver` class has been rewritten and its configuration
has been simplified. For more details on the source archiver, see the
":ref:`dev-manual/licenses:maintaining open source license compliance during your product's lifecycle`"
section in the Yocto Project Development Tasks Manual.
.. _migration-1.6-packaging-changes:
Packaging Changes
-----------------
The following packaging changes have been made:
- The ``binutils`` recipe no longer produces a ``binutils-symlinks``
package. ``update-alternatives`` is now used to handle the preferred
``binutils`` variant on the target instead.
- The tc (traffic control) utilities have been split out of the main
``iproute2`` package and put into the ``iproute2-tc`` package.
- The ``gtk-engines`` schemas have been moved to a dedicated
``gtk-engines-schemas`` package.
- The ``armv7a`` with thumb package architecture suffix has changed.
The suffix for these packages with the thumb optimization enabled is
"t2" as it should be. Use of this suffix was not the case in the 1.5
release. Architecture names will change within package feeds as a
result.
.. _migration-1.6-bitbake:
BitBake
-------
The following changes have been made to :term:`BitBake`.
.. _migration-1.6-matching-branch-requirement-for-git-fetching:
Matching Branch Requirement for Git Fetching
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
When fetching source from a Git repository using
:term:`SRC_URI`, BitBake will now validate the
:term:`SRCREV` value against the branch. You can specify
the branch using the following form::
SRC_URI = "git://server.name/repository;branch=branchname"
If you do not specify a branch, BitBake looks in the default "master" branch.
Alternatively, if you need to bypass this check (e.g. if you are
fetching a revision corresponding to a tag that is not on any branch),
you can add ";nobranch=1" to the end of the URL within :term:`SRC_URI`.
.. _migration-1.6-bitbake-deps:
Python Definition substitutions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
BitBake had some previously deprecated Python definitions within its
``bb`` module removed. You should use their sub-module counterparts
instead:
- ``bb.MalformedUrl``: Use ``bb.fetch.MalformedUrl``.
- ``bb.encodeurl``: Use ``bb.fetch.encodeurl``.
- ``bb.decodeurl``: Use ``bb.fetch.decodeurl``
- ``bb.mkdirhier``: Use ``bb.utils.mkdirhier``.
- ``bb.movefile``: Use ``bb.utils.movefile``.
- ``bb.copyfile``: Use ``bb.utils.copyfile``.
- ``bb.which``: Use ``bb.utils.which``.
- ``bb.vercmp_string``: Use ``bb.utils.vercmp_string``.
- ``bb.vercmp``: Use ``bb.utils.vercmp``.
.. _migration-1.6-bitbake-fetcher:
SVK Fetcher
~~~~~~~~~~~
The SVK fetcher has been removed from BitBake.
.. _migration-1.6-bitbake-console-output:
Console Output Error Redirection
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The BitBake console UI will now output errors to ``stderr`` instead of
``stdout``. Consequently, if you are piping or redirecting the output of
``bitbake`` to somewhere else, and you wish to retain the errors, you
will need to add ``2>&1`` (or something similar) to the end of your
``bitbake`` command line.
.. _migration-1.6-task-taskname-overrides:
``task-``\ taskname Overrides
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``task-``\ taskname overrides have been adjusted so that tasks whose
names contain underscores have the underscores replaced by hyphens for
the override so that they now function properly. For example, the task
override for :ref:`ref-tasks-populate_sdk` is
``task-populate-sdk``.
.. _migration-1.6-variable-changes:
Changes to Variables
--------------------
The following variables have changed. For information on the
OpenEmbedded build system variables, see the ":doc:`/ref-manual/variables`" Chapter.
.. _migration-1.6-variable-changes-TMPDIR:
``TMPDIR``
~~~~~~~~~~
:term:`TMPDIR` can no longer be on an NFS mount. NFS does
not offer full POSIX locking and inode consistency and can cause
unexpected issues if used to store :term:`TMPDIR`.
The check for this occurs on startup. If :term:`TMPDIR` is detected on an
NFS mount, an error occurs.
.. _migration-1.6-variable-changes-PRINC:
``PRINC``
~~~~~~~~~
The ``PRINC`` variable has been deprecated and triggers a warning if
detected during a build. For :term:`PR` increments on changes,
use the PR service instead. You can find out more about this service in
the ":ref:`dev-manual/packages:working with a pr service`"
section in the Yocto Project Development Tasks Manual.
.. _migration-1.6-variable-changes-IMAGE_TYPES:
``IMAGE_TYPES``
~~~~~~~~~~~~~~~
The "sum.jffs2" option for :term:`IMAGE_TYPES` has
been replaced by the "jffs2.sum" option, which fits the processing
order.
.. _migration-1.6-variable-changes-COPY_LIC_MANIFEST:
``COPY_LIC_MANIFEST``
~~~~~~~~~~~~~~~~~~~~~
The :term:`COPY_LIC_MANIFEST` variable must now
be set to "1" rather than any value in order to enable it.
.. _migration-1.6-variable-changes-COPY_LIC_DIRS:
``COPY_LIC_DIRS``
~~~~~~~~~~~~~~~~~
The :term:`COPY_LIC_DIRS` variable must now be set
to "1" rather than any value in order to enable it.
.. _migration-1.6-variable-changes-PACKAGE_GROUP:
``PACKAGE_GROUP``
~~~~~~~~~~~~~~~~~
The ``PACKAGE_GROUP`` variable has been renamed to
:term:`FEATURE_PACKAGES` to more accurately
reflect its purpose. You can still use ``PACKAGE_GROUP`` but the
OpenEmbedded build system produces a warning message when it encounters
the variable.
.. _migration-1.6-variable-changes-variable-entry-behavior:
Preprocess and Post Process Command Variable Behavior
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The following variables now expect a semicolon separated list of
functions to call and not arbitrary shell commands:
- :term:`ROOTFS_PREPROCESS_COMMAND`
- :term:`ROOTFS_POSTPROCESS_COMMAND`
- :term:`SDK_POSTPROCESS_COMMAND`
- :term:`POPULATE_SDK_POST_TARGET_COMMAND`
- :term:`POPULATE_SDK_POST_HOST_COMMAND`
- :term:`IMAGE_POSTPROCESS_COMMAND`
- :term:`IMAGE_PREPROCESS_COMMAND`
- :term:`ROOTFS_POSTUNINSTALL_COMMAND`
- :term:`ROOTFS_POSTINSTALL_COMMAND`
For
migration purposes, you can simply wrap shell commands in a shell
function and then call the function. Here is an example::
my_postprocess_function() {
echo "hello" > ${IMAGE_ROOTFS}/hello.txt
}
ROOTFS_POSTPROCESS_COMMAND += "my_postprocess_function; "
.. _migration-1.6-package-test-ptest:
Package Test (ptest)
--------------------
Package Tests (ptest) are built but not installed by default. For
information on using Package Tests, see the
":ref:`dev-manual/packages:testing packages with ptest`" section in the
Yocto Project Development Tasks Manual. See also the ":ref:`ref-classes-ptest`"
section.
.. _migration-1.6-build-changes:
Build Changes
-------------
Separate build and source directories have been enabled by default for
selected recipes where it is known to work and for all
recipes that inherit the :ref:`ref-classes-cmake` class. In
future releases the :ref:`ref-classes-autotools` class
will enable a separate :term:`Build Directory` by default as well. Recipes
building Autotools-based software that fails to build with a separate
:term:`Build Directory` should be changed to inherit from the
:ref:`autotools-brokensep <ref-classes-autotools>` class instead of
the :ref:`ref-classes-autotools` or ``autotools_stage`` classes.
.. _migration-1.6-building-qemu-native:
``qemu-native``
---------------
``qemu-native`` now builds without SDL-based graphical output support by
default. The following additional lines are needed in your
``local.conf`` to enable it::
PACKAGECONFIG_pn-qemu-native = "sdl"
ASSUME_PROVIDED += "libsdl-native"
.. note::
The default ``local.conf`` contains these statements. Consequently, if you
are building a headless system and using a default ``local.conf``
file, you will need comment these two lines out.
.. _migration-1.6-core-image-basic:
``core-image-basic``
--------------------
``core-image-basic`` has been renamed to ``core-image-full-cmdline``.
In addition to ``core-image-basic`` being renamed,
``packagegroup-core-basic`` has been renamed to
``packagegroup-core-full-cmdline`` to match.
.. _migration-1.6-licensing:
Licensing
---------
The top-level :term:`LICENSE` file has been changed to better describe the
license of the various components of :term:`OpenEmbedded-Core (OE-Core)`. However,
the licensing itself remains unchanged.
Normally, this change would not cause any side-effects. However, some
recipes point to this file within
:term:`LIC_FILES_CHKSUM` (as
``${COREBASE}/LICENSE``) and thus the accompanying checksum must be
changed from 3f40d7994397109285ec7b81fdeb3b58 to
4d92cd373abda3937c2bc47fbc49d690. A better alternative is to have
:term:`LIC_FILES_CHKSUM` point to a file describing the license that is
distributed with the source that the recipe is building, if possible,
rather than pointing to ``${COREBASE}/LICENSE``.
.. _migration-1.6-cflags-options:
``CFLAGS`` Options
------------------
The "-fpermissive" option has been removed from the default
:term:`CFLAGS` value. You need to take action on
individual recipes that fail when building with this option. You need to
either patch the recipes to fix the issues reported by the compiler, or
you need to add "-fpermissive" to :term:`CFLAGS` in the recipes.
.. _migration-1.6-custom-images:
Custom Image Output Types
-------------------------
Custom image output types, as selected using
:term:`IMAGE_FSTYPES`, must declare their
dependencies on other image types (if any) using a new
:term:`IMAGE_TYPEDEP` variable.
.. _migration-1.6-do-package-write-task:
Tasks
-----
The ``do_package_write`` task has been removed. The task is no longer
needed.
.. _migration-1.6-update-alternatives-provider:
``update-alternative`` Provider
-------------------------------
The default ``update-alternatives`` provider has been changed from
``opkg`` to ``opkg-utils``. This change resolves some troublesome
circular dependencies. The runtime package has also been renamed from
``update-alternatives-cworth`` to ``update-alternatives-opkg``.
.. _migration-1.6-virtclass-overrides:
``virtclass`` Overrides
-----------------------
The ``virtclass`` overrides are now deprecated. Use the equivalent class
overrides instead (e.g. ``virtclass-native`` becomes ``class-native``.)
.. _migration-1.6-removed-renamed-recipes:
Removed and Renamed Recipes
---------------------------
The following recipes have been removed:
- ``packagegroup-toolset-native`` --- this recipe is largely unused.
- ``linux-yocto-3.8`` --- support for the Linux yocto 3.8 kernel has been
dropped. Support for the 3.10 and 3.14 kernels have been added with
the ``linux-yocto-3.10`` and ``linux-yocto-3.14`` recipes.
- ``ocf-linux`` --- this recipe has been functionally replaced using
``cryptodev-linux``.
- ``genext2fs`` --- ``genext2fs`` is no longer used by the build system
and is unmaintained upstream.
- ``js`` --- this provided an ancient version of Mozilla's javascript
engine that is no longer needed.
- ``zaurusd`` --- the recipe has been moved to the ``meta-handheld``
layer.
- ``eglibc 2.17`` --- replaced by the ``eglibc 2.19`` recipe.
- ``gcc 4.7.2`` --- replaced by the now stable ``gcc 4.8.2``.
- ``external-sourcery-toolchain`` --- this recipe is now maintained in
the ``meta-sourcery`` layer.
- ``linux-libc-headers-yocto 3.4+git`` --- now using version 3.10 of the
``linux-libc-headers`` by default.
- ``meta-toolchain-gmae`` --- this recipe is obsolete.
- ``packagegroup-core-sdk-gmae`` --- this recipe is obsolete.
- ``packagegroup-core-standalone-gmae-sdk-target`` --- this recipe is
obsolete.
.. _migration-1.6-removed-classes:
Removed Classes
---------------
The following classes have become obsolete and have been removed:
- ``module_strip``
- ``pkg_metainfo``
- ``pkg_distribute``
- ``image-empty``
.. _migration-1.6-reference-bsps:
Reference Board Support Packages (BSPs)
---------------------------------------
The following reference BSPs changes occurred:
- The BeagleBoard (``beagleboard``) ARM reference hardware has been
replaced by the BeagleBone (``beaglebone``) hardware.
- The RouterStation Pro (``routerstationpro``) MIPS reference hardware
has been replaced by the EdgeRouter Lite (``edgerouter``) hardware.
The previous reference BSPs for the ``beagleboard`` and
``routerstationpro`` machines are still available in a new
``meta-yocto-bsp-old`` layer in the
:yocto_git:`Source Repositories <>` at
:yocto_git:`/meta-yocto-bsp-old/`.
poky/documentation/migration-guides/migration-1.7.rst
+222
View File
@@ -0,0 +1,222 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 1.7 (dizzy)
===================
This section provides migration information for moving to the Yocto
Project 1.7 Release (codename "dizzy") from the prior release.
.. _migration-1.7-changes-to-setting-qemu-packageconfig-options:
Changes to Setting QEMU ``PACKAGECONFIG`` Options in ``local.conf``
-------------------------------------------------------------------
The QEMU recipe now uses a number of
:term:`PACKAGECONFIG` options to enable various
optional features. The method used to set defaults for these options
means that existing ``local.conf`` files will need to be modified to
append to :term:`PACKAGECONFIG` for ``qemu-native`` and ``nativesdk-qemu``
instead of setting it. In other words, to enable graphical output for
QEMU, you should now have these lines in ``local.conf``::
PACKAGECONFIG_append_pn-qemu-native = " sdl"
PACKAGECONFIG_append_pn-nativesdk-qemu = " sdl"
.. _migration-1.7-minimum-git-version:
Minimum Git version
-------------------
The minimum :ref:`overview-manual/development-environment:git`
version required on the
build host is now 1.7.8 because the ``--list`` option is now required by
BitBake's Git fetcher. As always, if your host distribution does not
provide a version of Git that meets this requirement, you can use the
:term:`buildtools` tarball that does. See the
":ref:`ref-manual/system-requirements:required git, tar, python, make and gcc versions`"
section for more information.
.. _migration-1.7-autotools-class-changes:
Autotools Class Changes
-----------------------
The following :ref:`ref-classes-autotools` class changes occurred:
- *A separate :term:`Build Directory` is now used by default:* The
:ref:`ref-classes-autotools` class has been changed to use a directory for
building (:term:`B`), which is separate from the source directory
(:term:`S`). This is commonly referred to as ``B != S``, or
an out-of-tree build.
If the software being built is already capable of building in a
directory separate from the source, you do not need to do anything.
However, if the software is not capable of being built in this
manner, you will need to either patch the software so that it can
build separately, or you will need to change the recipe to inherit
the :ref:`autotools-brokensep <ref-classes-autotools>` class instead
of the :ref:`ref-classes-autotools` or ``autotools_stage`` classes.
- The ``--foreign`` option is no longer passed to ``automake`` when
running ``autoconf``: This option tells ``automake`` that a
particular software package does not follow the GNU standards and
therefore should not be expected to distribute certain files such as
``ChangeLog``, ``AUTHORS``, and so forth. Because the majority of
upstream software packages already tell ``automake`` to enable
foreign mode themselves, the option is mostly superfluous. However,
some recipes will need patches for this change. You can easily make
the change by patching ``configure.ac`` so that it passes "foreign"
to ``AM_INIT_AUTOMAKE()``. See :oe_git:`this
commit </openembedded-core/commit/?id=01943188f85ce6411717fb5bf702d609f55813f2>`
for an example showing how to make the patch.
.. _migration-1.7-binary-configuration-scripts-disabled:
Binary Configuration Scripts Disabled
-------------------------------------
Some of the core recipes that package binary configuration scripts now
disable the scripts due to the scripts previously requiring error-prone
path substitution. Software that links against these libraries using
these scripts should use the much more robust ``pkg-config`` instead.
The list of recipes changed in this version (and their configuration
scripts) is as follows::
directfb (directfb-config)
freetype (freetype-config)
gpgme (gpgme-config)
libassuan (libassuan-config)
libcroco (croco-6.0-config)
libgcrypt (libgcrypt-config)
libgpg-error (gpg-error-config)
libksba (ksba-config)
libpcap (pcap-config)
libpcre (pcre-config)
libpng (libpng-config, libpng16-config)
libsdl (sdl-config)
libusb-compat (libusb-config)
libxml2 (xml2-config)
libxslt (xslt-config)
ncurses (ncurses-config)
neon (neon-config)
npth (npth-config)
pth (pth-config)
taglib (taglib-config)
Additionally, support for ``pkg-config`` has been added to some recipes in the
previous list in the rare cases where the upstream software package does
not already provide it.
.. _migration-1.7-glibc-replaces-eglibc:
``eglibc 2.19`` Replaced with ``glibc 2.20``
--------------------------------------------
Because ``eglibc`` and ``glibc`` were already fairly close, this
replacement should not require any significant changes to other software
that links to ``eglibc``. However, there were a number of minor changes
in ``glibc 2.20`` upstream that could require patching some software
(e.g. the removal of the ``_BSD_SOURCE`` feature test macro).
``glibc 2.20`` requires version 2.6.32 or greater of the Linux kernel.
Thus, older kernels will no longer be usable in conjunction with it.
For full details on the changes in ``glibc 2.20``, see the upstream
release notes
`here <https://sourceware.org/ml/libc-alpha/2014-09/msg00088.html>`__.
.. _migration-1.7-kernel-module-autoloading:
Kernel Module Autoloading
-------------------------
The :term:`module_autoload_* <module_autoload>` variable is now
deprecated and a new
:term:`KERNEL_MODULE_AUTOLOAD` variable
should be used instead. Also, :term:`module_conf_* <module_conf>`
must now be used in conjunction with a new
:term:`KERNEL_MODULE_PROBECONF` variable.
The new variables no longer require you to specify the module name as
part of the variable name. This change not only simplifies usage but
also allows the values of these variables to be appropriately
incorporated into task signatures and thus trigger the appropriate tasks
to re-execute when changed. You should replace any references to
``module_autoload_*`` with :term:`KERNEL_MODULE_AUTOLOAD`, and add any
modules for which ``module_conf_*`` is specified to
:term:`KERNEL_MODULE_PROBECONF`.
.. _migration-1.7-qa-check-changes:
QA Check Changes
----------------
The following changes have occurred to the QA check process:
- Additional QA checks ``file-rdeps`` and ``build-deps`` have been
added in order to verify that file dependencies are satisfied (e.g.
package contains a script requiring ``/bin/bash``) and build-time
dependencies are declared, respectively. For more information, please
see the ":doc:`/ref-manual/qa-checks`" chapter.
- Package QA checks are now performed during a new
:ref:`ref-tasks-package_qa` task rather than being
part of the :ref:`ref-tasks-package` task. This allows
more parallel execution. This change is unlikely to be an issue
except for highly customized recipes that disable packaging tasks
themselves by marking them as ``noexec``. For those packages, you
will need to disable the :ref:`ref-tasks-package_qa` task as well.
- Files being overwritten during the
:ref:`ref-tasks-populate_sysroot` task now
trigger an error instead of a warning. Recipes should not be
overwriting files written to the sysroot by other recipes. If you
have these types of recipes, you need to alter them so that they do
not overwrite these files.
You might now receive this error after changes in configuration or
metadata resulting in orphaned files being left in the sysroot. If
you do receive this error, the way to resolve the issue is to delete
your :term:`TMPDIR` or to move it out of the way and
then re-start the build. Anything that has been fully built up to
that point and does not need rebuilding will be restored from the
shared state cache and the rest of the build will be able to proceed
as normal.
.. _migration-1.7-removed-recipes:
Removed Recipes
---------------
The following recipes have been removed:
- ``x-load``: This recipe has been superseded by U-Boot SPL for all
Cortex-based TI SoCs. For legacy boards, the ``meta-ti`` layer, which
contains a maintained recipe, should be used instead.
- ``ubootchart``: This recipe is obsolete. A ``bootchart2`` recipe has
been added to functionally replace it.
- ``linux-yocto 3.4``: Support for the linux-yocto 3.4 kernel has been
dropped. Support for the 3.10 and 3.14 kernels remains, while support
for version 3.17 has been added.
- ``eglibc`` has been removed in favor of ``glibc``. See the
":ref:`migration-1.7-glibc-replaces-eglibc`" section for more information.
.. _migration-1.7-miscellaneous-changes:
Miscellaneous Changes
---------------------
The following miscellaneous change occurred:
- The build history feature now writes ``build-id.txt`` instead of
``build-id``. Additionally, ``build-id.txt`` now contains the full
build header as printed by BitBake upon starting the build. You
should manually remove old "build-id" files from your existing build
history repositories to avoid confusion. For information on the build
history feature, see the
":ref:`dev-manual/build-quality:maintaining build output quality`"
section in the Yocto Project Development Tasks Manual.
poky/documentation/migration-guides/migration-1.8.rst
+183
View File
@@ -0,0 +1,183 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 1.8 (fido)
==================
This section provides migration information for moving to the Yocto
Project 1.8 Release (codename "fido") from the prior release.
.. _migration-1.8-removed-recipes:
Removed Recipes
---------------
The following recipes have been removed:
- ``owl-video``: Functionality replaced by ``gst-player``.
- ``gaku``: Functionality replaced by ``gst-player``.
- ``gnome-desktop``: This recipe is now available in ``meta-gnome`` and
is no longer needed.
- ``gsettings-desktop-schemas``: This recipe is now available in
``meta-gnome`` and is no longer needed.
- ``python-argparse``: The ``argparse`` module is already provided in
the default Python distribution in a package named
``python-argparse``. Consequently, the separate ``python-argparse``
recipe is no longer needed.
- ``telepathy-python, libtelepathy, telepathy-glib, telepathy-idle, telepathy-mission-control``:
All these recipes have moved to ``meta-oe`` and are consequently no
longer needed by any recipes in OpenEmbedded-Core.
- ``linux-yocto_3.10`` and ``linux-yocto_3.17``: Support for the
linux-yocto 3.10 and 3.17 kernels has been dropped. Support for the
3.14 kernel remains, while support for 3.19 kernel has been added.
- ``poky-feed-config-opkg``: This recipe has become obsolete and is no
longer needed. Use ``distro-feed-config`` from ``meta-oe`` instead.
- ``libav 0.8.x``: ``libav 9.x`` is now used.
- ``sed-native``: No longer needed. A working version of ``sed`` is
expected to be provided by the host distribution.
.. _migration-1.8-bluez:
BlueZ 4.x / 5.x Selection
-------------------------
Proper built-in support for selecting BlueZ 5.x in preference to the
default of 4.x now exists. To use BlueZ 5.x, simply add "bluez5" to your
:term:`DISTRO_FEATURES` value. If you had
previously added append files (``*.bbappend``) to make this selection,
you can now remove them.
Additionally, a ``bluetooth`` class has been added to make selection of
the appropriate bluetooth support within a recipe a little easier. If
you wish to make use of this class in a recipe, add something such as
the following::
inherit bluetooth
PACKAGECONFIG ??= "${@bb.utils.contains('DISTRO_FEATURES', 'bluetooth', '${BLUEZ}', '', d)}"
PACKAGECONFIG[bluez4] = "--enable-bluetooth,--disable-bluetooth,bluez4"
PACKAGECONFIG[bluez5] = "--enable-bluez5,--disable-bluez5,bluez5"
.. _migration-1.8-kernel-build-changes:
Kernel Build Changes
--------------------
The kernel build process was changed to place the source in a common shared work
area and to place build artifacts separately in the source code tree. In theory,
migration paths have been provided for most common usages in kernel recipes but
this might not work in all cases. In particular, users need to ensure that
``${S}`` (source files) and ``${B}`` (build artifacts) are used correctly in
functions such as :ref:`ref-tasks-configure` and :ref:`ref-tasks-install`. For
kernel recipes that do not inherit from :ref:`ref-classes-kernel-yocto` or
include ``linux-yocto.inc``, you might wish to refer to the ``linux.inc`` file
in the ``meta-oe`` layer for the kinds of changes you need to make. For reference,
here is the
:oe_git:`commit </meta-openembedded/commit/meta-oe/recipes-kernel/linux/linux.inc?id=fc7132ede27ac67669448d3d2845ce7d46c6a1ee>`
where the ``linux.inc`` file in ``meta-oe`` was updated.
Recipes that rely on the kernel source code and do not inherit the
:ref:`module <ref-classes-module>` classes might need to add explicit
dependencies on the :ref:`ref-tasks-shared_workdir` kernel task, for example::
do_configure[depends] += "virtual/kernel:do_shared_workdir"
.. _migration-1.8-ssl:
SSL 3.0 is Now Disabled in OpenSSL
----------------------------------
SSL 3.0 is now disabled when building OpenSSL. Disabling SSL 3.0 avoids
any lingering instances of the POODLE vulnerability. If you feel you
must re-enable SSL 3.0, then you can add an append file (``*.bbappend``)
for the ``openssl`` recipe to remove "-no-ssl3" from
:term:`EXTRA_OECONF`.
.. _migration-1.8-default-sysroot-poisoning:
Default Sysroot Poisoning
-------------------------
``gcc's`` default sysroot and include directories are now "poisoned". In
other words, the sysroot and include directories are being redirected to
a non-existent location in order to catch when host directories are
being used due to the correct options not being passed. This poisoning
applies both to the cross-compiler used within the build and to the
cross-compiler produced in the SDK.
If this change causes something in the build to fail, it almost
certainly means the various compiler flags and commands are not being
passed correctly to the underlying piece of software. In such cases, you
need to take corrective steps.
.. _migration-1.8-rebuild-improvements:
Rebuild Improvements
--------------------
Changes have been made to the :ref:`ref-classes-base`,
:ref:`ref-classes-autotools`, and :ref:`ref-classes-cmake` classes to clean out
generated files when the :ref:`ref-tasks-configure` task needs to be
re-executed.
One of the improvements is to attempt to run "make clean" during the
:ref:`ref-tasks-configure` task if a ``Makefile`` exists. Some software packages
do not provide a working clean target within their make files. If you
have such recipes, you need to set
:term:`CLEANBROKEN` to "1" within the recipe, for example::
CLEANBROKEN = "1"
.. _migration-1.8-qa-check-and-validation-changes:
QA Check and Validation Changes
-------------------------------
The following QA Check and Validation Changes have occurred:
- Usage of ``PRINC`` previously triggered a warning. It now triggers an
error. You should remove any remaining usage of ``PRINC`` in any
recipe or append file.
- An additional QA check has been added to detect usage of ``${D}`` in
:term:`FILES` values where :term:`D` values
should not be used at all. The same check ensures that ``$D`` is used
in ``pkg_preinst/pkg_postinst/pkg_prerm/pkg_postrm`` functions
instead of ``${D}``.
- :term:`S` now needs to be set to a valid value within a
recipe. If :term:`S` is not set in the recipe, the directory is not
automatically created. If :term:`S` does not point to a directory that
exists at the time the :ref:`ref-tasks-unpack` task
finishes, a warning will be shown.
- :term:`LICENSE` is now validated for correct
formatting of multiple licenses. If the format is invalid (e.g.
multiple licenses are specified with no operators to specify how the
multiple licenses interact), then a warning will be shown.
.. _migration-1.8-miscellaneous-changes:
Miscellaneous Changes
---------------------
The following miscellaneous changes have occurred:
- The ``send-error-report`` script now expects a "-s" option to be
specified before the server address. This assumes a server address is
being specified.
- The ``oe-pkgdata-util`` script now expects a "-p" option to be
specified before the ``pkgdata`` directory, which is now optional. If
the ``pkgdata`` directory is not specified, the script will run
BitBake to query :term:`PKGDATA_DIR` from the
build environment.
poky/documentation/migration-guides/migration-2.0.rst
+280
View File
@@ -0,0 +1,280 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 2.0 (jethro)
====================
This section provides migration information for moving to the Yocto
Project 2.0 Release (codename "jethro") from the prior release.
.. _migration-2.0-gcc-5:
GCC 5
-----
The default compiler is now GCC 5.2. This change has required fixes for
compilation errors in a number of other recipes.
One important example is a fix for when the Linux kernel freezes at boot
time on ARM when built with GCC 5. If you are using your own kernel
recipe or source tree and building for ARM, you will likely need to
apply this
`patch <https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit?id=a077224fd35b2f7fbc93f14cf67074fc792fbac2>`__.
The standard ``linux-yocto`` kernel source tree already has a workaround
for the same issue.
For further details, see https://gcc.gnu.org/gcc-5/changes.html
and the porting guide at
https://gcc.gnu.org/gcc-5/porting_to.html.
Alternatively, you can switch back to GCC 4.9 or 4.8 by setting
:term:`GCCVERSION` in your configuration, as follows::
GCCVERSION = "4.9%"
.. _migration-2.0-Gstreamer-0.10-removed:
Gstreamer 0.10 Removed
----------------------
Gstreamer 0.10 has been removed in favor of Gstreamer 1.x. As part of
the change, recipes for Gstreamer 0.10 and related software are now
located in ``meta-multimedia``. This change results in Qt4 having Phonon
and Gstreamer support in QtWebkit disabled by default.
.. _migration-2.0-removed-recipes:
Removed Recipes
---------------
The following recipes have been moved or removed:
- ``bluez4``: The recipe is obsolete and has been moved due to
``bluez5`` becoming fully integrated. The ``bluez4`` recipe now
resides in ``meta-oe``.
- ``gamin``: The recipe is obsolete and has been removed.
- ``gnome-icon-theme``: The recipe's functionally has been replaced by
``adwaita-icon-theme``.
- Gstreamer 0.10 Recipes: Recipes for Gstreamer 0.10 have been removed
in favor of the recipes for Gstreamer 1.x.
- ``insserv``: The recipe is obsolete and has been removed.
- ``libunique``: The recipe is no longer used and has been moved to
``meta-oe``.
- ``midori``: The recipe's functionally has been replaced by
``epiphany``.
- ``python-gst``: The recipe is obsolete and has been removed since it
only contains bindings for Gstreamer 0.10.
- ``qt-mobility``: The recipe is obsolete and has been removed since it
requires ``Gstreamer 0.10``, which has been replaced.
- ``subversion``: All 1.6.x versions of this recipe have been removed.
- ``webkit-gtk``: The older 1.8.3 version of this recipe has been
removed in favor of ``webkitgtk``.
.. _migration-2.0-bitbake-datastore-improvements:
BitBake datastore improvements
------------------------------
The method by which BitBake's datastore handles overrides has changed.
Overrides are now applied dynamically and ``bb.data.update_data()`` is
now a no-op. Thus, ``bb.data.update_data()`` is no longer required in
order to apply the correct overrides. In practice, this change is
unlikely to require any changes to Metadata. However, these minor
changes in behavior exist:
- All potential overrides are now visible in the variable history as
seen when you run the following::
$ bitbake -e
- ``d.delVar('VARNAME')`` and
``d.setVar('VARNAME', None)`` result in the variable and all
of its overrides being cleared out. Before the change, only the
non-overridden values were cleared.
.. _migration-2.0-shell-message-function-changes:
Shell Message Function Changes
------------------------------
The shell versions of the BitBake message functions (i.e. ``bbdebug``,
``bbnote``, ``bbwarn``, ``bbplain``, ``bberror``, and ``bbfatal``) are
now connected through to their BitBake equivalents ``bb.debug()``,
``bb.note()``, ``bb.warn()``, ``bb.plain()``, ``bb.error()``, and
``bb.fatal()``, respectively. Thus, those message functions that you
would expect to be printed by the BitBake UI are now actually printed.
In practice, this change means two things:
- If you now see messages on the console that you did not previously
see as a result of this change, you might need to clean up the calls
to ``bbwarn``, ``bberror``, and so forth. Or, you might want to
simply remove the calls.
- The ``bbfatal`` message function now suppresses the full error log in
the UI, which means any calls to ``bbfatal`` where you still wish to
see the full error log should be replaced by ``die`` or
``bbfatal_log``.
.. _migration-2.0-extra-development-debug-package-cleanup:
Extra Development/Debug Package Cleanup
---------------------------------------
The following recipes have had extra ``dev/dbg`` packages removed:
- ``acl``
- ``apmd``
- ``aspell``
- ``attr``
- ``augeas``
- ``bzip2``
- ``cogl``
- ``curl``
- ``elfutils``
- ``gcc-target``
- ``libgcc``
- ``libtool``
- ``libxmu``
- ``opkg``
- ``pciutils``
- ``rpm``
- ``sysfsutils``
- ``tiff``
- ``xz``
All of the above recipes now conform to the standard packaging scheme
where a single ``-dev``, ``-dbg``, and ``-staticdev`` package exists per
recipe.
.. _migration-2.0-recipe-maintenance-tracking-data-moved-to-oe-core:
Recipe Maintenance Tracking Data Moved to OE-Core
-------------------------------------------------
Maintenance tracking data for recipes that was previously part of
``meta-yocto`` has been moved to :term:`OpenEmbedded-Core (OE-Core)`. The change
includes ``package_regex.inc`` and ``distro_alias.inc``, which are
typically enabled when using the ``distrodata`` class. Additionally, the
contents of ``upstream_tracking.inc`` has now been split out to the
relevant recipes.
.. _migration-2.0-automatic-stale-sysroot-file-cleanup:
Automatic Stale Sysroot File Cleanup
------------------------------------
Stale files from recipes that no longer exist in the current
configuration are now automatically removed from sysroot as well as
removed from any other place managed by shared state. This automatic
cleanup means that the build system now properly handles situations such
as renaming the build system side of recipes, removal of layers from
``bblayers.conf``, and :term:`DISTRO_FEATURES`
changes.
Additionally, work directories for old versions of recipes are now
pruned. If you wish to disable pruning old work directories, you can set
the following variable in your configuration::
SSTATE_PRUNE_OBSOLETEWORKDIR = "0"
.. _migration-2.0-linux-yocto-kernel-metadata-repository-now-split-from-source:
``linux-yocto`` Kernel Metadata Repository Now Split from Source
----------------------------------------------------------------
The ``linux-yocto`` tree has up to now been a combined set of kernel
changes and configuration (meta) data carried in a single tree. While
this format is effective at keeping kernel configuration and source
modifications synchronized, it is not always obvious to developers how
to manipulate the Metadata as compared to the source.
Metadata processing has now been removed from the
:ref:`ref-classes-kernel-yocto` class and the external
Metadata repository ``yocto-kernel-cache``, which has always been used
to seed the ``linux-yocto`` "meta" branch. This separate ``linux-yocto``
cache repository is now the primary location for this data. Due to this
change, ``linux-yocto`` is no longer able to process combined trees.
Thus, if you need to have your own combined kernel repository, you must
do the split there as well and update your recipes accordingly. See the
``meta/recipes-kernel/linux/linux-yocto_4.1.bb`` recipe for an example.
.. _migration-2.0-additional-qa-checks:
Additional QA checks
--------------------
The following QA checks have been added:
- Added a "host-user-contaminated" check for ownership issues for
packaged files outside of ``/home``. The check looks for files that
are incorrectly owned by the user that ran BitBake instead of owned
by a valid user in the target system.
- Added an "invalid-chars" check for invalid (non-UTF8) characters in
recipe metadata variable values (i.e.
:term:`DESCRIPTION`,
:term:`SUMMARY`, :term:`LICENSE`, and
:term:`SECTION`). Some package managers do not support
these characters.
- Added an "invalid-packageconfig" check for any options specified in
:term:`PACKAGECONFIG` that do not match any
:term:`PACKAGECONFIG` option defined for the recipe.
.. _migration-2.0-miscellaneous:
Miscellaneous Changes
---------------------
These additional changes exist:
- ``gtk-update-icon-cache`` has been renamed to ``gtk-icon-utils``.
- The ``tools-profile`` :term:`IMAGE_FEATURES`
item as well as its corresponding packagegroup and
``packagegroup-core-tools-profile`` no longer bring in ``oprofile``.
Bringing in ``oprofile`` was originally added to aid compilation on
resource-constrained targets. However, this aid has not been widely
used and is not likely to be used going forward due to the more
powerful target platforms and the existence of better
cross-compilation tools.
- The :term:`IMAGE_FSTYPES` variable's default
value now specifies ``ext4`` instead of ``ext3``.
- All support for the ``PRINC`` variable has been removed.
- The ``packagegroup-core-full-cmdline`` packagegroup no longer brings
in ``lighttpd`` due to the fact that bringing in ``lighttpd`` is not
really in line with the packagegroup's purpose, which is to add full
versions of command-line tools that by default are provided by
``busybox``.
poky/documentation/migration-guides/migration-2.1.rst
+432
View File
@@ -0,0 +1,432 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 2.1 (krogoth)
=====================
This section provides migration information for moving to the Yocto
Project 2.1 Release (codename "krogoth") from the prior release.
.. _migration-2.1-variable-expansion-in-python-functions:
Variable Expansion in Python Functions
--------------------------------------
Variable expressions, such as ``${VARNAME}`` no longer expand
automatically within Python functions. Suppressing expansion was done to
allow Python functions to construct shell scripts or other code for
situations in which you do not want such expressions expanded. For any
existing code that relies on these expansions, you need to change the
expansions to expand the value of individual variables through
``d.getVar()``. To alternatively expand more complex expressions, use
``d.expand()``.
.. _migration-2.1-overrides-must-now-be-lower-case:
Overrides Must Now be Lower-Case
--------------------------------
The convention for overrides has always been for them to be lower-case
characters. This practice is now a requirement as BitBake's datastore
now assumes lower-case characters in order to give a slight performance
boost during parsing. In practical terms, this requirement means that
anything that ends up in :term:`OVERRIDES` must now
appear in lower-case characters (e.g. values for :term:`MACHINE`,
:term:`TARGET_ARCH`, :term:`DISTRO`, and also recipe names if
``_pn-``\ recipename overrides are to be effective).
.. _migration-2.1-expand-parameter-to-getvar-and-getvarflag-now-mandatory:
Expand Parameter to ``getVar()`` and ``getVarFlag()`` is Now Mandatory
----------------------------------------------------------------------
The expand parameter to ``getVar()`` and ``getVarFlag()`` previously
defaulted to False if not specified. Now, however, no default exists so
one must be specified. You must change any ``getVar()`` calls that do
not specify the final expand parameter to calls that do specify the
parameter. You can run the following ``sed`` command at the base of a
layer to make this change::
sed -e 's:\(\.getVar([^,()]*\)):\1, False):g' -i `grep -ril getVar *`
sed -e 's:\(\.getVarFlag([^,()]*,[^,()]*\)):\1, False):g' -i `grep -ril getVarFlag *`
.. note::
The reason for this change is that it prepares the way for changing
the default to True in a future Yocto Project release. This future
change is a much more sensible default than False. However, the
change needs to be made gradually as a sudden change of the default
would potentially cause side-effects that would be difficult to
detect.
.. _migration-2.1-makefile-environment-changes:
Makefile Environment Changes
----------------------------
:term:`EXTRA_OEMAKE` now defaults to "" instead of
"-e MAKEFLAGS=". Setting :term:`EXTRA_OEMAKE` to "-e MAKEFLAGS=" by default
was a historical accident that has required many classes (e.g.
:ref:`ref-classes-autotools`, ``module``) and recipes to override this default in order
to work with sensible build systems. When upgrading to the release, you
must edit any recipe that relies upon this old default by either setting
:term:`EXTRA_OEMAKE` back to "-e MAKEFLAGS=" or by explicitly setting any
required variable value overrides using :term:`EXTRA_OEMAKE`, which is
typically only needed when a Makefile sets a default value for a
variable that is inappropriate for cross-compilation using the "="
operator rather than the "?=" operator.
.. _migration-2.1-libexecdir-reverted-to-prefix-libexec:
``libexecdir`` Reverted to ``${prefix}/libexec``
------------------------------------------------
The use of ``${libdir}/${BPN}`` as ``libexecdir`` is different as
compared to all other mainstream distributions, which either uses
``${prefix}/libexec`` or ``${libdir}``. The use is also contrary to the
GNU Coding Standards (i.e.
https://www.gnu.org/prep/standards/html_node/Directory-Variables.html)
that suggest ``${prefix}/libexec`` and also notes that any
package-specific nesting should be done by the package itself. Finally,
having ``libexecdir`` change between recipes makes it very difficult for
different recipes to invoke binaries that have been installed into
``libexecdir``. The Filesystem Hierarchy Standard (i.e.
https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch04s07.html) now
recognizes the use of ``${prefix}/libexec/``, giving distributions the
choice between ``${prefix}/lib`` or ``${prefix}/libexec`` without
breaking FHS.
.. _migration-2.1-ac-cv-sizeof-off-t-no-longer-cached-in-site-files:
``ac_cv_sizeof_off_t`` is No Longer Cached in Site Files
--------------------------------------------------------
For recipes inheriting the :ref:`ref-classes-autotools`
class, ``ac_cv_sizeof_off_t`` is no longer cached in the site files for
``autoconf``. The reason for this change is because the
``ac_cv_sizeof_off_t`` value is not necessarily static per architecture
as was previously assumed. Rather, the value changes based on whether
large file support is enabled. For most software that uses ``autoconf``,
this change should not be a problem. However, if you have a recipe that
bypasses the standard :ref:`ref-tasks-configure` task
from the :ref:`ref-classes-autotools` class and the software the recipe is building
uses a very old version of ``autoconf``, the recipe might be incapable
of determining the correct size of ``off_t`` during :ref:`ref-tasks-configure`.
The best course of action is to patch the software as necessary to allow
the default implementation from the :ref:`ref-classes-autotools` class to work such
that ``autoreconf`` succeeds and produces a working configure script,
and to remove the overridden :ref:`ref-tasks-configure` task such that the default
implementation does get used.
.. _migration-2.1-image-generation-split-out-from-filesystem-generation:
Image Generation is Now Split Out from Filesystem Generation
------------------------------------------------------------
Previously, for image recipes the :ref:`ref-tasks-rootfs`
task assembled the filesystem and then from that filesystem generated
images. With this Yocto Project release, image generation is split into
separate :ref:`ref-tasks-image` tasks for clarity both in
operation and in the code.
For most cases, this change does not present any problems. However, if
you have made customizations that directly modify the :ref:`ref-tasks-rootfs` task
or that mention :ref:`ref-tasks-rootfs`, you might need to update those changes.
In particular, if you had added any tasks after :ref:`ref-tasks-rootfs`, you
should make edits so that those tasks are after the
:ref:`ref-tasks-image-complete` task rather than
after :ref:`ref-tasks-rootfs` so that your added tasks run at the correct
time.
A minor part of this restructuring is that the post-processing definitions and
functions have been moved from the :ref:`ref-classes-image` class to the
:ref:`rootfs-postcommands <ref-classes-rootfs*>` class. Functionally,
however, they remain unchanged.
.. _migration-2.1-removed-recipes:
Removed Recipes
---------------
The following recipes have been removed in the 2.1 release:
- ``gcc`` version 4.8: Versions 4.9 and 5.3 remain.
- ``qt4``: All support for Qt 4.x has been moved out to a separate
``meta-qt4`` layer because Qt 4 is no longer supported upstream.
- ``x11vnc``: Moved to the ``meta-oe`` layer.
- ``linux-yocto-3.14``: No longer supported.
- ``linux-yocto-3.19``: No longer supported.
- ``libjpeg``: Replaced by the ``libjpeg-turbo`` recipe.
- ``pth``: Became obsolete.
- ``liboil``: Recipe is no longer needed and has been moved to the
``meta-multimedia`` layer.
- ``gtk-theme-torturer``: Recipe is no longer needed and has been moved
to the ``meta-gnome`` layer.
- ``gnome-mime-data``: Recipe is no longer needed and has been moved to
the ``meta-gnome`` layer.
- ``udev``: Replaced by the ``eudev`` recipe for compatibility when
using ``sysvinit`` with newer kernels.
- ``python-pygtk``: Recipe became obsolete.
- ``adt-installer``: Recipe became obsolete. See the
":ref:`migration-guides/migration-2.1:adt removed`" section for more information.
.. _migration-2.1-class-changes:
Class Changes
-------------
The following classes have changed:
- ``autotools_stage``: Removed because the
:ref:`ref-classes-autotools` class now provides its
functionality. Recipes that inherited from ``autotools_stage`` should
now inherit from :ref:`ref-classes-autotools` instead.
- ``boot-directdisk``: Merged into the ``image-vm`` class. The
``boot-directdisk`` class was rarely directly used. Consequently,
this change should not cause any issues.
- ``bootimg``: Merged into the :ref:`ref-classes-image-live` class. The
``bootimg`` class was rarely directly used. Consequently, this change should
not cause any issues.
- ``packageinfo``: Removed due to its limited use by the Hob UI, which
has itself been removed.
.. _migration-2.1-build-system-ui-changes:
Build System User Interface Changes
-----------------------------------
The following changes have been made to the build system user interface:
- *Hob GTK+-based UI*: Removed because it is unmaintained and based on
the outdated GTK+ 2 library. The Toaster web-based UI is much more
capable and is actively maintained. See the
":ref:`toaster-manual/setup-and-use:using the toaster web interface`"
section in the Toaster User Manual for more information on this
interface.
- *"puccho" BitBake UI*: Removed because is unmaintained and no longer
useful.
.. _migration-2.1-adt-removed:
ADT Removed
-----------
The Application Development Toolkit (ADT) has been removed because its
functionality almost completely overlapped with the :ref:`standard
SDK <sdk-manual/using:using the standard sdk>` and the
:ref:`extensible SDK <sdk-manual/extensible:using the extensible sdk>`. For
information on these SDKs and how to build and use them, see the
:doc:`/sdk-manual/index` manual.
.. note::
The Yocto Project Eclipse IDE Plug-in is still supported and is not
affected by this change.
.. _migration-2.1-poky-reference-distribution-changes:
Poky Reference Distribution Changes
-----------------------------------
The following changes have been made for the Poky distribution:
- The ``meta-yocto`` layer has been renamed to ``meta-poky`` to better
match its purpose, which is to provide the Poky reference
distribution. The ``meta-yocto-bsp`` layer retains its original name
since it provides reference machines for the Yocto Project and it is
otherwise unrelated to Poky. References to ``meta-yocto`` in your
``conf/bblayers.conf`` should automatically be updated, so you should
not need to change anything unless you are relying on this naming
elsewhere.
- The :ref:`ref-classes-uninative` class is now enabled
by default in Poky. This class attempts to isolate the build system
from the host distribution's C library and makes re-use of native
shared state artifacts across different host distributions practical.
With this class enabled, a tarball containing a pre-built C library
is downloaded at the start of the build.
The :ref:`ref-classes-uninative` class is enabled through the
``meta/conf/distro/include/yocto-uninative.inc`` file, which for
those not using the Poky distribution, can include to easily enable
the same functionality.
Alternatively, if you wish to build your own ``uninative`` tarball,
you can do so by building the ``uninative-tarball`` recipe, making it
available to your build machines (e.g. over HTTP/HTTPS) and setting a
similar configuration as the one set by ``yocto-uninative.inc``.
- Static library generation, for most cases, is now disabled by default
in the Poky distribution. Disabling this generation saves some build
time as well as the size used for build output artifacts.
Disabling this library generation is accomplished through a
``meta/conf/distro/include/no-static-libs.inc``, which for those not
using the Poky distribution can easily include to enable the same
functionality.
Any recipe that needs to opt-out of having the ``--disable-static``
option specified on the configure command line either because it is
not a supported option for the configure script or because static
libraries are needed should set the following variable::
DISABLE_STATIC = ""
- The separate ``poky-tiny`` distribution now uses the musl C library
instead of a heavily pared down ``glibc``. Using musl results in a
smaller distribution and facilitates much greater maintainability
because musl is designed to have a small footprint.
If you have used ``poky-tiny`` and have customized the ``glibc``
configuration you will need to redo those customizations with musl
when upgrading to the new release.
.. _migration-2.1-packaging-changes:
Packaging Changes
-----------------
The following changes have been made to packaging:
- The ``runuser`` and ``mountpoint`` binaries, which were previously in
the main ``util-linux`` package, have been split out into the
``util-linux-runuser`` and ``util-linux-mountpoint`` packages,
respectively.
- The ``python-elementtree`` package has been merged into the
``python-xml`` package.
.. _migration-2.1-tuning-file-changes:
Tuning File Changes
-------------------
The following changes have been made to the tuning files:
- The "no-thumb-interwork" tuning feature has been dropped from the ARM
tune include files. Because interworking is required for ARM EABI,
attempting to disable it through a tuning feature no longer makes
sense.
.. note::
Support for ARM OABI was deprecated in gcc 4.7.
- The ``tune-cortexm*.inc`` and ``tune-cortexr4.inc`` files have been
removed because they are poorly tested. Until the OpenEmbedded build
system officially gains support for CPUs without an MMU, these tuning
files would probably be better maintained in a separate layer if
needed.
.. _migration-2.1-supporting-gobject-introspection:
Supporting GObject Introspection
--------------------------------
This release supports generation of GLib Introspective Repository (GIR)
files through GObject introspection, which is the standard mechanism for
accessing GObject-based software from runtime environments. You can
enable, disable, and test the generation of this data. See the
":ref:`dev-manual/gobject-introspection:enabling gobject introspection support`"
section in the Yocto Project Development Tasks Manual for more
information.
.. _migration-2.1-miscellaneous-changes:
Miscellaneous Changes
---------------------
These additional changes exist:
- The minimum Git version has been increased to 1.8.3.1. If your host
distribution does not provide a sufficiently recent version, you can
install the :term:`buildtools`, which will provide it. See the
:ref:`ref-manual/system-requirements:required git, tar, python, make and gcc versions`
section for more information on the :term:`buildtools` tarball.
- The buggy and incomplete support for the RPM version 4 package
manager has been removed. The well-tested and maintained support for
RPM version 5 remains.
- Previously, the following list of packages were removed if
package-management was not in
:term:`IMAGE_FEATURES`, regardless of any
dependencies::
update-rc.d
base-passwd
shadow
update-alternatives
run-postinsts
With the Yocto Project 2.1 release, these packages are
only removed if "read-only-rootfs" is in :term:`IMAGE_FEATURES`, since
they might still be needed for a read-write image even in the absence
of a package manager (e.g. if users need to be added, modified, or
removed at runtime).
- The
:ref:`devtool modify <sdk-manual/extensible:use \`\`devtool modify\`\` to modify the source of an existing component>`
command now defaults to extracting the source since that is most
commonly expected. The ``-x`` or ``--extract`` options are now no-ops. If
you wish to provide your own existing source tree, you will now need
to specify either the ``-n`` or ``--no-extract`` options when running
``devtool modify``.
- If the formfactor for a machine is either not supplied or does not
specify whether a keyboard is attached, then the default is to assume
a keyboard is attached rather than assume no keyboard. This change
primarily affects the Sato UI.
- The ``.debug`` directory packaging is now automatic. If your recipe
builds software that installs binaries into directories other than
the standard ones, you no longer need to take care of setting
``FILES_${PN}-dbg`` to pick up the resulting ``.debug`` directories
as these directories are automatically found and added.
- Inaccurate disk and CPU percentage data has been dropped from
:ref:`ref-classes-buildstats` output. This data has been replaced with
``getrusage()`` data and corrected IO statistics. You will probably
need to update any custom code that reads the :ref:`ref-classes-buildstats` data.
- The ``meta/conf/distro/include/package_regex.inc`` is now deprecated.
The contents of this file have been moved to individual recipes.
.. note::
Because this file will likely be removed in a future Yocto Project
release, it is suggested that you remove any references to the
file that might be in your configuration.
- The ``v86d/uvesafb`` has been removed from the ``genericx86`` and
``genericx86-64`` reference machines, which are provided by the
``meta-yocto-bsp`` layer. Most modern x86 boards do not rely on this
file and it only adds kernel error messages during startup. If you do
still need to support ``uvesafb``, you can simply add ``v86d`` to
your image.
- Build sysroot paths are now removed from debug symbol files. Removing
these paths means that remote GDB using an unstripped build system
sysroot will no longer work (although this was never documented to
work). The supported method to accomplish something similar is to set
``IMAGE_GEN_DEBUGFS`` to "1", which will generate a companion debug
image containing unstripped binaries and associated debug sources
alongside the image.
poky/documentation/migration-guides/migration-2.2.rst
+456
View File
@@ -0,0 +1,456 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 2.2 (morty)
===================
This section provides migration information for moving to the Yocto
Project 2.2 Release (codename "morty") from the prior release.
.. _migration-2.2-minimum-kernel-version:
Minimum Kernel Version
----------------------
The minimum kernel version for the target system and for SDK is now
3.2.0, due to the upgrade to ``glibc 2.24``. Specifically, for
AArch64-based targets the version is 3.14. For Nios II-based targets,
the minimum kernel version is 3.19.
.. note::
For x86 and x86_64, you can reset :term:`OLDEST_KERNEL`
to anything down to 2.6.32 if desired.
.. _migration-2.2-staging-directories-in-sysroot-simplified:
Staging Directories in Sysroot Has Been Simplified
--------------------------------------------------
The way directories are staged in sysroot has been simplified and
introduces the new :term:`SYSROOT_DIRS`,
:term:`SYSROOT_DIRS_NATIVE`, and ``SYSROOT_DIRS_BLACKLIST``
(replaced by :term:`SYSROOT_DIRS_IGNORE` in version 3.5). See the
:oe_lists:`v2 patch series on the OE-Core Mailing List
</pipermail/openembedded-core/2016-May/121365.html>`
for additional information.
.. _migration-2.2-removal-of-old-images-from-tmp-deploy-now-enabled:
Removal of Old Images and Other Files in ``tmp/deploy`` Now Enabled
-------------------------------------------------------------------
Removal of old images and other files in ``tmp/deploy/`` is now enabled
by default due to a new staging method used for those files. As a result
of this change, the ``RM_OLD_IMAGE`` variable is now redundant.
.. _migration-2.2-python-changes:
Python Changes
--------------
The following changes for Python occurred:
.. _migration-2.2-bitbake-now-requires-python-3.4:
BitBake Now Requires Python 3.4+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
BitBake requires Python 3.4 or greater.
.. _migration-2.2-utf-8-locale-required-on-build-host:
UTF-8 Locale Required on Build Host
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A UTF-8 locale is required on the build host due to Python 3. Since
C.UTF-8 is not a standard, the default is en_US.UTF-8.
.. _migration-2.2-metadata-now-must-use-python-3-syntax:
Metadata Must Now Use Python 3 Syntax
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The metadata is now required to use Python 3 syntax. For help preparing
metadata, see any of the many Python 3 porting guides available.
Alternatively, you can reference the conversion commits for BitBake and
you can use :term:`OpenEmbedded-Core (OE-Core)` as a guide for changes. Following are
particular areas of interest:
- subprocess command-line pipes needing locale decoding
- the syntax for octal values changed
- the ``iter*()`` functions changed name
- iterators now return views, not lists
- changed names for Python modules
.. _migration-2.2-target-python-recipes-switched-to-python-3:
Target Python Recipes Switched to Python 3
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Most target Python recipes have now been switched to Python 3.
Unfortunately, systems using RPM as a package manager and providing
online package-manager support through SMART still require Python 2.
.. note::
Python 2 and recipes that use it can still be built for the target as
with previous versions.
.. _migration-2.2-buildtools-tarball-includes-python-3:
``buildtools-tarball`` Includes Python 3
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The :term:`buildtools` tarball now includes Python 3.
.. _migration-2.2-uclibc-replaced-by-musl:
uClibc Replaced by musl
-----------------------
uClibc has been removed in favor of musl. Musl has matured, is better
maintained, and is compatible with a wider range of applications as
compared to uClibc.
.. _migration-2.2-B-no-longer-default-working-directory-for-tasks:
``${B}`` No Longer Default Working Directory for Tasks
------------------------------------------------------
``${``\ :term:`B`\ ``}`` is no longer the default working directory for tasks.
Consequently, any custom tasks you define now need to either have the
``[``\ :ref:`dirs <bitbake-user-manual/bitbake-user-manual-metadata:variable flags>`\ ``]``
flag set, or the task needs to change into the appropriate working directory
manually (e.g using ``cd`` for a shell task).
.. note::
The preferred method is to use the
[dirs]
flag.
.. _migration-2.2-runqemu-ported-to-python:
``runqemu`` Ported to Python
----------------------------
``runqemu`` has been ported to Python and has changed behavior in some
cases. Previous usage patterns continue to be supported.
The new ``runqemu`` is a Python script. Machine knowledge is no longer
hardcoded into ``runqemu``. You can choose to use the ``qemuboot``
configuration file to define the BSP's own arguments and to make it
bootable with ``runqemu``. If you use a configuration file, use the
following form::
image-name-machine.qemuboot.conf
The configuration file
enables fine-grained tuning of options passed to QEMU without the
``runqemu`` script hard-coding any knowledge about different machines.
Using a configuration file is particularly convenient when trying to use
QEMU with machines other than the ``qemu*`` machines in
:term:`OpenEmbedded-Core (OE-Core)`. The ``qemuboot.conf`` file is generated by the
``qemuboot`` class when the root filesystem is being built (i.e. build
rootfs). QEMU boot arguments can be set in BSP's configuration file and
the ``qemuboot`` class will save them to ``qemuboot.conf``.
If you want to use ``runqemu`` without a configuration file, use the
following command form::
$ runqemu machine rootfs kernel [options]
Supported machines are as follows:
- qemuarm
- qemuarm64
- qemux86
- qemux86-64
- qemuppc
- qemumips
- qemumips64
- qemumipsel
- qemumips64el
Consider the
following example, which uses the ``qemux86-64`` machine, provides a
root filesystem, provides an image, and uses the ``nographic`` option::
$ runqemu qemux86-64 tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.ext4 tmp/deploy/images/qemux86-64/bzImage nographic
Following is a list of variables that can be set in configuration files
such as ``bsp.conf`` to enable the BSP to be booted by ``runqemu``::
QB_SYSTEM_NAME: QEMU name (e.g. "qemu-system-i386")
QB_OPT_APPEND: Options to append to QEMU (e.g. "-show-cursor")
QB_DEFAULT_KERNEL: Default kernel to boot (e.g. "bzImage")
QB_DEFAULT_FSTYPE: Default FSTYPE to boot (e.g. "ext4")
QB_MEM: Memory (e.g. "-m 512")
QB_MACHINE: QEMU machine (e.g. "-machine virt")
QB_CPU: QEMU cpu (e.g. "-cpu qemu32")
QB_CPU_KVM: Similar to QB_CPU except used for kvm support (e.g. "-cpu kvm64")
QB_KERNEL_CMDLINE_APPEND: Options to append to the kernel's -append
option (e.g. "console=ttyS0 console=tty")
QB_DTB: QEMU dtb name
QB_AUDIO_DRV: QEMU audio driver (e.g. "alsa", set it when support audio)
QB_AUDIO_OPT: QEMU audio option (e.g. "-soundhw ac97,es1370"), which is used
when QB_AUDIO_DRV is set.
QB_KERNEL_ROOT: Kernel's root (e.g. /dev/vda)
QB_TAP_OPT: Network option for 'tap' mode (e.g.
"-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no -device virtio-net-device,netdev=net0").
runqemu will replace "@TAP@" with the one that is used, such as tap0, tap1 ...
QB_SLIRP_OPT: Network option for SLIRP mode (e.g. "-netdev user,id=net0 -device virtio-net-device,netdev=net0")
QB_ROOTFS_OPT: Used as rootfs (e.g.
"-drive id=disk0,file=@ROOTFS@,if=none,format=raw -device virtio-blk-device,drive=disk0").
runqemu will replace "@ROOTFS@" with the one which is used, such as
core-image-minimal-qemuarm64.ext4.
QB_SERIAL_OPT: Serial port (e.g. "-serial mon:stdio")
QB_TCPSERIAL_OPT: tcp serial port option (e.g.
" -device virtio-serial-device -chardev socket,id=virtcon,port=@PORT@,host=127.0.0.1 -device virtconsole,chardev=virtcon"
runqemu will replace "@PORT@" with the port number which is used.
To use ``runqemu``, set :term:`IMAGE_CLASSES` as
follows and run ``runqemu``:
.. note::
"QB" means "QEMU Boot".
.. note::
For command-line syntax, use ``runqemu help``.
::
IMAGE_CLASSES += "qemuboot"
.. _migration-2.2-default-linker-hash-style-changed:
Default Linker Hash Style Changed
---------------------------------
The default linker hash style for ``gcc-cross`` is now "sysv" in order
to catch recipes that are building software without using the
OpenEmbedded :term:`LDFLAGS`. This change could result in
seeing some "No GNU_HASH in the elf binary" QA issues when building such
recipes. You need to fix these recipes so that they use the expected
:term:`LDFLAGS`. Depending on how the software is built, the build system
used by the software (e.g. a Makefile) might need to be patched.
However, sometimes making this fix is as simple as adding the following
to the recipe::
TARGET_CC_ARCH += "${LDFLAGS}"
.. _migration-2.2-kernel-image-base-name-no-longer-uses-kernel-imagetype:
``KERNEL_IMAGE_BASE_NAME`` no Longer Uses ``KERNEL_IMAGETYPE``
--------------------------------------------------------------
The ``KERNEL_IMAGE_BASE_NAME`` variable no longer uses the
:term:`KERNEL_IMAGETYPE` variable to create the
image's base name. Because the OpenEmbedded build system can now build
multiple kernel image types, this part of the kernel image base name as
been removed leaving only the following::
KERNEL_IMAGE_BASE_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}-${DATETIME}"
If you have recipes or
classes that use ``KERNEL_IMAGE_BASE_NAME`` directly, you might need to
update the references to ensure they continue to work.
.. _migration-2.2-imgdeploydir-replaces-deploy-dir-image-for-most-use-cases:
``IMGDEPLOYDIR`` Replaces ``DEPLOY_DIR_IMAGE`` for Most Use Cases
-----------------------------------------------------------------
The :term:`IMGDEPLOYDIR` variable was introduced to allow sstate caching of
image creation results. Image recipes defining custom :term:`IMAGE_CMD` or
doing postprocessing on the generated images need to be adapted to use
:term:`IMGDEPLOYDIR` instead of :term:`DEPLOY_DIR_IMAGE`. :term:`IMAGE_MANIFEST`
creation and symlinking of the most recent image file will fail otherwise.
.. _migration-2.2-bitbake-changes:
BitBake Changes
---------------
The following changes took place for BitBake:
- The "goggle" UI and standalone image-writer tool have been removed as
they both require GTK+ 2.0 and were not being maintained.
- The Perforce fetcher now supports :term:`SRCREV` for
specifying the source revision to use, be it
``${``\ :term:`AUTOREV`\ ``}``, changelist number,
p4date, or label, in preference to separate
:term:`SRC_URI` parameters to specify these. This
change is more in-line with how the other fetchers work for source
control systems. Recipes that fetch from Perforce will need to be
updated to use :term:`SRCREV` in place of specifying the source revision
within :term:`SRC_URI`.
- Some of BitBake's internal code structures for accessing the recipe
cache needed to be changed to support the new multi-configuration
functionality. These changes will affect external tools that use
BitBake's tinfoil module. For information on these changes, see the
changes made to the scripts supplied with OpenEmbedded-Core:
:yocto_git:`1 </poky/commit/?id=189371f8393971d00bca0fceffd67cc07784f6ee>`
and
:yocto_git:`2 </poky/commit/?id=4a5aa7ea4d07c2c90a1654b174873abb018acc67>`.
- The task management code has been rewritten to avoid using ID
indirection in order to improve performance. This change is unlikely
to cause any problems for most users. However, the setscene
verification function as pointed to by
``BB_SETSCENE_VERIFY_FUNCTION`` needed to change signature.
Consequently, a new variable named ``BB_SETSCENE_VERIFY_FUNCTION2``
has been added allowing multiple versions of BitBake to work with
suitably written metadata, which includes OpenEmbedded-Core and Poky.
Anyone with custom BitBake task scheduler code might also need to
update the code to handle the new structure.
.. _migration-2.2-swabber-has-been-removed:
Swabber has Been Removed
------------------------
Swabber, a tool that was intended to detect host contamination in the
build process, has been removed, as it has been unmaintained and unused
for some time and was never particularly effective. The OpenEmbedded
build system has since incorporated a number of mechanisms including
enhanced QA checks that mean that there is less of a need for such a
tool.
.. _migration-2.2-removed-recipes:
Removed Recipes
---------------
The following recipes have been removed:
- ``augeas``: No longer needed and has been moved to ``meta-oe``.
- ``directfb``: Unmaintained and has been moved to ``meta-oe``.
- ``gcc``: Removed 4.9 version. Versions 5.4 and 6.2 are still present.
- ``gnome-doc-utils``: No longer needed.
- ``gtk-doc-stub``: Replaced by ``gtk-doc``.
- ``gtk-engines``: No longer needed and has been moved to
``meta-gnome``.
- ``gtk-sato-engine``: Became obsolete.
- ``libglade``: No longer needed and has been moved to ``meta-oe``.
- ``libmad``: Unmaintained and functionally replaced by ``libmpg123``.
``libmad`` has been moved to ``meta-oe``.
- ``libowl``: Became obsolete.
- ``libxsettings-client``: No longer needed.
- ``oh-puzzles``: Functionally replaced by ``puzzles``.
- ``oprofileui``: Became obsolete. OProfile has been largely supplanted
by perf.
- ``packagegroup-core-directfb.bb``: Removed.
- ``core-image-directfb.bb``: Removed.
- ``pointercal``: No longer needed and has been moved to ``meta-oe``.
- ``python-imaging``: No longer needed and moved to ``meta-python``
- ``python-pyrex``: No longer needed and moved to ``meta-python``.
- ``sato-icon-theme``: Became obsolete.
- ``swabber-native``: Swabber has been removed. See the :ref:`entry on
Swabber <migration-guides/migration-2.2:swabber has been removed>`.
- ``tslib``: No longer needed and has been moved to ``meta-oe``.
- ``uclibc``: Removed in favor of musl.
- ``xtscal``: No longer needed and moved to ``meta-oe``
.. _migration-2.2-removed-classes:
Removed Classes
---------------
The following classes have been removed:
- ``distutils-native-base``: No longer needed.
- ``distutils3-native-base``: No longer needed.
- ``sdl``: Only set :term:`DEPENDS` and
:term:`SECTION`, which are better set within the
recipe instead.
- ``sip``: Mostly unused.
- ``swabber``: See the :ref:`entry on
Swabber <migration-guides/migration-2.2:swabber has been removed>`.
.. _migration-2.2-minor-packaging-changes:
Minor Packaging Changes
-----------------------
The following minor packaging changes have occurred:
- ``grub``: Split ``grub-editenv`` into its own package.
- ``systemd``: Split container and vm related units into a new package,
systemd-container.
- ``util-linux``: Moved ``prlimit`` to a separate
``util-linux-prlimit`` package.
.. _migration-2.2-miscellaneous-changes:
Miscellaneous Changes
---------------------
The following miscellaneous changes have occurred:
- ``package_regex.inc``: Removed because the definitions
``package_regex.inc`` previously contained have been moved to their
respective recipes.
- Both ``devtool add`` and ``recipetool create`` now use a fixed
:term:`SRCREV` by default when fetching from a Git
repository. You can override this in either case to use
``${``\ :term:`AUTOREV`\ ``}`` instead by using the
``-a`` or ``--autorev`` command-line option
- ``distcc``: GTK+ UI is now disabled by default.
- ``packagegroup-core-tools-testapps``: Removed Piglit.
- :ref:`ref-classes-image`: Renamed COMPRESS(ION) to CONVERSION. This change
means that ``COMPRESSIONTYPES``, ``COMPRESS_DEPENDS`` and
``COMPRESS_CMD`` are deprecated in favor of ``CONVERSIONTYPES``,
``CONVERSION_DEPENDS`` and :term:`CONVERSION_CMD`. The ``COMPRESS*``
variable names will still work in the 2.2 release but metadata that
does not need to be backwards-compatible should be changed to use the
new names as the ``COMPRESS*`` ones will be removed in a future
release.
- ``gtk-doc``: A full version of ``gtk-doc`` is now made available.
However, some old software might not be capable of using the current
version of ``gtk-doc`` to build documentation. You need to change
recipes that build such software so that they explicitly disable
building documentation with ``gtk-doc``.
poky/documentation/migration-guides/migration-2.3.rst
+515
View File
@@ -0,0 +1,515 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 2.3 (pyro)
==================
This section provides migration information for moving to the Yocto
Project 2.3 Release (codename "pyro") from the prior release.
.. _migration-2.3-recipe-specific-sysroots:
Recipe-specific Sysroots
------------------------
The OpenEmbedded build system now uses one sysroot per recipe to resolve
long-standing issues with configuration script auto-detection of
undeclared dependencies. Consequently, you might find that some of your
previously written custom recipes are missing declared dependencies,
particularly those dependencies that are incidentally built earlier in a
typical build process and thus are already likely to be present in the
shared sysroot in previous releases.
Consider the following:
- *Declare Build-Time Dependencies:* Because of this new feature, you
must explicitly declare all build-time dependencies for your recipe.
If you do not declare these dependencies, they are not populated into
the sysroot for the recipe.
- *Specify Pre-Installation and Post-Installation Native Tool
Dependencies:* You must specifically specify any special native tool
dependencies of ``pkg_preinst`` and ``pkg_postinst`` scripts by using
the :term:`PACKAGE_WRITE_DEPS` variable.
Specifying these dependencies ensures that these tools are available
if these scripts need to be run on the build host during the
:ref:`ref-tasks-rootfs` task.
As an example, see the ``dbus`` recipe. You will see that this recipe
has a ``pkg_postinst`` that calls ``systemctl`` if "systemd" is in
:term:`DISTRO_FEATURES`. In the example,
``systemd-systemctl-native`` is added to :term:`PACKAGE_WRITE_DEPS`,
which is also conditional on "systemd" being in :term:`DISTRO_FEATURES`.
- Examine Recipes that Use ``SSTATEPOSTINSTFUNCS``: You need to
examine any recipe that uses ``SSTATEPOSTINSTFUNCS`` and determine
steps to take.
Functions added to ``SSTATEPOSTINSTFUNCS`` are still called as they
were in previous Yocto Project releases. However, since a separate
sysroot is now being populated for every recipe and if existing
functions being called through ``SSTATEPOSTINSTFUNCS`` are doing
relocation, then you will need to change these to use a
post-installation script that is installed by a function added to
:term:`SYSROOT_PREPROCESS_FUNCS`.
For an example, see the :ref:`ref-classes-pixbufcache` class in ``meta/classes/`` in
the :ref:`overview-manual/development-environment:yocto project source repositories`.
.. note::
The
SSTATEPOSTINSTFUNCS
variable itself is now deprecated in favor of the
do_populate_sysroot[postfuncs]
task. Consequently, if you do still have any function or functions
that need to be called after the sysroot component is created for
a recipe, then you would be well advised to take steps to use a
post installation script as described previously. Taking these
steps prepares your code for when
SSTATEPOSTINSTFUNCS
is removed in a future Yocto Project release.
- *Specify the Sysroot when Using Certain External Scripts:* Because
the shared sysroot is now gone, the scripts
``oe-find-native-sysroot`` and ``oe-run-native`` have been changed
such that you need to specify which recipe's
:term:`STAGING_DIR_NATIVE` is used.
.. note::
You can find more information on how recipe-specific sysroots work in
the ":ref:`ref-classes-staging`" section.
.. _migration-2.3-path-variable:
``PATH`` Variable
-----------------
Within the environment used to run build tasks, the environment variable
``PATH`` is now sanitized such that the normal native binary paths
(``/bin``, ``/sbin``, ``/usr/bin`` and so forth) are removed and a
directory containing symbolic links linking only to the binaries from
the host mentioned in the :term:`HOSTTOOLS` and
:term:`HOSTTOOLS_NONFATAL` variables is added
to ``PATH``.
Consequently, any native binaries provided by the host that you need to
call needs to be in one of these two variables at the configuration
level.
Alternatively, you can add a native recipe (i.e. ``-native``) that
provides the binary to the recipe's :term:`DEPENDS`
value.
.. note::
PATH
is not sanitized in the same way within ``devshell``.
If it were, you would have difficulty running host tools for
development and debugging within the shell.
.. _migration-2.3-scripts:
Changes to Scripts
------------------
The following changes to scripts took place:
- ``oe-find-native-sysroot``: The usage for the
``oe-find-native-sysroot`` script has changed to the following::
$ . oe-find-native-sysroot recipe
You must now supply a recipe for recipe
as part of the command. Prior to the Yocto Project 2.3 release, it
was not necessary to provide the script with the command.
- ``oe-run-native``: The usage for the ``oe-run-native`` script has
changed to the following::
$ oe-run-native native_recipe tool
You must
supply the name of the native recipe and the tool you want to run as
part of the command. Prior to the Yocto Project 2.3 release, it
was not necessary to provide the native recipe with the command.
- ``cleanup-workdir``: The ``cleanup-workdir`` script has been
removed because the script was found to be deleting files it should
not have, which lead to broken build trees. Rather than trying to
delete portions of :term:`TMPDIR` and getting it wrong,
it is recommended that you delete :term:`TMPDIR` and have it restored
from shared state (sstate) on subsequent builds.
- ``wipe-sysroot``: The ``wipe-sysroot`` script has been removed as
it is no longer needed with recipe-specific sysroots.
.. _migration-2.3-functions:
Changes to Functions
--------------------
The previously deprecated ``bb.data.getVar()``, ``bb.data.setVar()``,
and related functions have been removed in favor of ``d.getVar()``,
``d.setVar()``, and so forth.
You need to fix any references to these old functions.
.. _migration-2.3-bitbake-changes:
BitBake Changes
---------------
The following changes took place for BitBake:
- *BitBake's Graphical Dependency Explorer UI Replaced:* BitBake's
graphical dependency explorer UI ``depexp`` was replaced by
``taskexp`` ("Task Explorer"), which provides a graphical way of
exploring the ``task-depends.dot`` file. The data presented by Task
Explorer is much more accurate than the data that was presented by
``depexp``. Being able to visualize the data is an often requested
feature as standard ``*.dot`` file viewers cannot usual cope with the
size of the ``task-depends.dot`` file.
- *BitBake "-g" Output Changes:* The ``package-depends.dot`` and
``pn-depends.dot`` files as previously generated using the
``bitbake -g`` command have been removed. A ``recipe-depends.dot``
file is now generated as a collapsed version of ``task-depends.dot``
instead.
The reason for this change is because ``package-depends.dot`` and
``pn-depends.dot`` largely date back to a time before task-based
execution and do not take into account task-level dependencies
between recipes, which could be misleading.
- *Mirror Variable Splitting Changes:* Mirror variables including
:term:`MIRRORS`, :term:`PREMIRRORS`,
and :term:`SSTATE_MIRRORS` can now separate
values entirely with spaces. Consequently, you no longer need "\\n".
BitBake looks for pairs of values, which simplifies usage. There
should be no change required to existing mirror variable values
themselves.
- *The Subversion (SVN) Fetcher Uses an "ssh" Parameter and Not an
"rsh" Parameter:* The SVN fetcher now takes an "ssh" parameter
instead of an "rsh" parameter. This new optional parameter is used
when the "protocol" parameter is set to "svn+ssh". You can only use
the new parameter to specify the ``ssh`` program used by SVN. The SVN
fetcher passes the new parameter through the ``SVN_SSH`` environment
variable during the :ref:`ref-tasks-fetch` task.
See the
":ref:`bitbake-user-manual/bitbake-user-manual-fetching:subversion (svn) fetcher (\`\`svn://\`\`)`"
section in the BitBake User Manual for additional information.
- ``BB_SETSCENE_VERIFY_FUNCTION`` and ``BB_SETSCENE_VERIFY_FUNCTION2``
Removed: Because the mechanism they were part of is no longer
necessary with recipe-specific sysroots, the
``BB_SETSCENE_VERIFY_FUNCTION`` and ``BB_SETSCENE_VERIFY_FUNCTION2``
variables have been removed.
.. _migration-2.3-absolute-symlinks:
Absolute Symbolic Links
-----------------------
Absolute symbolic links (symlinks) within staged files are no longer
permitted and now trigger an error. Any explicit creation of symlinks
can use the ``lnr`` script, which is a replacement for ``ln -r``.
If the build scripts in the software that the recipe is building are
creating a number of absolute symlinks that need to be corrected, you
can inherit ``relative_symlinks`` within the recipe to turn those
absolute symlinks into relative symlinks.
.. _migration-2.3-gplv2-and-gplv3-moves:
GPLv2 Versions of GPLv3 Recipes Moved
-------------------------------------
Older GPLv2 versions of GPLv3 recipes have moved to a separate
``meta-gplv2`` layer.
If you use :term:`INCOMPATIBLE_LICENSE` to
exclude GPLv3 or set :term:`PREFERRED_VERSION`
to substitute a GPLv2 version of a GPLv3 recipe, then you must add the
``meta-gplv2`` layer to your configuration.
.. note::
You can ``find meta-gplv2`` layer in the OpenEmbedded layer index at
:oe_layer:`/meta-gplv2`.
These relocated GPLv2 recipes do not receive the same level of
maintenance as other core recipes. The recipes do not get security fixes
and upstream no longer maintains them. In fact, the upstream community
is actively hostile towards people that use the old versions of the
recipes. Moving these recipes into a separate layer both makes the
different needs of the recipes clearer and clearly identifies the number
of these recipes.
.. note::
The long-term solution might be to move to BSD-licensed replacements
of the GPLv3 components for those that need to exclude GPLv3-licensed
components from the target system. This solution will be investigated
for future Yocto Project releases.
.. _migration-2.3-package-management-changes:
Package Management Changes
--------------------------
The following package management changes took place:
- Smart package manager is replaced by DNF package manager. Smart has
become unmaintained upstream, is not ported to Python 3.x.
Consequently, Smart needed to be replaced. DNF is the only feasible
candidate.
The change in functionality is that the on-target runtime package
management from remote package feeds is now done with a different
tool that has a different set of command-line options. If you have
scripts that call the tool directly, or use its API, they need to be
fixed.
For more information, see the `DNF
Documentation <https://dnf.readthedocs.io/en/latest/>`__.
- Rpm 5.x is replaced with Rpm 4.x. This is done for two major reasons:
- DNF is API-incompatible with Rpm 5.x and porting it and
maintaining the port is non-trivial.
- Rpm 5.x itself has limited maintenance upstream, and the Yocto
Project is one of the very few remaining users.
- Berkeley DB 6.x is removed and Berkeley DB 5.x becomes the default:
- Version 6.x of Berkeley DB has largely been rejected by the open
source community due to its AGPLv3 license. As a result, most
mainstream open source projects that require DB are still
developed and tested with DB 5.x.
- In OE-core, the only thing that was requiring DB 6.x was Rpm 5.x.
Thus, no reason exists to continue carrying DB 6.x in OE-core.
- ``createrepo`` is replaced with ``createrepo_c``.
``createrepo_c`` is the current incarnation of the tool that
generates remote repository metadata. It is written in C as compared
to ``createrepo``, which is written in Python. ``createrepo_c`` is
faster and is maintained.
- Architecture-independent RPM packages are "noarch" instead of "all".
This change was made because too many places in DNF/RPM4 stack
already make that assumption. Only the filenames and the architecture
tag has changed. Nothing else has changed in OE-core system,
particularly in the :ref:`ref-classes-allarch` class.
- Signing of remote package feeds using ``PACKAGE_FEED_SIGN`` is not
currently supported. This issue will be fully addressed in a future
Yocto Project release. See :yocto_bugs:`defect 11209 </show_bug.cgi?id=11209>`
for more information on a solution to package feed signing with RPM
in the Yocto Project 2.3 release.
- OPKG now uses the libsolv backend for resolving package dependencies
by default. This is vastly superior to OPKG's internal ad-hoc solver
that was previously used. This change does have a small impact on
disk (around 500 KB) and memory footprint.
.. note::
For further details on this change, see the
:yocto_git:`commit message </poky/commit/?id=f4d4f99cfbc2396e49c1613a7d237b9e57f06f81>`.
.. _migration-2.3-removed-recipes:
Removed Recipes
---------------
The following recipes have been removed:
- ``linux-yocto 4.8``: Version 4.8 has been removed. Versions 4.1
(LTSI), 4.4 (LTS), 4.9 (LTS/LTSI) and 4.10 are now present.
- ``python-smartpm``: Functionally replaced by ``dnf``.
- ``createrepo``: Replaced by the ``createrepo-c`` recipe.
- ``rpmresolve``: No longer needed with the move to RPM 4 as RPM
itself is used instead.
- ``gstreamer``: Removed the GStreamer Git version recipes as they
have been stale. ``1.10.``\ x recipes are still present.
- ``alsa-conf-base``: Merged into ``alsa-conf`` since ``libasound``
depended on both. Essentially, no way existed to install only one of
these.
- ``tremor``: Moved to ``meta-multimedia``. Fixed-integer Vorbis
decoding is not needed by current hardware. Thus, GStreamer's ivorbis
plugin has been disabled by default eliminating the need for the
``tremor`` recipe in :term:`OpenEmbedded-Core (OE-Core)`.
- ``gummiboot``: Replaced by ``systemd-boot``.
.. _migration-2.3-wic-changes:
Wic Changes
-----------
The following changes have been made to Wic:
.. note::
For more information on Wic, see the
":ref:`dev-manual/wic:creating partitioned images using wic`"
section in the Yocto Project Development Tasks Manual.
- *Default Output Directory Changed:* Wic's default output directory is
now the current directory by default instead of the unusual
``/var/tmp/wic``.
The ``-o`` and ``--outdir`` options remain unchanged and are used to
specify your preferred output directory if you do not want to use the
default directory.
- *fsimage Plug-in Removed:* The Wic fsimage plugin has been removed as
it duplicates functionality of the rawcopy plugin.
.. _migration-2.3-qa-changes:
QA Changes
----------
The following QA checks have changed:
- ``unsafe-references-in-binaries``: The
``unsafe-references-in-binaries`` QA check, which was disabled by
default, has now been removed. This check was intended to detect
binaries in ``/bin`` that link to libraries in ``/usr/lib`` and have
the case where the user has ``/usr`` on a separate filesystem to
``/``.
The removed QA check was buggy. Additionally, ``/usr`` residing on a
separate partition from ``/`` is now a rare configuration.
Consequently, ``unsafe-references-in-binaries`` was removed.
- ``file-rdeps``: The ``file-rdeps`` QA check is now an error by
default instead of a warning. Because it is an error instead of a
warning, you need to address missing runtime dependencies.
For additional information, see the
:ref:`ref-classes-insane` class and the
":ref:`ref-manual/qa-checks:errors and warnings`" section.
.. _migration-2.3-miscellaneous-changes:
Miscellaneous Changes
---------------------
The following miscellaneous changes have occurred:
- In this release, a number of recipes have been changed to ignore the
``largefile`` :term:`DISTRO_FEATURES` item,
enabling large file support unconditionally. This feature has always
been enabled by default. Disabling the feature has not been widely
tested.
.. note::
Future releases of the Yocto Project will remove entirely the
ability to disable the
largefile
feature, which would make it unconditionally enabled everywhere.
- If the :term:`DISTRO_VERSION` value contains
the value of the :term:`DATE` variable, which is the
default between Poky releases, the :term:`DATE` value is explicitly
excluded from ``/etc/issue`` and ``/etc/issue.net``, which is
displayed at the login prompt, in order to avoid conflicts with
Multilib enabled. Regardless, the :term:`DATE` value is inaccurate if the
``base-files`` recipe is restored from shared state (sstate) rather
than rebuilt.
If you need the build date recorded in ``/etc/issue*`` or anywhere
else in your image, a better method is to define a post-processing
function to do it and have the function called from
:term:`ROOTFS_POSTPROCESS_COMMAND`.
Doing so ensures the value is always up-to-date with the created
image.
- Dropbear's ``init`` script now disables DSA host keys by default.
This change is in line with the systemd service file, which supports
RSA keys only, and with recent versions of OpenSSH, which deprecates
DSA host keys.
- The :ref:`ref-classes-buildhistory` class now
correctly uses tabs as separators between all columns in
``installed-package-sizes.txt`` in order to aid import into other
tools.
- The ``USE_LDCONFIG`` variable has been replaced with the "ldconfig"
:term:`DISTRO_FEATURES` feature. Distributions that previously set::
USE_LDCONFIG = "0"
should now instead use the following::
DISTRO_FEATURES_BACKFILL_CONSIDERED_append = " ldconfig"
- The default value of
:term:`COPYLEFT_LICENSE_INCLUDE` now
includes all versions of AGPL licenses in addition to GPL and LGPL.
.. note::
The default list is not intended to be guaranteed as a complete
safe list. You should seek legal advice based on what you are
distributing if you are unsure.
- Kernel module packages are now suffixed with the kernel version in
order to allow module packages from multiple kernel versions to
co-exist on a target system. If you wish to return to the previous
naming scheme that does not include the version suffix, use the
following::
KERNEL_MODULE_PACKAGE_SUFFIX = ""
- Removal of ``libtool`` ``*.la`` files is now enabled by default. The
``*.la`` files are not actually needed on Linux and relocating them
is an unnecessary burden.
If you need to preserve these ``.la`` files (e.g. in a custom
distribution), you must change :term:`INHERIT_DISTRO` such that
":ref:`ref-classes-remove-libtool`" is not included
in the value.
- Extensible SDKs built for GCC 5+ now refuse to install on a
distribution where the host GCC version is 4.8 or 4.9. This change
resulted from the fact that the installation is known to fail due to
the way the ``uninative`` shared state (sstate) package is built. See
the :ref:`ref-classes-uninative` class for additional information.
- All :ref:`ref-classes-native` and :ref:`ref-classes-nativesdk` recipes now
use a separate :term:`DISTRO_FEATURES` value instead of sharing the value
used by recipes for the target, in order to avoid unnecessary rebuilds.
The :term:`DISTRO_FEATURES` for :ref:`ref-classes-native` recipes
is :term:`DISTRO_FEATURES_NATIVE` added to an intersection of
:term:`DISTRO_FEATURES` and :term:`DISTRO_FEATURES_FILTER_NATIVE`.
For :ref:`ref-classes-nativesdk` recipes, the corresponding
variables are :term:`DISTRO_FEATURES_NATIVESDK` and
:term:`DISTRO_FEATURES_FILTER_NATIVESDK`.
- The ``FILESDIR`` variable, which was previously deprecated and rarely
used, has now been removed. You should change any recipes that set
``FILESDIR`` to set :term:`FILESPATH` instead.
- The ``MULTIMACH_HOST_SYS`` variable has been removed as it is no
longer needed with recipe-specific sysroots.
poky/documentation/migration-guides/migration-2.4.rst
+327
View File
@@ -0,0 +1,327 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 2.4 (rocko)
===================
This section provides migration information for moving to the Yocto
Project 2.4 Release (codename "rocko") from the prior release.
.. _migration-2.4-memory-resident-mode:
Memory Resident Mode
--------------------
A persistent mode is now available in BitBake's default operation,
replacing its previous "memory resident mode" (i.e.
``oe-init-build-env-memres``). Now you only need to set
:term:`BB_SERVER_TIMEOUT` to a timeout (in
seconds) and BitBake's server stays resident for that amount of time
between invocations. The ``oe-init-build-env-memres`` script has been
removed since a separate environment setup script is no longer needed.
.. _migration-2.4-packaging-changes:
Packaging Changes
-----------------
This section provides information about packaging changes that have
occurred:
- ``python3`` Changes:
- The main "python3" package now brings in all of the standard
Python 3 distribution rather than a subset. This behavior matches
what is expected based on traditional Linux distributions. If you
wish to install a subset of Python 3, specify ``python-core`` plus
one or more of the individual packages that are still produced.
- ``python3``: The ``bz2.py``, ``lzma.py``, and
``_compression.py`` scripts have been moved from the
``python3-misc`` package to the ``python3-compression`` package.
- ``binutils``: The ``libbfd`` library is now packaged in a separate
"libbfd" package. This packaging saves space when certain tools (e.g.
``perf``) are installed. In such cases, the tools only need
``libbfd`` rather than all the packages in ``binutils``.
- ``util-linux`` Changes:
- The ``su`` program is now packaged in a separate "util-linux-su"
package, which is only built when "pam" is listed in the
:term:`DISTRO_FEATURES` variable.
``util-linux`` should not be installed unless it is needed because
``su`` is normally provided through the shadow file format. The
main ``util-linux`` package has runtime dependencies (i.e.
:term:`RDEPENDS`) on the ``util-linux-su`` package
when "pam" is in :term:`DISTRO_FEATURES`.
- The ``switch_root`` program is now packaged in a separate
"util-linux-switch-root" package for small :term:`Initramfs` images that
do not need the whole ``util-linux`` package or the busybox
binary, which are both much larger than ``switch_root``. The main
``util-linux`` package has a recommended runtime dependency (i.e.
:term:`RRECOMMENDS`) on the
``util-linux-switch-root`` package.
- The ``ionice`` program is now packaged in a separate
"util-linux-ionice" package. The main ``util-linux`` package has a
recommended runtime dependency (i.e. :term:`RRECOMMENDS`) on the
``util-linux-ionice`` package.
- ``initscripts``: The ``sushell`` program is now packaged in a
separate "initscripts-sushell" package. This packaging change allows
systems to pull ``sushell`` in when ``selinux`` is enabled. The
change also eliminates needing to pull in the entire ``initscripts``
package. The main ``initscripts`` package has a runtime dependency
(i.e. :term:`RDEPENDS`) on the ``sushell`` package when "selinux" is in
:term:`DISTRO_FEATURES`.
- ``glib-2.0``: The ``glib-2.0`` package now has a recommended
runtime dependency (i.e. :term:`RRECOMMENDS`) on the ``shared-mime-info``
package, since large portions of GIO are not useful without the MIME
database. You can remove the dependency by using the
:term:`BAD_RECOMMENDATIONS` variable if
``shared-mime-info`` is too large and is not required.
- *Go Standard Runtime:* The Go standard runtime has been split out
from the main ``go`` recipe into a separate ``go-runtime`` recipe.
.. _migration-2.4-removed-recipes:
Removed Recipes
---------------
The following recipes have been removed:
- ``acpitests``: This recipe is not maintained.
- ``autogen-native``: No longer required by Grub, oe-core, or
meta-oe.
- ``bdwgc``: Nothing in OpenEmbedded-Core requires this recipe. It
has moved to meta-oe.
- ``byacc``: This recipe was only needed by rpm 5.x and has moved to
meta-oe.
- ``gcc (5.4)``: The 5.4 series dropped the recipe in favor of 6.3 /
7.2.
- ``gnome-common``: Deprecated upstream and no longer needed.
- ``go-bootstrap-native``: Go 1.9 does its own bootstrapping so this
recipe has been removed.
- ``guile``: This recipe was only needed by ``autogen-native`` and
``remake``. The recipe is no longer needed by either of these
programs.
- ``libclass-isa-perl``: This recipe was previously needed for LSB 4,
no longer needed.
- ``libdumpvalue-perl``: This recipe was previously needed for LSB 4,
no longer needed.
- ``libenv-perl``: This recipe was previously needed for LSB 4, no
longer needed.
- ``libfile-checktree-perl``: This recipe was previously needed for
LSB 4, no longer needed.
- ``libi18n-collate-perl``: This recipe was previously needed for LSB
4, no longer needed.
- ``libiconv``: This recipe was only needed for ``uclibc``, which was
removed in the previous release. ``glibc`` and ``musl`` have their
own implementations. ``meta-mingw`` still needs ``libiconv``, so it
has been moved to ``meta-mingw``.
- ``libpng12``: This recipe was previously needed for LSB. The
current ``libpng`` is 1.6.x.
- ``libpod-plainer-perl``: This recipe was previously needed for LSB
4, no longer needed.
- ``linux-yocto (4.1)``: This recipe was removed in favor of 4.4,
4.9, 4.10 and 4.12.
- ``mailx``: This recipe was previously only needed for LSB
compatibility, and upstream is defunct.
- ``mesa (git version only)``: The git version recipe was stale with
respect to the release version.
- ``ofono (git version only)``: The git version recipe was stale with
respect to the release version.
- ``portmap``: This recipe is obsolete and is superseded by
``rpcbind``.
- ``python3-pygpgme``: This recipe is old and unmaintained. It was
previously required by ``dnf``, which has switched to official
``gpgme`` Python bindings.
- ``python-async``: This recipe has been removed in favor of the
Python 3 version.
- ``python-gitdb``: This recipe has been removed in favor of the
Python 3 version.
- ``python-git``: This recipe was removed in favor of the Python 3
version.
- ``python-mako``: This recipe was removed in favor of the Python 3
version.
- ``python-pexpect``: This recipe was removed in favor of the Python
3 version.
- ``python-ptyprocess``: This recipe was removed in favor of Python
the 3 version.
- ``python-pycurl``: Nothing is using this recipe in
OpenEmbedded-Core (i.e. ``meta-oe``).
- ``python-six``: This recipe was removed in favor of the Python 3
version.
- ``python-smmap``: This recipe was removed in favor of the Python 3
version.
- ``remake``: Using ``remake`` as the provider of ``virtual/make`` is
broken. Consequently, this recipe is not needed in OpenEmbedded-Core.
.. _migration-2.4-kernel-device-tree-move:
Kernel Device Tree Move
-----------------------
Kernel Device Tree support is now easier to enable in a kernel recipe.
The Device Tree code has moved to a :ref:`ref-classes-kernel-devicetree` class.
Functionality is automatically enabled for any recipe that inherits the
:ref:`kernel <ref-classes-kernel>` class and sets the :term:`KERNEL_DEVICETREE`
variable. The previous mechanism for doing this,
``meta/recipes-kernel/linux/linux-dtb.inc``, is still available to avoid
breakage, but triggers a deprecation warning. Future releases of the
Yocto Project will remove ``meta/recipes-kernel/linux/linux-dtb.inc``.
It is advisable to remove any ``require`` statements that request
``meta/recipes-kernel/linux/linux-dtb.inc`` from any custom kernel
recipes you might have. This will avoid breakage in post 2.4 releases.
.. _migration-2.4-package-qa-changes:
Package QA Changes
------------------
The following package QA changes took place:
- The "unsafe-references-in-scripts" QA check has been removed.
- If you refer to ``${COREBASE}/LICENSE`` within
:term:`LIC_FILES_CHKSUM` you receive a
warning because this file is a description of the license for
OE-Core. Use ``${COMMON_LICENSE_DIR}/MIT`` if your recipe is
MIT-licensed and you cannot use the preferred method of referring to
a file within the source tree.
.. _migration-2.4-readme-changes:
``README`` File Changes
-----------------------
The following are changes to ``README`` files:
- The main Poky ``README`` file has been moved to the ``meta-poky``
layer and has been renamed ``README.poky``. A symlink has been
created so that references to the old location work.
- The ``README.hardware`` file has been moved to ``meta-yocto-bsp``. A
symlink has been created so that references to the old location work.
- A ``README.qemu`` file has been created with coverage of the
``qemu*`` machines.
.. _migration-2.4-miscellaneous-changes:
Miscellaneous Changes
---------------------
The following are additional changes:
- The ``ROOTFS_PKGMANAGE_BOOTSTRAP`` variable and any references to it
have been removed. You should remove this variable from any custom
recipes.
- The ``meta-yocto`` directory has been removed.
.. note::
In the Yocto Project 2.1 release
meta-yocto
was renamed to
meta-poky
and the
meta-yocto
subdirectory remained to avoid breaking existing configurations.
- The ``maintainers.inc`` file, which tracks maintainers by listing a
primary person responsible for each recipe in OE-Core, has been moved
from ``meta-poky`` to OE-Core (i.e. from
``meta-poky/conf/distro/include`` to ``meta/conf/distro/include``).
- The :ref:`ref-classes-buildhistory` class now makes
a single commit per build rather than one commit per subdirectory in
the repository. This behavior assumes the commits are enabled with
:term:`BUILDHISTORY_COMMIT` = "1", which
is typical. Previously, the :ref:`ref-classes-buildhistory` class made one commit
per subdirectory in the repository in order to make it easier to see
the changes for a particular subdirectory. To view a particular
change, specify that subdirectory as the last parameter on the
``git show`` or ``git diff`` commands.
- The ``x86-base.inc`` file, which is included by all x86-based machine
configurations, now sets :term:`IMAGE_FSTYPES`
using ``?=`` to "live" rather than appending with ``+=``. This change
makes the default easier to override.
- BitBake fires multiple "BuildStarted" events when multiconfig is
enabled (one per configuration). For more information, see the
":ref:`bitbake-user-manual/bitbake-user-manual-metadata:events`"
section in the BitBake User Manual.
- By default, the ``security_flags.inc`` file sets a
:term:`GCCPIE` variable with an option to enable
Position Independent Executables (PIE) within ``gcc``. Enabling PIE
in the GNU C Compiler (GCC), makes Return Oriented Programming (ROP)
attacks much more difficult to execute.
- OE-Core now provides a ``bitbake-layers`` plugin that implements a
"create-layer" subcommand. The implementation of this subcommand has
resulted in the ``yocto-layer`` script being deprecated and will
likely be removed in the next Yocto Project release.
- The ``vmdk``, ``vdi``, and ``qcow2`` image file types are now used in
conjunction with the "wic" image type through :term:`CONVERSION_CMD`.
Consequently, the equivalent image types are now ``wic.vmdk``,
``wic.vdi``, and ``wic.qcow2``, respectively.
- ``do_image_<type>[depends]`` has replaced ``IMAGE_DEPENDS_<type>``.
If you have your own classes that implement custom image types, then
you need to update them.
- OpenSSL 1.1 has been introduced. However, the default is still 1.0.x
through the :term:`PREFERRED_VERSION`
variable. This preference is set is due to the remaining
compatibility issues with other software. The
:term:`PROVIDES` variable in the openssl 1.0 recipe
now includes "openssl10" as a marker that can be used in
:term:`DEPENDS` within recipes that build software
that still depend on OpenSSL 1.0.
- To ensure consistent behavior, BitBake's "-r" and "-R" options (i.e.
prefile and postfile), which are used to read or post-read additional
configuration files from the command line, now only affect the
current BitBake command. Before these BitBake changes, these options
would "stick" for future executions.
poky/documentation/migration-guides/migration-2.5.rst
+310
View File
@@ -0,0 +1,310 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 2.5 (sumo)
==================
This section provides migration information for moving to the Yocto
Project 2.5 Release (codename "sumo") from the prior release.
.. _migration-2.5-packaging-changes:
Packaging Changes
-----------------
This section provides information about packaging changes that have
occurred:
- ``bind-libs``: The libraries packaged by the bind recipe are in a
separate ``bind-libs`` package.
- ``libfm-gtk``: The ``libfm`` GTK+ bindings are split into a
separate ``libfm-gtk`` package.
- ``flex-libfl``: The flex recipe splits out libfl into a separate
``flex-libfl`` package to avoid too many dependencies being pulled in
where only the library is needed.
- ``grub-efi``: The ``grub-efi`` configuration is split into a
separate ``grub-bootconf`` recipe. However, the dependency
relationship from ``grub-efi`` is through a virtual/grub-bootconf
provider making it possible to have your own recipe provide the
dependency. Alternatively, you can use a BitBake append file to bring
the configuration back into the ``grub-efi`` recipe.
- *armv7a Legacy Package Feed Support:* Legacy support is removed for
transitioning from ``armv7a`` to ``armv7a-vfp-neon`` in package
feeds, which was previously enabled by setting
``PKGARCHCOMPAT_ARMV7A``. This transition occurred in 2011 and active
package feeds should by now be updated to the new naming.
.. _migration-2.5-removed-recipes:
Removed Recipes
---------------
The following recipes have been removed:
- ``gcc``: The version 6.4 recipes are replaced by 7.x.
- ``gst-player``: Renamed to ``gst-examples`` as per upstream.
- ``hostap-utils``: This software package is obsolete.
- ``latencytop``: This recipe is no longer maintained upstream. The
last release was in 2009.
- ``libpfm4``: The only file that requires this recipe is
``oprofile``, which has been removed.
- ``linux-yocto``: The version 4.4, 4.9, and 4.10 recipes have been
removed. Versions 4.12, 4.14, and 4.15 remain.
- ``man``: This recipe has been replaced by modern ``man-db``
- ``mkelfimage``: This tool has been removed in the upstream coreboot
project, and is no longer needed with the removal of the ELF image
type.
- ``nativesdk-postinst-intercept``: This recipe is not maintained.
- ``neon``: This software package is no longer maintained upstream
and is no longer needed by anything in OpenEmbedded-Core.
- ``oprofile``: The functionality of this recipe is replaced by
``perf`` and keeping compatibility on an ongoing basis with ``musl``
is difficult.
- ``pax``: This software package is obsolete.
- ``stat``: This software package is not maintained upstream.
``coreutils`` provides a modern stat binary.
- ``zisofs-tools-native``: This recipe is no longer needed because
the compressed ISO image feature has been removed.
.. _migration-2.5-scripts-and-tools-changes:
Scripts and Tools Changes
-------------------------
The following are changes to scripts and tools:
- ``yocto-bsp``, ``yocto-kernel``, and ``yocto-layer``: The
``yocto-bsp``, ``yocto-kernel``, and ``yocto-layer`` scripts
previously shipped with poky but not in OpenEmbedded-Core have been
removed. These scripts are not maintained and are outdated. In many
cases, they are also limited in scope. The
``bitbake-layers create-layer`` command is a direct replacement for
``yocto-layer``. See the documentation to create a BSP or kernel
recipe in the ":ref:`bsp-guide/bsp:bsp kernel recipe example`" section.
- ``devtool finish``: ``devtool finish`` now exits with an error if
there are uncommitted changes or a rebase/am in progress in the
recipe's source repository. If this error occurs, there might be
uncommitted changes that will not be included in updates to the
patches applied by the recipe. A -f/--force option is provided for
situations that the uncommitted changes are inconsequential and you
want to proceed regardless.
- ``scripts/oe-setup-rpmrepo`` script: The functionality of
``scripts/oe-setup-rpmrepo`` is replaced by
``bitbake package-index``.
- ``scripts/test-dependencies.sh`` script: The script is largely made
obsolete by the recipe-specific sysroots functionality introduced in
the previous release.
.. _migration-2.5-bitbake-changes:
BitBake Changes
---------------
The following are BitBake changes:
- The ``--runall`` option has changed. There are two different
behaviors people might want:
- *Behavior A:* For a given target (or set of targets) look through
the task graph and run task X only if it is present and will be
built.
- *Behavior B:* For a given target (or set of targets) look through
the task graph and run task X if any recipe in the taskgraph has
such a target, even if it is not in the original task graph.
The ``--runall`` option now performs "Behavior B". Previously
``--runall`` behaved like "Behavior A". A ``--runonly`` option has
been added to retain the ability to perform "Behavior A".
- Several explicit "run this task for all recipes in the dependency
tree" tasks have been removed (e.g. ``fetchall``, ``checkuriall``,
and the ``*all`` tasks provided by the ``distrodata`` and
:ref:`ref-classes-archiver` classes). There is a BitBake option to complete this for
any arbitrary task. For example::
bitbake <target> -c fetchall
should now be replaced with::
bitbake <target> --runall=fetch
.. _migration-2.5-python-and-python3-changes:
Python and Python 3 Changes
---------------------------
The following are auto-packaging changes to Python and Python 3:
The script-managed ``python-*-manifest.inc`` files that were previously
used to generate Python and Python 3 packages have been replaced with a
JSON-based file that is easier to read and maintain. A new task is
available for maintainers of the Python recipes to update the JSON file
when upgrading to new Python versions. You can now edit the file
directly instead of having to edit a script and run it to update the
file.
One particular change to note is that the Python recipes no longer have
build-time provides for their packages. This assumes ``python-foo`` is
one of the packages provided by the Python recipe. You can no longer run
``bitbake python-foo`` or have a
:term:`DEPENDS` on ``python-foo``,
but doing either of the following causes the package to work as
expected::
IMAGE_INSTALL_append = " python-foo"
or ::
RDEPENDS_${PN} = "python-foo"
The earlier build-time provides behavior was a quirk of the
way the Python manifest file was created. For more information on this
change please see :yocto_git:`this commit
</poky/commit/?id=8d94b9db221d1def42f091b991903faa2d1651ce>`.
.. _migration-2.5-miscellaneous-changes:
Miscellaneous Changes
---------------------
The following are additional changes:
- The :ref:`ref-classes-kernel` class supports building packages for multiple kernels.
If your kernel recipe or ``.bbappend`` file mentions packaging at
all, you should replace references to the kernel in package names
with ``${KERNEL_PACKAGE_NAME}``. For example, if you disable
automatic installation of the kernel image using
``RDEPENDS_kernel-base = ""`` you can avoid warnings using
``RDEPENDS_${KERNEL_PACKAGE_NAME}-base = ""`` instead.
- The :ref:`ref-classes-buildhistory` class commits changes to the repository by
default so you no longer need to set ``BUILDHISTORY_COMMIT = "1"``.
If you want to disable commits you need to set
``BUILDHISTORY_COMMIT = "0"`` in your configuration.
- The ``beaglebone`` reference machine has been renamed to
``beaglebone-yocto``. The ``beaglebone-yocto`` BSP is a reference
implementation using only mainline components available in
OpenEmbedded-Core and ``meta-yocto-bsp``, whereas Texas Instruments
maintains a full-featured BSP in the ``meta-ti`` layer. This rename
avoids the previous name clash that existed between the two BSPs.
- The :ref:`ref-classes-update-alternatives` class no longer works with SysV ``init``
scripts because this usage has been problematic. Also, the
``sysklogd`` recipe no longer uses ``update-alternatives`` because it
is incompatible with other implementations.
- By default, the :ref:`ref-classes-cmake` class uses
``ninja`` instead of ``make`` for building. This improves build
performance. If a recipe is broken with ``ninja``, then the recipe
can set ``OECMAKE_GENERATOR = "Unix Makefiles"`` to change back to
``make``.
- The previously deprecated ``base_*`` functions have been removed in
favor of their replacements in ``meta/lib/oe`` and
``bitbake/lib/bb``. These are typically used from recipes and
classes. Any references to the old functions must be updated. The
following table shows the removed functions and their replacements:
+------------------------------+----------------------------------------------------------+
| *Removed* | *Replacement* |
+==============================+==========================================================+
| base_path_join() | oe.path.join() |
+------------------------------+----------------------------------------------------------+
| base_path_relative() | oe.path.relative() |
+------------------------------+----------------------------------------------------------+
| base_path_out() | oe.path.format_display() |
+------------------------------+----------------------------------------------------------+
| base_read_file() | oe.utils.read_file() |
+------------------------------+----------------------------------------------------------+
| base_ifelse() | oe.utils.ifelse() |
+------------------------------+----------------------------------------------------------+
| base_conditional() | oe.utils.conditional() |
+------------------------------+----------------------------------------------------------+
| base_less_or_equal() | oe.utils.less_or_equal() |
+------------------------------+----------------------------------------------------------+
| base_version_less_or_equal() | oe.utils.version_less_or_equal() |
+------------------------------+----------------------------------------------------------+
| base_contains() | bb.utils.contains() |
+------------------------------+----------------------------------------------------------+
| base_both_contain() | oe.utils.both_contain() |
+------------------------------+----------------------------------------------------------+
| base_prune_suffix() | oe.utils.prune_suffix() |
+------------------------------+----------------------------------------------------------+
| oe_filter() | oe.utils.str_filter() |
+------------------------------+----------------------------------------------------------+
| oe_filter_out() | oe.utils.str_filter_out() (or use the \_remove operator) |
+------------------------------+----------------------------------------------------------+
- Using ``exit 1`` to explicitly defer a postinstall script until first
boot is now deprecated since it is not an obvious mechanism and can
mask actual errors. If you want to explicitly defer a postinstall to
first boot on the target rather than at ``rootfs`` creation time, use
``pkg_postinst_ontarget()`` or call
``postinst_intercept delay_to_first_boot`` from ``pkg_postinst()``.
Any failure of a ``pkg_postinst()`` script (including ``exit 1``)
will trigger a warning during :ref:`ref-tasks-rootfs`.
For more information, see the
":ref:`dev-manual/new-recipe:post-installation scripts`"
section in the Yocto Project Development Tasks Manual.
- The ``elf`` image type has been removed. This image type was removed
because the ``mkelfimage`` tool that was required to create it is no
longer provided by coreboot upstream and required updating every time
``binutils`` updated.
- Support for .iso image compression (previously enabled through
``COMPRESSISO = "1"``) has been removed. The userspace tools
(``zisofs-tools``) are unmaintained and ``squashfs`` provides better
performance and compression. In order to build a live image with
squashfs+lz4 compression enabled you should now set
``LIVE_ROOTFS_TYPE = "squashfs-lz4"`` and ensure that ``live`` is in
:term:`IMAGE_FSTYPES`.
- Recipes with an unconditional dependency on ``libpam`` are only
buildable with ``pam`` in :term:`DISTRO_FEATURES`. If the dependency is
truly optional then it is recommended that the dependency be
conditional upon ``pam`` being in :term:`DISTRO_FEATURES`.
- For EFI-based machines, the bootloader (``grub-efi`` by default) is
installed into the image at /boot. Wic can be used to split the
bootloader into separate boot and root filesystem partitions if necessary.
- Patches whose context does not match exactly (i.e. where patch
reports "fuzz" when applying) will generate a warning. For an example
of this see :yocto_git:`this commit
</poky/commit/?id=cc97bc08125b63821ce3f616771830f77c456f57>`.
- Layers are expected to set ``LAYERSERIES_COMPAT_layername`` to match
the version(s) of OpenEmbedded-Core they are compatible with. This is
specified as codenames using spaces to separate multiple values (e.g.
"rocko sumo"). If a layer does not set
``LAYERSERIES_COMPAT_layername``, a warning will is shown. If a layer
sets a value that does not include the current version ("sumo" for
the 2.5 release), then an error will be produced.
- The ``TZ`` environment variable is set to "UTC" within the build
environment in order to fix reproducibility problems in some recipes.
poky/documentation/migration-guides/migration-2.6.rst
+447
View File
@@ -0,0 +1,447 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 2.6 (thud)
==================
This section provides migration information for moving to the Yocto
Project 2.6 Release (codename "thud") from the prior release.
.. _migration-2.6-gcc-changes:
GCC 8.2 is Now Used by Default
------------------------------
The GNU Compiler Collection version 8.2 is now used by default for
compilation. For more information on what has changed in the GCC 8.x
release, see https://gcc.gnu.org/gcc-8/changes.html.
If you still need to compile with version 7.x, GCC 7.3 is also provided.
You can select this version by setting the and can be selected by
setting the :term:`GCCVERSION` variable to "7.%" in
your configuration.
.. _migration-2.6-removed-recipes:
Removed Recipes
---------------
The following recipes have been removed:
- *beecrypt*: No longer needed since moving to RPM 4.
- *bigreqsproto*: Replaced by ``xorgproto``.
- *calibrateproto*: Removed in favor of ``xinput``.
- *compositeproto*: Replaced by ``xorgproto``.
- *damageproto*: Replaced by ``xorgproto``.
- *dmxproto*: Replaced by ``xorgproto``.
- *dri2proto*: Replaced by ``xorgproto``.
- *dri3proto*: Replaced by ``xorgproto``.
- *eee-acpi-scripts*: Became obsolete.
- *fixesproto*: Replaced by ``xorgproto``.
- *fontsproto*: Replaced by ``xorgproto``.
- *fstests*: Became obsolete.
- *gccmakedep*: No longer used.
- *glproto*: Replaced by ``xorgproto``.
- *gnome-desktop3*: No longer needed. This recipe has moved to ``meta-oe``.
- *icon-naming-utils*: No longer used since the Sato theme was removed in 2016.
- *inputproto*: Replaced by ``xorgproto``.
- *kbproto*: Replaced by ``xorgproto``.
- *libusb-compat*: Became obsolete.
- *libuser*: Became obsolete.
- *libnfsidmap*: No longer an external requirement since ``nfs-utils`` 2.2.1. ``libnfsidmap`` is now integrated.
- *libxcalibrate*: No longer needed with ``xinput``
- *mktemp*: Became obsolete. The ``mktemp`` command is provided by both ``busybox`` and ``coreutils``.
- *ossp-uuid*: Is not being maintained and has mostly been replaced by ``uuid.h`` in ``util-linux``.
- *pax-utils*: No longer needed. Previous QA tests that did use this recipe are now done at build time.
- *pcmciautils*: Became obsolete.
- *pixz*: No longer needed. ``xz`` now supports multi-threaded compression.
- *presentproto*: Replaced by ``xorgproto``.
- *randrproto*: Replaced by ``xorgproto``.
- *recordproto*: Replaced by ``xorgproto``.
- *renderproto*: Replaced by ``xorgproto``.
- *resourceproto*: Replaced by ``xorgproto``.
- *scrnsaverproto*: Replaced by ``xorgproto``.
- *trace-cmd*: Became obsolete. ``perf`` replaced this recipe's functionally.
- *videoproto*: Replaced by ``xorgproto``.
- *wireless-tools*: Became obsolete. Superseded by ``iw``.
- *xcmiscproto*: Replaced by ``xorgproto``.
- *xextproto*: Replaced by ``xorgproto``.
- *xf86dgaproto*: Replaced by ``xorgproto``.
- *xf86driproto*: Replaced by ``xorgproto``.
- *xf86miscproto*: Replaced by ``xorgproto``.
- *xf86-video-omapfb*: Became obsolete. Use kernel modesetting driver instead.
- *xf86-video-omap*: Became obsolete. Use kernel modesetting driver instead.
- *xf86vidmodeproto*: Replaced by ``xorgproto``.
- *xineramaproto*: Replaced by ``xorgproto``.
- *xproto*: Replaced by ``xorgproto``.
- *yasm*: No longer needed since previous usages are now satisfied by ``nasm``.
.. _migration-2.6-packaging-changes:
Packaging Changes
-----------------
The following packaging changes have been made:
- *cmake*: ``cmake.m4`` and ``toolchain`` files have been moved to
the main package.
- *iptables*: The ``iptables`` modules have been split into
separate packages.
- *alsa-lib*: ``libasound`` is now in the main ``alsa-lib`` package
instead of ``libasound``.
- *glibc*: ``libnss-db`` is now in its own package along with a
``/var/db/makedbs.sh`` script to update databases.
- *python and python3*: The main package has been removed from
the recipe. You must install specific packages or ``python-modules``
/ ``python3-modules`` for everything.
- *systemtap*: Moved ``systemtap-exporter`` into its own package.
.. _migration-2.6-xorg-protocol-dependencies:
XOrg Protocol dependencies
--------------------------
The ``*proto`` upstream repositories have been combined into one
"xorgproto" repository. Thus, the corresponding recipes have also been
combined into a single ``xorgproto`` recipe. Any recipes that depend
upon the older ``*proto`` recipes need to be changed to depend on the
newer ``xorgproto`` recipe instead.
For names of recipes removed because of this repository change, see the
:ref:`migration-guides/migration-2.6:removed recipes` section.
.. _migration-2.6-distutils-distutils3-fetching-dependencies:
``distutils`` and ``distutils3`` Now Prevent Fetching Dependencies During the ``do_configure`` Task
---------------------------------------------------------------------------------------------------
Previously, it was possible for Python recipes that inherited the
``distutils`` and ``distutils3`` classes to fetch code
during the :ref:`ref-tasks-configure` task to satisfy
dependencies mentioned in ``setup.py`` if those dependencies were not
provided in the sysroot (i.e. recipes providing the dependencies were
missing from :term:`DEPENDS`).
.. note::
This change affects classes beyond just the two mentioned (i.e. ``distutils``
and ``distutils3``). Any recipe that inherits ``distutils*`` classes are
affected. For example, the ``setuptools`` and :ref:`ref-classes-setuptools3`
recipes are affected since they inherit the ``distutils*`` classes.
Fetching these types of dependencies that are not provided in the
sysroot negatively affects the ability to reproduce builds. This type of
fetching is now explicitly disabled. Consequently, any missing
dependencies in Python recipes that use these classes now result in an
error during the :ref:`ref-tasks-configure` task.
.. _migration-2.6-linux-yocto-configuration-audit-issues-now-correctly-reported:
``linux-yocto`` Configuration Audit Issues Now Correctly Reported
-----------------------------------------------------------------
Due to a bug, the kernel configuration audit functionality was not
writing out any resulting warnings during the build. This issue is now
corrected. You might notice these warnings now if you have a custom
kernel configuration with a ``linux-yocto`` style kernel recipe.
.. _migration-2.6-image-kernel-artifact-naming-changes:
Image/Kernel Artifact Naming Changes
------------------------------------
The following changes have been made:
- Name variables (e.g. :term:`IMAGE_NAME`) use a new
:term:`IMAGE_VERSION_SUFFIX` variable instead of
:term:`DATETIME`. Using :term:`IMAGE_VERSION_SUFFIX`
allows easier and more direct changes.
The :term:`IMAGE_VERSION_SUFFIX` variable is set in the ``bitbake.conf``
configuration file as follows::
IMAGE_VERSION_SUFFIX = "-${DATETIME}"
- Several variables have changed names for consistency::
Old Variable Name New Variable Name
========================================================
KERNEL_IMAGE_BASE_NAME KERNEL_IMAGE_NAME
KERNEL_IMAGE_SYMLINK_NAME KERNEL_IMAGE_LINK_NAME
MODULE_TARBALL_BASE_NAME MODULE_TARBALL_NAME
MODULE_TARBALL_SYMLINK_NAME MODULE_TARBALL_LINK_NAME
INITRAMFS_BASE_NAME INITRAMFS_NAME
- The ``MODULE_IMAGE_BASE_NAME`` variable has been removed. The module
tarball name is now controlled directly with the
:term:`MODULE_TARBALL_NAME` variable.
- The :term:`KERNEL_DTB_NAME` and
:term:`KERNEL_DTB_LINK_NAME` variables
have been introduced to control kernel Device Tree Binary (DTB)
artifact names instead of mangling ``KERNEL_IMAGE_*`` variables.
- The :term:`KERNEL_FIT_NAME` and
:term:`KERNEL_FIT_LINK_NAME` variables
have been introduced to specify the name of flattened image tree
(FIT) kernel images similar to other deployed artifacts.
- The :term:`MODULE_TARBALL_NAME` and
:term:`MODULE_TARBALL_LINK_NAME`
variable values no longer include the "module-" prefix or ".tgz"
suffix. These parts are now hardcoded so that the values are
consistent with other artifact naming variables.
- Added the :term:`INITRAMFS_LINK_NAME`
variable so that the symlink can be controlled similarly to other
artifact types.
- :term:`INITRAMFS_NAME` now uses
"${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}" instead
of "${PV}-${PR}-${MACHINE}-${DATETIME}", which makes it consistent
with other variables.
.. _migration-2.6-serial-console-deprecated:
``SERIAL_CONSOLE`` Deprecated
-----------------------------
The ``SERIAL_CONSOLE`` variable has been functionally replaced by the
:term:`SERIAL_CONSOLES` variable for some time. With the Yocto Project 2.6
release, ``SERIAL_CONSOLE`` has been officially deprecated.
``SERIAL_CONSOLE`` will continue to work as before for the 2.6 release.
However, for the sake of future compatibility, it is recommended that
you replace all instances of ``SERIAL_CONSOLE`` with :term:`SERIAL_CONSOLES`.
.. note::
The only difference in usage is that :term:`SERIAL_CONSOLES`
expects entries to be separated using semicolons as compared to
``SERIAL_CONSOLE``, which expects spaces.
.. _migration-2.6-poky-sets-unknown-configure-option-to-qa-error:
Configure Script Reports Unknown Options as Errors
--------------------------------------------------
If the configure script reports an unknown option, this now triggers a
QA error instead of a warning. Any recipes that previously got away with
specifying such unknown options now need to be fixed.
.. _migration-2.6-override-changes:
Override Changes
----------------
The following changes have occurred:
- The ``virtclass-native`` and ``virtclass-nativesdk`` Overrides Have
Been Removed: The ``virtclass-native`` and ``virtclass-nativesdk``
overrides have been deprecated since 2012 in favor of
``class-native`` and ``class-nativesdk``, respectively. Both
``virtclass-native`` and ``virtclass-nativesdk`` are now dropped.
.. note::
The ``virtclass-multilib-`` overrides for multilib are still valid.
- The ``forcevariable`` Override Now Has a Higher Priority Than
``libc`` Overrides: The ``forcevariable`` override is documented to
be the highest priority override. However, due to a long-standing
quirk of how :term:`OVERRIDES` is set, the ``libc``
overrides (e.g. ``libc-glibc``, ``libc-musl``, and so forth)
erroneously had a higher priority. This issue is now corrected.
It is likely this change will not cause any problems. However, it is
possible with some unusual configurations that you might see a change
in behavior if you were relying on the previous behavior. Be sure to
check how you use ``forcevariable`` and ``libc-*`` overrides in your
custom layers and configuration files to ensure they make sense.
- The ``build-${BUILD_OS}`` Override Has Been Removed: The
``build-${BUILD_OS}``, which is typically ``build-linux``, override
has been removed because building on a host operating system other
than a recent version of Linux is neither supported nor recommended.
Dropping the override avoids giving the impression that other host
operating systems might be supported.
- The "_remove" operator now preserves whitespace. Consequently, when
specifying list items to remove, be aware that leading and trailing
whitespace resulting from the removal is retained.
See the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:removal (override style syntax)`"
section in the BitBake User Manual for a detailed example.
.. _migration-2.6-systemd-configuration-now-split-out-to-system-conf:
``systemd`` Configuration is Now Split Into ``systemd-conf``
------------------------------------------------------------
The configuration for the ``systemd`` recipe has been moved into a
``system-conf`` recipe. Moving this configuration to a separate recipe
avoids the ``systemd`` recipe from becoming machine-specific for cases
where machine-specific configurations need to be applied (e.g. for
``qemu*`` machines).
Currently, the new recipe packages the following files::
${sysconfdir}/machine-id
${sysconfdir}/systemd/coredump.conf
${sysconfdir}/systemd/journald.conf
${sysconfdir}/systemd/logind.conf
${sysconfdir}/systemd/system.conf
${sysconfdir}/systemd/user.conf
If you previously used bbappend files to append the ``systemd`` recipe to
change any of the listed files, you must do so for the ``systemd-conf``
recipe instead.
.. _migration-2.6-automatic-testing-changes:
Automatic Testing Changes
-------------------------
This section provides information about automatic testing changes:
- ``TEST_IMAGE`` Variable Removed: Prior to this release, you set the
``TEST_IMAGE`` variable to "1" to enable automatic testing for
successfully built images. The ``TEST_IMAGE`` variable no longer
exists and has been replaced by the
:term:`TESTIMAGE_AUTO` variable.
- Inheriting the :ref:`ref-classes-testimage` and :ref:`ref-classes-testsdk`
classes: best practices now dictate that you use the :term:`IMAGE_CLASSES`
variable rather than the :term:`INHERIT` variable when you inherit the
:ref:`ref-classes-testimage` and :ref:`ref-classes-testsdk` classes used
for automatic testing.
.. _migration-2.6-openssl-changes:
OpenSSL Changes
---------------
`OpenSSL <https://www.openssl.org/>`__ has been upgraded from 1.0 to
1.1. By default, this upgrade could cause problems for recipes that have
both versions in their dependency chains. The problem is that both
versions cannot be installed together at build time.
.. note::
It is possible to have both versions of the library at runtime.
.. _migration-2.6-bitbake-changes:
BitBake Changes
---------------
The server logfile ``bitbake-cookerdaemon.log`` is now always placed in
the :term:`Build Directory` instead of the current directory.
.. _migration-2.6-security-changes:
Security Changes
----------------
The Poky distribution now uses security compiler flags by default.
Inclusion of these flags could cause new failures due to stricter
checking for various potential security issues in code.
.. _migration-2.6-post-installation-changes:
Post Installation Changes
-------------------------
You must explicitly mark post installs to defer to the target. If you
want to explicitly defer a postinstall to first boot on the target
rather than at root filesystem creation time, use ``pkg_postinst_ontarget()`` or
call ``postinst_intercept delay_to_first_boot`` from ``pkg_postinst()``.
Any failure of a ``pkg_postinst()`` script (including exit 1) triggers
an error during the :ref:`ref-tasks-rootfs` task.
For more information on post-installation behavior, see the
":ref:`dev-manual/new-recipe:post-installation scripts`"
section in the Yocto Project Development Tasks Manual.
.. _migration-2.6-python-3-profile-guided-optimizations:
Python 3 Profile-Guided Optimization
------------------------------------
The ``python3`` recipe now enables profile-guided optimization. Using
this optimization requires a little extra build time in exchange for
improved performance on the target at runtime. Additionally, the
optimization is only enabled if the current
:term:`MACHINE` has support for user-mode emulation in
QEMU (i.e. "qemu-usermode" is in
:term:`MACHINE_FEATURES`, which it is by
default).
If you wish to disable Python profile-guided optimization regardless of
the value of :term:`MACHINE_FEATURES`, then ensure that
:term:`PACKAGECONFIG` for the ``python3`` recipe
does not contain "pgo". You could accomplish the latter using the
following at the configuration level::
PACKAGECONFIG_remove_pn-python3 = "pgo"
Alternatively, you can set :term:`PACKAGECONFIG` using an append file
for the ``python3`` recipe.
.. _migration-2.6-miscellaneous-changes:
Miscellaneous Changes
---------------------
The following miscellaneous changes occurred:
- Default to using the Thumb-2 instruction set for armv7a and above. If
you have any custom recipes that build software that needs to be
built with the ARM instruction set, change the recipe to set the
instruction set as follows::
ARM_INSTRUCTION_SET = "arm"
- ``run-postinsts`` no longer uses ``/etc/*-postinsts`` for
``dpkg/opkg`` in favor of built-in postinst support. RPM behavior
remains unchanged.
- The ``NOISO`` and ``NOHDD`` variables are no longer used. You now
control building ``*.iso`` and ``*.hddimg`` image types directly by
using the :term:`IMAGE_FSTYPES` variable.
- The ``scripts/contrib/mkefidisk.sh`` has been removed in favor of
Wic.
- ``kernel-modules`` has been removed from
:term:`RRECOMMENDS` for ``qemumips`` and
``qemumips64`` machines. Removal also impacts the ``x86-base.inc``
file.
.. note::
``genericx86`` and ``genericx86-64`` retain ``kernel-modules`` as part of
the :term:`RRECOMMENDS` variable setting.
- The ``LGPLv2_WHITELIST_GPL-3.0`` variable has been removed. If you
are setting this variable in your configuration, set or append it to
the ``WHITELIST_GPL-3.0`` variable instead.
- ``${ASNEEDED}`` is now included in the
:term:`TARGET_LDFLAGS` variable directly. The
remaining definitions from ``meta/conf/distro/include/as-needed.inc``
have been moved to corresponding recipes.
- Support for DSA host keys has been dropped from the OpenSSH recipes.
If you are still using DSA keys, you must switch over to a more
secure algorithm as recommended by OpenSSH upstream.
- The ``dhcp`` recipe now uses the ``dhcpd6.conf`` configuration file
in ``dhcpd6.service`` for IPv6 DHCP rather than re-using
``dhcpd.conf``, which is now reserved for IPv4.
poky/documentation/migration-guides/migration-2.7.rst
+181
View File
@@ -0,0 +1,181 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 2.7 (warrior)
=====================
This section provides migration information for moving to the Yocto
Project 2.7 Release (codename "warrior") from the prior release.
.. _migration-2.7-bitbake-changes:
BitBake Changes
---------------
The following changes have been made to BitBake:
- BitBake now checks anonymous Python functions and pure Python
functions (e.g. ``def funcname:``) in the metadata for tab
indentation. If found, BitBake produces a warning.
- BitBake now checks
:term:`BBFILE_COLLECTIONS` for duplicate
entries and triggers an error if any are found.
.. _migration-2.7-eclipse-support-dropped:
Eclipse Support Removed
-----------------------
Support for the Eclipse IDE has been removed. Support continues for
those releases prior to 2.7 that did include support. The 2.7 release
does not include the Eclipse Yocto plugin.
.. _migration-2.7-qemu-native-splits-system-and-user-mode-parts:
``qemu-native`` Splits the System and User-Mode Parts
-----------------------------------------------------
The system and user-mode parts of ``qemu-native`` are now split.
``qemu-native`` provides the user-mode components and
``qemu-system-native`` provides the system components. If you have
recipes that depend on QEMU's system emulation functionality at build
time, they should now depend upon ``qemu-system-native`` instead of
``qemu-native``.
.. _migration-2.7-upstream-tracking.inc-removed:
The ``upstream-tracking.inc`` File Has Been Removed
---------------------------------------------------
The previously deprecated ``upstream-tracking.inc`` file is now removed.
Any ``UPSTREAM_TRACKING*`` variables are now set in the corresponding
recipes instead.
Remove any references you have to the ``upstream-tracking.inc`` file in
your configuration.
.. _migration-2.7-distro-features-libc-removed:
The ``DISTRO_FEATURES_LIBC`` Variable Has Been Removed
------------------------------------------------------
The ``DISTRO_FEATURES_LIBC`` variable is no longer used. The ability to
configure glibc using kconfig has been removed for quite some time
making the ``libc-*`` features set no longer effective.
Remove any references you have to ``DISTRO_FEATURES_LIBC`` in your own
layers.
.. _migration-2.7-license-values:
License Value Corrections
-------------------------
The following corrections have been made to the
:term:`LICENSE` values set by recipes:
- *socat*: Corrected :term:`LICENSE` to be "GPLv2" rather than "GPLv2+".
- *libgfortran*: Set license to "GPL-3.0-with-GCC-exception".
- *elfutils*: Removed "Elfutils-Exception" and set to "GPLv2" for shared libraries
.. _migration-2.7-packaging-changes:
Packaging Changes
-----------------
This section provides information about packaging changes.
- ``bind``: The ``nsupdate`` binary has been moved to the
``bind-utils`` package.
- Debug split: The default debug split has been changed to create
separate source packages (i.e. ``package_name-dbg`` and
``package_name-src``). If you are currently using ``dbg-pkgs`` in
:term:`IMAGE_FEATURES` to bring in debug
symbols and you still need the sources, you must now also add
``src-pkgs`` to :term:`IMAGE_FEATURES`. Source packages remain in the
target portion of the SDK by default, unless you have set your own
value for :term:`SDKIMAGE_FEATURES` that
does not include ``src-pkgs``.
- Mount all using ``util-linux``: ``/etc/default/mountall`` has moved
into the -mount sub-package.
- Splitting binaries using ``util-linux``: ``util-linux`` now splits
each binary into its own package for fine-grained control. The main
``util-linux`` package pulls in the individual binary packages using
the :term:`RRECOMMENDS` and
:term:`RDEPENDS` variables. As a result, existing
images should not see any changes assuming
:term:`NO_RECOMMENDATIONS` is not set.
- ``netbase/base-files``: ``/etc/hosts`` has moved from ``netbase`` to
``base-files``.
- ``tzdata``: The main package has been converted to an empty meta
package that pulls in all ``tzdata`` packages by default.
- ``lrzsz``: This package has been removed from
``packagegroup-self-hosted`` and
``packagegroup-core-tools-testapps``. The X/Y/ZModem support is less
likely to be needed on modern systems. If you are relying on these
packagegroups to include the ``lrzsz`` package in your image, you now
need to explicitly add the package.
.. _migration-2.7-removed-recipes:
Removed Recipes
---------------
The following recipes have been removed:
- *gcc*: Drop version 7.3 recipes. Version 8.3 now remains.
- *linux-yocto*: Drop versions 4.14 and 4.18 recipes. Versions 4.19 and 5.0 remain.
- *go*: Drop version 1.9 recipes. Versions 1.11 and 1.12 remain.
- *xvideo-tests*: Became obsolete.
- *libart-lgpl*: Became obsolete.
- *gtk-icon-utils-native*: These tools are now provided by gtk+3-native
- *gcc-cross-initial*: No longer needed. gcc-cross/gcc-crosssdk is now used instead.
- *gcc-crosssdk-initial*: No longer needed. gcc-cross/gcc-crosssdk is now used instead.
- *glibc-initial*: Removed because the benefits of having it for site_config are currently outweighed by the cost of building the recipe.
.. _migration-2.7-removed-classes:
Removed Classes
---------------
The following classes have been removed:
- *distutils-tools*: This class was never used.
- *bugzilla.bbclass*: Became obsolete.
- *distrodata*: This functionally has been replaced by a more modern tinfoil-based implementation.
.. _migration-2.7-miscellaneous-changes:
Miscellaneous Changes
---------------------
The following miscellaneous changes occurred:
- The ``distro`` subdirectory of the Poky repository has been removed
from the top-level ``scripts`` directory.
- Perl now builds for the target using
`perl-cross <https://arsv.github.io/perl-cross/>`_ for better
maintainability and improved build performance. This change should
not present any problems unless you have heavily customized your Perl
recipe.
- ``arm-tunes``: Removed the "-march" option if mcpu is already added.
- ``update-alternatives``: Convert file renames to
:term:`PACKAGE_PREPROCESS_FUNCS`
- ``base/pixbufcache``: Obsolete ``sstatecompletions`` code has been
removed.
- :ref:`ref-classes-native` class: :term:`RDEPENDS` handling has been enabled.
- ``inetutils``: This recipe has rsh disabled.
poky/documentation/migration-guides/migration-3.0.rst
+323
View File
@@ -0,0 +1,323 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 3.0 (zeus)
==================
This section provides migration information for moving to the Yocto
Project 3.0 Release (codename "zeus") from the prior release.
.. _migration-3.0-init-system-selection:
Init System Selection
---------------------
Changing the init system manager previously required setting a number of
different variables. You can now change the manager by setting the
``INIT_MANAGER`` variable and the corresponding include files (i.e.
``conf/distro/include/init-manager-*.conf``). Include files are provided
for four values: "none", "sysvinit", "systemd", and "mdev-busybox". The
default value, "none", for ``INIT_MANAGER`` should allow your current
settings to continue working. However, it is advisable to explicitly set
``INIT_MANAGER``.
.. _migration-3.0-lsb-support-removed:
LSB Support Removed
-------------------
Linux Standard Base (LSB) as a standard is not current, and is not well
suited for embedded applications. Support can be continued in a separate
layer if needed. However, presently LSB support has been removed from
the core.
As a result of this change, the ``poky-lsb`` derivative distribution
configuration that was also used for testing alternative configurations
has been replaced with a ``poky-altcfg`` distribution that has LSB parts
removed.
.. _migration-3.0-removed-recipes:
Removed Recipes
---------------
The following recipes have been removed.
- ``core-image-lsb-dev``: Part of removed LSB support.
- ``core-image-lsb``: Part of removed LSB support.
- ``core-image-lsb-sdk``: Part of removed LSB support.
- ``cve-check-tool``: Functionally replaced by the ``cve-update-db``
recipe and :ref:`ref-classes-cve-check` class.
- ``eglinfo``: No longer maintained. ``eglinfo`` from ``mesa-demos`` is
an adequate and maintained alternative.
- ``gcc-8.3``: Version 8.3 removed. Replaced by 9.2.
- ``gnome-themes-standard``: Only needed by gtk+ 2.x, which has been
removed.
- ``gtk+``: GTK+ 2 is obsolete and has been replaced by gtk+3.
- ``irda-utils``: Has become obsolete. IrDA support has been removed
from the Linux kernel in version 4.17 and later.
- ``libnewt-python``: ``libnewt`` Python support merged into main
``libnewt`` recipe.
- ``libsdl``: Replaced by newer ``libsdl2``.
- ``libx11-diet``: Became obsolete.
- ``libxx86dga``: Removed obsolete client library.
- ``libxx86misc``: Removed. Library is redundant.
- ``linux-yocto``: Version 5.0 removed, which is now redundant (5.2 /
4.19 present).
- ``lsbinitscripts``: Part of removed LSB support.
- ``lsb``: Part of removed LSB support.
- ``lsbtest``: Part of removed LSB support.
- ``openssl10``: Replaced by newer ``openssl`` version 1.1.
- ``packagegroup-core-lsb``: Part of removed LSB support.
- ``python-nose``: Removed the Python 2.x version of the recipe.
- ``python-numpy``: Removed the Python 2.x version of the recipe.
- ``python-scons``: Removed the Python 2.x version of the recipe.
- ``source-highlight``: No longer needed.
- ``stress``: Replaced by ``stress-ng``.
- ``vulkan``: Split into ``vulkan-loader``, ``vulkan-headers``, and
``vulkan-tools``.
- ``weston-conf``: Functionality moved to ``weston-init``.
.. _migration-3.0-packaging-changes:
Packaging Changes
-----------------
The following packaging changes have occurred.
- The :wikipedia:`Epiphany <GNOME_Web>` browser
has been dropped from ``packagegroup-self-hosted`` as it has not been
needed inside ``build-appliance-image`` for quite some time and was
causing resource problems.
- ``libcap-ng`` Python support has been moved to a separate
``libcap-ng-python`` recipe to streamline the build process when the
Python bindings are not needed.
- ``libdrm`` now packages the file ``amdgpu.ids`` into a separate
``libdrm-amdgpu`` package.
- ``python3``: The ``runpy`` module is now in the ``python3-core``
package as it is required to support the common "python3 -m" command
usage.
- ``distcc`` now provides separate ``distcc-client`` and
``distcc-server`` packages as typically one or the other are needed,
rather than both.
- ``python*-setuptools`` recipes now separately package the
``pkg_resources`` module in a ``python-pkg-resources`` /
``python3-pkg-resources`` package as the module is useful independent
of the rest of the setuptools package. The main ``python-setuptools``
/ ``python3-setuptools`` package depends on this new package so you
should only need to update dependencies unless you want to take
advantage of the increased granularity.
.. _migration-3.0-cve-checking:
CVE Checking
------------
``cve-check-tool`` has been functionally replaced by a new
``cve-update-db`` recipe and functionality built into the :ref:`ref-classes-cve-check`
class. The result uses NVD JSON data feeds rather than the deprecated
XML feeds that ``cve-check-tool`` was using, supports CVSSv3 scoring,
and makes other improvements.
Additionally, the ``CVE_CHECK_CVE_WHITELIST`` variable has been replaced
by ``CVE_CHECK_WHITELIST`` (replaced by :term:`CVE_CHECK_IGNORE` in version 3.5).
.. _migration-3.0-bitbake-changes:
BitBake Changes
---------------
The following BitBake changes have occurred.
- ``addtask`` statements now properly validate dependent tasks.
Previously, an invalid task was silently ignored. With this change,
the invalid task generates a warning.
- Other invalid ``addtask`` and ``deltask`` usages now trigger these
warnings: "multiple target tasks arguments with addtask / deltask",
and "multiple before/after clauses".
- The "multiconfig" prefix is now shortened to "mc". "multiconfig" will
continue to work, however it may be removed in a future release.
- The ``bitbake -g`` command no longer generates a
``recipe-depends.dot`` file as the contents (i.e. a reprocessed
version of ``task-depends.dot``) were confusing.
- The ``bb.build.FuncFailed`` exception, previously raised by
``bb.build.exec_func()`` when certain other exceptions have occurred,
has been removed. The real underlying exceptions will be raised
instead. If you have calls to ``bb.build.exec_func()`` in custom
classes or ``tinfoil-using`` scripts, any references to
``bb.build.FuncFailed`` should be cleaned up.
- Additionally, the ``bb.build.exec_func()`` no longer accepts the
"pythonexception" parameter. The function now always raises
exceptions. Remove this argument in any calls to
``bb.build.exec_func()`` in custom classes or scripts.
- The ``BB_SETSCENE_VERIFY_FUNCTION2`` variable is no longer used. In
the unlikely event that you have any references to it, they should be
removed.
- The ``RunQueueExecuteScenequeue`` and ``RunQueueExecuteTasks`` events
have been removed since setscene tasks are now executed as part of
the normal runqueue. Any event handling code in custom classes or
scripts that handles these two events need to be updated.
- The arguments passed to functions used with
:term:`BB_HASHCHECK_FUNCTION`
have changed. If you are using your own custom hash check function,
see :yocto_git:`/poky/commit/?id=40a5e193c4ba45c928fccd899415ea56b5417725`
for details.
- Task specifications in ``BB_TASKDEPDATA`` and class implementations
used in signature generator classes now use "<fn>:<task>" everywhere
rather than the "." delimiter that was being used in some places.
This change makes it consistent with all areas in the code. Custom
signature generator classes and code that reads ``BB_TASKDEPDATA``
need to be updated to use ':' as a separator rather than '.'.
.. _migration-3.0-sanity-checks:
Sanity Checks
-------------
The following sanity check changes occurred.
- :term:`SRC_URI` is now checked for usage of two
problematic items:
- "${PN}" prefix/suffix use --- warnings always appear if ${PN} is
used. You must fix the issue regardless of whether multiconfig or
anything else that would cause prefixing/suffixing to happen.
- Github archive tarballs --- these are not guaranteed to be stable.
Consequently, it is likely that the tarballs will be refreshed and
thus the :term:`SRC_URI` checksums will fail to apply. It is recommended
that you fetch either an official release tarball or a specific
revision from the actual Git repository instead.
Either one of these items now trigger a warning by default. If you
wish to disable this check, remove ``src-uri-bad`` from
:term:`WARN_QA`.
- The ``file-rdeps`` runtime dependency check no longer expands
:term:`RDEPENDS` recursively as there is no mechanism
to ensure they can be fully computed, and thus races sometimes result
in errors either showing up or not. Thus, you might now see errors
for missing runtime dependencies that were previously satisfied
recursively. Here is an example: package A contains a shell script
starting with ``#!/bin/bash`` but has no dependency on bash. However,
package A depends on package B, which does depend on bash. You need
to add the missing dependency or dependencies to resolve the warning.
- Setting ``DEPENDS_${PN}`` anywhere (i.e. typically in a recipe) now
triggers an error. The error is triggered because
:term:`DEPENDS` is not a package-specific variable
unlike RDEPENDS. You should set :term:`DEPENDS` instead.
- systemd currently does not work well with the musl C library because
only upstream officially supports linking the library with glibc.
Thus, a warning is shown when building systemd in conjunction with
musl.
.. _migration-3.0-miscellaneous-changes:
Miscellaneous Changes
---------------------
The following miscellaneous changes have occurred.
- The ``gnome`` class has been removed because it now does very little.
You should update recipes that previously inherited this class to do
the following::
inherit gnomebase gtk-icon-cache gconf mime
- The ``meta/recipes-kernel/linux/linux-dtb.inc`` file has been
removed. This file was previously deprecated in favor of setting
:term:`KERNEL_DEVICETREE` in any kernel
recipe and only produced a warning. Remove any ``include`` or
``require`` statements pointing to this file.
- :term:`TARGET_CFLAGS`,
:term:`TARGET_CPPFLAGS`,
:term:`TARGET_CXXFLAGS`, and
:term:`TARGET_LDFLAGS` are no longer exported
to the external environment. This change did not require any changes
to core recipes, which is a good indicator that no changes will be
required. However, if for some reason the software being built by one
of your recipes is expecting these variables to be set, then building
the recipe will fail. In such cases, you must either export the
variable or variables in the recipe or change the scripts so that
exporting is not necessary.
- You must change the host distro identifier used in
:term:`NATIVELSBSTRING` to use all lowercase
characters even if it does not contain a version number. This change
is necessary only if you are not using
:ref:`ref-classes-uninative` and :term:`SANITY_TESTED_DISTROS`.
- In the ``base-files`` recipe, writing the hostname into
``/etc/hosts`` and ``/etc/hostname`` is now done within the main
:ref:`ref-tasks-install` function rather than in the
``do_install_basefilesissue`` function. The reason for the change is
because ``do_install_basefilesissue`` is more easily overridden
without having to duplicate the hostname functionality. If you have
done the latter (e.g. in a ``base-files`` bbappend), then you should
remove it from your customized ``do_install_basefilesissue``
function.
- The ``wic --expand`` command now uses commas to separate "key:value"
pairs rather than hyphens.
.. note::
The wic command-line help is not updated.
You must update any scripts or commands where you use
``wic --expand`` with multiple "key:value" pairs.
- UEFI image variable settings have been moved from various places to a
central ``conf/image-uefi.conf``. This change should not influence
any existing configuration as the ``meta/conf/image-uefi.conf`` in
the core metadata sets defaults that can be overridden in the same
manner as before.
- ``conf/distro/include/world-broken.inc`` has been removed. For cases
where certain recipes need to be disabled when using the musl C
library, these recipes now have ``COMPATIBLE_HOST_libc-musl`` set
with a comment that explains why.
poky/documentation/migration-guides/migration-3.1.rst
+277
View File
@@ -0,0 +1,277 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 3.1 (dunfell)
=====================
This section provides migration information for moving to the Yocto
Project 3.1 Release (codename "dunfell") from the prior release.
.. _migration-3.1-minimum-system-requirements:
Minimum system requirements
---------------------------
The following versions / requirements of build host components have been
updated:
- gcc 5.0
- python 3.5
- tar 1.28
- ``rpcgen`` is now required on the host (part of the ``libc-dev-bin``
package on Ubuntu, Debian and related distributions, and the
``glibc`` package on RPM-based distributions).
Additionally, the ``makeinfo`` and ``pod2man`` tools are *no longer*
required on the host.
.. _migration-3.1-mpc8315e-rdb-removed:
mpc8315e-rdb machine removed
----------------------------
The MPC8315E-RDB machine is old/obsolete and unobtainable, thus given
the maintenance burden the ``mpc8315e-rdb`` machine configuration that
supported it has been removed in this release. The removal does leave a
gap in official PowerPC reference hardware support; this may change in
future if a suitable machine with accompanying support resources is
found.
.. _migration-3.1-python-2-removed:
Python 2 removed
----------------
Due to the expiration of upstream support in January 2020, support for
Python 2 has now been removed; it is recommended that you use Python 3
instead. If absolutely needed there is a meta-python2 community layer
containing Python 2, related classes and various Python 2-based modules,
however it should not be considered as supported.
.. _migration-3.1-reproducible-builds:
Reproducible builds now enabled by default
------------------------------------------
In order to avoid unnecessary differences in output files (aiding binary
reproducibility), the Poky distribution configuration
(``DISTRO = "poky"``) now inherits the ``reproducible_build`` class by
default.
.. _migration-3.1-ptest-feature-impact:
Impact of ptest feature is now more significant
-----------------------------------------------
The Poky distribution configuration (``DISTRO = "poky"``) enables ptests
by default to enable runtime testing of various components. In this
release, a dependency needed to be added that has resulted in a
significant increase in the number of components that will be built just
when building a simple image such as core-image-minimal. If you do not
need runtime tests enabled for core components, then it is recommended
that you remove "ptest" from
:term:`DISTRO_FEATURES` to save a significant
amount of build time e.g. by adding the following in your configuration::
DISTRO_FEATURES_remove = "ptest"
.. _migration-3.1-removed-recipes:
Removed recipes
---------------
The following recipes have been removed:
- ``chkconfig``: obsolete
- ``console-tools``: obsolete
- ``enchant``: replaced by ``enchant2``
- ``foomatic-filters``: obsolete
- ``libidn``: no longer needed, moved to meta-oe
- ``libmodulemd``: replaced by ``libmodulemd-v1``
- ``linux-yocto``: drop 4.19, 5.2 version recipes (5.4 now provided)
- ``nspr``: no longer needed, moved to meta-oe
- ``nss``: no longer needed, moved to meta-oe
- ``python``: Python 2 removed (Python 3 preferred)
- ``python-setuptools``: Python 2 version removed (python3-setuptools
preferred)
- ``sysprof``: no longer needed, moved to meta-oe
- ``texi2html``: obsolete
- ``u-boot-fw-utils``: functionally replaced by ``libubootenv``
.. _migration-3.1-features-check:
features_check class replaces distro_features_check
---------------------------------------------------
The ``distro_features_check`` class has had its functionality expanded,
now supporting ``ANY_OF_MACHINE_FEATURES``,
``REQUIRED_MACHINE_FEATURES``, ``CONFLICT_MACHINE_FEATURES``,
``ANY_OF_COMBINED_FEATURES``, ``REQUIRED_COMBINED_FEATURES``,
``CONFLICT_COMBINED_FEATURES``. As a result the class has now been
renamed to ``features_check``; the ``distro_features_check`` class still
exists but generates a warning and redirects to the new class. In
preparation for a future removal of the old class it is recommended that
you update recipes currently inheriting ``distro_features_check`` to
inherit :ref:`ref-classes-features_check` instead.
.. _migration-3.1-removed-classes:
Removed classes
---------------
The following classes have been removed:
- ``distutils-base``: moved to meta-python2
- ``distutils``: moved to meta-python2
- ``libc-common``: merged into the glibc recipe as nothing else used
it.
- ``python-dir``: moved to meta-python2
- ``pythonnative``: moved to meta-python2
- ``setuptools``: moved to meta-python2
- ``tinderclient``: dropped as it was obsolete.
.. _migration-3.1-src-uri-checksums:
SRC_URI checksum behaviour
--------------------------
Previously, recipes by tradition included both SHA256 and MD5 checksums
for remotely fetched files in :term:`SRC_URI`, even
though only one is actually mandated. However, the MD5 checksum does not
add much given its inherent weakness; thus when a checksum fails only
the SHA256 sum will now be printed. The md5sum will still be verified if
it is specified.
.. _migration-3.1-npm:
npm fetcher changes
-------------------
The npm fetcher has been completely reworked in this release. The npm
fetcher now only fetches the package source itself and no longer the
dependencies; there is now also an npmsw fetcher which explicitly
fetches the shrinkwrap file and the dependencies. This removes the
slightly awkward ``NPM_LOCKDOWN`` and ``NPM_SHRINKWRAP`` variables which
pointed to local files; the lockdown file is no longer needed at all.
Additionally, the package name in ``npm://`` entries in
:term:`SRC_URI` is now specified using a ``package``
parameter instead of the earlier ``name`` which overlapped with the
generic ``name`` parameter. All recipes using the npm fetcher will need
to be changed as a result.
An example of the new scheme::
SRC_URI = "npm://registry.npmjs.org;package=array-flatten;version=1.1.1 \
npmsw://${THISDIR}/npm-shrinkwrap.json"
Another example where the sources are fetched from git rather than an npm repository::
SRC_URI = "git://github.com/foo/bar.git;protocol=https \
npmsw://${THISDIR}/npm-shrinkwrap.json"
devtool and recipetool have also been updated to match with the npm
fetcher changes. Other than producing working and more complete recipes
for npm sources, there is also a minor change to the command line for
devtool: the ``--fetch-dev`` option has been renamed to ``--npm-dev`` as
it is npm-specific.
.. _migration-3.1-packaging-changes:
Packaging changes
-----------------
- ``intltool`` has been removed from ``packagegroup-core-sdk`` as it is
rarely needed to build modern software --- gettext can do most of the
things it used to be needed for. ``intltool`` has also been removed
from ``packagegroup-core-self-hosted`` as it is not needed to for
standard builds.
- git: ``git-am``, ``git-difftool``, ``git-submodule``, and
``git-request-pull`` are no longer perl-based, so are now installed
with the main ``git`` package instead of within ``git-perltools``.
- The ``ldconfig`` binary built as part of glibc has now been moved to
its own ``ldconfig`` package (note no ``glibc-`` prefix). This
package is in the :term:`RRECOMMENDS` of the main
``glibc`` package if ``ldconfig`` is present in
:term:`DISTRO_FEATURES`.
- ``libevent`` now splits each shared library into its own package (as
Debian does). Since these are shared libraries and will be pulled in
through the normal shared library dependency handling, there should
be no impact to existing configurations other than less unnecessary
libraries being installed in some cases.
- linux-firmware now has a new package for ``bcm4366c`` and includes
available NVRAM config files into the ``bcm43340``, ``bcm43362``,
``bcm43430`` and ``bcm4356-pcie`` packages.
- ``harfbuzz`` now splits the new ``libharfbuzz-subset.so`` library
into its own package to reduce the main package size in cases where
``libharfbuzz-subset.so`` is not needed.
.. _migration-3.1-package-qa-warnings:
Additional warnings
-------------------
Warnings will now be shown at :ref:`ref-tasks-package_qa` time in the following
circumstances:
- A recipe installs ``.desktop`` files containing ``MimeType`` keys but
does not inherit the new :ref:`ref-classes-mime-xdg` class
- A recipe installs ``.xml`` files into ``${datadir}/mime/packages``
but does not inherit the :ref:`ref-classes-mime` class
.. _migration-3.1-x86-live-wic:
``wic`` image type now used instead of ``live`` by default for x86
------------------------------------------------------------------
``conf/machine/include/x86-base.inc`` (inherited by most x86 machine
configurations) now specifies ``wic`` instead of ``live`` by default in
:term:`IMAGE_FSTYPES`. The ``live`` image type will
likely be removed in a future release so it is recommended that you use
``wic`` instead.
.. _migration-3.1-misc:
Miscellaneous changes
---------------------
- The undocumented ``SRC_DISTRIBUTE_LICENSES`` variable has now been
removed in favour of a new ``AVAILABLE_LICENSES`` variable which is
dynamically set based upon license files found in
``${COMMON_LICENSE_DIR}`` and ``${LICENSE_PATH}``.
- The tune definition for big-endian microblaze machines is now
``microblaze`` instead of ``microblazeeb``.
- ``newlib`` no longer has built-in syscalls. ``libgloss`` should then
provide the syscalls, ``crt0.o`` and other functions that are no
longer part of ``newlib`` itself. If you are using
``TCLIBC = "newlib"`` this now means that you must link applications
with both ``newlib`` and ``libgloss``, whereas before ``newlib``
would run in many configurations by itself.
poky/documentation/migration-guides/migration-3.2.rst
+340
View File
@@ -0,0 +1,340 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 3.2 (gatesgarth)
========================
This section provides migration information for moving to the Yocto
Project 3.2 Release (codename "gatesgarth") from the prior release.
.. _migration-3.2-minimum-system-requirements:
Minimum system requirements
---------------------------
``gcc`` version 6.0 is now required at minimum on the build host. For older
host distributions where this is not available, you can use the
:term:`buildtools-extended` tarball (easily installable using
``scripts/install-buildtools``).
.. _migration-3.2-removed-recipes:
Removed recipes
---------------
The following recipes have been removed:
- ``bjam-native``: replaced by ``boost-build-native``
- ``avahi-ui``: folded into the main ``avahi`` recipe --- the GTK UI can be disabled using :term:`PACKAGECONFIG` for ``avahi``.
- ``build-compare``: no longer needed with the removal of the ``packagefeed-stability`` class
- ``dhcp``: obsolete, functionally replaced by ``dhcpcd`` and ``kea``
- ``libmodulemd-v1``: replaced by ``libmodulemd``
- ``packagegroup-core-device-devel``: obsolete
.. _migration-3.2-removed-classes:
Removed classes
---------------
The following classes (.bbclass files) have been removed:
- ``spdx``: obsolete --- the Yocto Project is a strong supporter of SPDX, but this class was old code using a dated approach and had the potential to be misleading. The ``meta-sdpxscanner`` layer is a much more modern and active approach to handling this and is recommended as a replacement.
- ``packagefeed-stability``: this class had become obsolete with the advent of hash equivalence and reproducible builds.
pseudo path filtering and mismatch behaviour
--------------------------------------------
pseudo now operates on a filtered subset of files. This is a significant change
to the way pseudo operates within OpenEmbedded --- by default, pseudo monitors and
logs (adds to its database) any file created or modified whilst in a ``fakeroot``
environment. However, there are large numbers of files that we simply don't care
about the permissions of whilst in that ``fakeroot`` context, for example ${:term:`S`}, ${:term:`B`}, ${:term:`T`},
${:term:`SSTATE_DIR`}, the central sstate control directories, and others.
As of this release, new functionality in pseudo is enabled to ignore these
directory trees (controlled using a new :term:`PSEUDO_IGNORE_PATHS` variable)
resulting in a cleaner database with less chance of "stray" mismatches if files
are modified outside pseudo context. It also should reduce some overhead from
pseudo as the interprocess round trip to the server is avoided.
There is a possible complication where some existing recipe may break, for
example, a recipe was found to be writing to ``${B}/install`` for
``make install`` in :ref:`ref-tasks-install` and since ``${B}`` is listed as not to be tracked,
there were errors trying to ``chown root`` for files in this location. Another
example was the ``tcl`` recipe where the source directory :term:`S` is set to a
subdirectory of the source tree but files were written out to the directory
structure above that subdirectory. For these types of cases in your own recipes,
extend :term:`PSEUDO_IGNORE_PATHS` to cover additional paths that pseudo should not
be monitoring.
In addition, pseudo's behaviour on mismatches has now been changed --- rather
than doing what turns out to be a rather dangerous "fixup" if it sees a file
with a different path but the same inode as another file it has previously seen,
pseudo will throw an ``abort()`` and direct you to a :yocto_wiki:`wiki page </Pseudo_Abort>`
that explains how to deal with this.
.. _migration-3.2-multilib-mlprefix:
``MLPREFIX`` now required for multilib when runtime dependencies conditionally added
------------------------------------------------------------------------------------
In order to solve some previously intractable problems with runtime
dependencies and multilib, a change was made that now requires the :term:`MLPREFIX`
value to be explicitly prepended to package names being added as
dependencies (e.g. in :term:`RDEPENDS` and :term:`RRECOMMENDS` values)
where the dependency is conditionally added.
If you have anonymous Python or in-line Python conditionally adding
dependencies in your custom recipes, and you intend for those recipes to
work with multilib, then you will need to ensure that ``${MLPREFIX}``
is prefixed on the package names in the dependencies, for example
(from the ``glibc`` recipe)::
RRECOMMENDS_${PN} = "${@bb.utils.contains('DISTRO_FEATURES', 'ldconfig', '${MLPREFIX}ldconfig', '', d)}"
This also applies when conditionally adding packages to :term:`PACKAGES` where
those packages have dependencies, for example (from the ``alsa-plugins`` recipe)::
PACKAGES += "${@bb.utils.contains('PACKAGECONFIG', 'pulseaudio', 'alsa-plugins-pulseaudio-conf', '', d)}"
...
RDEPENDS_${PN}-pulseaudio-conf += "\
${MLPREFIX}libasound-module-conf-pulse \
${MLPREFIX}libasound-module-ctl-pulse \
${MLPREFIX}libasound-module-pcm-pulse \
"
.. _migration-3.2-packagegroup-core-device-devel:
packagegroup-core-device-devel no longer included in images built for qemu* machines
------------------------------------------------------------------------------------
``packagegroup-core-device-devel`` was previously added automatically to
images built for ``qemu*`` machines, however the purpose of the group and what
it should contain is no longer clear, and in general, adding userspace
development items to images is best done at the image/class level; thus this
packagegroup was removed.
This packagegroup previously pulled in the following:
- ``distcc-config``
- ``nfs-export-root``
- ``bash``
- ``binutils-symlinks``
If you still need any of these in your image built for a ``qemu*`` machine
then you will add them explicitly to :term:`IMAGE_INSTALL` or another
appropriate place in the dependency chain for your image (if you have not
already done so).
.. _migration-3.2-dhcp:
DHCP server/client replaced
---------------------------
The ``dhcp`` software package has become unmaintained and thus has been
functionally replaced by ``dhcpcd`` (client) and ``kea`` (server). You will
need to replace references to the recipe/package names as appropriate --- most
commonly, at the package level ``dhcp-client`` should be replaced by
``dhcpcd`` and ``dhcp-server`` should be replaced by ``kea``. If you have any
custom configuration files for these they will need to be adapted --- refer to
the upstream documentation for ``dhcpcd`` and ``kea`` for further details.
.. _migration-3.2-packaging-changes:
Packaging changes
-----------------
- ``python3``: the ``urllib`` Python package has now moved into the core package, as it is used more commonly than just netclient (e.g. email, xml, mimetypes, pydoc). In addition, the ``pathlib`` module is now also part of the core package.
- ``iptables``: ``iptables-apply`` and ``ip6tables-apply`` have been split out to their own package to avoid a bash dependency in the main ``iptables`` package
.. _migration-3.2-package-qa-checks:
Package QA check changes
------------------------
Previously, the following package QA checks triggered warnings, however they can
be indicators of genuine underlying problems and are therefore now treated as
errors:
- :ref:`already-stripped <qa-check-already-stripped>`
- :ref:`compile-host-path <qa-check-compile-host-path>`
- :ref:`installed-vs-shipped <qa-check-installed-vs-shipped>`
- :ref:`ldflags <qa-check-ldflags>`
- :ref:`pn-overrides <qa-check-pn-overrides>`
- :ref:`rpaths <qa-check-rpaths>`
- :ref:`staticdev <qa-check-staticdev>`
- :ref:`unknown-configure-option <qa-check-unknown-configure-option>`
- :ref:`useless-rpaths <qa-check-useless-rpaths>`
In addition, the following new checks were added and default to triggering an error:
- :ref:`shebang-size <qa-check-shebang-size>`: Check for shebang (#!) lines
longer than 128 characters, which can give an error at runtime depending on
the operating system.
- :ref:`unhandled-features-check <qa-check-unhandled-features-check>`: Check
if any of the variables supported by the :ref:`ref-classes-features_check`
class is set while not inheriting the class itself.
- :ref:`missing-update-alternatives <qa-check-missing-update-alternatives>`:
Check if the recipe sets the :term:`ALTERNATIVE` variable for any of its
packages, and does not inherit the :ref:`ref-classes-update-alternatives`
class.
- A trailing slash or duplicated slashes in the value of :term:`S` or :term:`B`
will now trigger a warning so that they can be removed and path comparisons
can be more reliable --- remove any instances of these in your recipes if the
warning is displayed.
.. _migration-3.2-src-uri-file-globbing:
Globbing no longer supported in ``file://`` entries in ``SRC_URI``
------------------------------------------------------------------
Globbing (``*`` and ``?`` wildcards) in ``file://`` URLs within :term:`SRC_URI`
did not properly support file checksums, thus changes to the source files
would not always change the :ref:`ref-tasks-fetch` task checksum, and consequently would
not ensure that the changed files would be incorporated in subsequent builds.
Unfortunately it is not practical to make globbing work generically here, so
the decision was taken to remove support for globs in ``file://`` URLs.
If you have any usage of these in your recipes, then you will now need to
either add each of the files that you expect to match explicitly, or
alternatively if you still need files to be pulled in dynamically, put the
files into a subdirectory and reference that instead.
.. _migration-3.2-deploydir-clean:
deploy class now cleans ``DEPLOYDIR`` before ``do_deploy``
----------------------------------------------------------
:ref:`ref-tasks-deploy` as implemented in the :ref:`ref-classes-deploy` class
now cleans up ${:term:`DEPLOYDIR`} before running, just as
:ref:`ref-tasks-install` cleans up ${:term:`D`} before running. This reduces
the risk of :term:`DEPLOYDIR` being accidentally contaminated by files from
previous runs, possibly even with different config, in case of incremental
builds.
Most recipes and classes that inherit the :ref:`ref-classes-deploy` class or
interact with :ref:`ref-tasks-deploy` are unlikely to be affected by this
unless they add ``prefuncs`` to :ref:`ref-tasks-deploy` *which also* put files
into ``${DEPLOYDIR}`` --- these should be refactored to use
``do_deploy_prepend`` instead.
.. _migration-3.2-nativesdk-sdk-provides-dummy:
Custom SDK / SDK-style recipes need to include ``nativesdk-sdk-provides-dummy``
-------------------------------------------------------------------------------
All :ref:`ref-classes-nativesdk` packages require ``/bin/sh`` due
to their postinstall scriptlets, thus this package has to be dummy-provided
within the SDK and ``nativesdk-sdk-provides-dummy`` now does this. If you have
a custom SDK recipe (or your own SDK-style recipe similar to e.g.
``buildtools-tarball``), you will need to ensure
``nativesdk-sdk-provides-dummy`` or an equivalent is included in
:term:`TOOLCHAIN_HOST_TASK`.
``ld.so.conf`` now moved back to main ``glibc`` package
-------------------------------------------------------
There are cases where one doesn't want ``ldconfig`` on target (e.g. for
read-only root filesystems, it's rather pointless), yet one still
needs ``/etc/ld.so.conf`` to be present at image build time:
When some recipe installs libraries to a non-standard location, and
therefore installs in a file in ``/etc/ld.so.conf.d/foo.conf``, we
need ``/etc/ld.so.conf`` containing::
include /etc/ld.so.conf.d/*.conf
in order to get those other locations picked up.
Thus ``/etc/ld.so.conf`` is now in the main ``glibc`` package so that
there's always an ``ld.so.conf`` present when the build-time ``ldconfig``
runs towards the end of image construction.
The ``ld.so.conf`` and ``ld.so.conf.d/*.conf`` files do not take up
significant space (at least not compared to the ~700kB ``ldconfig`` binary), and they
might be needed in case ``ldconfig`` is installable, so they are left
in place after the image is built. Technically it would be possible to
remove them if desired, though it would not be trivial if you still
wanted the build-time ldconfig to function (:term:`ROOTFS_POSTPROCESS_COMMAND`
will not work as ``ldconfig`` is run after the functions referred to
by that variable).
.. _migration-3.2-virgl:
Host DRI drivers now used for GL support within ``runqemu``
-----------------------------------------------------------
``runqemu`` now uses the mesa-native libraries everywhere virgl is used
(i.e. when ``gl``, ``gl-es`` or ``egl-headless`` options are specified),
but instructs them to load DRI drivers from the host. Unfortunately this
may not work well with proprietary graphics drivers such as those from
Nvidia; if you are using such drivers then you may need to switch to an
alternative (such as Nouveau in the case of Nvidia hardware) or avoid
using the GL options.
.. _migration-3.2-initramfs-suffix:
Initramfs images now use a blank suffix
---------------------------------------
The reference :term:`Initramfs` images (``core-image-minimal-initramfs``,
``core-image-tiny-initramfs`` and ``core-image-testmaster-initramfs``) now
set an empty string for :term:`IMAGE_NAME_SUFFIX`, which otherwise defaults
to ``".rootfs"``. These images aren't root filesystems and thus the rootfs
label didn't make sense. If you are looking for the output files generated
by these image recipes directly then you will need to adapt to the new
naming without the ``.rootfs`` part.
.. _migration-3.2-image-artifact-names:
Image artifact name variables now centralised in image-artifact-names class
---------------------------------------------------------------------------
The defaults for the following image artifact name variables have been moved
from ``bitbake.conf`` to a new ``image-artifact-names`` class:
- :term:`IMAGE_BASENAME`
- :term:`IMAGE_LINK_NAME`
- :term:`IMAGE_NAME`
- :term:`IMAGE_NAME_SUFFIX`
- :term:`IMAGE_VERSION_SUFFIX`
Image-related classes now inherit this class, and typically these variables
are only referenced within image recipes so those will be unaffected by this
change. However if you have references to these variables in either a recipe
that is not an image or a class that is enabled globally, then those will
now need to be changed to ``inherit image-artifact-names``.
.. _migration-3.2-misc:
Miscellaneous changes
---------------------
- Support for the long-deprecated ``PACKAGE_GROUP`` variable has now been removed --- replace any remaining instances with :term:`FEATURE_PACKAGES`.
- The ``FILESPATHPKG`` variable, having been previously deprecated, has now been removed. Replace any remaining references with appropriate use of :term:`FILESEXTRAPATHS`.
- Erroneous use of ``inherit +=`` (instead of ``INHERIT +=``) in a configuration file now triggers an error instead of silently being ignored.
- ptest support has been removed from the ``kbd`` recipe, as upstream has moved to autotest which is difficult to work with in a cross-compilation environment.
- ``oe.utils.is_machine_specific()`` and ``oe.utils.machine_paths()`` have been removed as their utility was questionable. In the unlikely event that you have references to these in your own code, then the code will need to be reworked.
- The ``i2ctransfer`` module is now disabled by default when building ``busybox`` in order to be consistent with disabling the other i2c tools there. If you do wish the i2ctransfer module to be built in BusyBox then add ``CONFIG_I2CTRANSFER=y`` to your custom BusyBox configuration.
- In the ``Upstream-Status`` header convention for patches, ``Accepted`` has been replaced with ``Backport`` as these almost always mean the same thing i.e. the patch is already upstream and may need to be removed in a future recipe upgrade. If you are adding these headers to your own patches then use ``Backport`` to indicate that the patch has been sent upstream.
- The ``tune-supersparc.inc`` tune file has been removed as it does not appear to be widely used and no longer works.
poky/documentation/migration-guides/migration-3.3.rst
+170
View File
@@ -0,0 +1,170 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 3.3 (hardknott)
=======================
This section provides migration information for moving to the Yocto
Project 3.3 Release (codename "hardknott") from the prior release.
.. _migration-3.3-minimum-system-requirements:
Minimum system requirements
---------------------------
You will now need at least Python 3.6 installed on your build host. Most recent
distributions provide this, but should you be building on a distribution that
does not have it, you can use the :term:`buildtools` tarball (easily installable
using ``scripts/install-buildtools``) --- see
:ref:`ref-manual/system-requirements:required git, tar, python, make and gcc versions`
for details.
.. _migration-3.3-removed-recipes:
Removed recipes
---------------
The following recipes have been removed:
- ``go-dep``: obsolete with the advent of go modules
- ``gst-validate``: replaced by ``gst-devtools``
- ``linux-yocto``: removed 5.8 version recipes (5.4 / 5.10 still provided)
- ``vulkan-demos``: replaced by ``vulkan-samples``
.. _migration-3.3-common-license-only-versions:
Single version common license file naming
-----------------------------------------
Some license files in ``meta/files/common-licenses`` have been renamed to match
current SPDX naming conventions:
- AGPL-3.0 -> AGPL-3.0-only
- GPL-1.0 -> GPL-1.0-only
- GPL-2.0 -> GPL-2.0-only
- GPL-3.0 -> GPL-3.0-only
- LGPL-2.0 -> LGPL-2.0-only
- LGPL-2.1 -> LGPL-2.1-only
- LGPL-3.0 -> LGPL-3.0-only
Additionally, corresponding "-or-later" suffixed files have been added e.g.
``GPL-2.0-or-later``.
It is not required that you change :term:`LICENSE` values as there are mappings
from the original names in place; however, in rare cases where you have a recipe
which sets :term:`LIC_FILES_CHKSUM` to point to file(s) in
``meta/files/common-licenses`` (which in any case is not recommended) you will
need to update those.
.. _migration-3.3-python3targetconfig:
New ``python3targetconfig`` class
---------------------------------
A new :ref:`ref-classes-python3targetconfig` class has
been created for situations where you would previously have inherited the
:ref:`ref-classes-python3native` class but need access to
target configuration data (such as correct installation directories). Recipes
where this situation applies should be changed to inherit
:ref:`ref-classes-python3targetconfig` instead of
:ref:`ref-classes-python3native`. This also adds a dependency
on target ``python3``, so it should only be used where appropriate in order to
avoid unnecessarily lengthening builds.
Some example recipes where this change has been made: ``gpgme``, ``libcap-ng``,
``python3-pycairo``.
.. _migration-3.3-distutils-path:
``setup.py`` path for Python modules
------------------------------------
In a Python module, sometimes ``setup.py`` can be buried deep in the
source tree. Previously this was handled in recipes by setting :term:`S` to
point to the subdirectory within the source where ``setup.py`` is located.
However with the recent :ref:`pseudo <overview-manual/concepts:fakeroot and pseudo>`
changes, some Python modules make changes to files beneath ``${S}``, for
example::
S = "${WORKDIR}/git/python/pythonmodule"
then in ``setup.py`` it works with source code in a relative fashion, such
as ``../../src``. This causes pseudo to fail as it isn't able to track
the paths properly. This release introduces a new ``DISTUTILS_SETUP_PATH``
variable so that recipes can specify it explicitly, for example::
S = "${WORKDIR}/git"
DISTUTILS_SETUP_PATH = "${S}/python/pythonmodule"
Recipes that inherit from ``distutils3`` (or :ref:`ref-classes-setuptools3`
which itself inherits ``distutils3``) that also set :term:`S` to point to a
Python module within a subdirectory in the aforementioned manner should be
changed to set ``DISTUTILS_SETUP_PATH`` instead.
.. _migration-3.3-bitbake:
BitBake changes
---------------
- BitBake is now configured to use a default ``umask`` of ``022`` for all tasks
(specified via a new :term:`BB_DEFAULT_UMASK` variable). If needed, ``umask`` can
still be set on a per-task basis via the ``umask`` varflag on the task
function, but that is unlikely to be necessary in most cases.
- If a version specified in :term:`PREFERRED_VERSION` is not available this
will now trigger a warning instead of just a note, making such issues more
visible.
.. _migration-3.3-packaging:
Packaging changes
-----------------
The following packaging changes have been made; in all cases the main package
still depends upon the split out packages so you should not need to do anything
unless you want to take advantage of the improved granularity:
- ``dbus``: ``-common`` and ``-tools`` split out
- ``iproute2``: split ``ip`` binary to its own package
- ``net-tools``: split ``mii-tool`` into its own package
- ``procps``: split ``ps`` and ``sysctl`` into their own packages
- ``rpm``: split build and extra functionality into separate packages
- ``sudo``: split ``sudo`` binary into ``sudo-sudo`` and libs into ``sudo-lib``
- ``systemtap``: examples, Python scripts and runtime material split out
- ``util-linux``: ``libuuid`` has been split out to its own
``util-linux-libuuid`` recipe (and corresponding packages) to avoid circular
dependencies if ``libgcrypt`` support is enabled in ``util-linux``.
(``util-linux`` depends upon ``util-linux-libuuid``.)
.. _migration-3.3-misc:
Miscellaneous changes
---------------------
- The default poky :term:`DISTRO_VERSION` value now uses the core metadata's
git hash (i.e. :term:`METADATA_REVISION`) rather than the date (i.e.
:term:`DATE`) to reduce one small source of non-reproducibility. You can
of course specify your own :term:`DISTRO_VERSION` value as desired
(particularly if you create your own custom distro configuration).
- ``adwaita-icon-theme`` version 3.34.3 has been added back, and is selected
as the default via :term:`PREFERRED_VERSION` in
``meta/conf/distro/include/default-versions.inc`` due to newer versions
not working well with ``librsvg`` 2.40. ``librsvg`` is not practically
upgradeable at the moment as it has been ported to Rust, and Rust is not
(yet) in OE-Core, but this will change in a future release.
- ``ffmpeg`` is now configured to disable GPL-licensed portions by default
to make it harder to accidentally violate the GPL. To explicitly enable GPL
licensed portions, add ``gpl`` to :term:`PACKAGECONFIG` for ``ffmpeg``
using a bbappend (or use ``PACKAGECONFIG_append_pn-ffmpeg = " gpl"`` in
your configuration.)
- ``connman`` is now set to conflict with ``systemd-networkd`` as they
overlap functionally and may interfere with each other at runtime.
- Canonical SPDX license names are now used in image license manifests in
order to avoid aliases of the same license from showing up together (e.g.
``GPLv2`` and ``GPL-2.0``)
poky/documentation/migration-guides/migration-3.4.rst
+275
View File
@@ -0,0 +1,275 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Migration notes for 3.4 (honister)
----------------------------------
This section provides migration information for moving to the Yocto
Project 3.4 Release (codename "honister") from the prior release.
Override syntax changes
~~~~~~~~~~~~~~~~~~~~~~~
In this release, the ``:`` character replaces the use of ``_`` to
refer to an override, most commonly when making a conditional assignment
of a variable. This means that an entry like::
SRC_URI_qemux86 = "file://somefile"
now becomes::
SRC_URI:qemux86 = "file://somefile"
since ``qemux86`` is an override. This applies to any use of override
syntax, so the following::
SRC_URI_append = " file://somefile"
SRC_URI_append_qemux86 = " file://somefile2"
SRC_URI_remove_qemux86-64 = "file://somefile3"
SRC_URI_prepend_qemuarm = "file://somefile4 "
FILES_${PN}-ptest = "${bindir}/xyz"
IMAGE_CMD_tar = "tar"
BASE_LIB_tune-cortexa76 = "lib"
SRCREV_pn-bash = "abc"
BB_TASK_NICE_LEVEL_task-testimage = '0'
would now become::
SRC_URI:append = " file://somefile"
SRC_URI:append:qemux86 = " file://somefile2"
SRC_URI:remove:qemux86-64 = "file://somefile3"
SRC_URI:prepend:qemuarm = "file://somefile4 "
FILES:${PN}-ptest = "${bindir}/xyz"
IMAGE_CMD:tar = "tar"
BASE_LIB:tune-cortexa76 = "lib"
SRCREV:pn-bash = "abc"
BB_TASK_NICE_LEVEL:task-testimage = '0'
This also applies to
:ref:`variable queries to the datastore <bitbake-user-manual/bitbake-user-manual-metadata:functions for accessing datastore variables>`,
for example using ``getVar`` and similar so ``d.getVar("RDEPENDS_${PN}")``
becomes ``d.getVar("RDEPENDS:${PN}")``.
Whilst some of these are fairly obvious such as :term:`MACHINE` and :term:`DISTRO`
overrides, some are less obvious, for example the packaging variables such as
:term:`RDEPENDS`, :term:`FILES` and so on taking package names (e.g. ``${PN}``,
``${PN}-ptest``) as overrides. These overrides are not always in
:term:`OVERRIDES` but applied conditionally in specific contexts
such as packaging. ``task-<taskname>`` is another context specific override, the
context being specific tasks in that case. Tune overrides are another special
case where some code does use them as overrides but some does not. We plan to try
and make the tune code use overrides more consistently in the future.
There are some variables which do not use override syntax which include the
suffix to variables in ``layer.conf`` files such as :term:`BBFILE_PATTERN`,
:term:`SRCREV`\ ``_xxx`` where ``xxx`` is a name from :term:`SRC_URI` and
:term:`PREFERRED_VERSION`\ ``_xxx``. In particular, ``layer.conf`` suffixes
may be the same as a :term:`DISTRO` override causing some confusion. We do
plan to try and improve consistency as these issues are identified.
To help with migration of layers, a script has been provided in OE-Core.
Once configured with the overrides used by a layer, this can be run as::
<oe-core>/scripts/contrib/convert-overrides.py <layerdir>
.. note::
Please read the notes in the script as it isn't entirely automatic and it isn't
expected to handle every case. In particular, it needs to be told which overrides
the layer uses (usually machine and distro names/overrides) and the result should
be carefully checked since it can be a little enthusiastic and will convert
references to ``_append``, ``_remove`` and ``_prepend`` in function and variable
names.
For reference, this conversion is important as it allows BitBake to more reliably
determine what is an override and what is not, as underscores are also used in
variable names without intending to be overrides. This should allow us to proceed
with other syntax improvements and simplifications for usability. It also means
BitBake no longer has to guess and maintain large lookup lists just in case
e.g. ``functionname`` in ``my_functionname`` is an override, and thus should improve
efficiency.
New host dependencies
~~~~~~~~~~~~~~~~~~~~~
The ``lz4c``, ``pzstd`` and ``zstd`` commands are now required to be
installed on the build host to support LZ4 and Zstandard compression
functionality. These are typically provided by ``lz4`` and ``zstd``
packages in most Linux distributions. Alternatively they are available
as part of :term:`buildtools` tarball if your distribution does not provide
them. For more information see
:ref:`ref-manual/system-requirements:required packages for the build host`.
Removed recipes
~~~~~~~~~~~~~~~
The following recipes have been removed in this release:
- ``assimp``: problematic from a licensing perspective and no longer
needed by anything else
- ``clutter-1.0``: legacy component moved to meta-gnome
- ``clutter-gst-3.0``: legacy component moved to meta-gnome
- ``clutter-gtk-1.0``: legacy component moved to meta-gnome
- ``cogl-1.0``: legacy component moved to meta-gnome
- ``core-image-clutter``: removed along with clutter
- ``linux-yocto``: removed version 5.4 recipes (5.14 and 5.10 still
provided)
- ``mklibs-native``: not actively tested and upstream mklibs still
requires Python 2
- ``mx-1.0``: obsolete (last release 2012) and isn't used by anything in
any known layer
- ``packagegroup-core-clutter``: removed along with clutter
Removed classes
~~~~~~~~~~~~~~~
- ``clutter``: moved to meta-gnome along with clutter itself
- ``image-mklibs``: not actively tested and upstream mklibs still
requires Python 2
- ``meta``: no longer useful. Recipes that need to skip installing
packages should inherit :ref:`ref-classes-nopackages` instead.
Prelinking disabled by default
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Recent tests have shown that prelinking works only when PIE is not
enabled (see `here <https://rlbl.me/prelink-1>`__ and `here <https://rlbl.me/prelink-2>`__),
and as PIE is both a desirable security feature, and the only
configuration provided and tested by the Yocto Project, there is
simply no sense in continuing to enable prelink.
There's also a concern that no one is maintaining the code, and there
are open bugs (including :yocto_bugs:`this serious one </show_bug.cgi?id=14429>`).
Given that prelink does intricate address arithmetic and rewriting
of binaries the best option is to disable the feature. It is recommended
that you consider disabling this feature in your own configuration if
it is currently enabled.
Virtual runtime provides
~~~~~~~~~~~~~~~~~~~~~~~~
Recipes shouldn't use the ``virtual/`` string in :term:`RPROVIDES` and
:term:`RDEPENDS` --- it is confusing because ``virtual/`` has no special
meaning in :term:`RPROVIDES` and :term:`RDEPENDS` (unlike in the
corresponding build-time :term:`PROVIDES` and :term:`DEPENDS`).
Tune files moved to architecture-specific directories
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The tune files found in ``conf/machine/include`` have now been moved
into their respective architecture name directories under that same
location; e.g. x86 tune files have moved into an ``x86`` subdirectory,
MIPS tune files have moved into a ``mips`` subdirectory, etc.
The ARM tunes have an extra level (``armv8a``, ``armv8m``, etc.) and
some have been renamed to make them uniform with the rest of the tunes.
See :yocto_git:`this commit </poky/commit/?id=1d381f21f5f13aa0c4e1a45683ed656ebeedd37d>`
for reference.
If you have any references to tune files (e.g. in custom machine
configuration files) they will need to be updated.
Extensible SDK host extension
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For a normal SDK, some layers append to :term:`TOOLCHAIN_HOST_TASK`
unconditionally which is fine, until the eSDK tries to override the
variable to its own values. Instead of installing packages specified
in this variable it uses native recipes instead --- a very different
approach. This has led to confusing errors when binaries are added
to the SDK but not relocated.
To avoid these issues, a new :term:`TOOLCHAIN_HOST_TASK_ESDK` variable has
been created. If you wish to extend what is installed in the host
portion of the eSDK then you will now need to set this variable.
Package/recipe splitting
~~~~~~~~~~~~~~~~~~~~~~~~
- ``perl-cross`` has been split out from the main ``perl`` recipe to
its own ``perlcross`` recipe for maintenance reasons. If you have
bbappends for the perl recipe then these may need extending.
- The ``wayland`` recipe now packages its binaries in a
``wayland-tools`` package rather than putting them into
``wayland-dev``.
- Xwayland has been split out of the xserver-xorg tree and thus is now
in its own ``xwayland`` recipe. If you need Xwayland in your image
then you may now need to add it explicitly.
- The ``rpm`` package no longer has ``rpm-build`` in its :term:`RRECOMMENDS`;
if by chance you still need rpm package building functionality in
your image and you have not already done so then you should add
``rpm-build`` to your image explicitly.
- The Python ``statistics`` standard module is now packaged in its own
``python3-statistics`` package instead of ``python3-misc`` as
previously.
Image / SDK generation changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Recursive dependencies on the :ref:`ref-tasks-build` task are now disabled when
building SDKs. These are generally not needed; in the unlikely event
that you do encounter problems then it will probably be as a result of
missing explicit dependencies that need to be added.
- Errors during "complementary" package installation (e.g. for ``*-dbg``
and ``*-dev`` packages) during image construction are no longer
ignored. Historically some of these packages had installation problems,
that is no longer the case. In the unlikely event that you see errors
as a result, you will need to fix the installation/packaging issues.
- When building an image, only packages that will be used in building
the image (i.e. the first entry in :term:`PACKAGE_CLASSES`) will be
produced if multiple package types are enabled (which is not a typical
configuration). If in your CI system you need to have the original
behaviour, use ``bitbake --runall build <target>``.
- The ``-lic`` package is no longer automatically added to
:term:`RRECOMMENDS` for every other package when
:term:`LICENSE_CREATE_PACKAGE` is set to "1". If you wish all license
packages to be installed corresponding to packages in your image, then
you should instead add the new ``lic-pkgs`` feature to
:term:`IMAGE_FEATURES`.
Miscellaneous
~~~~~~~~~~~~~
- Certificates are now properly checked when BitBake fetches sources
over HTTPS. If you receive errors as a result for your custom recipes,
you will need to use a mirror or address the issue with the operators
of the server in question.
- ``avahi`` has had its GTK+ support disabled by default. If you wish to
re-enable it, set ``AVAHI_GTK = "gtk3"`` in a bbappend for the
``avahi`` recipe or in your custom distro configuration file.
- Setting the ``BUILD_REPRODUCIBLE_BINARIES`` variable to "0" no longer
uses a strangely old fallback date of April 2011, it instead disables
building reproducible binaries as you would logically expect.
- Setting noexec/nostamp/fakeroot varflags to any value besides "1" will
now trigger a warning. These should be either set to "1" to enable, or
not set at all to disable.
- The previously deprecated ``COMPRESS_CMD`` and
``CVE_CHECK_CVE_WHITELIST`` variables have been removed. Use
:term:`CONVERSION_CMD` and ``CVE_CHECK_WHITELIST`` (replaced by
:term:`CVE_CHECK_IGNORE` in version 3.5) respectively
instead.
- The obsolete ``oe_machinstall`` function previously provided in the
:ref:`ref-classes-utils` class has been removed. For
machine-specific installation it is recommended that you use the
built-in override support in the fetcher or overrides in general
instead.
- The ``-P`` (``--clear-password``) option can no longer be used with
``useradd`` and ``usermod`` entries in :term:`EXTRA_USERS_PARAMS`.
It was being implemented using a custom patch to the ``shadow`` recipe
which clashed with a ``-P`` option that was added upstream in
``shadow`` version 4.9, and in any case is fundamentally insecure.
Hardcoded passwords are still supported but they need to be hashed, see
examples in :term:`EXTRA_USERS_PARAMS`.
poky/documentation/migration-guides/migration-4.0.rst
+271
View File
@@ -0,0 +1,271 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 4.0 (kirkstone)
=======================
Migration notes for 4.0 (kirkstone)
-----------------------------------
This section provides migration information for moving to the Yocto
Project 4.0 Release (codename "kirkstone") from the prior release.
.. _migration-4.0-inclusive-language:
Inclusive language improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To use more `inclusive language <https://inclusivenaming.org/>`__
in the code and documentation, some variables have been renamed, and
some have been deleted where they are no longer needed. In many cases the
new names are also easier to understand. BitBake will stop with an error when
renamed or removed variables still exist in your recipes or configuration.
Please note that the change applies also to environmental variables, so
make sure you use a fresh environment for your build.
The following variables have changed their names:
- ``BB_ENV_WHITELIST`` became :term:`BB_ENV_PASSTHROUGH`
- ``BB_ENV_EXTRAWHITE`` became :term:`BB_ENV_PASSTHROUGH_ADDITIONS`
- ``BB_HASHBASE_WHITELIST`` became :term:`BB_BASEHASH_IGNORE_VARS`
- ``BB_HASHCONFIG_WHITELIST`` became :term:`BB_HASHCONFIG_IGNORE_VARS`
- ``BB_HASHTASK_WHITELIST`` became ``BB_TASKHASH_IGNORE_TASKS``
- ``BB_SETSCENE_ENFORCE_WHITELIST`` became ``BB_SETSCENE_ENFORCE_IGNORE_TASKS``
- ``CVE_CHECK_PN_WHITELIST`` became :term:`CVE_CHECK_SKIP_RECIPE`
- ``CVE_CHECK_WHITELIST`` became :term:`CVE_CHECK_IGNORE`
- ``ICECC_USER_CLASS_BL`` became :term:`ICECC_CLASS_DISABLE`
- ``ICECC_SYSTEM_CLASS_BL`` became :term:`ICECC_CLASS_DISABLE`
- ``ICECC_USER_PACKAGE_WL`` became :term:`ICECC_RECIPE_ENABLE`
- ``ICECC_USER_PACKAGE_BL`` became :term:`ICECC_RECIPE_DISABLE`
- ``ICECC_SYSTEM_PACKAGE_BL`` became :term:`ICECC_RECIPE_DISABLE`
- ``LICENSE_FLAGS_WHITELIST`` became :term:`LICENSE_FLAGS_ACCEPTED`
- ``MULTI_PROVIDER_WHITELIST`` became :term:`BB_MULTI_PROVIDER_ALLOWED`
- ``PNBLACKLIST`` became :term:`SKIP_RECIPE`
- ``SDK_LOCAL_CONF_BLACKLIST`` became :term:`ESDK_LOCALCONF_REMOVE`
- ``SDK_LOCAL_CONF_WHITELIST`` became :term:`ESDK_LOCALCONF_ALLOW`
- ``SDK_INHERIT_BLACKLIST`` became :term:`ESDK_CLASS_INHERIT_DISABLE`
- ``SSTATE_DUPWHITELIST`` became ``SSTATE_ALLOW_OVERLAP_FILES``
- ``SYSROOT_DIRS_BLACKLIST`` became :term:`SYSROOT_DIRS_IGNORE`
- ``UNKNOWN_CONFIGURE_WHITELIST`` became :term:`UNKNOWN_CONFIGURE_OPT_IGNORE`
- ``WHITELIST_<license>`` became :term:`INCOMPATIBLE_LICENSE_EXCEPTIONS`
In addition, ``BB_STAMP_WHITELIST``, ``BB_STAMP_POLICY``, ``INHERIT_BLACKLIST``,
``TUNEABI``, ``TUNEABI_WHITELIST``, and ``TUNEABI_OVERRIDE`` have been removed.
Many internal variable names have been also renamed accordingly.
In addition, in the ``cve-check`` output, the CVE issue status ``Whitelisted``
has been renamed to ``Ignored``.
The :term:`BB_DISKMON_DIRS` variable value now uses the term ``HALT``
instead of ``ABORT``.
A :oe_git:`convert-variable-renames.py
</openembedded-core/tree/scripts/contrib/convert-variable-renames.py>`
script is provided to convert your recipes and configuration,
and also warns you about the use of problematic words. The script performs
changes and you need to review them before committing. An example warning
looks like::
poky/scripts/lib/devtool/upgrade.py needs further work at line 275 since it contains abort
Fetching changes
~~~~~~~~~~~~~~~~
- Because of the uncertainty in future default branch names in git repositories,
it is now required to add a branch name to all URLs described
by ``git://`` and ``gitsm://`` :term:`SRC_URI` entries. For example::
SRC_URI = "git://git.denx.de/u-boot.git;branch=master"
A :oe_git:`convert-srcuri </openembedded-core/tree/scripts/contrib/convert-srcuri.py>`
script to convert your recipes is available in :term:`OpenEmbedded-Core (OE-Core)`
and in :term:`Poky`.
- Because of `GitHub dropping support for the git:
protocol <https://github.blog/2021-09-01-improving-git-protocol-security-github/>`__,
recipes now need to use ``;protocol=https`` at the end of GitHub
URLs. The same ``convert-srcuri`` script mentioned above can be used to convert
your recipes.
- Network access from tasks is now disabled by default on kernels which support
this feature (on most recent distros such as CentOS 8 and Debian 11 onwards).
This means that tasks accessing the network need to be marked as such with the ``network``
flag. For example::
do_mytask[network] = "1"
This is allowed by default from :ref:`ref-tasks-fetch` but not from any of our other standard
tasks. Recipes shouldn't be accessing the network outside of :ref:`ref-tasks-fetch` as it
usually undermines fetcher source mirroring, image and licence manifests, software
auditing and supply chain security.
License changes
~~~~~~~~~~~~~~~
- The ambiguous "BSD" license has been removed from the ``common-licenses`` directory.
Each recipe that fetches or builds BSD-licensed code should specify the proper
version of the BSD license in its :term:`LICENSE` value.
- :term:`LICENSE` variable values should now use `SPDX identifiers <https://spdx.org/licenses/>`__.
If they do not, by default a warning will be shown. A
:oe_git:`convert-spdx-licenses.py </openembedded-core/tree/scripts/contrib/convert-spdx-licenses.py>`
script can be used to update your recipes.
- :term:`INCOMPATIBLE_LICENSE` should now use `SPDX identifiers <https://spdx.org/licenses/>`__.
Additionally, wildcarding is now limited to specifically supported values -
see the :term:`INCOMPATIBLE_LICENSE` documentation for further information.
- The ``AVAILABLE_LICENSES`` variable has been removed. This variable was a performance
liability and is highly dependent on which layers are added to the configuration,
which can cause signature issues for users. In addition the ``available_licenses()``
function has been removed from the :ref:`ref-classes-license` class as
it is no longer needed.
Removed recipes
~~~~~~~~~~~~~~~
The following recipes have been removed in this release:
- ``dbus-test``: merged into main dbus recipe
- ``libid3tag``: moved to meta-oe - no longer needed by anything in OE-Core
- ``libportal``: moved to meta-gnome - no longer needed by anything in OE-Core
- ``linux-yocto``: removed version 5.14 recipes (5.15 and 5.10 still provided)
- ``python3-nose``: has not changed since 2016 upstream, and no longer needed by anything in OE-Core
- ``rustfmt``: not especially useful as a standalone recipe
Python changes
~~~~~~~~~~~~~~
- ``distutils`` has been deprecated upstream in Python 3.10 and thus the ``distutils*``
classes have been moved to ``meta-python``. Recipes that inherit the ``distutils*``
classes should be updated to inherit ``setuptools*`` equivalents instead.
- The Python package build process is now based on `wheels <https://pythonwheels.com/>`__.
Here are the new Python packaging classes that should be used:
:ref:`ref-classes-python_flit_core`, :ref:`ref-classes-python_setuptools_build_meta`
and :ref:`ref-classes-python_poetry_core`.
- The :ref:`ref-classes-setuptools3` class :ref:`ref-tasks-install` task now
installs the ``wheel`` binary archive. In current versions of ``setuptools`` the
legacy ``setup.py install`` method is deprecated. If the ``setup.py`` cannot be used
with wheels, for example it creates files outside of the Python module or standard
entry points, then :ref:`ref-classes-setuptools3_legacy` should
be used instead.
Prelink removed
~~~~~~~~~~~~~~~
Prelink has been dropped by ``glibc`` upstream in 2.36. It already caused issues with
binary corruption, has a number of open bugs and is of questionable benefit
without disabling load address randomization and PIE executables.
We disabled prelinking by default in the honister (3.4) release, but left it able
to be enabled if desired. However, without glibc support it cannot be maintained
any further, so all of the prelinking functionality has been removed in this release.
If you were enabling the ``image-prelink`` class in :term:`INHERIT`, :term:`IMAGE_CLASSES`,
:term:`USER_CLASSES` etc in your configuration, then you will need to remove the
reference(s).
Reproducible as standard
~~~~~~~~~~~~~~~~~~~~~~~~
Reproducibility is now considered as standard functionality, thus the
``reproducible`` class has been removed and its previous contents merged into the
:ref:`ref-classes-base` class. If you have references in your configuration to
``reproducible`` in :term:`INHERIT`, :term:`USER_CLASSES` etc. then they should be
removed.
Additionally, the ``BUILD_REPRODUCIBLE_BINARIES`` variable is no longer used.
Specifically for the kernel, if you wish to enable build timestamping functionality
that is normally disabled for reproducibility reasons, you can do so by setting
a new :term:`KERNEL_DEBUG_TIMESTAMPS` variable to "1".
Supported host distribution changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Support for :wikipedia:`AlmaLinux <AlmaLinux>`
hosts replacing :wikipedia:`CentOS <CentOS>`.
The following distribution versions were dropped: CentOS 8, Ubuntu 16.04 and Fedora 30, 31 and 32.
- ``gcc`` version 7.5 is now required at minimum on the build host. For older
host distributions where this is not available, you can use the
:term:`buildtools-extended` tarball (easily installable using
``scripts/install-buildtools``).
:append/:prepend in combination with other operators
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The ``append``, ``prepend`` and ``remove`` operators can now only be combined with
``=`` and ``:=`` operators. To the exception of the ``append`` plus ``+=`` and
``prepend`` plus ``=+`` combinations, all combinations could be factored up to the
``append``, ``prepend`` or ``remove`` in the combination. This brought a lot of
confusion on how the override style syntax operators work and should be used.
Therefore, those combinations should be replaced by a single ``append``,
``prepend`` or ``remove`` operator without any additional change.
For the ``append`` plus ``+=`` (and ``prepend`` plus ``=+``) combinations,
the content should be prefixed (respectively suffixed) by a space to maintain
the same behavior. You can learn more about override style syntax operators
(``append``, ``prepend`` and ``remove``) in the BitBake documentation:
:ref:`bitbake-user-manual/bitbake-user-manual-metadata:appending and prepending (override style syntax)`
and :ref:`bitbake-user-manual/bitbake-user-manual-metadata:removal (override style syntax)`.
Miscellaneous changes
~~~~~~~~~~~~~~~~~~~~~
- ``blacklist.bbclass`` is removed and the functionality moved to the
:ref:`ref-classes-base` class with a more descriptive
``varflag`` variable named :term:`SKIP_RECIPE` which will use the `bb.parse.SkipRecipe()`
function. The usage remains the same, for example::
SKIP_RECIPE[my-recipe] = "Reason for skipping recipe"
- :ref:`ref-classes-allarch` packagegroups can no longer depend on packages
which use :term:`PKG` renaming such as :ref:`ref-classes-debian`. Such packagegroups
recipes should be changed to avoid inheriting :ref:`ref-classes-allarch`.
- The ``lnr`` script has been removed. ``lnr`` implemented the same behaviour as `ln --relative --symbolic`,
since at the time of creation `--relative` was only available in coreutils 8.16
onwards which was too new for the older supported distros. Current supported host
distros have a new enough version of coreutils, so it is no longer needed. If you have
any calls to ``lnr`` in your recipes or classes, they should be replaced with
`ln --relative --symbolic` or `ln -rs` if you prefer the short version.
- The ``package_qa_handle_error()`` function formerly in the :ref:`ref-classes-insane`
class has been moved and renamed - if you have any references in your own custom
classes they should be changed to ``oe.qa.handle_error()``.
- When building ``perl``, Berkeley db support is no longer enabled by default, since
Berkeley db is largely obsolete. If you wish to reenable it, you can append ``bdb``
to :term:`PACKAGECONFIG` in a ``perl`` bbappend or ``PACKAGECONFIG:pn-perl`` at
the configuration level.
- For the ``xserver-xorg`` recipe, the ``xshmfence``, ``xmlto`` and ``systemd`` options
previously supported in :term:`PACKAGECONFIG` have been removed, as they are no
longer supported since the move from building it with autotools to meson in this release.
- For the ``libsdl2`` recipe, various X11 features are now disabled by default (primarily
for reproducibility purposes in the native case) with options in :term:`EXTRA_OECMAKE`
within the recipe. These can be changed within a bbappend if desired. See the
``libsdl2`` recipe for more details.
- The ``cortexa72-crc`` and ``cortexa72-crc-crypto`` tunes have been removed since
the crc extension is now enabled by default for cortexa72. Replace any references to
these with ``cortexa72`` and ``cortexa72-crypto`` respectively.
- The Python development shell (previously known as ``devpyshell``) feature has been
renamed to ``pydevshell``. To start it you should now run::
bitbake <target> -c pydevshell
- The ``packagegroups-core-full-cmdline-libs`` packagegroup is no longer produced, as
libraries should normally be brought in via dependencies. If you have any references
to this then remove them.
- The :term:`TOPDIR` variable and the current working directory are no longer modified
when parsing recipes. Any code depending on the previous behaviour will no longer
work - change any such code to explicitly use appropriate path variables instead.
- In order to exclude the kernel image from the image rootfs,
:term:`RRECOMMENDS`\ ``:${KERNEL_PACKAGE_NAME}-base`` should be set instead of
:term:`RDEPENDS`\ ``:${KERNEL_PACKAGE_NAME}-base``.
poky/documentation/migration-guides/migration-4.1.rst
+216
View File
@@ -0,0 +1,216 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 4.1 (langdale)
======================
Migration notes for 4.1 (langdale)
-----------------------------------
This section provides migration information for moving to the Yocto
Project 4.1 Release (codename "langdale") from the prior release.
.. _migration-4.1-make-4.0:
make 4.0 is now the minimum required make version
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
glibc now requires ``make`` 4.0 to build, thus it is now the version required to
be installed on the build host. A new :term:`buildtools-make` tarball has been
introduced to provide just make 4.0 for host distros without a current/working
make 4.x version; if you also need other tools you can use the updated
:term:`buildtools` tarball. For more information see
:ref:`ref-manual/system-requirements:required packages for the build host`.
.. _migration-4.1-complementary-deps:
Complementary package installation ignores recommends
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
When installing complementary packages (e.g. ``-dev`` and ``-dbg`` packages when
building an SDK, or if you have added ``dev-deps`` to :term:`IMAGE_FEATURES`),
recommends (as defined by :term:`RRECOMMENDS`) are no longer installed.
If you wish to double-check the contents of your images after this change, see
:ref:`Checking Image / SDK Changes <migration-general-buildhistory>`. If needed
you can explicitly install items by adding them to :term:`IMAGE_INSTALL` in
image recipes or :term:`TOOLCHAIN_TARGET_TASK` for the SDK.
.. _migration-4.1-dev-recommends:
dev dependencies are now recommends
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The default for ``${PN}-dev`` package is now to use :term:`RRECOMMENDS` instead
of :term:`RDEPENDS` to pull in the main package. This takes advantage of a
change to complimentary package installation to not follow :term:`RRECOMMENDS`
(as mentioned above) and for example means an SDK for an image with both openssh
and dropbear components will now build successfully.
.. _migration-4.1-dropbear-sftp:
dropbear now recommends openssh-sftp-server
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
openssh has switched the scp client to use the sftp protocol instead of scp to
move files. This means scp from Fedora 36 and other current distributions will
no longer be able to move files to/from a system running dropbear with no sftp
server installed.
The sftp server from openssh is small (200kb uncompressed) and standalone, so
adding it to the packagegroup seems to be the best way to preserve the
functionality for user sanity. However, if you wish to avoid this dependency,
you can either:
A. Use ``dropbear`` in :term:`IMAGE_INSTALL` instead of
``packagegroup-core-ssh-dropbear`` (or ``ssh-server-dropbear`` in
:term:`IMAGE_FEATURES`), or
B. Add ``openssh-sftp-server`` to :term:`BAD_RECOMMENDATIONS`.
.. _migration-4.1-classes-split:
Classes now split by usage context
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A split directory structure has now been set up for ``.bbclass`` files - classes
that are intended to be inherited only by recipes (e.g. ``inherit`` in a recipe
file, :term:`IMAGE_CLASSES` or :term:`KERNEL_CLASSES`) should be in a
``classes-recipe`` subdirectory and classes that are intended to be inherited
globally (e.g. via ``INHERIT +=``, :term:`PACKAGE_CLASSES`, :term:`USER_CLASSES`
or :term:`INHERIT_DISTRO`) should be in ``classes-global``. Classes in the
existing ``classes`` subdirectory will continue to work in any context as before.
Other than knowing where to look when manually browsing the class files, this is
not likely to require any changes to your configuration. However, if in your
configuration you were using some classes in the incorrect context, you will now
receive an error during parsing. For example, the following in ``local.conf`` will
now cause an error::
INHERIT += "testimage"
Since :ref:`ref-classes-testimage` is a class intended solely to
affect image recipes, this would be correctly specified as::
IMAGE_CLASSES += "testimage"
.. _migration-4.1-local-file-error:
Missing local files in SRC_URI now triggers an error
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If a file referenced in :term:`SRC_URI` does not exist, in 4.1 this will trigger
an error at parse time where previously this only triggered a warning. In the past
you could ignore these warnings for example if you have multiple build
configurations (e.g. for several different target machines) and there were recipes
that you were not building in one of the configurations. If you have this scenario
you will now need to conditionally add entries to :term:`SRC_URI` where they are
valid, or use :term:`COMPATIBLE_MACHINE` / :term:`COMPATIBLE_HOST` to prevent the
recipe from being available (and therefore avoid it being parsed) in configurations
where the files aren't available.
.. _migration-4.1-qa-checks:
QA check changes
~~~~~~~~~~~~~~~~
- The :ref:`buildpaths <qa-check-buildpaths>` QA check is now enabled by default
in :term:`WARN_QA`, and thus any build system paths found in output files will
trigger a warning. If you see these warnings for your own recipes, for full
binary reproducibility you should make the necessary changes to the recipe build
to remove these paths. If you wish to disable the warning for a particular
recipe you can use :term:`INSANE_SKIP`, or for the entire build you can adjust
:term:`WARN_QA`. For more information, see the :ref:`buildpaths QA check
<qa-check-buildpaths>` section.
- ``do_qa_staging`` now checks shebang length in all directories specified by
:term:`SYSROOT_DIRS`, since there is a maximum length defined in the kernel. For
native recipes which write scripts to the sysroot, if the shebang line in one of
these scripts is too long you will get an error. This can be skipped using
:term:`INSANE_SKIP` if necessary, but the best course of action is of course to
fix the script. There is now also a ``create_cmdline_shebang_wrapper`` function
that you can call e.g. from ``do_install`` (or ``do_install:append``) within a
recipe to create a wrapper to fix such scripts - see the ``libcheck`` recipe
for an example usage.
Miscellaneous changes
~~~~~~~~~~~~~~~~~~~~~
- ``mount.blacklist`` has been renamed to ``mount.ignorelist`` in
``udev-extraconf``. If you are customising this file via ``udev-extraconf`` then
you will need to update your ``udev-extraconf`` ``.bbappend`` as appropriate.
- ``help2man-native`` has been removed from implicit sysroot dependencies. If a
recipe needs ``help2man-native`` it should now be explicitly added to
:term:`DEPENDS` within the recipe.
- For images using systemd, the reboot watchdog timeout has been set to 60
seconds (from the upstream default of 10 minutes). If you wish to override this
you can set :term:`WATCHDOG_TIMEOUT` to the desired timeout in seconds. Note
that the same :term:`WATCHDOG_TIMEOUT` variable also specifies the timeout used
for the ``watchdog`` tool (if that is being built).
- The :ref:`ref-classes-image-buildinfo` class now writes to
``${sysconfdir}/buildinfo`` instead of ``${sysconfdir}/build`` by default (i.e.
the default value of :term:`IMAGE_BUILDINFO_FILE` has been changed). If you have
code that reads this from images at build or runtime you will need to update it
or specify your own value for :term:`IMAGE_BUILDINFO_FILE`.
- In the :ref:`ref-classes-archiver` class, the default
``ARCHIVER_OUTDIR`` value no longer includes the :term:`MACHINE` value in order
to avoid the archive task running multiple times in a multiconfig setup. If you
have custom code that does something with the files archived by the
:ref:`ref-classes-archiver` class then you may need to adjust it to
the new structure.
- If you are not using `systemd` then udev is now configured to use labels
(``LABEL`` or ``PARTLABEL``) to set the mount point for the device. For example::
/run/media/rootfs-sda2
instead of::
/run/media/sda2
- ``icu`` no longer provides the ``icu-config`` configuration tool - upstream
have indicated ``icu-config`` is deprecated and should no longer be used. Code
with references to it will need to be updated, for example to use ``pkg-config``
instead.
- The ``rng-tools`` systemd service name has changed from ``rngd`` to ``rng-tools``
- The ``largefile`` :term:`DISTRO_FEATURES` item has been removed, large file
support is now always enabled where it was previously optional.
- The Python ``zoneinfo`` module is now split out to its own ``python3-zoneinfo``
package.
- The :term:`PACKAGECONFIG` option to enable wpa_supplicant in the ``connman``
recipe has been renamed to "wpa-supplicant". If you have set :term:`PACKAGECONFIG` for
the ``connman`` recipe to include this option you will need to update
your configuration. Related to this, the :term:`WIRELESS_DAEMON` variable
now expects the new ``wpa-supplicant`` naming and affects ``packagegroup-base``
as well as ``connman``.
- The ``wpa-supplicant`` recipe no longer uses a static (and stale) ``defconfig``
file, instead it uses the upstream version with appropriate edits for the
:term:`PACKAGECONFIG`. If you are customising this file you will need to
update your customisations.
- With the introduction of picobuild in
:ref:`ref-classes-python_pep517`, The ``PEP517_BUILD_API``
variable is no longer supported. If you have any references to this variable
you should remove them.
.. _migration-4.1-removed-recipes:
Removed recipes
~~~~~~~~~~~~~~~
The following recipes have been removed in this release:
- ``alsa-utils-scripts``: merged into alsa-utils
- ``cargo-cross-canadian``: optimised out
- ``lzop``: obsolete, unmaintained upstream
- ``linux-yocto (5.10)``: 5.15 and 5.19 are currently provided
- ``rust-cross``: optimised out
- ``rust-crosssdk``: optimised out
- ``rust-tools-cross-canadian``: optimised out
- ``xf86-input-keyboard``: obsolete (replaced by libinput/evdev)
poky/documentation/migration-guides/migration-4.2.rst
+276
View File
@@ -0,0 +1,276 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 4.2 (mickledore)
========================
Migration notes for 4.2 (mickledore)
------------------------------------
This section provides migration information for moving to the Yocto
Project 4.2 Release (codename "mickledore") from the prior release.
.. _migration-4.2-supported-distributions:
Supported distributions
~~~~~~~~~~~~~~~~~~~~~~~
This release supports running BitBake on new GNU/Linux distributions:
- Fedora 36 and 37
- AlmaLinux 8.7 and 9.1
- OpenSuse 15.4
On the other hand, some earlier distributions are no longer supported:
- Debian 10.x
- Fedora 34 and 35
- AlmaLinux 8.5
See :ref:`all supported distributions <system-requirements-supported-distros>`.
.. _migration-4.2-python-3.8:
Python 3.8 is now the minimum required Python version version
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
BitBake and OpenEmbedded-Core now require Python 3.8 or newer,
making it a requirement to use a distribution providing at least this
version, or to install a :term:`buildtools` tarball.
.. _migration-4.2-gcc-8.0:
gcc 8.0 is now the minimum required GNU C compiler version
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This version, released in 2018, is a minimum requirement
to build the ``mesa-native`` recipe and as the latter is in the
default dependency chain when building QEMU this has now been
made a requirement for all builds.
In the event that your host distribution does not provide this
or a newer version of gcc, you can install a
:term:`buildtools-extended` tarball.
.. _migration-4.2-new-nvd-api:
Fetching the NVD vulnerability database through the 2.0 API
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This new version adds a new fetcher for the NVD database using the 2.0 API,
as the 1.0 API will be retired in 2023.
The implementation changes as little as possible, keeping the current
database format (but using a different database file for the transition
period), with a notable exception of not using the META table.
Here are minor changes that you may notice:
- The database starts in 1999 instead of 2002
- The complete fetch is longer (30 minutes typically)
.. _migration-4.2-rust-crate-checksums:
Rust: mandatory checksums for crates
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This release now supports checksums for Rust crates and makes
them mandatory for each crate in a recipe. See :yocto_git:`python3_bcrypt recipe changes
</poky/commit/?h=mickledore&id=0dcb5ab3462fdaaf1646b05a00c7150eea711a9a>`
for example.
The ``cargo-update-recipe-crates`` utility
:yocto_git:`has been extended </poky/commit/?h=mickledore&id=eef7fbea2c5bf59369390be4d5efa915591b7b22>`
to include such checksums. So, in case you need to add the list of checksums
to a recipe just inheriting the :ref:`ref-classes-cargo` class so far, you can
follow these steps:
#. Make the recipe inherit :ref:`ref-classes-cargo-update-recipe-crates`
#. Remove all ``crate://`` lines from the recipe
#. Create an empty ``${BPN}-crates.inc`` file and make your recipe require it
#. Execute ``bitbake -c update_crates your_recipe``
#. Copy and paste the output of BitBake about the missing checksums into the
``${BPN}-crates.inc`` file.
.. _migration-4.2-addpylib:
Python library code extensions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
BitBake in this release now supports a new ``addpylib`` directive to enable
Python libraries within layers.
This directive should be added to your layer configuration
as in the below example from ``meta/conf/layer.conf``::
addpylib ${LAYERDIR}/lib oe
Layers currently adding a lib directory to extend Python library code should now
use this directive as :term:`BBPATH` is not going to be added automatically by
OE-Core in future. Note that the directives are immediate operations, so it does
make modules available for use sooner than the current BBPATH-based approach.
For more information, see :ref:`bitbake-user-manual/bitbake-user-manual-metadata:extending python library code`.
.. _migration-4.2-removed-variables:
Removed variables
~~~~~~~~~~~~~~~~~
The following variables have been removed:
- ``SERIAL_CONSOLE``, deprecated since version 2.6, replaced by :term:`SERIAL_CONSOLES`.
- ``PACKAGEBUILDPKGD``, a mostly internal variable in the ref:`ref-classes-package`
class was rarely used to customise packaging. If you were using this in your custom
recipes or bbappends, you will need to switch to using :term:`PACKAGE_PREPROCESS_FUNCS`
or :term:`PACKAGESPLITFUNCS` instead.
.. _migration-4.2-removed-recipes:
Removed recipes
~~~~~~~~~~~~~~~
The following recipes have been removed in this release:
- ``python3-picobuild``: after switching to ``python3-build``
- ``python3-strict-rfc3339``: unmaintained and not needed by anything in
:oe_git:`openembedded-core </openembedded-core>`
or :oe_git:`meta-openembedded </meta-openembedded>`.
- ``linux-yocto``: removed version 5.19 recipes (6.1 and 5.15 still provided)
.. _migration-4.2-removed-classes:
Removed classes
~~~~~~~~~~~~~~~
The following classes have been removed in this release:
- ``rust-bin``: no longer used
- ``package_tar``: could not be used for actual packaging, and thus not particularly useful.
LAYERSERIES_COMPAT for custom layers and devtool workspace
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Some layer maintainers have been setting :term:`LAYERSERIES_COMPAT` in their
layer's ``conf/layer.conf`` to the value of ``LAYERSERIES_CORENAMES`` to
effectively bypass the compatibility check - this is no longer permitted.
Layer maintainers should set :term:`LAYERSERIES_COMPAT` appropriately to
help users understand the compatibility status of the layer.
Additionally, the :term:`LAYERSERIES_COMPAT` value for the devtool workspace
layer is now set at the time of creation, thus if you upgrade with the
workspace layer enabled and you wish to retain it, you will need to manually
update the :term:`LAYERSERIES_COMPAT` value in ``workspace/conf/layer.conf``
(or remove the path from :term:`BBLAYERS` in ``conf/bblayers.conf`` and
delete/move the ``workspace`` directory out of the way if you no longer
need it).
.. _migration-4.2-runqemu-slirp:
runqemu now limits slirp host port forwarding to localhost
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
With default slirp port forwarding configuration in runqemu, qemu
previously listened on TCP ports 2222 and 2323 on all IP addresses
available on the build host. Most use cases with runqemu only need
it for localhost and it is not safe to run qemu images with root
login without password enabled and listening on all available,
possibly Internet reachable network interfaces. Thus, in this
release we limit qemu port forwarding to localhost (127.0.0.1).
However, if you need the qemu machine to be reachable from the
network, then it can be enabled via ``conf/local.conf`` or machine
config variable ``QB_SLIRP_OPT``::
QB_SLIRP_OPT = "-netdev user,id=net0,hostfwd=tcp::2222-:22"
.. _migration-4.2-patch-qa:
Patch QA checks
~~~~~~~~~~~~~~~
The QA checks for patch fuzz and Upstream-Status have been reworked
slightly in this release. The Upstream-Status checking is now configurable
from :term:`WARN_QA` / :term:`ERROR_QA` (``patch-status-core`` for the
core layer, and ``patch-status-noncore`` for other layers).
The ``patch-fuzz`` and ``patch-status-core`` checks are now in the default
value of :term:`ERROR_QA` so that they will cause the build to fail
if triggered. If you prefer to avoid this you will need to adjust the value
of :term:`ERROR_QA` in your configuration as desired.
.. _migration-4.2-mesa:
Native/nativesdk mesa usage and graphics drivers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This release includes mesa 23.0, and with that mesa release it is not longer
possible to use drivers from the host system, as mesa upstream has added strict
checks for matching builds between drivers and libraries that load them.
This is particularly relevant when running QEMU built within the build
system. A check has been added to runqemu so that there is a helpful error
when there is no native/nativesdk opengl/virgl support available.
To support this, a number of drivers have been enabled when building ``mesa-native``.
The one major dependency pulled in by this change is ``llvm-native`` which will
add a few minutes to the build on a modern machine. If this is undesirable, you
can set the value of :term:`DISTRO_FEATURES_NATIVE` in your configuration such
that ``opengl`` is excluded.
.. _migration-4.2-misc-changes:
Miscellaneous changes
~~~~~~~~~~~~~~~~~~~~~
- The :term:`IMAGE_NAME` variable is now set based on :term:`IMAGE_LINK_NAME`. This
means that if you are setting :term:`IMAGE_LINK_NAME` to "" to disable unversioned
image symlink creation, you also now need to set :term:`IMAGE_NAME` to still have
a reasonable value e.g.::
IMAGE_LINK_NAME = ""
IMAGE_NAME = "${IMAGE_BASENAME}${IMAGE_MACHINE_SUFFIX}${IMAGE_VERSION_SUFFIX}"
- In ``/etc/os-release``, the ``VERSION_CODENAME`` field is now used instead of
``DISTRO_CODENAME`` (though its value is still set from the :term:`DISTRO_CODENAME`
variable) for better conformance to standard os-release usage. If you have runtime
code reading this from ``/etc/os-release`` it may need to be updated.
- The kmod recipe now enables OpenSSL support by default in order to support module
signing. If you do not need this and wish to reclaim some space/avoid the dependency
you should set :term:`PACKAGECONFIG` in a kmod bbappend (or ``PACKAGECONFIG:pn-kmod``
at the configuration level) to exclude ``openssl``.
- The ``OEBasic`` signature handler (see :term:`BB_SIGNATURE_HANDLER`) has been
removed. It is unlikely that you would have selected to use this, but if you have
you will need to remove this setting.
- The :ref:`ref-classes-package` class now checks if package names conflict via
``PKG:${PN}`` override during ``do_package``. If you receive the associated error
you will need to address the :term:`PKG` usage so that the conflict is resolved.
- openssh no longer uses :term:`RRECOMMENDS` to pull in ``rng-tools``, since rngd
is no longer needed as of Linux kernel 5.6. If you still need ``rng-tools``
installed for other reasons, you should add ``rng-tools`` explicitly to your
image. If you additionally need rngd to be started as a service you will also
need to add the ``rng-tools-service`` package as that has been split out.
- The cups recipe no longer builds with the web interface enabled, saving ~1.8M of
space in the final image. If you wish to enable it, you should set
:term:`PACKAGECONFIG` in a cups bbappend (or ``PACKAGECONFIG:pn-cups`` at the
configuration level) to include ``webif``.
- The :ref:`ref-classes-scons` class now passes a ``MAXLINELENGTH`` argument to
scons in order to fix an issue with scons and command line lengths when ccache is
enabled. However, some recipes may be using older scons versions which don't support
this argument. If that is the case you can set the following in the recipe in order
to disable this::
SCONS_MAXLINELENGTH = ""
poky/documentation/migration-guides/migration-4.3.rst
+119
View File
@@ -0,0 +1,119 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
Release 4.3 (nanbield)
========================
Migration notes for 4.3 (nanbield)
------------------------------------
This section provides migration information for moving to the Yocto
Project 4.3 Release (codename "nanbield") from the prior release.
.. _migration-4.3-supported-kernel-versions:
Supported kernel versions
~~~~~~~~~~~~~~~~~~~~~~~~~
The :term:`OLDEST_KERNEL` setting has been changed to "5.15" in this release, meaning that
out the box, older kernels are not supported. There were two reasons for this.
Firstly it allows glibc optimisations that improve the performance of the system
by removing compatibility code and using modern kernel APIs exclusively. The second
issue was this allows 64 bit time support even on 32 bit platforms and resolves Y2038
issues.
It is still possible to override this value and build for older kernels, this is just
no longer the default supported configuration. This setting does not affect which
kernel versions SDKs will run against and does not affect which versions of the kernel
can be used to run builds.
Layername override implications
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Code can now know which layer a recipe is coming from through the newly added
:term:`FILE_LAYERNAME` variable and the ``layer-<layername> override``. This is being used
for enabling QA checks on a per layer basis. For existing code this has the
side effect that the QA checks will apply to things being bbappended to recipes
from other layers. Those other layers would need to have patch upstream status
entries for patches being bbappended for example.
.. _migration-4.3-supported-distributions:
Supported distributions
~~~~~~~~~~~~~~~~~~~~~~~
This release supports running BitBake on new GNU/Linux distributions:
On the other hand, some earlier distributions are no longer supported:
See :ref:`all supported distributions <system-requirements-supported-distros>`.
.. _migration-4.3-go-changes:
Go language changes
~~~~~~~~~~~~~~~~~~~
- Support for the Glide package manager has been removed, as ``go mod``
has become the standard.
Systemd changes
~~~~~~~~~~~~~~~
Upstream systemd is now more strict on filesystem layout and the ``usrmerge``
feature is therefore required alongside systemd. The Poky test configurations
have been updated accordingly for systemd.
.. _migration-4.3-recipe-changes:
Recipe changes
~~~~~~~~~~~~~~
- Runtime testing of ptest now fails if no test results are returned by
any given ptest.
.. _migration-4.3-class-changes:
Class changes
~~~~~~~~~~~~~
- The ``perl-version`` class no longer provides the ``PERLVERSION`` and ``PERLARCH`` variables
as there were no users in any core layer. The functions for this functionality
are still available.
.. _migration-4.3-removed-variables:
Removed variables
~~~~~~~~~~~~~~~~~
The following variables have been removed:
- ``PERLARCH``
- ``PERLVERSION``
.. _migration-4.3-removed-recipes:
Removed recipes
~~~~~~~~~~~~~~~
The following recipes have been removed in this release:
- ``glide``, as explained in :ref:`migration-4.3-go-changes`.
.. _migration-4.3-removed-classes:
Removed classes
~~~~~~~~~~~~~~~
The following classes have been removed in this release:
.. _migration-4.3-misc-changes:
Miscellaneous changes
~~~~~~~~~~~~~~~~~~~~~
- The ``-crosssdk`` suffix and any :term:`MLPREFIX` were removed from
``virtual/XXX`` provider/dependencies where a ``PREFIX`` was used as well,
as we don't need both and it made automated dependency rewriting
unnecessarily complex. In general this only affects internal toolchain
dependencies so isn't end user visible.

Some files were not shown because too many files have changed in this diff Show More