jim.lai/OpenBMC
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Your Name
2026-04-23 17:07:55 +08:00
commit b7e39e063b
16725 changed files with 1625565 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
poky/meta/recipes-devtools/python/python3/0001-Avoid-shebang-overflow-on-python-config.py.patch
+31
View File
@@ -0,0 +1,31 @@
From 78f482b91d94b44a02e02c4580166757119061ea Mon Sep 17 00:00:00 2001
From: Paulo Neves <ptsneves@gmail.com>
Date: Tue, 7 Jun 2022 16:16:41 +0200
Subject: [PATCH] Avoid shebang overflow on python-config.py
The whole native path may be too big, leading to shebang
overflow. Let's just use the env shebang.
Denial reason: [1]
Upstream-Status: Denied [distribution]
[1] https://github.com/python/cpython/pull/93760#pullrequestreview-1005365737
---
Makefile.pre.in | 2 ++
1 file changed, 2 insertions(+)
diff --git a/Makefile.pre.in b/Makefile.pre.in
index 55c7c46..1f6500a 100644
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -2115,6 +2115,8 @@ python-config: $(srcdir)/Misc/python-config.in Misc/python-config.sh
@ # Substitution happens here, as the completely-expanded BINDIR
@ # is not available in configure
sed -e "s,@EXENAME@,$(BINDIR)/python$(LDVERSION)$(EXE)," < $(srcdir)/Misc/python-config.in >python-config.py
+ @ # Otherwise we might get huge shebangs with native paths
+ sed -i -e '1s|^#!.*|#!/usr/bin/env python3|' python-config.py
@ # Replace makefile compat. variable references with shell script compat. ones; $(VAR) -> ${VAR}
LC_ALL=C sed -e 's,\$$(\([A-Za-z0-9_]*\)),\$$\{\1\},g' < Misc/python-config.sh >python-config
@ # On Darwin, always use the python version of the script, the shell
poky/meta/recipes-devtools/python/python3/0001-Do-not-add-usr-lib-termcap-to-linker-flags-to-avoid-.patch
+25
View File
@@ -0,0 +1,25 @@
From 93ae2ed3fc8be0245e35063c4f63626792f4cd0c Mon Sep 17 00:00:00 2001
From: Alexander Kanavin <alex.kanavin@gmail.com>
Date: Fri, 25 Jan 2019 19:04:13 +0100
Subject: [PATCH] Do not add /usr/lib/termcap to linker flags to avoid host
contamination
Upstream-Status: Inappropriate [oe-core specific]
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
---
setup.py | 1 -
1 file changed, 1 deletion(-)
diff --git a/setup.py b/setup.py
index 15d0d45..37ed244 100644
--- a/setup.py
+++ b/setup.py
@@ -1109,7 +1109,6 @@ class PyBuildExt(build_ext):
'termcap'):
readline_libs.append('termcap')
self.add(Extension('readline', ['readline.c'],
- library_dirs=['/usr/lib/termcap'],
libraries=readline_libs))
else:
self.missing.append('readline')
poky/meta/recipes-devtools/python/python3/0001-Don-t-search-system-for-headers-libraries.patch
+27
View File
@@ -0,0 +1,27 @@
From aa8f1709c54557d2b51a9a37d15ccc3de62e90cb Mon Sep 17 00:00:00 2001
From: Jeremy Puhlman <jpuhlman@mvista.com>
Date: Wed, 4 Mar 2020 00:06:42 +0000
Subject: [PATCH] Don't search system for headers/libraries
Upstream-Status: Inappropriate [oe-core specific]
Signed-off-by: Jeremy Puhlman <jpuhlman@mvista.com>
---
setup.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/setup.py b/setup.py
index 6811951..65ae476 100644
--- a/setup.py
+++ b/setup.py
@@ -877,8 +877,8 @@ class PyBuildExt(build_ext):
add_dir_to_list(self.compiler.include_dirs,
sysconfig.get_config_var("INCLUDEDIR"))
- system_lib_dirs = ['/lib64', '/usr/lib64', '/lib', '/usr/lib']
- system_include_dirs = ['/usr/include']
+ system_lib_dirs = []
+ system_include_dirs = []
# lib_dirs and inc_dirs are used to search for files;
# if a file is found in one of those directories, it can
# be assumed that no additional -I,-L directives are needed.
poky/meta/recipes-devtools/python/python3/0001-Lib-pty.py-handle-stdin-I-O-errors-same-way-as-maste.patch
+47
View File
@@ -0,0 +1,47 @@
From 7b0a14e7320078ac891d415cab9b7568e3f52ad8 Mon Sep 17 00:00:00 2001
From: Alexander Kanavin <alex@linutronix.de>
Date: Thu, 16 Sep 2021 16:35:37 +0200
Subject: [PATCH] Lib/pty.py: handle stdin I/O errors same way as master I/O
errors
reading stdin can throw the same I/O errors as reading from master fd does,
e.g. when running under Yocto's test harness:
======================================================================
ERROR: test_spawn_doesnt_hang (test.test_pty.PtyTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3.10/test/test_pty.py", line 316, in test_spawn_doesnt_hang
pty.spawn([sys.executable, '-c', 'print("hi there")'])
File "/usr/lib/python3.10/pty.py", line 181, in spawn
_copy(master_fd, master_read, stdin_read)
File "/usr/lib/python3.10/pty.py", line 157, in _copy
data = stdin_read(STDIN_FILENO)
File "/usr/lib/python3.10/pty.py", line 132, in _read
return os.read(fd, 1024)
OSError: [Errno 5] Input/output error
So let's treat both channels the same.
Upstream-Status: Submitted [https://github.com/python/cpython/pull/28388]
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
Lib/pty.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/Lib/pty.py b/Lib/pty.py
index fefb63a..4cef056 100644
--- a/Lib/pty.py
+++ b/Lib/pty.py
@@ -184,7 +184,10 @@ def _copy(master_fd, master_read=_read, stdin_read=_read):
i_buf = i_buf[n:]
if stdin_avail and STDIN_FILENO in rfds:
- data = stdin_read(STDIN_FILENO)
+ try:
+ data = stdin_read(STDIN_FILENO)
+ except OSError:
+ data = b""
if not data:
stdin_avail = False
else:
poky/meta/recipes-devtools/python/python3/0001-Lib-sysconfig.py-use-prefix-value-from-build-configu.patch
+32
View File
@@ -0,0 +1,32 @@
From 512c617bd00b74b30a80dd56a12391de46e2b6cf Mon Sep 17 00:00:00 2001
From: Alexander Kanavin <alex@linutronix.de>
Date: Fri, 10 Sep 2021 12:28:31 +0200
Subject: [PATCH] Lib/sysconfig.py: use prefix value from build configuration
file
This allows correctly substituting them for target installs using
native python.
Upstream-Status: Inappropriate [oe-core cross builds]
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
Lib/sysconfig.py | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py
index 93c6f73..ff399e2 100644
--- a/Lib/sysconfig.py
+++ b/Lib/sysconfig.py
@@ -668,6 +668,11 @@ def get_config_vars(*args):
_CONFIG_VARS['VPATH'] = sys._vpath
if os.name == 'posix':
_init_posix(_CONFIG_VARS)
+ _CONFIG_VARS['installed_base'] = _CONFIG_VARS['prefix']
+ _CONFIG_VARS['base'] = _CONFIG_VARS['prefix']
+ _CONFIG_VARS['installed_platbase'] = _CONFIG_VARS['prefix']
+ _CONFIG_VARS['platbase'] = _CONFIG_VARS['prefix']
+ _CONFIG_VARS['platlibdir'] = _CONFIG_VARS['PLATLIBDIR']
if _HAS_USER_BASE:
# Setting 'userbase' is done below the call to the
# init function to enable using 'get_config_var' in
poky/meta/recipes-devtools/python/python3/0001-Makefile-do-not-compile-.pyc-in-parallel.patch
+65
View File
@@ -0,0 +1,65 @@
From c960837b8fd83074bab5148236f3d0595468cea4 Mon Sep 17 00:00:00 2001
From: Alexander Kanavin <alex.kanavin@gmail.com>
Date: Thu, 16 Jan 2020 12:34:20 +0100
Subject: [PATCH] Makefile: do not compile .pyc in parallel
This was found to lock up builds, break reproducibility, and produce strange file ownership
races.
The upstream commit introducing the change was:
https://github.com/python/cpython/commit/1a2dd82f56bd813aacc570e172cefe55a8a41504
The build lock up issue is reported here:
https://bugs.python.org/issue45945
The repro failures are documented here:
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20211130-yr_o1a8d/packages/diff-html/
Upstream-Status: Inappropriate [see issues above]
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
---
Makefile.pre.in | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/Makefile.pre.in b/Makefile.pre.in
index edd70d4..5e13ba2 100644
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -1601,30 +1601,30 @@ libinstall: build_all $(srcdir)/Modules/xxmodule.c
fi
-PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
$(PYTHON_FOR_BUILD) -Wi $(DESTDIR)$(LIBDEST)/compileall.py \
- -j0 -d $(LIBDEST) -f \
+ -d $(LIBDEST) -f \
-x 'bad_coding|badsyntax|site-packages|lib2to3/tests/data' \
$(DESTDIR)$(LIBDEST)
-PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
$(PYTHON_FOR_BUILD) -Wi -O $(DESTDIR)$(LIBDEST)/compileall.py \
- -j0 -d $(LIBDEST) -f \
+ -d $(LIBDEST) -f \
-x 'bad_coding|badsyntax|site-packages|lib2to3/tests/data' \
$(DESTDIR)$(LIBDEST)
-PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
$(PYTHON_FOR_BUILD) -Wi -OO $(DESTDIR)$(LIBDEST)/compileall.py \
- -j0 -d $(LIBDEST) -f \
+ -d $(LIBDEST) -f \
-x 'bad_coding|badsyntax|site-packages|lib2to3/tests/data' \
$(DESTDIR)$(LIBDEST)
-PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
$(PYTHON_FOR_BUILD) -Wi $(DESTDIR)$(LIBDEST)/compileall.py \
- -j0 -d $(LIBDEST)/site-packages -f \
+ -d $(LIBDEST)/site-packages -f \
-x badsyntax $(DESTDIR)$(LIBDEST)/site-packages
-PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
$(PYTHON_FOR_BUILD) -Wi -O $(DESTDIR)$(LIBDEST)/compileall.py \
- -j0 -d $(LIBDEST)/site-packages -f \
+ -d $(LIBDEST)/site-packages -f \
-x badsyntax $(DESTDIR)$(LIBDEST)/site-packages
-PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
$(PYTHON_FOR_BUILD) -Wi -OO $(DESTDIR)$(LIBDEST)/compileall.py \
- -j0 -d $(LIBDEST)/site-packages -f \
+ -d $(LIBDEST)/site-packages -f \
-x badsyntax $(DESTDIR)$(LIBDEST)/site-packages
-PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
$(PYTHON_FOR_BUILD) -m lib2to3.pgen2.driver $(DESTDIR)$(LIBDEST)/lib2to3/Grammar.txt
poky/meta/recipes-devtools/python/python3/0001-Makefile.pre-use-qemu-wrapper-when-gathering-profile.patch
+26
View File
@@ -0,0 +1,26 @@
From 9f85089cc3a21d5ff235bb37c6c9758f2b70497d Mon Sep 17 00:00:00 2001
From: Alexander Kanavin <alex.kanavin@gmail.com>
Date: Wed, 30 Jan 2019 12:41:04 +0100
Subject: [PATCH] Makefile.pre: use qemu wrapper when gathering profile data
Upstream-Status: Inappropriate [oe-core specific]
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
---
Makefile.pre.in | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/Makefile.pre.in b/Makefile.pre.in
index f0aedb7..edd70d4 100644
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -519,8 +519,7 @@ build_all_generate_profile:
$(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS_NODIST) $(PGO_PROF_GEN_FLAG)" LDFLAGS_NODIST="$(LDFLAGS_NODIST) $(PGO_PROF_GEN_FLAG)" LIBS="$(LIBS)"
run_profile_task:
- @ # FIXME: can't run for a cross build
- $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true
+ ./pgo-wrapper ./python -m test.regrtest --pgo test_grammar test_opcodes test_dict test_builtin test_exceptions test_types test_support || true
build_all_merge_profile:
$(LLVM_PROF_MERGER)
poky/meta/recipes-devtools/python/python3/0001-Skip-failing-tests-due-to-load-variability-on-YP-AB.patch
+50
View File
@@ -0,0 +1,50 @@
From 01e02fb4720ecbbc44c694ee1b7fb6d5d95b5fe2 Mon Sep 17 00:00:00 2001
From: Yi Fan Yu <yifan.yu@windriver.com>
Date: Thu, 1 Apr 2021 13:08:37 -0700
Subject: [PATCH] Skip failing tests due to load variability on YP AB
Skip these tests until AB-INT is solved.
[YOCTO #14296]
Upstream-Status: Inappropriate [OE-Specific]
Signed-off-by: Yi Fan Yu <yifan.yu@windriver.com>
---
Lib/test/_test_multiprocessing.py | 2 ++
Lib/test/test_time.py | 1 +
2 files changed, 3 insertions(+)
diff --git a/Lib/test/_test_multiprocessing.py b/Lib/test/_test_multiprocessing.py
index 599c3f2..23328be 100644
--- a/Lib/test/_test_multiprocessing.py
+++ b/Lib/test/_test_multiprocessing.py
@@ -618,6 +618,7 @@ class _TestProcess(BaseTestCase):
close_queue(q)
+ @unittest.skip('timing related test, dependent on load')
def test_many_processes(self):
if self.TYPE == 'threads':
self.skipTest('test not appropriate for {}'.format(self.TYPE))
@@ -4890,6 +4891,7 @@ class TestWait(unittest.TestCase):
sem.release()
time.sleep(period)
+ @unittest.skip('timing related test, dependent on load')
def test_wait_integer(self):
from multiprocessing.connection import wait
diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py
index 884b142..542e980 100644
--- a/Lib/test/test_time.py
+++ b/Lib/test/test_time.py
@@ -492,6 +492,7 @@ class TimeTestCase(unittest.TestCase):
@unittest.skipIf(
support.is_wasi, "process_time not available on WASI"
)
+ @unittest.skip('timing related test, dependent on load')
def test_process_time(self):
# process_time() should not include time spend during a sleep
start = time.process_time()
poky/meta/recipes-devtools/python/python3/0001-Update-test_sysconfig-for-posix_user-purelib.patch
+39
View File
@@ -0,0 +1,39 @@
From b9f825b298b555c4770024d1f68ef1df65aad20a Mon Sep 17 00:00:00 2001
From: Wentao Zhang <wentao.zhang@windriver.com>
Date: Mon, 20 Mar 2023 13:39:52 +0800
Subject: [PATCH] Update test_sysconfig for posix_user purelib
Steps to trigger the failed test:
Edit local.conf to add something as follows:
BASELIB = "lib64"
IMAGE_INSTALL:append = " python3-tests".
bitbake core-image-sato
runqemu qemux86-64 nographic slirp
Reproducer:
$python3 -m test test_sysconfig
Update test_sysconfig.test_user_similar() for the posix_user scheme:
"purelib" doesn't use sys.platlibdir.
Upstream-Status: Inappropriate [oe-core specific]
Signed-off-by: Wentao Zhang <wentao.zhang@windriver.com>
---
Lib/test/test_sysconfig.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Lib/test/test_sysconfig.py b/Lib/test/test_sysconfig.py
index d96371d..20aea4b 100644
--- a/Lib/test/test_sysconfig.py
+++ b/Lib/test/test_sysconfig.py
@@ -373,7 +373,7 @@ class TestSysConfig(unittest.TestCase):
expected = os.path.normpath(global_path.replace(base, user, 1))
# bpo-44860: platlib of posix_user doesn't use sys.platlibdir,
# whereas posix_prefix does.
- if name == 'platlib':
+ if name == 'platlib' or name == 'purelib':
# Replace "/lib64/python3.11/site-packages" suffix
# with "/lib/python3.11/site-packages".
py_version_short = sysconfig.get_python_version()
--
2.25.1
poky/meta/recipes-devtools/python/python3/0001-python3-use-cc_basename-to-replace-CC-for-checking-c.patch
+121
View File
@@ -0,0 +1,121 @@
From 4ba40ee527f844a804be571e52d9dc5447ae4cdd Mon Sep 17 00:00:00 2001
From: Changqing Li <changqing.li@windriver.com>
Date: Mon, 22 Oct 2018 15:19:51 +0800
Subject: [PATCH] python3: use cc_basename to replace CC for checking compiler
When working path contains "clang"/"gcc"/"icc", it might be part of $CC
because of the "--sysroot" parameter. That could cause judgement error
about clang/gcc/icc compilers. e.g.
When "icc" is containded in working path, below errors are reported when
compiling python3:
x86_64-wrs-linux-gcc: error: strict: No such file or directory
x86_64-wrs-linux-gcc: error: unrecognized command line option '-fp-model'
Here use cc_basename to replace CC for checking compiler to avoid such
kind of issue.
Upstream-Status: Submitted [https://github.com/python/cpython/pull/96399]
Signed-off-by: Li Zhou <li.zhou@windriver.com>
patch originally from Li Zhou, I just rework it to new version
Signed-off-by: Changqing Li <changqing.li@windriver.com>
---
configure.ac | 19 ++++++++++---------
1 file changed, 10 insertions(+), 9 deletions(-)
diff --git a/configure.ac b/configure.ac
index 90008bc..bf56195 100644
--- a/configure.ac
+++ b/configure.ac
@@ -134,6 +134,7 @@ AC_CONFIG_HEADERS([pyconfig.h])
AC_CANONICAL_HOST
AC_SUBST(build)
AC_SUBST(host)
+LT_INIT
AS_VAR_IF([cross_compiling], [maybe],
[AC_MSG_ERROR([Cross compiling required --host=HOST-TUPLE and --build=ARCH])]
@@ -887,7 +888,7 @@ AC_MSG_RESULT($with_cxx_main)
preset_cxx="$CXX"
if test -z "$CXX"
then
- case "$CC" in
+ case "$cc_basename" in
gcc) AC_PATH_TOOL(CXX, [g++], [g++], [notfound]) ;;
cc) AC_PATH_TOOL(CXX, [c++], [c++], [notfound]) ;;
clang|*/clang) AC_PATH_TOOL(CXX, [clang++], [clang++], [notfound]) ;;
@@ -1300,7 +1301,7 @@ rmdir CaseSensitiveTestDir
case $ac_sys_system in
hp*|HP*)
- case $CC in
+ case $cc_basename in
cc|*/cc) CC="$CC -Ae";;
esac;;
esac
@@ -1834,7 +1835,7 @@ esac
],
[AC_MSG_RESULT(no)])
if test "$Py_LTO" = 'true' ; then
- case $CC in
+ case $cc_basename in
*clang*)
LDFLAGS_NOLTO="-fno-lto"
dnl Clang linker requires -flto in order to link objects with LTO information.
@@ -1955,7 +1956,7 @@ then
fi
fi
LLVM_PROF_ERR=no
-case $CC in
+case $cc_basename in
*clang*)
# Any changes made here should be reflected in the GCC+Darwin case below
PGO_PROF_GEN_FLAG="-fprofile-instr-generate"
@@ -2016,7 +2017,7 @@ esac
# compiler and platform. BASECFLAGS tweaks need to be made even if the
# user set OPT.
-case $CC in
+case $cc_basename in
*clang*)
cc_is_clang=1
;;
@@ -2235,7 +2236,7 @@ yes)
# ICC doesn't recognize the option, but only emits a warning
## XXX does it emit an unused result warning and can it be disabled?
- AS_CASE([$CC],
+ AS_CASE([$cc_basename],
[*icc*], [ac_cv_disable_unused_result_warning=no]
[PY_CHECK_CC_WARNING([disable], [unused-result])])
AS_VAR_IF([ac_cv_disable_unused_result_warning], [yes],
@@ -2477,7 +2478,7 @@ yes)
;;
esac
-case "$CC" in
+case "$cc_basename" in
*icc*)
# ICC needs -fp-model strict or floats behave badly
CFLAGS_NODIST="$CFLAGS_NODIST -fp-model strict"
@@ -3319,7 +3320,7 @@ then
then
LINKFORSHARED="-Wl,--export-dynamic"
fi;;
- SunOS/5*) case $CC in
+ SunOS/5*) case $cc_basename in
*gcc*)
if $CC -Xlinker --help 2>&1 | grep export-dynamic >/dev/null
then
@@ -6410,7 +6411,7 @@ if test "$ac_cv_gcc_asm_for_x87" = yes; then
# Some versions of gcc miscompile inline asm:
# http://gcc.gnu.org/bugzilla/show_bug.cgi?id=46491
# http://gcc.gnu.org/ml/gcc/2010-11/msg00366.html
- case $CC in
+ case $cc_basename in
*gcc*)
AC_MSG_CHECKING(for gcc ipa-pure-const bug)
saved_cflags="$CFLAGS"
poky/meta/recipes-devtools/python/python3/0001-setup.py-Do-not-detect-multiarch-paths-when-cross-co.patch
+42
View File
@@ -0,0 +1,42 @@
From dc966f1278c1077938626d682666767d2c8d0c72 Mon Sep 17 00:00:00 2001
From: Khem Raj <raj.khem@gmail.com>
Date: Sat, 9 Apr 2022 18:29:47 +0000
Subject: [PATCH] setup.py: Do not detect multiarch paths when cross-compiling
add_multiarch_paths() function relies on host tools like dpkg-configure
to operate, which is not good when cross compiling, since it ends up
adding native paths in includes in certain cases, e.g. when building
for aarch64 targets using aarch64 build hosts running debian-like
distributions e.g. ubuntu, it ends up adding native multiarch paths
-I/usr/include/aarch64-linux-gnu during cross compile and since arches
are so similar, cross compiler (epecially clang) is inhererently configured
with multiarch ends up adding these paths to compiler cmdline which
works ok with gcc since headers are similar but clang barfs on some gcc
extentions and build fails due to missing gnu extentions but it silently
compiles when using cross gcc.
Fixes python3 cross build by not running this funciton when cross compiling
Upstream-Status: Inappropriate [OE-Specific]
Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
setup.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/setup.py b/setup.py
index 2e7f263..f7a3d39 100644
--- a/setup.py
+++ b/setup.py
@@ -840,7 +840,8 @@ class PyBuildExt(build_ext):
# only change this for cross builds for 3.3, issues on Mageia
if CROSS_COMPILING:
self.add_cross_compiling_paths()
- self.add_multiarch_paths()
+ if not CROSS_COMPILING:
+ self.add_multiarch_paths()
self.add_ldflags_cppflags()
def init_inc_lib_dirs(self):
--
2.25.1
poky/meta/recipes-devtools/python/python3/0001-sysconfig.py-use-platlibdir-also-for-purelib.patch
+28
View File
@@ -0,0 +1,28 @@
From 9162460d81ccc725fb04a14b27d0bf4afcfb69c9 Mon Sep 17 00:00:00 2001
From: Alexander Kanavin <alex@linutronix.de>
Date: Sun, 12 Sep 2021 21:44:36 +0200
Subject: [PATCH] sysconfig.py: use platlibdir also for purelib
This is needed in multilib configurations where hardcoding 'lib'
is not correct.
Upstream-Status: Inappropriate [oe-core specific]
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
Lib/sysconfig.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py
index daf9f00..e64bcdc 100644
--- a/Lib/sysconfig.py
+++ b/Lib/sysconfig.py
@@ -27,7 +27,7 @@ _INSTALL_SCHEMES = {
'posix_prefix': {
'stdlib': '{installed_base}/{platlibdir}/python{py_version_short}',
'platstdlib': '{platbase}/{platlibdir}/python{py_version_short}',
- 'purelib': '{base}/lib/python{py_version_short}/site-packages',
+ 'purelib': '{base}/{platlibdir}/python{py_version_short}/site-packages',
'platlib': '{platbase}/{platlibdir}/python{py_version_short}/site-packages',
'include':
'{installed_base}/include/python{py_version_short}{abiflags}',
poky/meta/recipes-devtools/python/python3/0001-test_ctypes.test_find-skip-without-tools-sdk.patch
+34
View File
@@ -0,0 +1,34 @@
From 13aa6449c47980c7270dad2527c3911517bf34e6 Mon Sep 17 00:00:00 2001
From: Tim Orling <timothy.t.orling@intel.com>
Date: Fri, 18 Jun 2021 11:56:50 -0700
Subject: [PATCH] test_ctypes.test_find: skip without tools-sdk
These tests need full packagegroup-core-buildessential, the
easiest way to dynamically check for that is looking for
'tools-sdk' in IMAGE_FEATURES.
Upstream-Status: Inappropriate [oe-specific]
Signed-off-by: Tim Orling <timothy.t.orlign@intel.com>
---
Lib/ctypes/test/test_find.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/Lib/ctypes/test/test_find.py b/Lib/ctypes/test/test_find.py
index 1ff9d01..59def26 100644
--- a/Lib/ctypes/test/test_find.py
+++ b/Lib/ctypes/test/test_find.py
@@ -113,10 +113,12 @@ class FindLibraryLinux(unittest.TestCase):
# LD_LIBRARY_PATH)
self.assertEqual(find_library(libname), 'lib%s.so' % libname)
+ @unittest.skip("Needs IMAGE_FEATURE += \"tools-sdk\"")
def test_find_library_with_gcc(self):
with unittest.mock.patch("ctypes.util._findSoname_ldconfig", lambda *args: None):
self.assertNotEqual(find_library('c'), None)
+ @unittest.skip("Needs IMAGE_FEATURE += \"tools-sdk\"")
def test_find_library_with_ld(self):
with unittest.mock.patch("ctypes.util._findSoname_ldconfig", lambda *args: None), \
unittest.mock.patch("ctypes.util._findLib_gcc", lambda *args: None):
poky/meta/recipes-devtools/python/python3/0001-test_locale.py-correct-the-test-output-format.patch
+46
View File
@@ -0,0 +1,46 @@
From 46856e692377d21be3562f6f90c242f5c9594ae2 Mon Sep 17 00:00:00 2001
From: Mingli Yu <mingli.yu@windriver.com>
Date: Mon, 5 Aug 2019 15:57:39 +0800
Subject: [PATCH] test_locale.py: correct the test output format
Before this patch:
# python3 -m test -v test_locale
[snip]
test_getsetlocale_issue1813 (test.test_locale.TestMiscellaneous) ... testing with ('tr_TR', 'ISO8859-9') ok
[snip]
After this patch:
# python3 -m test -v test_locale
[snip]
test_getsetlocale_issue1813 (test.test_locale.TestMiscellaneous) ... testing with ('tr_TR', 'ISO8859-9')... ok
[snip]
Make the test ended with "... ok" is common in python
unittest world, we should make it keep consistent
with other test cases in case it may be ignored to
record in the report if we use the common filter
"... ok".
Upstream-Status: Submitted [https://github.com/python/cpython/pull/15132]
Rebased for 3.9.4, still not accepted upstream Signed-off-by: Alejandro Hernandez <alejandro@enedino.org>
Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
---
Lib/test/test_locale.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Lib/test/test_locale.py b/Lib/test/test_locale.py
index f844e62..04df0c2 100644
--- a/Lib/test/test_locale.py
+++ b/Lib/test/test_locale.py
@@ -564,7 +564,7 @@ class TestMiscellaneous(unittest.TestCase):
self.skipTest('test needs Turkish locale')
loc = locale.getlocale(locale.LC_CTYPE)
if verbose:
- print('testing with %a' % (loc,), end=' ', flush=True)
+ print('testing with %a...' % (loc,), end=' ', flush=True)
try:
locale.setlocale(locale.LC_CTYPE, loc)
except locale.Error as exc:
poky/meta/recipes-devtools/python/python3/0017-setup.py-do-not-report-missing-dependencies-for-disa.patch
+38
View File
@@ -0,0 +1,38 @@
From 311cf9abc213fcd76795cc3a25814a15fb552065 Mon Sep 17 00:00:00 2001
From: Alexander Kanavin <alex.kanavin@gmail.com>
Date: Mon, 7 Oct 2019 13:22:14 +0200
Subject: [PATCH] setup.py: do not report missing dependencies for disabled
modules
Reporting those missing dependencies is misleading as the modules would not
have been built anyway. This particularly matters in oe-core's automated
build completeness checker which relies on the report.
Upstream-Status: Inappropriate [oe-core specific]
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alejandro Hernandez Samaniego <alejandro@enedino.org>
---
setup.py | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/setup.py b/setup.py
index 934cf2e..ccf83b4 100644
--- a/setup.py
+++ b/setup.py
@@ -517,6 +517,14 @@ class PyBuildExt(build_ext):
print("%-*s %-*s %-*s" % (longest, e, longest, f,
longest, g))
+ # There is no need to report missing module dependencies,
+ # if the modules have been disabled in the first place.
+ # cannot use mods_disabled here, because remove_configured_extensions adds
+ # only disabled extensions into it (doesn't cover _dbm, _gdbm, readline
+ # we support disabling through PACKAGECONFIG)
+ sysconf_dis = sysconfig.get_config_var('MODDISABLED_NAMES').split()
+ self.missing = list(set(self.missing) - set(sysconf_dis))
+
if self.missing:
print()
print("The necessary bits to build these optional modules were not "
poky/meta/recipes-devtools/python/python3/0020-configure.ac-setup.py-do-not-add-a-curses-include-pa.patch
+49
View File
@@ -0,0 +1,49 @@
From 788cd0464ee2b175493a0167ceee8c0045ce323c Mon Sep 17 00:00:00 2001
From: Alexander Kanavin <alex.kanavin@gmail.com>
Date: Sun, 16 Feb 2020 17:50:25 +0100
Subject: [PATCH] configure.ac, setup.py: do not add a curses include path from
the host
This leads to host contamination, and particularly can cause
curses modules to fail at runtime if the host curses is configured
differently to native curses (observed on current OpenSuse Tumbleweed
as dnf failures).
Upstream-Status: Inappropriate [oe-core specific]
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
---
configure.ac | 6 ------
setup.py | 2 --
2 files changed, 8 deletions(-)
diff --git a/configure.ac b/configure.ac
index e5e3df8..bfdd987 100644
--- a/configure.ac
+++ b/configure.ac
@@ -5092,12 +5092,6 @@ then
[Define if you have struct stat.st_mtimensec])
fi
-# first curses header check
-ac_save_cppflags="$CPPFLAGS"
-if test "$cross_compiling" = no; then
- CPPFLAGS="$CPPFLAGS -I/usr/include/ncursesw"
-fi
-
AC_CHECK_HEADERS(curses.h ncurses.h)
# On Solaris, term.h requires curses.h
diff --git a/setup.py b/setup.py
index 62f0e18..c190002 100644
--- a/setup.py
+++ b/setup.py
@@ -1169,8 +1169,6 @@ class PyBuildExt(build_ext):
panel_library = 'panel'
if curses_library == 'ncursesw':
curses_defines.append(('HAVE_NCURSESW', '1'))
- if not CROSS_COMPILING:
- curses_includes.append('/usr/include/ncursesw')
# Bug 1464056: If _curses.so links with ncursesw,
# _curses_panel.so must link with panelw.
panel_library = 'panelw'
poky/meta/recipes-devtools/python/python3/12-distutils-prefix-is-inside-staging-area.patch
+58
View File
@@ -0,0 +1,58 @@
From 843574d5a5b0818e83e20f8c0389d567bd4733fb Mon Sep 17 00:00:00 2001
From: Khem Raj <raj.khem@gmail.com>
Date: Tue, 14 May 2013 15:00:26 -0700
Subject: [PATCH] python3: Add target and native recipes
Upstream-Status: Inappropriate [embedded specific]
02/2015 Rebased for Python 3.4.2
The proper prefix is inside our staging area.
Signed-Off: Michael 'Mickey' Lauer <mickey@vanille-media.de>
Signed-off-by: Phil Blundell <philb@gnu.org>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
---
Lib/distutils/sysconfig.py | 14 +++++++++++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py
index 03b8558..57d193d 100644
--- a/Lib/distutils/sysconfig.py
+++ b/Lib/distutils/sysconfig.py
@@ -272,7 +272,9 @@ def get_python_inc(plat_specific=0, prefix=None):
If 'prefix' is supplied, use it instead of sys.base_prefix or
sys.base_exec_prefix -- i.e., ignore 'plat_specific'.
"""
- if prefix is None:
+ if prefix is None and os.environ.get('STAGING_INCDIR', ""):
+ prefix = os.environ['STAGING_INCDIR'].rstrip('include')
+ elif prefix is None:
prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX
if os.name == "posix":
if python_build:
@@ -315,7 +317,13 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None):
If 'prefix' is supplied, use it instead of sys.base_prefix or
sys.base_exec_prefix -- i.e., ignore 'plat_specific'.
"""
- if prefix is None:
+ if os.environ.get('STAGING_LIBDIR', ""):
+ lib_basename = os.environ['STAGING_LIBDIR'].split('/')[-1]
+ else:
+ lib_basename = "lib"
+ if prefix is None and os.environ.get('STAGING_LIBDIR', ""):
+ prefix = os.environ['STAGING_LIBDIR'].rstrip(lib_basename)
+ elif prefix is None:
if standard_lib:
prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX
else:
@@ -329,7 +337,7 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None):
else:
# Pure Python
libdir = "lib"
- libpython = os.path.join(prefix, libdir,
+ libpython = os.path.join(prefix, lib_basename,
"python" + get_python_version())
if standard_lib:
return libpython
poky/meta/recipes-devtools/python/python3/avoid_warning_about_tkinter.patch
+30
View File
@@ -0,0 +1,30 @@
From 627b8fe6b3c11e8bb1bb1ad1d6b816b79b8dd2ce Mon Sep 17 00:00:00 2001
From: Andrei Gherzan <andrei@gherzan.ro>
Date: Mon, 28 Jan 2019 15:57:54 +0000
Subject: [PATCH] _tkinter module needs tk module along with tcl. tk is not yet
integrated in yocto so we skip the check for this module. Avoid a warning by
not adding this module to missing variable.
Upstream-Status: Inappropriate [distribution]
Also simply disable the tk module since its not in DEPENDS.
Signed-off-by: Andrei Gherzan <andrei@gherzan.ro>
---
setup.py | 1 -
1 file changed, 1 deletion(-)
diff --git a/setup.py b/setup.py
index 7555dcd..f29ac86 100644
--- a/setup.py
+++ b/setup.py
@@ -1364,7 +1364,6 @@ class PyBuildExt(build_ext):
self.detect_decimal()
self.detect_ctypes()
self.detect_multiprocessing()
- self.detect_tkinter()
self.detect_uuid()
# Uncomment the next line if you want to play with xxmodule.c
--
2.30.2
poky/meta/recipes-devtools/python/python3/cgi_py.patch
+32
View File
@@ -0,0 +1,32 @@
From 5b0d1212d661e9a8a36738279fc9109f96eebd25 Mon Sep 17 00:00:00 2001
From: Mark Hatle <mark.hatle@windriver.com>
Date: Wed, 21 Sep 2011 20:55:33 -0500
Subject: [PATCH] Lib/cgi.py: Update the script as mentioned in the comment
Upstream-Status: Inappropriate [distribution]
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
Lib/cgi.py | 11 +----------
1 file changed, 1 insertion(+), 10 deletions(-)
diff --git a/Lib/cgi.py b/Lib/cgi.py
index 6cb8cf2..a873ff3 100755
--- a/Lib/cgi.py
+++ b/Lib/cgi.py
@@ -1,13 +1,4 @@
-#! /usr/local/bin/python
-
-# NOTE: the above "/usr/local/bin/python" is NOT a mistake. It is
-# intentionally NOT "/usr/bin/env python". On many systems
-# (e.g. Solaris), /usr/local/bin is not in $PATH as passed to CGI
-# scripts, and /usr/local/bin is the default directory where Python is
-# installed, so /usr/bin/env would be unable to find python. Granted,
-# binary installations by Linux vendors often install Python in
-# /usr/bin. So let those vendors patch cgi.py to match their choice
-# of installation.
+#! /usr/bin/env python
"""Support module for CGI (Common Gateway Interface) scripts.
poky/meta/recipes-devtools/python/python3/check_build_completeness.py
Executable
+17
View File
@@ -0,0 +1,17 @@
#!/usr/bin/env python3
import sys
logfile = open(sys.argv[1]).read()
necessary_bits = logfile.find("The necessary bits to build these optional modules were not found")
to_find_bits = logfile.find("To find the necessary bits, look in setup.py in detect_modules() for the module's name.")
if necessary_bits != -1:
print("%s" %(logfile[necessary_bits:to_find_bits]))
failed_to_build = logfile.find("Failed to build these modules:")
if failed_to_build != -1:
failed_to_build_end = logfile.find("\n\n", failed_to_build)
print("%s" %(logfile[failed_to_build:failed_to_build_end]))
if necessary_bits != -1 or failed_to_build != -1:
sys.exit(1)
poky/meta/recipes-devtools/python/python3/create_manifest3.py
+444
View File
@@ -0,0 +1,444 @@
# This script is used as a bitbake task to create a new python manifest
# $ bitbake python -c create_manifest
#
# Our goal is to keep python-core as small as posible and add other python
# packages only when the user needs them, hence why we split upstream python
# into several packages.
#
# In a very simplistic way what this does is:
# Launch python and see specifically what is required for it to run at a minimum
#
# Go through the python-manifest file and launch a separate task for every single
# one of the files on each package, this task will check what was required for that
# specific module to run, these modules will be called dependencies.
# The output of such task will be a list of the modules or dependencies that were
# found for that file.
#
# Such output will be parsed by this script, we will look for each dependency on the
# manifest and if we find that another package already includes it, then we will add
# that package as an RDEPENDS to the package we are currently checking; in case we dont
# find the current dependency on any other package we will add it to the current package
# as part of FILES.
#
#
# This way we will create a new manifest from the data structure that was built during
# this process, on this new manifest each package will contain specifically only
# what it needs to run.
#
# There are some caveats which we try to deal with, such as repeated files on different
# packages, packages that include folders, wildcards, and special packages.
# Its also important to note that this method only works for python files, and shared
# libraries. Static libraries, header files and binaries need to be dealt with manually.
#
# This script differs from its python2 version mostly on how shared libraries are handled
# The manifest file for python3 has an extra field which contains the cached files for
# each package.
# Tha method to handle cached files does not work when a module includes a folder which
# itself contains the pycache folder, gladly this is almost never the case.
#
# Author: Alejandro Enedino Hernandez Samaniego <alejandro at enedino dot org>
import sys
import subprocess
import json
import os
import collections
if '-d' in sys.argv:
debugFlag = '-d'
else:
debugFlag = ''
# Get python version from ${PYTHON_MAJMIN}
pyversion = str(sys.argv[1])
# Hack to get native python search path (for folders), not fond of it but it works for now
pivot = 'recipe-sysroot-native'
for p in sys.path:
if pivot in p:
nativelibfolder = p[:p.find(pivot)+len(pivot)]
# Empty dict to hold the whole manifest
new_manifest = collections.OrderedDict()
# Check for repeated files, folders and wildcards
allfiles = []
repeated = []
wildcards = []
hasfolders = []
allfolders = []
def isFolder(value):
value = value.replace('${PYTHON_MAJMIN}',pyversion)
if os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib')) or os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib64')) or os.path.isdir(value.replace('${libdir}',nativelibfolder+'/usr/lib32')):
return True
else:
return False
def isCached(item):
if '__pycache__' in item:
return True
else:
return False
def prepend_comments(comments, json_manifest):
with open(json_manifest, 'r+') as manifest:
json_contents = manifest.read()
manifest.seek(0, 0)
manifest.write(comments + json_contents)
def print_indent(msg, offset):
for l in msg.splitlines():
msg = ' ' * offset + l
print(msg)
# Read existing JSON manifest
with open('python3-manifest.json') as manifest:
# The JSON format doesn't allow comments so we hack the call to keep the comments using a marker
manifest_str = manifest.read()
json_start = manifest_str.find('# EOC') + 6 # EOC + \n
manifest.seek(0)
comments = manifest.read(json_start)
manifest_str = manifest.read()
old_manifest = json.loads(manifest_str, object_pairs_hook=collections.OrderedDict)
#
# First pass to get core-package functionality, because we base everything on the fact that core is actually working
# Not exactly the same so it should not be a function
#
print_indent('Getting dependencies for package: core', 0)
# This special call gets the core dependencies and
# appends to the old manifest so it doesnt hurt what it
# currently holds.
# This way when other packages check for dependencies
# on the new core package, they will still find them
# even when checking the old_manifest
output = subprocess.check_output([sys.executable, 'get_module_deps3.py', 'python-core-package', '%s' % debugFlag]).decode('utf8')
for coredep in output.split():
coredep = coredep.replace(pyversion,'${PYTHON_MAJMIN}')
if isCached(coredep):
if coredep not in old_manifest['core']['cached']:
old_manifest['core']['cached'].append(coredep)
else:
if coredep not in old_manifest['core']['files']:
old_manifest['core']['files'].append(coredep)
# The second step is to loop through the existing files contained in the core package
# according to the old manifest, identify if they are modules, or some other type
# of file that we cant import (directories, binaries, configs) in which case we
# can only assume they were added correctly (manually) so we ignore those and
# pass them to the manifest directly.
for filedep in old_manifest['core']['files']:
if isFolder(filedep):
if isCached(filedep):
if filedep not in old_manifest['core']['cached']:
old_manifest['core']['cached'].append(filedep)
else:
if filedep not in old_manifest['core']['files']:
old_manifest['core']['files'].append(filedep)
continue
if '${bindir}' in filedep:
if filedep not in old_manifest['core']['files']:
old_manifest['core']['files'].append(filedep)
continue
if filedep == '':
continue
if '${includedir}' in filedep:
if filedep not in old_manifest['core']['files']:
old_manifest['core']['files'].append(filedep)
continue
# Get actual module name , shouldnt be affected by libdir/bindir, etc.
pymodule = os.path.splitext(os.path.basename(os.path.normpath(filedep)))[0]
# We now know that were dealing with a python module, so we can import it
# and check what its dependencies are.
# We launch a separate task for each module for deterministic behavior.
# Each module will only import what is necessary for it to work in specific.
# The output of each task will contain each module's dependencies
print_indent('Getting dependencies for module: %s' % pymodule, 2)
output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule, '%s' % debugFlag]).decode('utf8')
print_indent('The following dependencies were found for module %s:\n' % pymodule, 4)
print_indent(output, 6)
for pymodule_dep in output.split():
pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}')
if isCached(pymodule_dep):
if pymodule_dep not in old_manifest['core']['cached']:
old_manifest['core']['cached'].append(pymodule_dep)
else:
if pymodule_dep not in old_manifest['core']['files']:
old_manifest['core']['files'].append(pymodule_dep)
# At this point we are done with the core package.
# The old_manifest dictionary is updated only for the core package because
# all others will use this a base.
print('\n\nChecking for directories...\n')
# To improve the script speed, we check which packages contain directories
# since we will be looping through (only) those later.
for pypkg in old_manifest:
for filedep in old_manifest[pypkg]['files']:
if isFolder(filedep):
print_indent('%s is a directory' % filedep, 2)
if pypkg not in hasfolders:
hasfolders.append(pypkg)
if filedep not in allfolders:
allfolders.append(filedep)
# This is the main loop that will handle each package.
# It works in a similar fashion than the step before, but
# we will now be updating a new dictionary that will eventually
# become the new manifest.
#
# The following loops though all packages in the manifest,
# through all files on each of them, and checks whether or not
# they are modules and can be imported.
# If they can be imported, then it checks for dependencies for
# each of them by launching a separate task.
# The output of that task is then parsed and the manifest is updated
# accordingly, wether it should add the module on FILES for the current package
# or if that module already belongs to another package then the current one
# will RDEPEND on it
for pypkg in old_manifest:
# Use an empty dict as data structure to hold data for each package and fill it up
new_manifest[pypkg] = collections.OrderedDict()
new_manifest[pypkg]['summary'] = old_manifest[pypkg]['summary']
new_manifest[pypkg]['rdepends'] = []
new_manifest[pypkg]['files'] = []
new_manifest[pypkg]['cached'] = old_manifest[pypkg]['cached']
# All packages should depend on core
if pypkg != 'core':
new_manifest[pypkg]['rdepends'].append('core')
new_manifest[pypkg]['cached'] = []
print('\n')
print('--------------------------')
print('Handling package %s' % pypkg)
print('--------------------------')
# Handle special cases, we assume that when they were manually added
# to the manifest we knew what we were doing.
special_packages = ['misc', 'modules', 'dev', 'tests']
if pypkg in special_packages or 'staticdev' in pypkg:
print_indent('Passing %s package directly' % pypkg, 2)
new_manifest[pypkg] = old_manifest[pypkg]
continue
for filedep in old_manifest[pypkg]['files']:
# We already handled core on the first pass, we can ignore it now
if pypkg == 'core':
if filedep not in new_manifest[pypkg]['files']:
new_manifest[pypkg]['files'].append(filedep)
continue
# Handle/ignore what we cant import
if isFolder(filedep):
new_manifest[pypkg]['files'].append(filedep)
# Asyncio (and others) are both the package and the folder name, we should not skip those...
path,mod = os.path.split(filedep)
if mod != pypkg:
continue
if '${bindir}' in filedep:
if filedep not in new_manifest[pypkg]['files']:
new_manifest[pypkg]['files'].append(filedep)
continue
if filedep == '':
continue
if '${includedir}' in filedep:
if filedep not in new_manifest[pypkg]['files']:
new_manifest[pypkg]['files'].append(filedep)
continue
# Get actual module name , shouldnt be affected by libdir/bindir, etc.
# We need to check if the imported module comes from another (e.g. sqlite3.dump)
path, pymodule = os.path.split(filedep)
path = os.path.basename(path)
pymodule = os.path.splitext(os.path.basename(pymodule))[0]
# If this condition is met, it means we need to import it from another module
# or its the folder itself (e.g. unittest)
if path == pypkg:
if pymodule:
pymodule = path + '.' + pymodule
else:
pymodule = path
# We now know that were dealing with a python module, so we can import it
# and check what its dependencies are.
# We launch a separate task for each module for deterministic behavior.
# Each module will only import what is necessary for it to work in specific.
# The output of each task will contain each module's dependencies
print_indent('\nGetting dependencies for module: %s' % pymodule, 2)
output = subprocess.check_output([sys.executable, 'get_module_deps3.py', '%s' % pymodule, '%s' % debugFlag]).decode('utf8')
print_indent('The following dependencies were found for module %s:\n' % pymodule, 4)
print_indent(output, 6)
reportFILES = []
reportRDEPS = []
for pymodule_dep in output.split():
# Warning: This first part is ugly
# One of the dependencies that was found, could be inside of one of the folders included by another package
# We need to check if this happens so we can add the package containing the folder as an rdependency
# e.g. Folder encodings contained in codecs
# This would be solved if no packages included any folders
# This can be done in two ways:
# 1 - We assume that if we take out the filename from the path we would get
# the folder string, then we would check if folder string is in the list of folders
# This would not work if a package contains a folder which contains another folder
# e.g. path/folder1/folder2/filename folder_string= path/folder1/folder2
# folder_string would not match any value contained in the list of folders
#
# 2 - We do it the other way around, checking if the folder is contained in the path
# e.g. path/folder1/folder2/filename folder_string= path/folder1/folder2
# is folder_string inside path/folder1/folder2/filename?,
# Yes, it works, but we waste a couple of milliseconds.
pymodule_dep = pymodule_dep.replace(pyversion,'${PYTHON_MAJMIN}')
inFolders = False
for folder in allfolders:
# The module could have a directory named after it, e.g. xml, if we take out the filename from the path
# we'll end up with ${libdir}, and we want ${libdir}/xml
if isFolder(pymodule_dep):
check_path = pymodule_dep
else:
check_path = os.path.dirname(pymodule_dep)
if folder in check_path :
inFolders = True # Did we find a folder?
folderFound = False # Second flag to break inner for
# Loop only through packages which contain folders
for pypkg_with_folder in hasfolders:
if (folderFound == False):
# print('Checking folder %s on package %s' % (pymodule_dep,pypkg_with_folder))
for folder_dep in old_manifest[pypkg_with_folder]['files'] or folder_dep in old_manifest[pypkg_with_folder]['cached']:
if folder_dep == folder:
print ('%s directory found in %s' % (folder, pypkg_with_folder))
folderFound = True
if pypkg_with_folder not in new_manifest[pypkg]['rdepends'] and pypkg_with_folder != pypkg:
new_manifest[pypkg]['rdepends'].append(pypkg_with_folder)
else:
break
# A folder was found so we're done with this item, we can go on
if inFolders:
continue
# No directories beyond this point
# We might already have this module on the dictionary since it could depend on a (previously checked) module
if pymodule_dep not in new_manifest[pypkg]['files'] and pymodule_dep not in new_manifest[pypkg]['cached']:
# Handle core as a special package, we already did it so we pass it to NEW data structure directly
if pypkg == 'core':
print('Adding %s to %s FILES' % (pymodule_dep, pypkg))
if pymodule_dep.endswith('*'):
wildcards.append(pymodule_dep)
if isCached(pymodule_dep):
new_manifest[pypkg]['cached'].append(pymodule_dep)
else:
new_manifest[pypkg]['files'].append(pymodule_dep)
# Check for repeated files
if pymodule_dep not in allfiles:
allfiles.append(pymodule_dep)
else:
if pymodule_dep not in repeated:
repeated.append(pymodule_dep)
else:
# Last step: Figure out if we this belongs to FILES or RDEPENDS
# We check if this module is already contained on another package, so we add that one
# as an RDEPENDS, or if its not, it means it should be contained on the current
# package, and we should add it to FILES
for possible_rdep in old_manifest:
# Debug
# print('Checking %s ' % pymodule_dep + ' in %s' % possible_rdep)
if pymodule_dep in old_manifest[possible_rdep]['files'] or pymodule_dep in old_manifest[possible_rdep]['cached']:
# Since were nesting, we need to check its not the same pypkg
if(possible_rdep != pypkg):
if possible_rdep not in new_manifest[pypkg]['rdepends']:
# Add it to the new manifest data struct as RDEPENDS since it contains something this module needs
reportRDEPS.append('Adding %s to %s RDEPENDS, because it contains %s\n' % (possible_rdep, pypkg, pymodule_dep))
new_manifest[pypkg]['rdepends'].append(possible_rdep)
break
else:
# Since this module wasnt found on another package, it is not an RDEP,
# so we add it to FILES for this package.
# A module shouldn't contain itself (${libdir}/python3/sqlite3 shouldnt be on sqlite3 files)
if os.path.basename(pymodule_dep) != pypkg:
reportFILES.append(('Adding %s to %s FILES\n' % (pymodule_dep, pypkg)))
if isCached(pymodule_dep):
new_manifest[pypkg]['cached'].append(pymodule_dep)
else:
new_manifest[pypkg]['files'].append(pymodule_dep)
if pymodule_dep.endswith('*'):
wildcards.append(pymodule_dep)
if pymodule_dep not in allfiles:
allfiles.append(pymodule_dep)
else:
if pymodule_dep not in repeated:
repeated.append(pymodule_dep)
print('\n')
print('#################################')
print('Summary for module %s' % pymodule)
print('FILES found for module %s:' % pymodule)
print(''.join(reportFILES))
print('RDEPENDS found for module %s:' % pymodule)
print(''.join(reportRDEPS))
print('#################################')
print('The following FILES contain wildcards, please check if they are necessary')
print(wildcards)
print('The following FILES contain folders, please check if they are necessary')
print(hasfolders)
# Sort it just so it looks nicer
for pypkg in new_manifest:
new_manifest[pypkg]['files'].sort()
new_manifest[pypkg]['cached'].sort()
new_manifest[pypkg]['rdepends'].sort()
# Create the manifest from the data structure that was built
with open('python3-manifest.json.new','w') as outfile:
json.dump(new_manifest,outfile, indent=4)
outfile.write('\n')
prepend_comments(comments,'python3-manifest.json.new')
if (repeated):
error_msg = '\n\nERROR:\n'
error_msg += 'The following files were found in more than one package),\n'
error_msg += 'this is likely to happen when new files are introduced after an upgrade,\n'
error_msg += 'please check which package should get it,\n modify the manifest accordingly and re-run the create_manifest task:\n'
error_msg += '\n'.join(repeated)
error_msg += '\n'
sys.exit(error_msg)
poky/meta/recipes-devtools/python/python3/crosspythonpath.patch
+34
View File
@@ -0,0 +1,34 @@
From 7d5fc6a86103d9bd4a274e9fd31b6987e39998a1 Mon Sep 17 00:00:00 2001
From: Ricardo Ribalda <ricardo@ribalda.com>
Date: Tue, 18 Nov 2014 03:35:33 -0500
Subject: [PATCH] configure.ac: add CROSSPYTHONPATH into PYTHONPATH for
PYTHON_FOR_BUILD
When building x86->x86 the system will try to execute .so and related items
from the default PYTHONPATH. This will fail if the target CPU contains
instructions that the host CPU does not have, add CROSSPYTHONPATH
into PYTHONPATH so we can prepend the list to find correct libs.
Upstream-Status: Inappropriate [OE-Core integration specific]
Credits-to: Mark Hatle <mark.hatle@windriver.com>
Credits-to: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Ricardo Ribalda <ricardo@ribalda.com>
---
configure.ac | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/configure.ac b/configure.ac
index 085fc0b..22790d7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -163,7 +163,7 @@ AC_ARG_WITH(
dnl Build Python interpreter is used for regeneration and freezing.
ac_cv_prog_PYTHON_FOR_REGEN=$with_build_python
PYTHON_FOR_FREEZE="$with_build_python"
- PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata_$(ABIFLAGS)_$(MACHDEP)_$(MULTIARCH) '$with_build_python
+ PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(CROSSPYTHONPATH):$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata_$(ABIFLAGS)_$(MACHDEP)_$(MULTIARCH) '$with_build_python
AC_MSG_RESULT([$with_build_python])
], [
AS_VAR_IF([cross_compiling], [yes],
poky/meta/recipes-devtools/python/python3/deterministic_imports.patch
+32
View File
@@ -0,0 +1,32 @@
There are two issues here. Firstly, the modules are accessed in on disk order. This
means behaviour seen on one system might not reproduce on another and is a real headache.
Secondly, empty directories left behind by previous modules might be looked at. This
has caused a long string of different issues for us.
As a result, patch this to a behaviour which works for us.
Upstream-Status: Pending [need to talk to upstream to see if they'll take one or both fixes]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Index: Python-3.10.4/Lib/importlib/metadata/__init__.py
===================================================================
--- Python-3.10.4.orig/Lib/importlib/metadata/__init__.py
+++ Python-3.10.4/Lib/importlib/metadata/__init__.py
@@ -819,7 +819,14 @@ class Lookup:
self.infos = FreezableDefaultDict(list)
self.eggs = FreezableDefaultDict(list)
- for child in path.children():
+ for child in sorted(path.children()):
+ childpath = pathlib.Path(path.root, child)
+ try:
+ if childpath.is_dir() and not any(childpath.iterdir()):
+ # Empty directories aren't interesting
+ continue
+ except PermissionError:
+ continue
low = child.lower()
if low.endswith((".dist-info", ".egg-info")):
# rpartition is faster than splitext and suitable for this purpose.
poky/meta/recipes-devtools/python/python3/get_module_deps3.py
+174
View File
@@ -0,0 +1,174 @@
# This script is launched on separate task for each python module
# It checks for dependencies for that specific module and prints
# them out, the output of this execution will have all dependencies
# for a specific module, which will be parsed an dealt on create_manifest.py
#
# Author: Alejandro Enedino Hernandez Samaniego <alejandro at enedino dot org>
import sys
import os
# We can get a log per module, for all the dependencies that were found, but its messy.
if '-d' in sys.argv:
debug = True
else:
debug = False
# We can get a list of the modules which are currently required to run python
# so we run python-core and get its modules, we then import what we need
# and check what modules are currently running, if we substract them from the
# modules we had initially, we get the dependencies for the module we imported.
# We use importlib to achieve this, so we also need to know what modules importlib needs
import importlib
core_deps = set(sys.modules)
def fix_path(dep_path):
import os
# We DONT want the path on our HOST system
pivot = 'recipe-sysroot-native'
dep_path = dep_path[dep_path.find(pivot)+len(pivot):]
if '/usr/bin' in dep_path:
dep_path = dep_path.replace('/usr/bin','${bindir}')
# Handle multilib, is there a better way?
if '/usr/lib32' in dep_path:
dep_path = dep_path.replace('/usr/lib32','${libdir}')
if '/usr/lib64' in dep_path:
dep_path = dep_path.replace('/usr/lib64','${libdir}')
if '/usr/lib' in dep_path:
dep_path = dep_path.replace('/usr/lib','${libdir}')
if '/usr/include' in dep_path:
dep_path = dep_path.replace('/usr/include','${includedir}')
if '__init__.' in dep_path:
dep_path = os.path.split(dep_path)[0]
return dep_path
# Module to import was passed as an argument
current_module = str(sys.argv[1]).rstrip()
if debug == True:
log = open('temp/log_%s' % current_module.strip('.*'),'w')
log.write('Module %s generated the following dependencies:\n' % current_module)
try:
m = importlib.import_module(current_module)
# handle python packages which may not include all modules in the __init__
if hasattr(m, '__file__') and os.path.basename(m.__file__) == "__init__.py":
modulepath = os.path.dirname(m.__file__)
for i in os.listdir(modulepath):
if i.startswith("_") or not(i.endswith(".py")):
continue
submodule = "{}.{}".format(current_module, i[:-3])
try:
importlib.import_module(submodule)
except:
pass # ignore all import or other exceptions raised during import
except ImportError as e:
if debug == True:
log.write('Module was not found\n')
pass
# Get current module dependencies, dif will contain a list of specific deps for this module
module_deps = set(sys.modules)
# We handle the core package (1st pass on create_manifest.py) as a special case
if current_module == 'python-core-package':
dif = core_deps
else:
# We know this is not the core package, so there must be a difference.
dif = module_deps-core_deps
# Check where each dependency came from
for item in dif:
# Main module returns script filename, __main matches mp_main__ as well
if 'main__' in item:
continue
dep_path = ''
try:
if debug == True:
log.write('\nCalling: sys.modules[' + '%s' % item + '].__file__\n')
dep_path = sys.modules['%s' % item].__file__
except AttributeError as e:
# Deals with thread (builtin module) not having __file__ attribute
if debug == True:
log.write(item + ' ')
log.write(str(e))
log.write('\n')
pass
except NameError as e:
# Deals with NameError: name 'dep_path' is not defined
# because module is not found (wasn't compiled?), e.g. bddsm
if debug == True:
log.write(item+' ')
log.write(str(e))
pass
if dep_path == '':
continue
if debug == True:
log.write('Dependency path found:\n%s\n' % dep_path)
# Site-customize is a special case since we (OpenEmbedded) put it there manually
if 'sitecustomize' in dep_path:
dep_path = '${libdir}/python${PYTHON_MAJMIN}/sitecustomize.py'
# Prints out result, which is what will be used by create_manifest
print (dep_path)
continue
dep_path = fix_path(dep_path)
import sysconfig
soabi = sysconfig.get_config_var('SOABI')
# Check if its a shared library and deconstruct it
if soabi in dep_path:
if debug == True:
log.write('Shared library found in %s\n' % dep_path)
dep_path = dep_path.replace(soabi,'*')
print (dep_path)
continue
if "_sysconfigdata" in dep_path:
dep_path = dep_path.replace(sysconfig._get_sysconfigdata_name(), "_sysconfigdata*")
if debug == True:
log.write(dep_path+'\n')
# Prints out result, which is what will be used by create_manifest
print (dep_path)
cpython_tag = sys.implementation.cache_tag
cached = ''
# Theres no naive way to find *.pyc files on python3
try:
if debug == True:
log.write('\nCalling: sys.modules[' + '%s' % item + '].__cached__\n')
cached = sys.modules['%s' % item].__cached__
except AttributeError as e:
# Deals with thread (builtin module) not having __cached__ attribute
if debug == True:
log.write(item + ' ')
log.write(str(e))
log.write('\n')
pass
except NameError as e:
# Deals with NameError: name 'cached' is not defined
if debug == True:
log.write(item+' ')
log.write(str(e))
pass
if cached is not None:
if debug == True:
log.write(cached + '\n')
cached = fix_path(cached)
cached = cached.replace(cpython_tag,'*')
if "_sysconfigdata" in cached:
cached = cached.replace(sysconfig._get_sysconfigdata_name(), "_sysconfigdata*")
print (cached)
if debug == True:
log.close()
poky/meta/recipes-devtools/python/python3/makerace.patch
+32
View File
@@ -0,0 +1,32 @@
From dde5cb74f55b6dd39d25cff639d16940d9dad505 Mon Sep 17 00:00:00 2001
From: Richard Purdie <richard.purdie@linuxfoundation.org>
Date: Tue, 13 Jul 2021 23:19:29 +0100
Subject: [PATCH] python3: Fix make race
libainstall installs python-config.py but the .pyc cache files are generated
by the libinstall target. This means some builds may not generate the pyc files
for python-config.py depending on the order things happen in. This means builds
are not always reproducible.
Add a dependency to avoid the race.
Upstream-Status: Pending
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
Makefile.pre.in | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Makefile.pre.in b/Makefile.pre.in
index c6d7e85..205af6c 100644
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -2045,7 +2045,7 @@ TESTSUBDIRS= ctypes/test \
unittest/test/testmock
TEST_MODULES=@TEST_MODULES@
-libinstall: all $(srcdir)/Modules/xxmodule.c
+libinstall: all $(srcdir)/Modules/xxmodule.c libainstall
@for i in $(SCRIPTDIR) $(LIBDEST); \
do \
if test ! -d $(DESTDIR)$$i; then \
poky/meta/recipes-devtools/python/python3/python3-manifest.json
+1361
View File
File diff suppressed because it is too large Load Diff
poky/meta/recipes-devtools/python/python3/reformat_sysconfig.py
+21
View File
@@ -0,0 +1,21 @@
#! /usr/bin/env python3
#
# SPDX-License-Identifier: MIT
#
# Copyright 2019 by Garmin Ltd. or its subsidiaries
#
# A script to reformat python sysconfig
import sys
import pprint
l = {}
g = {}
with open(sys.argv[1], 'r') as f:
exec(f.read(), g, l)
with open(sys.argv[1], 'w') as f:
for k in sorted(l.keys()):
f.write('%s = ' % k)
pprint.pprint(l[k], stream=f, width=1)
f.write('\n')
poky/meta/recipes-devtools/python/python3/run-ptest
+3
View File
@@ -0,0 +1,3 @@
#!/bin/sh
{ SETUPTOOLS_USE_DISTUTILS=nonlocal python3 -m test -v -j 4 || echo "FAIL: python3" ; } | sed -u -e '/\.\.\. ok/ s/^/PASS: /g' -r -e '/\.\.\. (ERROR|FAIL)/ s/^/FAIL: /g' -e '/\.\.\. skipped/ s/^/SKIP: /g' -e 's/ \.\.\. ok//g' -e 's/ \.\.\. ERROR//g' -e 's/ \.\.\. FAIL//g' -e 's/ \.\.\. skipped//g'