will make my sources available as trytond modules.
This has been the aftermath of many years struggling with the legacy setup.py files from the Tryton ecosystem causing multiple issues to pipenv and poetry resolvers and venv managers.
Eventually, I discovered I didn’t need to package the modules at all, which was the comon practice in my organisation for non-tryton applications, and I was happy everafter (until the issues you linked rose).
I had gone as far as prototyping a Hatch plugin (GitHub - pypa/hatch: Modern, extensible Python project management plugin) that could replace all those setup.py files, but abandoned that initiative because I never felt Tryton has any interest in keeping up with the Python packaging evolution. I still believe it is doable.
BUT
This post kind of startled me. Like, how can a project that aims for modularity consider removing the only module registration mechanism? I’d expect moves in the quite opposite direction!
I am using the entrypoints mechanism to register plugins with trytond. Main reason is that I was not able to have my code inside of trytond.modules without trouble. My guess is that this is because it does not support to be a namespace package. So I ended up putting my code into a separate package and registering it via the entrypoint mechanism.
The main issue was with the development setup. In a regular installation I would not have the issue and be able to throw it all into trytond.modules.
In general I agree with the pytest example from the post above, it would be weird if tryton or any other project would impose package names upon me.
Actually the topic did stick with me and I just realized that I have defined my tiny customization module completely different than the typical trytond module is defined and used a pyproject.toml instead:
[project]
name = "bv-tryton-meta-botech"
version = "0.8.0"
authors = [
{ name="Johannes Bornhold", email="joh@bo-tech.de" },
]
description = "Meta package with Tryton adjustments for the use at bo-tech."
readme = "README.md"
requires-python = ">=3.8"
classifiers = [
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
]
[project.entry-points."trytond.modules"]
meta_botech = "bv.tryton.meta_botech"
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["src/bv"]
Have to admit that I am not too deep into the recent developments of packaging in Python, still, so far this works out just fine thanks to the entrypoint support.
@ced would you mind sharing a few hints about the complexities / issues around the entrypoint support? Just wondering if there may be other options to reduce the maintenance burden.
Since I was bugged again by not being able to I load a module in certain setups, I spend quite some time to analyze the root cause — which is a combination of different issues:
Official Tryton modules define themself as Python package trytond.modules.xxx and provide an entry-point of type trytond.module pointing to this very package. Many third-party modules I’ve seen follow this convention (although the distribution package might have different a name like mycoop-xxx). AFAIK this is the case ever since.
Anyhow, trytond.modules is not a namespace-module (since there is an __init__.py and this does not set up a namespace either). Which means: Python will search trytond.modules.xxx in the same directory as the file trytond.modules.__init__.py. This of course will fail except if the package was linked or copied there.
In Tryton 6.x, trytond implemented its own mechanisms to load the entry-point — and there even was a comment on why ep.load() can not be used. AFAIR this worked acceptable well — except of the drawback that the name of the directory, the module lives in must match the name of the module. (The later inhibits e.g. “src layout”.)
Now, as a work-around it was documented to install editable packages using --use-pep517 - which works: This option will create a file like __editable___trytond_xxx_7_0_0_finder.py, which in effect makes trytond.modules a namespace package
Conclusion: IMHO for 3rd-party modules the best solution is to use an entry-point and a custom package-namespace.
Anyhow, this yields to another issue: When swapping package mycoop-party-superextensions by yourcorp-party-superextensions, tests of modules using module ….party.superextensions would need to ba changed — jsut because the implementation changed.
There are three solutions I see:
Make trytond.modules a real namespace package — not good since this would require changes to all modules using functions from trytond.modules.
Introduce another “modules” namespace, e.g. trytond.modules.3rd_party, trytond.3rd_party — this would require changes to 3rd-party modules.
Make trytond.modules extend its own __path__ by collecting the paths of all trytond.module entry-points — anyhow I did not find a reasonable way to get these values from ep.dist.
Since all official modules provide entry-points, too, the code could be stripped down to only use entry-points. (And also use importlib.resources.files() for accessing the files.)
This is interesting, since I thought about such thing, too. Meanwhile I implemented Tryton Community / setuptools · GitLab (which still relies on setuptools, though).
There is nothing wrong with setuptools per se, only with the deprecated implicit setuptools.build_meta:__legacy__ fallback. For a pure PEP-517 resolver/installation frontend trying to circumvent that backend can easily go wrong.
Ultimately hatch, even from my 0-knowledge about it, felt much easier and natural to extend than the actual PEP-517 setuptools.build_meta backend, and did not feature tons of complexities accumulated throughout the Python history that were anyway unnecessary to package a pure python trytond module: we just need to build metadata mixing tryton.cfg and pyproject.toml and providing some sensible defaults!
While very breaking code-wise, a straightforward trytond.modules.import_module("mymod") was an easy replacement for from trytond.modules import mymod for me when I needed it.
I would understand reluctance against that since we would have to maintain yet an additional module loading system.
I have mixed feelings with this idea. Generally I’d be against tampering that, but importlib replacements for distutils entrypoints might have a nicer interface that could make easier that kind of hacking.
This! @ced why isn’t this the present solution? Which are the pitfalls or blockers? Can we work towards this?
as_file() indeed uses a temporary copy. Anyhow files() (which is what I proposed) returns a PosixPath instance. Thus no copy and no performance impact.
Thus it would be interesting to know in which setup you experience not getting a PosixPath. (I can imagine that dev-mode might not work if installed using zipped eggs, anyhow this could be worked around easily.)
