Howto deal with Pallets Projects dependencies

It is now clear that the Pallets Projects is no more following semver.
They break API compatibility in feature release, ex:

We also have Issue 10086: Add support for werkzeug 2.0 - Tryton issue tracker which raises multiple times concern about the maintenance of older branch. Indeed the project has stated multiple times that once a new “branch” is released, all previous are deprecated and no more maintained.
And now Issue 11353: Werkzeug new release breaks the test - Tryton issue tracker.

With such behavior, I think we could not rely on the release number to have any meaning and we should consider that it is like a pre-stable project.
So I think we should try our best to make all supported Tryton series compatible with the latest version of the pallets projects releases but also with the largest possible older release (by using fallback, API test etc.).
If we agree on this rule, we should backport all the fixes.

Here another statement with regard to the subject Importing jinja2 fails with ImportError from MarkupSafe #1596 .

I fully agree.

1 Like

I think we do not have any other option as it is clear they do not follow semver and they do not want to support older series.

Agree but as it is clear that they only support latest released serie we should make our latest version compatible only with newer releases and remove all the code compatible to older releases. Does it make sense for you?

No it is not because they are bad at maintaining older releases that we should be bad.
For me we must define a minimal supported version. For werkzeug I would say 0.11 which was released 2 years before our 5.0. which seems reasonable guess about what people would have on system when they upgraded to 5.0. We may update to 0.15 for 6.0 series.

Now I’m wondering if we should not have a dedicated test which force to install the lower supported version to ensure we are still compatible with it.

Also I would be in favor of remove dependencies to werkzeug for modules. Instead of importing from werkzeug we should provide a cleaned API for the tools we need in modules like: exceptions, response and redirect. This will help us to maintain support for various version of werkzeug in a single place.

Does it make sense to use the version included on debian stable on the time of the LTS version is released? That will be a good reference for stable points.

And what about 6.2 and 6.4 series? As far as 1.0 and 2.0 is should we use that version or just keep the old dependencies?

Makes sense for me, but probably just for trytond and not for all modules.

It also make sense. But do you want to backport such API or just use it for newer series? I will consider backporting it to ease the maintenance of older series.

Well it is an indication but we should not tie ourselves to rules that we do not control.

For me supporting the most version possible is the best. So it will be a trade-off between code complexity and maintenance. But at some point it will be easier for maintenance that between LTS we do not change the minimal version supported.

I think we can go with only new series because back porting will create a dependency between bugfix release of module and bugfix release of trytond.
Also for now what we used in those module has not been a problem.

And why not dropping Werkzeug, in favor of something more reliable?

What we need is:

  • a Request object
  • a Response object
  • some routing system
  • a minimal server for development
  • a project that follow semver

As there is no shortage of WSGI framework in python from the minimalist bottle to Pyramid which is a complete web framework, there’s probably one that is well suited for our needs.

We do not need a web framework but a WSGI library which allows use to expose a WSGI app to any WSGI server.
So for me the only alternative I know is wsgiref but we will need to create some tools to simplify the management (like werkzeug did).

I know but we don’t have to use all the bell and whistle from the project we chose just like we don’t use the debugger of Werkzeug.

I guess that’s what bottle is doing also since they only depend on the standard lib. So we might as well reuse what they’re doing and contribute back if necessary.

Pyramid is using https://webob.org/

Which also doesn’t depend on anything except the standard library.

I think that the test-system must be updated to filter out the deprecation warnings. Functionality is not removed all of a sudden, but there are deprecation warnings before. So you can update your code or pin the to the current version.

So instead of moving away, maybe a change in the test-system will do the trick. You can even think of automatically creating an issue when a deprecation warning pops up (be careful with that it may flood the issue tracker).

And what will be the cost of such migration?

How do you make that tests are failing because our code is using a deprecated feature of a library and not when it is a external dependency?

Pinning dependency is not a solution because Tryton is a public project so we can not force everyone to use the exact version we choose. But also pinning prevent security update.
For example Pallets Project has this argument to justify to not follow the semver, they say that user must pin their dependencies. But they do not even follow their own recommendation because their own projects (which depend on each other) are not pinned. And so some version of Flask does not work with some version of werkzeug but there is no information in the requirements about which versions work with which one.

I don’t know but it’s less than re-implementing everything from scratch and maybe comparable to providing a compatibility layer to werkzeug.

But the risk is higher since it’s a library that we haven’t work with until now.

On the other hand we wouldn’t have the burden of supporting their changes every time (provided that the behaviour of the webob devs is not the same as the one from the pallet project).

I didn’t say that the test should be failing because of a deprecation. When you look for example at https://drone.tryton.org/tryton/trytond/1054/10 you will find deprecation warnings in the log. You can filter these out and make them available somewhere else. Because when the tests are ‘green’ you generally don’t look at the log and are missing this deprecation warnings. And this goes beyond the Pallets Project.

That’s an issue.

I think this is the best solution.

I see one problem here: We have versions already released with werkzeug so untils those versions are active maintained we will need to support werkzeug. As 6.0 series is LTS version we need to keep compatibility with werkzeug for 4 years.

The change to new library will make harder to support LTS versions.

Indeed switching would make supporting previous versions really cumbersome.

As it seems to have a good consensus on this. I will backport to all supported series, the fixes:

And I have created Issue 11407: Remove direct dependency to werkzeug in modules - Tryton issue tracker for future series.