Provide development packages via trydevpi again

Feature
1

Proposal

  • Provide some means for making source development packages available for pip install.
  • Use trydevpi for providing the development packages.

Rational

Use-case

Developers neeed to use the latest development packages in CI to prepare migration to the next LTS version.

Need

  • Having development versions of the modules and trytond available is a precondition for running test against latest development tip/trunk.

  • Packages actually need to be build to be available to pip install, as using just a checkout of the mono-repository is not enough:

    • pip install --find-links requires either a html page linking to packages or a directory containing packages. Pointing to source directories are not supported.
    • pip install --index-url and pip install --extra-index both require an URL of a Package Index. 0Pointing to source directories are not supported.

    • Using editable installs (pip install -e) in CI not only requires a clone of the mono-repo, but also to create a list of recursive dependencies.

    • VCS URLs also requires to create a list of recursive dependencies. And given that Tryton uses Mercurial it is uncertain whether this still clones the whole mono-repository.

  • With the de-activation of trydevpi, development packages are no longer available easily. This results in every development team needs to create some means of making dev packages available for its projects. Doing so is a waste of rare development resources and leads to redundant evaluation, evaluation even if nothing changed and redundant data.

    Even if development packages are used less often than they used to be, it seams to be much more efficient providing them centrally and updating them only if actually changed. Otherwise each development team would need to run scheduled jobs (aka polling the upstream repo) and eventually build the packages.

What to provide

  • Source packages (sdist) are already build as part of Tryton’s CI process (in stage preparation). These artifacts could be reused.

  • Providing source packages is enough, given development packages are not used that often.
    Providing source packages only also saves CI resources.

  • Providing docker images seems overdone, as it would require additional steps for building the image. Also a single image might not suffice, as Tryton supports several versions of Python.

How to provide

  • trydevpi is still running, reactivating it should not be much of a problem

  • Means for uploading packages to trydevpi are already known

  • Using heptapods built-in package registry looks obvious, but has one major draw back: Existing package versions can not be updated. Thus updating requires quite some gitlab magic to delete the package first, which increases complexity

Implementation

Extend the .gitlab-ci.yml file by a task uploading the package to trydevpi.
Run only on branch default and if tests passed
Upload only changed packages.
Details to be discussed after proposal has been accepted…

2

This sounds like the way to go for those (???) who tries to develop based on a moving target.
This would also allow such team to use explicit changeset of the tree and update it at their own speed. This would at least provide reproductible tests.

I do not think. For me providing such service would be a bad service which makes think that it is a good idea to develop this way. It may even wrongly promote to deploy using such mechanism.

3

Hello,

When giving a try to next version, I usually clone the trunk and run it from source. It works or it doesn’t. Usually it works. I guess there are better methods, but can you tell me in a few words the benefit of your proposal ?

4

My point is about automated testing and migration.

We are currently preparing migrating our modules to 7.0, which is expected very soon. Most changes for 7.0 are in 6.9 already, thus we can already start migration and have our migration done rather soon after release. We have set up CI for testing our modules. And of course we want to test the migrated modules against 6.9.

For testing against 6.9 we need 6.9/default-branch packages, of course.

And instead of every integrator building 6.9/default-branch packages by its own, this should be provided by the Foundation as a central service. This would be much more efficient in regard to developer resources, computing resources and storage than if every integrator would develop and maintain its own infrastructure for doing so.

1 Like
5

When following default-branch, reproducible tests are not relevant. Relevant is to spot breaking changes early.

Just put a warning sign there, it you think developers are stupid :slight_smile: Anyway it’s not up to you to decide what is the develoment module others want to adhere to.

6

Ok, seems I didn’t understand what trydevpi was about.
Thank you for your answer.

7

Until the Tryton main project provides a trydevpi again, we’ve set up one at tryton.community. See Trydevpi available at tryton.community