How to run uwsgi with different virtualenv

Support System Administrator
1

The point I’d like to discuss with the community is about uwsgi and virtualenv.

I use virtualenv to run several trytond servers with different python and dependencies sets.
Trytond servers run in parallel and listen to different ports.

I want to use uwsgi in this context, for some of these Trytond servers. Other less-critical servers will stay on vanilla werkzeug for now.
I plan to pip install uwsgi in every environment and set a uwsgi config in every environment.

Does it make sens for you ? Do some of you have an experience of something similar ? Will my uwsgi implement impact the way I start workers for a database (sorry if it is a completely stupid question) ?

Thank you for any opinion/piece of advice.

2

I’m doing exactly the same, but then with Gunicorn so I don’t need to install development packages to get uwsgi compiled. Also to me it seems that Gunicorn out-of-the-box is faster then uwsgi. This is how I install Tryton in production environments.

So in short: I create a virtual environment, install Tryton and the different modules, install Gunicorn, create the different configuration files and start running. I also created some systemd service and socket files to start and run Tryton automatically. I create a configuration for NginX so TLS/SSL can be handled by NginX and requests are then offloaded to Tryton. Instead of listening and sending to ports, I create sockets in a specific directory so NginX is allowed to read / write to them. The cron and workers are also started through systemd service files.

Most of the time there is a production version and a testversion. Both are running on the same PostgreSQL databaseserver, but with different databases and users to prevent things messing up.

I have even a special database server VM where all the databases live for a company. So not only Tryton but several others. This all internal on the same hardware.

3

Thank you for this description.

What does the Trytond start command look like with Gunicorn ?

Do you use also proteus in this context ?

4

I installed Gunicorn with gevent, keep that in mind. Also you have to have a gunicorn.conf.py in the root of your virtual environment. In that file you have at minimum:

import os
os.environ['TRYTOND_CONFIG'] = '/<your-absolute-location>/trytond.conf'

Then you can start Gunicorn with:

/<your-absolute-location>/bin/gunicorn --workers=5 --worker-class=gevent 'trytond.application:app'

Above is the command from my systemd file, maybe you have to bind to a port as well. Read Gunicorn - WSGI server — Gunicorn 21.2.0 documentation and Running Gunicorn — Gunicorn 21.2.0 documentation

Proteus is a script which cannot be run by Gunicorn. I use Proteus very often on a client, which works well. If you want to run Proteus in the background you have to embed it into Flask or another framework. You then start that framework also with Gunicorn.

5

Thank you very much for your detailled information
.
Your Gunicorn command is very explicit and tells a lot to me.

It seems (from documentations) that all traffic must then go through the wsgi server.

My understanding of Proteus was that it had a direct access to the database and I don’t remember if I always had a trytond process alive when running a proteus script.

In this case, Proteus would bypass the wsgi server layer. Am I right ?

Should I stop the trytond/wsgi service, and eventually restart it without wsgi layer, while I execute a Proteus script ?

Thank you to anybody for any complement.

6

I’m using Proteus always as it was a client. So not connecting directly to the database, but using the Tryton service. A little idea:

from proteus import Model, config

# Tryton server connection settings
HOST = 'my-server'
PORT = '80'
DB = 'mydatabase'
USER = 'the user'
PASSWORD = 'very secret something'

if __name__ == '__main__':
    config.set_xmlrpc(
        url='http://{user}:{password}@{host}:{port}/{db}/'.format(
            user=USER, password=PASSWORD, host=HOST, port=PORT, db=DB))

    Sale = Model.get('sale.sale')
    # etc .... you know the drill

With this you connect to Gunicorn or NginX not bypassing anything and acting as a client.

Yes, but then Proteus has to run on the same server as well. In this case it imports Tryton as a library and running in a completely different process.

This is not needed. Just connect Proteus as you always do and eventually connect directly to the database.

7

Thank you.
Indeed, my specific policy requires Proteus to run in the same virtualenv as trytond process.

Thank you again for all these effective answers.

8

I put in my trytond configuration directly into the gunicorn.conf.py file.
Like this:

PYTHONOPTIMIZE=2
daemon = True
threads = 3
bind = ['<ip>']
wsgi_app = 'trytond.application:app'
raw_env = [
        'TRYTOND_DATABASE_URI=postgresql://<usr>@/',
        'TRYTOND_DATABASE_PATH=/var/tryton/run',
        'TRYTOND_EMAIL__FROM=<email>',
        'TRYTOND_WEB__ROOT=/var/tryton/root',
]

Is there a reason to rely on trytond.conf?

PS: It is not clear to me why between database and uri there is supposed to be one underscore as they are supposed to have two according to config documentation, but it seems to work.

PS: trytond.application.app (as stated in the documentation) did not work with gunicorn. I do not know what the deal is, but trytond.application:app does work. Don’t ask me how many days I spent until I found this.

9

First quick and dirty try, SUCCESS !
Thanks to all information I got here, my uwsgi implementation starts at first try.
I have now to understand why/how it works, how it fails, how I monitor it, and adapt my provisioning script to automate all I want to automate.
I suppose also I’ll have to check reliability and performances.
I’ll come back here if something goes wrong or share my results. Thank you.

10
  • using workers and cron
  • running trytond-console
  • running trytond-admin
  • lots of options in the configuration
  • readability and easily move the configuration around

Which documentation? The gunicorn documentation shows clearly the : . Did you read the uwsgi documentation or moved from uwsgi to gunicorn? It can be a difference between gunicorn and uwsgi.

11

I think I globally understand wsgi.
uwsgi has a lot of options.
For example, how to pass arguments to trytond:
usage: trytond [-h] [–version] [-c FILE [FILE …]] [-v] [–dev] [-d DATABASE [DATABASE …]] [–logconf FILE] [–pidfile FILE] [–coroutine]
or how to identify all processes related to a virtualenv.
Much homework which depends on my specific settings.

12

On my ubuntu server, I just installed the uwsgi package - apt install uwsgi. No need to compile. Did I miss something?

I don’t know which is the better choice, but the uwsgi install process seems not to be an argument.

13

I’m using RockyLinux 8.8 which comes with python 3.6. But in my virtual environment I have python 3.9 because the newest version of Tryton don’t support python 3.6.
When installing uwsgi through the package manager (dnf in my case) I get uwsgi and it’s plugins for python 3.6 and it won’t work and find Tryton in my virtual environment. So you have to pip install uwsgi in your virtual environment which then needs all the compile tools to compile uwsgi. That’s why I moved to Gunicorn instead and never looked back.

14

This is not a virtualenv installation.
Lukio’s how-to suggested a system install of uwsgi would do the trick for virtualenvs, I’m pretty sure it can, because uwsgi has some advanced capabilities, but didn’t work enough to understand how.

15

I may extract part of my settings and share them. They are a bit specific, but if you look for an example, just ask.

16

This answer refers Multiple instances with VENV, nginx, uwsgi, sao, certbot - #5 by herrdeh but addresses the same topic.

Using an alternative to Werkzeug in production is a recommendation (from werkzeug), indeed.

Now this is my understanding (please community, correct me):

You don’t need virtualenvs to run different copies of trytond, sao, proteus. Start them directly from code copy. They will all run in the same system wide environment which must be compatible with all servers.

I use virtualenvs to set dependencies specifically according to each trytond service requirements. Specific python3 versions in particular.

I made the assumption that uswgi and its python3 extension are python-version sensitive. This is how I explain that pip install requires recompilation, depending on the python version used in the environment.
Therefore, I doubt a system-wide uwsgi can serve different virtualenvs if they differ by their python version. (but again, Lukio seemed affirmative in How to run trytond with nginx + supervisord + uwsgi)

17

You are right in your understanding and analysis. But running multiple instances of Tryton system wide will become a pain very fast. Even when your python version is not different from the system default, just run inside a virtual environment because you can then add dependencies and when you mess things up, just delete the environment and start again. That’s the sole reason why virtual environments exists in the first place.

Completely true. When you have a different version of Python inside your virtual environment you have to recompile uwsgi. That’s why I switched to Gunicorn (uwsgi alternative) which doesn’t need that

Again, true, but sometimes it’s not needed to have a different Python version in your virtual environment. In that case you can use your system-wide-uwsgi server with the different virtual environments.

18

Yes, with some care it works.
But system update and Tryton upgrade may be much more complex.

19

For the record 7.0 series will use Gunicorn in the docker image: Draft: Add 7.0 series (!14) · Merge requests · Tryton / Tryton Docker · GitLab

1 Like