Proteus and trytond_import_countries. How to install?

splegla · June 26, 2020, 2:20pm

Hi,

I’ve just installed tryton and a few modules. I went to configure the party and company but it asks for country and currency. I then found that in order to populate these I need to run trytond_import_countries and trytond_import_currencies. In order to do that I needed to install proteus. My problem comes when I try to run either of those as I don’t know the correct format to specify the database. The -c /etc/tryton/trytond.conf is straightforward. I’m running postgresql locally installed on port 5432.

How do I go about actually successfully run these scripts?

Thanks

Eamonn

csotelo · June 26, 2020, 3:06pm

Have you tried xmlrpc?

config.set_xmlrpc(’http://admin:admin@localhost:8000/foo’)

I think is the most easy way to access to tryton using proteus

ced · June 26, 2020, 3:12pm

Running them with --help should give you the proper information:

$ import_countries.py --help
usage: import_countries.py [-h] [-d DATABASE] [-c CONFIG_FILE]

optional arguments:
  -h, --help            show this help message and exit
  -d DATABASE, --database DATABASE
  -c CONFIG_FILE, --config CONFIG_FILE
                        the trytond config file

splegla · June 26, 2020, 4:41pm

I have used that. That’s how I knew that I had to pass -d and -c flag. I’ve tried this:

$ trytond_import_countries -d 'postgresql://tryton:5432/' -c /etc/tryton/trytond.conf

That’s specifying the full uri as given the trytond.conf

That results in a KeyError: 'DB_NAME'

If I try just passing the database name like this:

trytond_import_countries -d 'tryton' -c /etc/tryton/trytond.conf

then I get this error:

raise IOError('Database "%s" doesn\'t exist!' % path)
OSError: Database "/var/lib/tryton/tryton.sqlite" doesn't exist!

It’s looking for a sqlite database but I’m using postgres. Is there some other configuration that needs to be done other than just installing proteus and if so what?

Thanks

Eamnn

ced · June 26, 2020, 4:49pm

This is related to Issue 9365: Support TRYTOND_DATABASE_URI as database basis for scripts - Tryton issue tracker
So prior of this issue, database should be a full URI with database name like postgresql://tryton:5432/tryton.

pokoli · June 26, 2020, 4:55pm

Do you think we should backport the new feature?
I found that there are some users that are strugled with this issue (specially when using the docker image).

ced · June 26, 2020, 4:56pm

As you said it is a new feature so it should not be backported, especially because there is a way to make it work.

splegla · June 26, 2020, 5:04pm

@ced I’ve tried that with the uri suggested and still get the same error like it’s looking for a sqlite database

`$ trytond_import_countries -d 'postgresql://tryton:5432/tryton' -c /etc/tryton/trytond.conf`

and the full traceback:

Traceback (most recent call last):
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/bin/trytond_import_countries", line 8, in <module>
    sys.exit(run())
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/modules/country/scripts/import_countries.py", line 173, in run
    main(args.database, args.config_file)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/modules/country/scripts/import_countries.py", line 149, in main
    config.set_trytond(database, config_file=config_file)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/proteus/config.py", line 293, in set_trytond
    _CONFIG.current = TrytondConfig(database, user, config_file=config_file)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/proteus/config.py", line 246, in __init__
    self.pool.init()
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/pool.py", line 161, in init
    restart = not load_modules(self.database_name, self, update=update,
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/modules/__init__.py", line 426, in load_modules
    with Transaction().start(database_name, 0):
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/transaction.py", line 107, in start
    database = backend.Database(database_name).connect()
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/backend/sqlite/database.py", line 344, in connect
    raise IOError('Database "%s" doesn\'t exist!' % path)
OSError: Database "/var/lib/tryton/tryton.sqlite" doesn't exist!

josesalvador · June 26, 2020, 5:12pm

Tryton is looking for a sqlite database:

Could you show us the content of /etct/tryton/trytond.conf file?

splegla · June 26, 2020, 6:32pm

Hi Jose,

One thing I did notice in my trytond.conf that was highlighted by your post was that /var/lib/tryton doesn’t exist. As I’m only testing this in a virtualenv I’ve changed the path to fall under my home directory. Here’s the trytond.conf

# /etc/tryton/trytond.conf - Configuration file for Tryton Server (trytond)
#
# See the Tryton documentation for a description of the options:
# https://docs.tryton.org/projects/server/en/latest/topics/configuration.html#topics-configuration

[database]
# Database related settings

# The URI to connect to the SQL database (following RFC-3986)
# uri = database://username:password@host:port/
# (Internal default: sqlite:// (i.e. a local SQLite database))
#
# PostgreSQL via Unix domain sockets
# (e.g. PostgreSQL database running on the same machine (localhost))
#uri = postgresql://tryton:tryton@/
#
#Default setting for a local postgres database
#uri = postgresql:///

#
# PostgreSQL via TCP/IP
# (e.g. connecting to a PostgreSQL database running on a remote machine or
# by means of md5 authentication. Needs PostgreSQL to be configured to accept
# those connections (pg_hba.conf).)
uri = postgresql://tryton:tryton@localhost:5432/

# The path to the directory where the Tryton Server stores files.
# The server must have write permissions to this directory.
# (Internal default: /var/lib/trytond)
path = /home/eamonn/.trytond

# Shall available databases be listed in the client?
list = True

# The number of retries of the Tryton Server when there are errors
# in a request to the database
#retry = 5

# The primary language, that is used to store entries in translatable
# fields into the database.
language = en_GB

[ssl]
# SSL settings
# Activation of SSL for all available protocols.
# Uncomment the following settings for key and certificate
# to enable SSL.

# The path to the private key
#privatekey = /etc/ssl/private/ssl-cert-snakeoil.key

# The path to the certificate
#certificate = /etc/ssl/certs/ssl-cert-snakeoil.pem

[jsonrpc]
# Settings for the JSON-RPC network interface

# The IP/host and port number of the interface
# (Internal default: localhost:8000)
#
# Listen on all interfaces (IPv4)

listen = 0.0.0.0:8000

#
# Listen on all interfaces (IPv4 and IPv6)
#listen = [::]:8000

# The hostname for this interface
#hostname =

# The root path to retrieve data for GET requests
#data = jsondata

[xmlrpc]
# Settings for the XML-RPC network interface

# The IP/host and port number of the interface
#listen = localhost:8069

[webdav]
# Settings for the WebDAV network interface

# The IP/host and port number of the interface
#listen = localhost:8080
listen = 0.0.0.0:8080

[session]
# Session settings

# The time (in seconds) until an inactive session expires
timeout = 3600

# The server administration password used by the client for
# the execution of database management tasks. It is encrypted
# using using the Unix crypt(3) routine. A password can be
# generated using the following command line (on one line):
# $ python -c 'import getpass,crypt,random,string; \
# print crypt.crypt(getpass.getpass(), \
# "".join(random.sample(string.ascii_letters + string.digits, 8)))'
# Example password with 'admin'
#super_pwd = jkUbZGvFNeugk

[email]
# Mail settings

# The URI to connect to the SMTP server.
# Available protocols are:
# - smtp: simple SMTP
# - smtp+tls: SMTP with STARTTLS
# - smtps: SMTP with SSL
#uri = smtp://localhost:25
uri = smtp://localhost:25

# The From address used by the Tryton Server to send emails.
from = tryton@<your-domain.tld>

[report]
# Report settings

# Unoconv parameters for connection to the unoconv service.
#unoconv = pipe,name=trytond;urp;StarOffice.ComponentContext

# Module settings
#
# Some modules are reading configuration parameters from this
# configuration file. These settings only apply when those modules
# are installed.
#
#[ldap_authentication]
# The URI to connect to the LDAP server.
#uri = ldap://host:port/dn?attributes?scope?filter?extensions
# A basic default URL could look like
#uri = ldap://localhost:389/

[web]
# Path for the web-frontend
#root = /usr/lib/node-modules/tryton-sao
listen = 0.0.0.0:8000
root = /home/eamonn/projects/tryton/sao

ced · June 26, 2020, 7:33pm

For me, the configuration file is not correctly read or used. It is probably an access right issue.

josesalvador · June 26, 2020, 8:51pm

Follow the @ced advice:

Your configuration file does not match your output result.

splegla · June 26, 2020, 9:30pm

As per my last response I did make a change in the config file to change the location of the path field from /var/lib/trytond to /home/eamonn/.trytond so that explains the difference in output. There error is the same just now says it’s looking for the sqlite database at /home/eamonn/.trytond

If I stop trytond and don’t pass the configuration file the sao errors out saying it can’t find sqlite. When I add the configuration file it does access postgresql and not look for sqlite and I can access tryton through the sao client. This would seem to show that it does indeed read the configuration file.

Could the issue be that tryton didn’t properly initialise because the path location in the trytond.conf was pointing to location that didn’t exist let alone be writable? If I look at the new path /home/eamonn/.tryton there are no files there yet the conf is commented to say that the path directory is where the Tryton server stores files. So perhaps these are missing or maybe it’s just empty because I’ve done little with Tryton yet so nothing needed to be stored there yet.

ced · June 26, 2020, 10:44pm

I made somemore testing to try to reproduce your behavior. The only way to get the script using sqlite instead of postgresql backend with the command line is by having the configuration file setting uri to sqlite:///.
Also the script does not really require a configuration file if the database uri is passed as parameter.

splegla · June 26, 2020, 11:12pm

Many thanks for your continued help. If I run without the config file parameter:

$ trytond_import_countries -d 'postgresql://tryton:5432/tryton'

I get the same type of error just this time it’s not looking at the path in the config file, because it wasn’t passed to it, but instead is looking at /home/eamonn/db/ which doesn’t exist:

Traceback (most recent call last):
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/bin/trytond_import_countries", line 8, in <module>
    sys.exit(run())
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/modules/country/scripts/import_countries.py", line 173, in run
    main(args.database, args.config_file)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/modules/country/scripts/import_countries.py", line 149, in main
    config.set_trytond(database, config_file=config_file)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/proteus/config.py", line 293, in set_trytond
    _CONFIG.current = TrytondConfig(database, user, config_file=config_file)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/proteus/config.py", line 246, in __init__
    self.pool.init()
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/pool.py", line 161, in init
    restart = not load_modules(self.database_name, self, update=update,
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/modules/__init__.py", line 426, in load_modules
    with Transaction().start(database_name, 0):
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/transaction.py", line 107, in start
    database = backend.Database(database_name).connect()
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/backend/sqlite/database.py", line 344, in connect
    raise IOError('Database "%s" doesn\'t exist!' % path)
OSError: Database "/home/eamonn/db/tryton.sqlite" doesn't exist!

Seems something isn’t right here as it’s still looking for sqlite even without a config file and with me passing the database argument to it which should be pointing at postgresql.

dave · June 27, 2020, 9:34am

Hi,

I am seeing exactly the same problem.

This was on version 5.6.2, using a trytond.conf containing:

[database]
uri = postgresql://tryton:5432/tryton

Running the command:

$ trytond_import_countries -d 'postgresql://tryton:5432/tryton' -c trytond.conf

On line 107 of trytond/transaction.py the config['database']['uri'] is postgresql://tryton:5432/tryton, and the database_name is tryton.

However, at this point the backend/__init__.py has already been imported, and at the time it got imported, on line 17 of backend/init.py config['database']['uri'] was sqlite://.

I think what is happening is that trytond_import_countries is importing the run function from the trytond_country module, and this is having the knock on effect of importing the backend before the config['database']['uri'] is setup:

Traceback (most recent call last):
  File "/home/tryton/reviews/test/.venv/bin/trytond_import_countries", line 5, in <module>
    from trytond.modules.country.scripts.import_countries import run
  File "/home/tryton/reviews/test/.venv/lib/python3.8/site-packages/trytond/modules/__init__.py", line 17, in <module>
    from trytond.cache import Cache
  File "/home/tryton/reviews/test/.venv/lib/python3.8/site-packages/trytond/cache.py", line 16, in <module>
    from trytond import backend
  File "/home/tryton/reviews/test/.venv/lib/python3.8/site-packages/trytond/backend/__init__.py", line 16, in <module>
    raise Exception(config['database']['uri'])
Exception: sqlite://

dave · June 27, 2020, 9:35am

I have just created Issue 9446: Incorrect database uri used when running trytond_import_countries - Tryton issue tracker for it.

dave · June 27, 2020, 9:39am

You can get around this issue by setting the TRYTOND_DATABASE_URI or TRYTOND_CONFIG environment variables before running the script, e.g.:

$ TRYTOND_DATABASE_URI='postgresql://tryton:5432/tryton' trytond_import_countries -d 'postgresql://tryton:5432/tryton'

splegla · June 27, 2020, 10:33am

Thanks @dave for the detective work. It has got me further. It no longer moans about sqlite however environment variable at the start I had to set to be the same format as in the config as it gave the following error:

connection to "tryton" failed
Traceback (most recent call last):
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/backend/postgresql/database.py", line 174, in __new__
    inst._connpool = ThreadedConnectionPool(
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/psycopg2/pool.py", line 161, in __init__
    AbstractConnectionPool.__init__(
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/psycopg2/pool.py", line 59, in __init__
    self._connect()
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/psycopg2/pool.py", line 63, in _connect
    conn = psycopg2.connect(*self._args, **self._kwargs)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/psycopg2/__init__.py", line 127, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not translate host name "tryton" to address: Name or service not known

Traceback (most recent call last):
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/bin/trytond_import_countries", line 8, in <module>
    sys.exit(run())
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/modules/country/scripts/import_countries.py", line 173, in run
    main(args.database, args.config_file)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/modules/country/scripts/import_countries.py", line 149, in main
    config.set_trytond(database, config_file=config_file)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/proteus/config.py", line 293, in set_trytond
    _CONFIG.current = TrytondConfig(database, user, config_file=config_file)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/proteus/config.py", line 246, in __init__
    self.pool.init()
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/pool.py", line 161, in init
    restart = not load_modules(self.database_name, self, update=update,
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/modules/__init__.py", line 426, in load_modules
    with Transaction().start(database_name, 0):
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/transaction.py", line 107, in start
    database = backend.Database(database_name).connect()
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/backend/postgresql/database.py", line 174, in __new__
    inst._connpool = ThreadedConnectionPool(
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/psycopg2/pool.py", line 161, in __init__
    AbstractConnectionPool.__init__(
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/psycopg2/pool.py", line 59, in __init__
    self._connect()
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/psycopg2/pool.py", line 63, in _connect
    conn = psycopg2.connect(*self._args, **self._kwargs)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/psycopg2/__init__.py", line 127, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not translate host name "tryton" to address: Name or service not known

So I changed to this:

$ TRYTOND_DATABASE_URI='postgresql://tryton:tryton@localhost:5432/' trytond_import_countries -d 'postgresql://tryton:5432/tryton'

however it seems I made an error initially on install as I set the language in the config to en_GB but judging by the error below that’s not a valid translation:

Update countries
Traceback (most recent call last):
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/bin/trytond_import_countries", line 8, in <module>
    sys.exit(run())
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/modules/country/scripts/import_countries.py", line 173, in run
    main(args.database, args.config_file)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/modules/country/scripts/import_countries.py", line 151, in main
    do_import()
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/modules/country/scripts/import_countries.py", line 156, in do_import
    countries = update_countries(countries)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/modules/country/scripts/import_countries.py", line 60, in update_countries
    Country.save(records)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/proteus/__init__.py", line 104, in newfunc
    return self.func(owner, *args, **kwargs)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/proteus/__init__.py", line 847, in save
    ids = proxy.create(values, context)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/proteus/config.py", line 188, in __call__
    result = rpc.result(meth(*args, **kwargs))
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/modules/country/country.py", line 70, in create
    return super(Country, cls).create(vlist)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/model/modelsql.py", line 159, in wrapper
    return func(cls, *args, **kwargs)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/model/modelsql.py", line 658, in create
    Translation.set_ids(name, 'model', Transaction().language,
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/ir/translation.py", line 494, in set_ids
    cls.save(to_save)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/model/descriptors.py", line 33, in newfunc
    return self.func(owner, *args, **kwargs)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/model/modelstorage.py", line 1705, in save
    news = cls.create([save_values[r] for r in to_create])
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/ir/translation.py", line 613, in create
    return super(Translation, cls).create(vlist)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/model/modelsql.py", line 159, in wrapper
    return func(cls, *args, **kwargs)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/model/modelsql.py", line 674, in create
    cls._validate(sub_records)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/model/modelstorage.py", line 76, in wrapper
    return func(*args, **kwargs)
  File "/home/eamonn/.local/share/virtualenvs/tryton-dhcBaQGN/lib/python3.8/site-packages/trytond/model/modelstorage.py", line 1293, in _validate
    raise SelectionValidationError(gettext(
trytond.model.modelstorage.SelectionValidationError: The value "en_GB" for field "Language" in "Translation" is not one of the allowed options. -

So I tried stopping trytond, updating the config to en_US, restarting the server and running again but the same error so my guess is that en_GB has now been stored somewhere. Any ideas how to get round that now or easier just blow the virtualenv away and start again.

Thanks

Eamonn

dave · June 27, 2020, 11:06am

Yes, I think it gets stored in the database when the database is first initialized.

You should be able to sort this without getting rid of your virtualenv, by:

commenting out the language in your config file (see note below),
dropping the database (just be aware you will loose any data you’ve entered into Tryton doing this)
recreating the database
running trytond-admin -c <config_file> -d <database_name> --all

Also I would suggest using en as the language (which also happens to be the default Configuration file for Tryton — Tryton server).