Trytond-admin 6.6 stuck updating database

trytond-admin gets stuck updating without any error message, also it is not possible to Ctrl+C out of it when it gets stuck.

dot$ . ../venv66/bin/activate
(venv66) dot$ env |grep TRYTOND
TRYTOND_WEB__ROOT=/var/tryton/production/root/
TRYTOND_DATABASE__PATH=/var/tryton/production/run
TRYTOND_WEB__LISTEN=*obfuscated*
TRYTOND_DATABASE__URI=postgresql://tryton@/
(venv66) dot$ 
(venv66) dot$ trytond-admin -d test --all -vvv 2> logfile1 & 
(venv66) dot$ tail logfile1                                                                                                                                                                                                                                                                                                   
88451 17158793241296 [2022-11-02 11:02:08,872] DEBUG trytond.backend.postgresql.database b'SELECT "a"."id" AS "id", "a"."create_date" AS "create_date", "a"."create_uid" AS "create_uid", "a"."db_id" AS "db_id", "a"."fs_id" AS "fs_id", "a"."fs_values" AS "fs_values", "a"."model" AS "model", "a"."module" AS "module", "a"."noupdate" AS "noupdate", "a"."values" AS "values", "a"."write_date" AS "write_date", "a"."write_uid" AS "write_uid", CAST(EXTRACT(\'EPOCH\' FROM COALESCE("a"."write_date", "a"."create_date")) AS VARCHAR) AS "_timestamp" FROM "ir_model_data" AS "a" WHERE (false AND ("a"."module" = \'carrier_weight\')) ORDER BY "a"."id" DESC'
88451 17158793241296 [2022-11-02 11:02:08,873] DEBUG trytond.backend.postgresql.database b'SELECT "a"."id" AS "id", "a"."create_date" AS "create_date", "a"."create_uid" AS "create_uid", "a"."db_id" AS "db_id", "a"."fs_id" AS "fs_id", "a"."fs_values" AS "fs_values", "a"."model" AS "model", "a"."module" AS "module", "a"."noupdate" AS "noupdate", "a"."values" AS "values", "a"."write_date" AS "write_date", "a"."write_uid" AS "write_uid", CAST(EXTRACT(\'EPOCH\' FROM COALESCE("a"."write_date", "a"."create_date")) AS VARCHAR) AS "_timestamp" FROM "ir_model_data" AS "a" WHERE (("a"."fs_id" IN (\'msg_sale_shipment_cost_unique\')) AND ("a"."module" = \'sale_shipment_cost\')) ORDER BY "a"."id" DESC'
88451 17158793241296 [2022-11-02 11:02:08,873] INFO trytond.convert Deleting 350@ir.message from sale_shipment_cost.msg_sale_shipment_cost_unique
88451 17158793241296 [2022-11-02 11:02:08,877] DEBUG trytond.backend.postgresql.database b'SELECT "a"."id" AS "id", "a"."create_date" AS "create_date", "a"."create_uid" AS "create_uid", "a"."db_id" AS "db_id", "a"."fs_id" AS "fs_id", "a"."fs_values" AS "fs_values", "a"."model" AS "model", "a"."module" AS "module", "a"."noupdate" AS "noupdate", "a"."values" AS "values", "a"."write_date" AS "write_date", "a"."write_uid" AS "write_uid", CAST(EXTRACT(\'EPOCH\' FROM COALESCE("a"."write_date", "a"."create_date")) AS VARCHAR) AS "_timestamp" FROM "ir_model_data" AS "a" WHERE (("a"."model" = \'ir.message\') AND ("a"."db_id" IN (350)) AND ("a"."noupdate" = true)) ORDER BY "a"."id" ASC'
88451 17158793241296 [2022-11-02 11:02:08,879] DEBUG trytond.backend.postgresql.database b'DELETE FROM "ir_message" WHERE (("id" IN (350)))'
88451 17158793241296 [2022-11-02 11:02:08,880] DEBUG trytond.backend.postgresql.database b'SELECT "a"."id" AS "id", "a"."create_date" AS "create_date", "a"."create_uid" AS "create_uid", "a"."fuzzy" AS "fuzzy", "a"."lang" AS "lang", "a"."module" AS "module", "a"."name" AS "name", "a"."overriding_module" AS "overriding_module", "a"."res_id" AS "res_id", "a"."src" AS "src", "a"."type" AS "type", "a"."value" AS "value", "a"."write_date" AS "write_date", "a"."write_uid" AS "write_uid", CAST(EXTRACT(\'EPOCH\' FROM COALESCE("a"."write_date", "a"."create_date")) AS VARCHAR) AS "_timestamp" FROM "ir_translation" AS "a" WHERE (("a"."type" = \'model\') AND ("a"."name" LIKE \'ir.message,%\' ESCAPE \'\\\') AND ("a"."res_id" IN (350))) ORDER BY "a"."id" ASC'
88451 17158793241296 [2022-11-02 11:02:08,884] DEBUG trytond.backend.postgresql.database b'SELECT "a"."model" FROM "ir_model_data" AS "a" GROUP BY "a"."model"'
88451 17158793241296 [2022-11-02 11:02:08,886] DEBUG trytond.backend.postgresql.database b'DELETE FROM "ir_translation" WHERE (("id" IN (11640)))'
88451 17158793241296 [2022-11-02 11:02:08,886] DEBUG trytond.backend.postgresql.database b'SELECT "a"."name", "a"."id", "a"."timestamp" FROM "ir_cache" AS "a" WHERE ("a"."name" = \'ir.translation.get_report\') LIMIT 1'
88451 17158793241296 [2022-11-02 11:02:08,887] DEBUG trytond.backend.postgresql.database b'UPDATE "ir_cache" AS "a" SET "timestamp" = CURRENT_TIMESTAMP WHERE ("a"."name" = \'ir.translation.get_report\')'
dot# kdump -f ktrace.out |tail -n20
 88451 python   CALL  futex(0xf9b1843a560,0x82<FUTEX_WAKE|FUTEX_PRIVATE_FLAG>,1,0,0)
 88451 python   RET   futex 0
 88451 python   CALL  connect(3,0xf9ac64ba7e8,48)
 88451 python   STRU  struct sockaddr { AF_UNIX, "/var/tryton/production/run/trytond-stat.socket" }
 88451 python   NAMI  "/var/tryton/production/run/trytond-stat.socket"
 88451 python   RET   connect -1 errno 2 No such file or directory
 88451 python   CALL  futex(0xf9b184569f0,0x82<FUTEX_WAKE|FUTEX_PRIVATE_FLAG>,1,0,0)
 88451 python   RET   futex 0
 88451 python   CALL  futex(0xf9b1843a560,0x82<FUTEX_WAKE|FUTEX_PRIVATE_FLAG>,1,0,0)
 88451 python   RET   futex 0
 88451 python   CALL  close(3)
 88451 python   RET   close 0
 88451 python   CALL  futex(0xf9b184569f0,0x82<FUTEX_WAKE|FUTEX_PRIVATE_FLAG>,1,0,0)
 88451 python   RET   futex 0
 88451 python   CALL  futex(0xf9b1843a560,0x82<FUTEX_WAKE|FUTEX_PRIVATE_FLAG>,1,0,0)
 88451 python   RET   futex 0
 88451 python   CALL  select(0,0,0,0,0xf9ac64bab10)
 88451 python   STRU  struct timeval { 60 }
 88451 python   PSIG  SIGTERM SIG_DFL
 88451 python   STRU  struct pollfd { fd=5, events=0x9<POLLIN|POLLERR>, revents=0<> }

I run trytond-admin without trytond running in order to avoid interference, I assume this is the right way, maybe it should be specified in documentation. I also tested with trytond running, same result.
Venv was created using pip on 2nd November.
I have a bunch of reports that were created inside of tryton that do not come from a module.

I would have filed a bug report, but I am not sure if I am doing something wrong or if there is a problem in trytond.

1 Like

Update 1: The last line displayed changed after I updated modules list with -m

54735 7859537781456 [2022-11-02 12:09:42,333] DEBUG trytond.backend.postgresql.database b'SELECT "a"."name", "a"."id", "a"."timestamp" FROM "ir_cache" AS "a" WHERE ("a"."name" = \'ir_model_data.has_model\') LIMIT 1'
54735 7859537781456 [2022-11-02 12:09:42,334] DEBUG trytond.backend.postgresql.database b'UPDATE "ir_cache" AS "a" SET "timestamp" = CURRENT_TIMESTAMP WHERE ("a"."name" = \'ir_model_data.has_model\')'

Update 2: trytond-admin -v -u ir -d test gets stuck, all other modules update without problem.

Update 3: While in the blocked state, when I kill trytond-admin the postgresql log shows me:

2022-11-02 14:56:35.358 EET [20290] LOG:  unexpected EOF on client connection with an open transaction
2022-11-02 14:56:35.361 EET [56700] LOG:  could not send data to client: Broken pipe
2022-11-02 14:56:35.361 EET [56700] FATAL:  connection to client lost

Indeed there is a dead lock when updating some modules: Issue 11855: Dead lock after updating indexes - Tryton issue tracker

Same problem after upgrade to 6.6, trytond-admin gets stuck updating.
Reboot solved and everythings seems to work as expected but I’m not sure upgrade have been done properly.
Do I have to worry about consequences?

Without the patch and not going the full update, you will be missing the new indexes and the removal of some records.

ok so could you explain how to apply the patch?
My server is installed with a “classic” python virtuaenv

Thanks, upgrade complete.