# Transactional queue
**URL:** https://discuss.tryton.org/t/transactional-queue/461
**Category:** Feature
**Created:** [December 13, 2017, 10:21am UTC](https://discuss.tryton.org/t/transactional-queue/461 "2017-12-13T10:21:38Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@ced](https://discuss.tryton.org/u/ced)
#### Post date: [December 13, 2017, 10:21am UTC](https://discuss.tryton.org/t/transactional-queue/461/1 "2017-12-13T10:21:38Z")
## Rational
Some actions may take a long time because they do a lot of things, for example processing a sale or a purchase. We have already separated the validation and the processing and [added a cron task to process](https://bugs.tryton.org/issue6487). But this does not scale up very well (one cron task at a time) and has a long delay (cron run every hours).
Solution like [celery\_tryton](https://pypi.python.org/pypi/celery_tryton) can not be used in core module because of the complexity of the dependencies and setup.
## Proposal
We need a way to post tasks in a queue and process them by workers as soon as possible. The queue must be transactional and scale well. The queue manager should be replaceable (ex: by Celery) like the `Cache` is.
For PostgreSQL back-end, we could use (or be inspired) by [pq](https://pypi.python.org/pypi/pq). For other backends like SQLite, we could just run the task directly as there are no LISTEN/NOTIFY support.
The posted task which should be a method on a `Model` should of course take parameters (`*args`, `**kwargs`) from the call but also store the user and context. The `instantiate` keyword will indicate which argument should be instantiate using the Model class.
A task will have no returned value because they are executed after the transaction (and the answer to the request).
The worker processes will wait for task to be posted in the queue and execute them one at a time. If an operational transaction happens, they will retry the task (same as in the dispatcher). For all other error, it will generate a log entry.
### API
General low level:
```
Transaction().queue.put({'model': 'sale.sale', 'method': 'process', 'user': 1, 'args': [1], 'kwargs': {'context': {}}, 'instantiate': 0})
```
From Model:
```
Sale. __queue__.process([sale])
```
## Implementation
- [Transactional queue](https://bugs.tryton.org/issue7406)
---
### Author:  [@pokoli](https://discuss.tryton.org/u/pokoli)
#### Post date: [December 13, 2017, 11:59am UTC](https://discuss.tryton.org/t/transactional-queue/461/2 "2017-12-13T11:59:14Z")
Some doubts:
- How will the workers run when celery is not installed?
- Do we support scheduling tasks?
- Do we support priority?
- Do we manage retry on failure?
---
### Author:  [@ced](https://discuss.tryton.org/u/ced)
#### Post date: [December 13, 2017, 1:46pm UTC](https://discuss.tryton.org/t/transactional-queue/461/3 "2017-12-13T13:46:31Z")
> [@pokoli](#):
>
> How will the workers run when celery is not installed?
Like in pq using LISTEN/NOTIFY and [advisory lock](http://www.postgresql.org/docs/current/static/explicit-locking.html#ADVISORY-LOCKS).
> [@pokoli](#):
>
> Do we support scheduling tasks?
I think at one point the cron will just post the tasks in the queue.
> [@pokoli](#):
>
> Do we support priority?
Why not but it can come later.
> [@pokoli](#):
>
> Do we manage retry on failure?
Of course we have to retry on transactional error. But for other, we just log (or send an email to the user). The code who will be using such asynchronous task should implement a way to manually re-post the task.
---
### Author:  [@pokoli](https://discuss.tryton.org/u/pokoli)
#### Post date: [December 13, 2017, 2:39pm UTC](https://discuss.tryton.org/t/transactional-queue/461/4 "2017-12-13T14:39:23Z")
> [@ced](#):
>
> How will the workers run when celery is not installed?
>
> Like in pq using LISTEN/NOTIFY and advisory lock.
I mean if workers will be exectued on a separated process, by cron or how. It’s not clear from the description.
---
### Author:  [@ced](https://discuss.tryton.org/u/ced)
#### Post date: [December 13, 2017, 3:15pm UTC](https://discuss.tryton.org/t/transactional-queue/461/5 "2017-12-13T15:15:49Z")
Workers will be new processes.
---
### Author:  [@pokoli](https://discuss.tryton.org/u/pokoli)
#### Post date: [May 21, 2018, 12:57pm UTC](https://discuss.tryton.org/t/transactional-queue/461/6 "2018-05-21T12:57:58Z")
The implementation is available on [issue7406](https://bugs.tryton.org/issue7406)
---
### Author:  [@dabada83](https://discuss.tryton.org/u/dabada83)
#### Post date: [May 21, 2018, 2:36pm UTC](https://discuss.tryton.org/t/transactional-queue/461/7 "2018-05-21T14:36:01Z")
There is an issue with URL
---
### Author:  [@pokoli](https://discuss.tryton.org/u/pokoli)
#### Post date: [May 21, 2018, 2:48pm UTC](https://discuss.tryton.org/t/transactional-queue/461/8 "2018-05-21T14:48:25Z")
> [@dabada83](#):
>
> There is and issue with URL
Thanks for the notice, I’ve already edited and fixed it
---
### Author:  [@ced](https://discuss.tryton.org/u/ced)
#### Post date: [December 21, 2018, 5:00pm UTC](https://discuss.tryton.org/t/transactional-queue/461/9 "2018-12-21T17:00:03Z")
This topic was automatically closed after 14 days. New replies are no longer allowed.