)
500 0.001 0.000 4.061 0.008 multivalue.py:55(_multivalue_getter)
500 0.002 0.000 4.057 0.008 model.py:21(get_multivalue)
500 0.001 0.000 4.053 0.008 multivalue.py:43(get_multivalue)
500 0.001 0.000 4.002 0.008 multivalue.py:37(__values)
500 0.001 0.000 3.446 0.007 model.py:12(multivalue_records)
43929 0.311 0.000 2.730 0.000 modelstorage.py:1493(instantiate)
--- cut ---
```
Only 50% of instances created, and the execution time drops to 2.5985 seconds (still a lot of time, but 30% less).
Time measured with cprofile disabled.
What do you think? Should we make create\_uid/write\_uid use lazy loading?
Other tricks we should use to lower the number of instances created? I’m wondering if the cache should just store id’s by default and create instances only if the field is actually accessed in any of the records of the cache.
---
### Author:  [@ced](https://discuss.tryton.org/u/ced)
#### Post date: [March 17, 2020, 8:51am UTC](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458/2 "2020-03-17T08:51:54Z")
Indeed it will probably be better to lazy instanciate all `Many2One`. By that I mean that for now, we have only one boolean attribute for `loading` which is used for two different thing (but linked). The pre-SELECT of column in search query (to fill the cache) and the instantiation/setup in `ModelStorage. __getattr__ `. I think `Many2One` are a special case which does not fit well in the binary choice. It should be pre-SELECTed but not instantiated. The `Reference` field is also in the same situation (but by default it is lazy because less often used).
So I think we may introduce a third option.
---
### Author:  [@albert](https://discuss.tryton.org/u/albert)
#### Post date: [March 17, 2020, 10:57pm UTC](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458/3 "2020-03-17T22:57:30Z")
> [@ced](#):
>
> Indeed it will probably be better to lazy instanciate all `Many2One` . By that I mean that for now, we have only one boolean attribute for `loading` which is used for two different thing (but linked). The pre-SELECT of column in search query (to fill the cache) and the instantiation/setup in `ModelStorage. __getattr__ ` . I think `Many2One` are a special case which does not fit well in the binary choice. It should be pre-SELECTed but not instantiated. The `Reference` field is also in the same situation (but by default it is lazy because less often used).
I don’t understand the need for a third option. I understand that a third option could be useful if preSELECTING _and_ instantiating brought some kind of performance advantage in some cases. But AFAIU instantiating on demand would be as fast as pre-instantiating. Isn’t it?
---
### Author:  [@ced](https://discuss.tryton.org/u/ced)
#### Post date: [March 18, 2020, 8:07am UTC](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458/4 "2020-03-18T08:07:22Z")
Instantiating has virtue also. If you know that the field will be used anyway, it avoid going through the ` __getattr__ ` machinery.
Indeed I even think it will be great to have a way to customize on runtime for a specific _browse_ case the `loading` attribute of each field.
---
### Author:  [@albert](https://discuss.tryton.org/u/albert)
#### Post date: [March 20, 2020, 2:42pm UTC](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458/5 "2020-03-20T14:42:33Z")
> [@ced](#):
>
> Indeed I even think it will be great to have a way to customize on runtime for a specific _browse_ case the `loading` attribute of each field.
Yes, I think that would be great.
---
### Author:  [@albert](https://discuss.tryton.org/u/albert)
#### Post date: [March 22, 2020, 1:02am UTC](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458/6 "2020-03-22T01:02:01Z")
I did another test in another process. In this case, in the receive() button of a stock.shipment.in with a shipment with 250 lines.
Execution time on version 5.4: 186 seconds
Then, I changed the loading to ‘lazy’ for **all** Many2One fields and execution time goes down to just 7 seconds.
---
### Author:  [@albert](https://discuss.tryton.org/u/albert)
#### Post date: [March 22, 2020, 1:29am UTC](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458/7 "2020-03-22T01:29:08Z")
I must clarify that both measures where with profiling enabled and that the first execution (the one taking 186 seconds) already had applied a patch that made create\_uid and write\_uid load lazily.
---
### Author:  [@sergyo](https://discuss.tryton.org/u/sergyo)
#### Post date: [March 23, 2020, 11:30pm UTC](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458/8 "2020-03-23T23:30:59Z")
> [@albert](#):
>
> I did another test in another process. In this case, in the receive() button of a stock.shipment.in with a shipment with 250 lines.
>
> Execution time on version 5.4: 186 seconds
>
> Then, I changed the loading to ‘lazy’ for **all** Many2One fields and execution time goes down to just 7 seconds.
I was very impressive about this results so I decided to make a similar test in 5.0 with a shipment (in) with 200 lines. But I haven’t got those great results …
- with current loading configuration (eager) execution time was: 166 seconds
- with lazy loading as default value for all many2one execution time was: 106 seconds.
It is a 36% of improvement on execution time, not bad … but too far from your results. I will test on 5.4.
---
### Author:  [@pokoli](https://discuss.tryton.org/u/pokoli)
#### Post date: [March 24, 2020, 8:54am UTC](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458/9 "2020-03-24T08:54:51Z")
Probable the performance depends on the number of diferent products on the supplier lines.
It will be great if you can share a script (in proteus or using trytond-console) in order to create the shipment with 200 lines and test the computation of 200 lines. This will allow everyone to test with the same values and have more similar results.
---
### Author:  [@albert](https://discuss.tryton.org/u/albert)
#### Post date: [March 24, 2020, 9:38am UTC](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458/10 "2020-03-24T09:38:21Z")
> [@sergyo](#):
>
> but too far from your results. I will test on 5.4.
That’s simply because:
- Version 5.4 executes the purchase’s “process” method in a worker. And although I did not have the workers enabled
- I only tested part of the execution of the receive method
But if you use verson 5.4 and use the workers you may get similar results.
---
### Author:  [@pokoli](https://discuss.tryton.org/u/pokoli)
#### Post date: [March 24, 2020, 10:38am UTC](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458/11 "2020-03-24T10:38:11Z")
> [@albert](#):
>
> Version 5.4 executes the purchase’s “process” method in a worker. And although I did not have the workers enabled
If you do not have any workers there should be no diference as the task is executed add the end of the transaction.
---
### Author:  [@albert](https://discuss.tryton.org/u/albert)
#### Post date: [March 24, 2020, 10:53am UTC](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458/12 "2020-03-24T10:53:03Z")
> [@pokoli](#):
>
> If you do not have any workers there should be no diference as the task is executed add the end of the transaction.
Version 4.8 executes process in batch and on the write operation. It is not comparable.
---
### Author:  [@sergyo](https://discuss.tryton.org/u/sergyo)
#### Post date: [March 24, 2020, 1:11pm UTC](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458/13 "2020-03-24T13:11:02Z")
> [@pokoli](#):
>
> Probable the performance depends on the number of diferent products on the supplier lines.
All moves are from same product.
> [@pokoli](#):
>
> It will be great if you can share a script (in proteus or using trytond-console) in order to create the shipment with 200 lines and test the computation of 200 lines. This will allow everyone to test with the same values and have more similar results.
I did a little script for adding moves to an existing shipment in a filled database. I will try to completed it to execute on an empty one.
> [@albert](#):
>
> - Version 5.4 executes the purchase’s “process” method in a worker. And although I did not have the workers enabled
Indeed 5.0 has workers as well and I tested [without enable them](https://docs.tryton.org/projects/server/en/5.0/topics/configuration.html#worker).
But as I said on the other post I want to test it on 5.4 to discard code refactoring among versions.
> [@albert](#):
>
> - I only tested part of the execution of the receive method
Same here: receive an shipment in with no purchase.
---
### Author:  [@albert](https://discuss.tryton.org/u/albert)
#### Post date: [March 24, 2020, 2:50pm UTC](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458/14 "2020-03-24T14:50:08Z")
> [@sergyo](#):
>
> Indeed 5.0 has workers as well and I tested [without enable them](https://docs.tryton.org/projects/server/en/5.0/topics/configuration.html#worker).
> But as I said on the other post I want to test it on 5.4 to discard code refactoring among versions.
Probably the difference is due to the modules installed.
For example, in my database there’s the production and many other modules installed. Production itself adds several m2o fields to stock moves, so in my case it’s going to be slower without the patch. And thus the patch will have a more positive impact.
There could be other factors: the size of the database, the hardware…
I don’t think it’s worth trying to reproduce exactly the same performance improvement. +30% is already a huge difference and you can easily reproduce such a gain in lots of places if you have enough records and m2o fields in the model (although create\_uid/write\_uid are already enough to see the difference).
---
### Author:  [@pokoli](https://discuss.tryton.org/u/pokoli)
#### Post date: [March 24, 2020, 7:09pm UTC](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458/15 "2020-03-24T19:09:10Z")
> [@albert](#):
>
> I don’t think it’s worth trying to reproduce exactly the same performance improvement. +30% is already a huge difference and you can easily reproduce such a gain in lots of places if you have enough records and m2o fields in the model (although create\_uid/write\_uid are already enough to see the difference).
I can not reproduce your performance on latest trunk. I’ve created a proteus script (see below) to create 200 products and add a shipment which recives 4 units of one of them. I’ve run on a postgresql database with stock, sale, stock\_lot and production modules activated. Here are my results:
With eager loading:
```
Shipment created 0:00:07.425063
Shipment received 0:00:02.891001
Shipment done 0:00:01.724375
```
With lazy loading:
```
Shipment created 0:00:06.377097
Shipment received 0:00:02.791547
Shipment done 0:00:01.463134
```
Here is the script:
```python
import datetime
from decimal import Decimal
from proteus import Model, config
config.set_trytond()
Product = Model.get('product.product')
products = Product.find([])
if not products:
print('Creating products')
Uom = Model.get('product.uom')
unit, = Uom.find([('symbol', '=', 'u')])
Template = Model.get('product.template')
templates = []
for i in range(200):
template = Template()
template.name = 'Product %s' % (i + 1)
template.type = 'goods'
template.default_uom = unit
template.list_price = Decimal(0)
product, = template.products
product.code = 'PROD%s' % (i + 1)
product.cost_price = Decimal(0)
templates.append(template)
Template.save(templates)
products = Product.find([])
ShipmentIn = Model.get('stock.shipment.in')
Location = Model.get('stock.location')
warehouse_loc, = Location.find([('code', '=', 'WH')])
Party = Model.get('party.party')
party, = Party.find([], limit=1)
start = datetime.datetime.now()
print('Creating shipment')
shipment = ShipmentIn()
shipment.supplier = party
for product in products:
move = shipment.inventory_moves.new()
move.from_location = party.supplier_location
move.to_location = shipment.warehouse.input_location
move.product = product
move.quantity = 4.0
move.unit_price = product.cost_price
shipment.save()
print('Shipment created', (datetime.datetime.now() - start))
start = datetime.datetime.now()
shipment.click('receive')
print('Shipment received', (datetime.datetime.now() - start))
start = datetime.datetime.now()
shipment.click('done')
print('Shipment done', (datetime.datetime.now() - start))
```
---
### Author:  [@albert](https://discuss.tryton.org/u/albert)
#### Post date: [March 24, 2020, 7:30pm UTC](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458/16 "2020-03-24T19:30:05Z")
> [@pokoli](#):
>
> I can not reproduce your performance on latest trunk. I’ve created a proteus script (see below) to create 200 products and add a shipment which recives 4 units of one of them. I’ve run on a postgresql database with stock, sale, stock\_lot and production modules activated. Here are my results:
As I said, the performance improvement will highly depend on the number of modules you’ve installed.
Your simple script already shows 15% performance improvement in several of the steps you showed.
Add several m2o fields and you’ll probably see a bigger difference.
---
### Author:  [@pokoli](https://discuss.tryton.org/u/pokoli)
#### Post date: [March 24, 2020, 8:27pm UTC](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458/17 "2020-03-24T20:27:42Z")
> [@albert](#):
>
> As I said, the performance improvement will highly depend on the number of modules you’ve installed.
But the issue may also come because a module has something that is not properly implemented. So the module must be fixed and not anything else.
> [@albert](#):
>
> Add several m2o fields and you’ll probably see a bigger difference.
Which fields do I have to add? AFAIK there aren’t so much Many2One fields on the _stock.shipmen.in_ and _stock.move_ models.
What it’s strange to me is that receiving a shipment with 260 lines lasts for 160seconds. This is a huge amount of time and it reduced.
Are you using average cost price? This may add some seconds to compute the cost price but probably not as much.
BTW I’ve just modified the script to also create the purchase and here are the results:
With lazy loading:
```
Purchase created 0:00:49.893981
Purchase confirmed 0:00:01.399725
Creating shipment
Shipment created 0:00:00.426459
Shipment received 0:00:01.103032
Shipment done 0:00:00.494885
```
With eager loading:
```
Purchase created 0:00:52.913379
Purchase confirmed 0:00:01.365182
Creating shipment
Shipment created 0:00:00.453029
Shipment received 0:00:01.195862
Shipment done 0:00:00.490862
```
Note that i’ve reduced the number of products to 100 because the for each purchase line proteus calls on\_change\_lines to recompute the amounts and this was slowing the testing.
But for the shipment reception part I get very similar results comparing the patch and without the patch.
Here is the modified script if you want to test it:
```python
import datetime
from decimal import Decimal
from proteus import Model, config
config.set_trytond()
Product = Model.get('product.product')
products = Product.find([('purchasable', '=', 'True')])
if not products:
print('Creating products')
Uom = Model.get('product.uom')
unit, = Uom.find([('symbol', '=', 'u')])
Template = Model.get('product.template')
Category = Model.get('product.category')
category = Category()
category.accounting = True
category.name = 'category'
category.save()
templates = []
for i in range(100):
template = Template()
template.name = 'Product %s' % (i + 1)
template.type = 'goods'
template.default_uom = unit
template.purchasable = True
template.salable = True
template.list_price = Decimal(0)
template.account_category = category
product, = template.products
product.code = 'PROD%s' % (i + 1)
product.cost_price = Decimal(0)
templates.append(template)
Template.save(templates)
products = Product.find([('purchasable', '=', True)])
Purchase = Model.get('purchase.purchase')
ShipmentIn = Model.get('stock.shipment.in')
Location = Model.get('stock.location')
Move = Model.get('stock.move')
warehouse_loc, = Location.find([('code', '=', 'WH')])
Party = Model.get('party.party')
party, = Party.find([], limit=1)
start = datetime.datetime.now()
print('Create purchase')
purchase = Purchase()
purchase.party = party
purchase.purchase_date = datetime.date.today()
for product in products:
line = purchase.lines.new()
line.product = product
line.quantity = 4.0
print(product.rec_name)
purchase.save()
print('Purchase created', (datetime.datetime.now() - start))
start = datetime.datetime.now()
purchase.click('quote')
purchase.click('confirm')
print('Purchase confirmed', (datetime.datetime.now() - start))
start = datetime.datetime.now()
print('Creating shipment')
shipment = ShipmentIn()
shipment.supplier = party
for move in purchase.moves:
incoming_move = Move(id=move.id)
shipment.incoming_moves.append(incoming_move)
shipment.save()
print('Shipment created', (datetime.datetime.now() - start))
start = datetime.datetime.now()
shipment.click('receive')
print('Shipment received', (datetime.datetime.now() - start))
start = datetime.datetime.now()
shipment.click('done')
print('Shipment done', (datetime.datetime.now() - start))
```
It will be great to share some way to reproduce your performance issues.
---
### Author:  [@pokoli](https://discuss.tryton.org/u/pokoli)
#### Post date: [April 13, 2020, 9:54pm UTC](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458/18 "2020-04-13T21:54:21Z")
I’m wondering if [this changeset](https://hg.tryton.org/trytond/rev/f4ea454f5131) is what produces most diference on the patch.
I’ve done my test with this change applied so if fields are already on the cache they will not be loaded providing a speedup!
---
### Author:  [@ced](https://discuss.tryton.org/u/ced)
#### Post date: [January 24, 2021, 12:52am UTC](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458/19 "2021-01-24T00:52:46Z")
Here is [Instantiate only requested field attribute (#10026) · Issues · Tryton / Tryton · GitLab](https://bugs.tryton.org/issue10026) which should bring the best of the two loading behavior for the `Many2One` and `Reference` without any change in the field definition.
---
### Author:  [@pokoli](https://discuss.tryton.org/u/pokoli)
#### Post date: [January 24, 2021, 7:44am UTC](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458/20 "2021-01-24T07:44:50Z")
Thanks for proposing a patch. Did you test any the provided scripts to see if there is any performance gain in the mentionen cases?
[Next page](https://discuss.tryton.org/t/should-create-uid-write-uid-use-lazy-loading/2458.md?page=2)