Extended record logs

Rational

Sometimes it is useful to have a full picture of what changes happened on a record. This is often the case to understand how a document ended in a specific state (for example a sale on which it was applied a amendment).
For now we have the basic create/write logs on the records which is often not enough. And we may have the history but it is not always activated and it can be very data consuming.
So we would need an intermediary log system which provides enough to understand the sequence of events but not too much to slow or fill the database.

Proposal

We add a new ResourceMixin kind of model to log external event that happen to a record.
By external we mean from direct call of user which means when the _check_access is set.
By events we mean the create, write and delete calls but also the ModelView.button, Wizard.create and Workflow.transition.
The Log model is append only and we use the create_uid and create_date as log data.
The event is stored in a Selection: create, write, delete, button, wizard, transition.
As extra data for button we store the button name, for wizard the wizard name and for the transition the state.
The transition is the only event that will not be linked to a user (because it is either triggered by a button which will already have an entry or as callback of other event which should not be assigned to the user).

The “View Logs…” popup of the clients is extended to display the list of log entries. If the model has history activated, opening the log entry opens the record at the datetime of the entry.

The flag ModelStorage._log allows to deactivate for the model this mechanism. All ir.* models have the log deactivated.

Implementation

1 Like

How will a user be able to show “View Logs” for deleted records? IIUC “View Logs” works record based and not model based.

I do not think he will be but that’s not really a problem.

Why not add processing errors, too. With this, the record logs would solve many questions about why a special event raises an error.

For me it would be a conceptual error to include logs for something that did not happen.
And again the unexpected error can not always be linked to a single record.

So why collect logs about delete? Is it because the logs are collected in an own model and can be shown independent of the existence of the record?

For completeness and it can be useful for forensic.

The log will not be deleted so it is possible to show it even if the target has been deleted.

If I understand correctly the values stored by a create or write operation will not be logged, will they?

No if you want that you must activate the history.

Ok.

However, I think that recording that information for almost every model and record in the database and without a process that periodically removes those records, it will probably be too much in many cases.

Of course, everyone will be free to remove the data from the log table whenever they want but it looks to me that we should provide a ir.cron process that cleans that information (based on a configuration parameter) even if it is inactive by default. Just like it was done for ir.queue.

I don’t think many people will want to keep logs older than say 5 years, or 2 years or whatever.

I do not think that clearing database should be managed by a cron task. We do not do it for history table so I do not think we should for this.
The ir.queue is a little bit different because it does not really contain information, it is just an internal communication tool. But there is a cron task to clear them because it is as simple as executing a DELETE query based on the create_date.