The invoices should be sent by using a WSDL web-service. There is one for issued invoices and other for received invoices. All the requests to this web-services should be made with a certificate authentication.
Proposal
Add a new module named account_es_sii. This module will automatically send the invoices to the web-services when an invoices is posted. In order to keep it transactional a separate record wil be created to store the comunincation details. A cron job will read this records and send them to the tax authorities.
The account.invoice.sii modul will be used to store the comunication details and will have the following fields:
Invoice: The invoice to be sent.
State: A selection field indicating the reply of the web-service. Can be one of the following:
Pending:
Accepted
Accepted with errors
Rejected
Canceled
CSV: A character to identify the communication
Error: In case of error the returned message from the web-service.
As the invoice may be rejected, it should be possible to resend it (once the error is corrected) by duplicating the record.
The web-service also requires to identify the type of taxes with a special codes. New fields will be added on tax (and tax template) to let the user set the codes. The values will be set for the taxes defined on the account_es module.
On the account configuration model we will add the following fields
SII URL: The URL from where to read the webservice URL.
SII Environment: None, Staging and Production
The Public and private keys used for authentification will be stored on the filesystem and they should be specified by adding its url to the tryton configuration system.
I do not think a task queue is the proper way because this is a process with side-effect that will not be really transactional. For me, it is very similar to account_fr_chorus which uses a cron because it retries automatically on failure and commit after each successful sent.
I would prefer like for Chorus to have a separate model to store communication data.
What is the workflow behind those states?
Did not they reuse an existing codification?
I think it is better stored on the account.configuration.
Do they provide a staging server for testing?
I prefer to store keys on the file system and get them via configuration because it is sensitive information that should not be part of the backup. And they may need to be renewed by an automatic service like “Let’s encrypt”.
This is the document describing this among another lot of things.
Summarzing, every single WSDL service has end points for staging and production environments. An example of part of one of the WSDL:
<wsdl:service name="siiService">
<!-- Entorno de PRODUCCION día 1 de julio 2018-->
<wsdl:port name="SuministroFactEmitidas" binding="siiWdsl:siiBinding">
<soap:address location="https://www1.agenciatributaria.gob.es/wlpl/SSII-FACT/ws/fe/SiiFactFEV1SOAP"/>
</wsdl:port>
<!--
Entorno de PRODUCCION día 1 de julio 2018 para acceso con certificado de sello
-->
<wsdl:port name="SuministroFactEmitidasSello" binding="siiWdsl:siiBinding">
<soap:address location="https://www10.agenciatributaria.gob.es/wlpl/SSII-FACT/ws/fe/SiiFactFEV1SOAP"/>
</wsdl:port>
<!--
Entorno de PRUEBAS (tambien valido para acceso con certificado de sello)
-->
<wsdl:port name="SuministroFactEmitidasPruebas" binding="siiWdsl:siiBinding">
<soap:address location="https://www7.aeat.es/wlpl/SSII-FACT/ws/fe/SiiFactFEV1SOAP"/>
</wsdl:port>
</wsdl:service>
contains the end point for the staging environment.
@pokoli is proposing to encrypt this sensitive information so no problem for the backup. Anyway if it is finally decided to store this data in file system we have to give the user the UI to create/modify/delete this data so it is user information.
I think (but I am not sure) that the process to obtain this type of certificate in Spain is not automatizable.
Normally all invoices should be accepted but it there should be ones that may be rejected (for example by some invalid data). If so the invoice should be resended or updated via the webservice.
So if we use a separate model we may allow to re-create a record for the same invoice and resend them (the webservice allows to update an existing invoice using a diferent operation type)
For the User Point of view the important part is that all the invoices should be sent to the tax authorities in a period of 4 days.
No, there is a new codification for each kind of taxes. There are 16 types for sale invoices and 14 types for sale invoices. That’s why we need new fields.
No, the certificate is used to authenticate the company. So it’s end user data.
The main issue here is that we need to store one certificate per each company in the system. In this webservice there is no login/password auth. All the authenfication is done via the certificate that is generated by the tax authorities. This certificate is used for all the comunications with the tax authority, for example to upload the tax reports from it’s website you should use the same certificate.
The problem I see with storing them on the configuration is that it does not allow to set diferent certificates for each company in the system. If file system storage is prefered we should store them by using a binary field (on company) which is stored on the filesistem.
Another issue with certificates is that the user receives a single file (with .p12 format and password protected) with the public and private key inside. Then the keys should be extracted (this can be done with load_pkcs12, dump_privatekey and dump_publickey functions from pyopenssl package.
For the user point of view the easiest setup will be to have a wizard that request for the file and the password. Then the cerver can extract public an dprivate keys and store them wherever we decide it’s the best place.
One good think about the SII webservice is that it allows to send multiple invoices in a bunch (10.000 maximum) so we don’t need to have one commit per record but we can have one commit for each 10.000 invoices.
I mean is not part of a normalized standard instead of a custom codification just for one service.
I do not agree. A certificate can authenticate multiple entities. This is even strange that there is not an additional user/password authentication.
At best you could have Tryton generate the certificate on the filesystem, provides the certificate signing request file and get back the public key signed. But I think it will be over-specify while tools like openssl supports that better.
How can it be possible? This is not secure if you can not choose your secret.
They should not use certificate but just plain secret if they do not use the properties of certificate.
This is not how certificate should be used. A different certificate should be allowed for each purpose.
In such case, for me there would be no need to have special care about this. It should be treaded as a shared secret without caring it is a certificate because it the usage it is not.
It depends really on how it behaves with failure. Is all rollbacked or not? The 10000 records should be locked any way and commit must be done after the network communication.
Thinking about this I guess the end user must have an option to force the process of sending pending invoices instead of waiting for the Cron job for all those cases of last time error correction (due to the max law period of 4 days to send the invoices).
Well, I just choose the password when you download the certificate from the tax authority website. So the password is what is choosen by the user. This is one of the reasons to have a process to extract public/private keys from the server, otherwise the user should told the secret to the systems administrator.
When browsing the tax authority website they do not have any login, they request the certificate (that they emitted). In order to autentify you have to add your user certificates on the browser and the browser will ask you which identify you want to use with the website. For example I have two certificates on my laptop, on which identifies my as person and another which identifies my as representative of my company. Whenever I enter the website I choose which identifiy I want to use.
We do not need to generate any certificate. This is the reponsability of the FNMT
Indeed there is only one purpose: “Identify a party with spanish tax authorities”. That’s why there is only one certificate per party which is emited by the administration.
Sorry but I do not understand what do you mean. What do you mean by “treating it as a shared secret”?
The request is fully rejected if not valid. If valid it contains for each invoice the state (accepted/rejected) and it’s errors. So yes, we can commit all the responses after network communication.
Why? if the cron job is executed every 5 minutes the invoices will be always sent in time by the cron job. It’s the user responsability to create (and update if needed) the invoices in time to be accepted by tax authority.
BTW the webservice will never reject an invoice which is out of comunication period. The comunication period of 4 days may only generate a fine for the company for the delays.
No normalized format is used, indeed no format at all is used. The tax authority only accepts comunication with the SOAP webserivce and they designed it to accept all the invoice details as parameters of ti. To ease the comunication we can use zeep, which expectes to receive a dict of values that can directly generated from the invoice.
Any way, the system administrator will have access to this data no matter what you design (except if you put the submission process on the user system).
That’s why I said it is not secure and it is not how certificates are designed.
The certificate is not used as a certificate for all the above reason.
But I do not think there is any need to discuss this technical semantic further.
This certificate is just like the secret_key of Stripe (with a different way to use it). So we must manage the certificate like we do for similar concept. We just store them in the database as it is linked to the company.
And in this case as the data received by the user is protected by a password, I would store the password and the “certificate” as-is.
About the fernet usage, I think it is out of the scope of this blueprint. I think we have already one (I can not find it) blueprint to store encrypted data in the database. Once implemented it can be used for this password (and other modules).
Even for the codification? They just invented a specific codification only for them?
I still do not understand most of those states. For me when you send something to someone, it has a binary state: sent or not. What are the other cases? Should they really be managed?
No as the secret is not stored on the website, it’s only used to encrypt the cert so anyone that does not know this secret can not extract the keys (so can not use it).
Ok, we store this information but then we will need to extract public and private key it each time an invoice is sent. I’m not very fan of storing the password (as we have to store it on plain text). That’s why I propsed to store only the public key and the private key which is the only data we require and use a wizard to update this values by asking the user for the right data. As this may be an implementation detail I’m ok leaving the discussion here and we can come back latter when we have some code to discuss about.
Ok, Forget about the encription part and keep it simple.
I only found the blueprint about client side transforming, but I do not see it usefull for this use case.
Yes, the codification is specific for sending the invoices with the webservice. But if this changes in the future I think we can move the fields to the account_es module.
Here we have Pending for not set, and all the others are sent + the state on the other system.
If you prefer we can seperate it storing only the state on the other system, and all the records with empty state are the one that should be sent to tax authorithy.
Yes, it is very important to manage the “Rejected” and "Accepted with errors"status because it’s the user responsabiliy to do something in order to resend the invoice until he gets a reply with the “Accepted” state.
For the canceled one, it is the response that the tax authority gives when an invoice is cancelled.
From the user point of view they need a screen to search for invoices with are not in accepted state and have the possibility to re-send the invoice. Here is detailed:
But after a second thought I am wondering if it won’t be better to follow the account_fr_chrous design and have a single record for the invoice (with an unique constraint) but allow to re-schedule a record to be resended. This way we always have the last state of the tax_authority system which is what is relevant for the end user.
I agree better a cron job. Although will be great to add a button in order to users can force a manual sent.
I prefer this design instead of duplicate records, but it should keep a history of changes (when was sent for the first time, the state, …). Perhaps enabling history changes on the model for a set of fields.
Will be great to have access to the JSON on the form view (as a text field or a binary widget to download).
Indeed the certificate identifies the party (so it is per party).
For intracommunitary invoices the type of operation (Detalle Operaciones) must be sent. This is untaxed amount and tax amount per goods and services. How do you think to implement it? I suppose:
goods: lines with product of types goods and assets
service: lines with product of types service
when there is no product on invoice line: ¿?
As you know the web service allows to sent invoice modification and invoice cancelation. So this actions should be considered to re-schedule the record to be resended.
For me the table recording the sending of the invoice should have unique constraint on the invoice to ensure it is sent only once.
So all other states are indeed just logs.
Could you give an example for each case? I do not understand how it could be accepted with errors.
For me the invoice is sent or not. If it is not sent the cron must sent it.
Send status should be a boolean. It is or not, all other cases are not sent and logging data.
I do not see the point. User can run cron at any time.
When could an invoice be modified. In Tryton is it never.
What is invoice cancelation? For me, it makes no sense to change an posted invoice.
Please could you provide a use where an invoice should be re-sent.
It is when there are errors on invoice’s message but they are considered as non critical and the invoice is accepted with this special state.
Well I don’t agree due to an invoice sent and rejected is sent. Moreover in this case if the invoice is marked as not sent the job will send it indefinitely until user fix it.
Account user normally does not have acces to Cron jobs.
AFAIK Tryton allows (or will allow) to cancel in/out invoices.
You’re right about modifications.
When state is rejected/accepted with errors that user must fix any configuration, and when invoice is cancelled.