Skip to content

Keep auto top-up invoices when billing e-invoicing is slow - #2988

Open
OlegPhenomenon wants to merge 1 commit into
masterfrom
2948-autotopup-invoice-billing-timeout
Open

OlegPhenomenon wants to merge 1 commit into
masterfrom
2948-autotopup-invoice-billing-timeout

Conversation

@OlegPhenomenon

Copy link
Copy Markdown
Contributor

Problem

On 23.07 registrars:reload_balance issued an auto top-up invoice (144677). It then sent the e-invoice to Billing synchronously, inside a DB transaction that wrapped all registrars. SendEInvoiceJob.set(wait: 1.minute).perform_now ignores wait. Billing waited 60.5 seconds for Omniva, the registry got Net::ReadTimeout and the transaction rolled the invoice back. Billing kept it and sent the e-invoice. The next run issued 144678, and the payment for 144677 arrived for an invoice the registry didn't have.

Side error: Billing's e_invoice_response callback for 144677 hit nil.update in EInvoiceResponseController. The registry returned a 500 HTML page, and Billing failed with JSON::ParserError: unexpected character: '<!DOCTYPE'.

The Billing side, which no longer waits for Omniva inside the request, is in internetee/eis_billing_system#274.

Changes

  • Registrar#issue_prepayment_invoice enqueues SendEInvoiceJob with perform_later instead of running it with perform_now. This covers every caller: the auto top-up task, admin and REPP add credit.
  • SendEInvoiceJob uses find_by instead of find, so a job for an invoice whose transaction was rolled back does nothing instead of retrying in Sidekiq.
  • registrars:reload_balance no longer has the transaction around the whole loop. A failure for one registrar is printed to the cron log and the rest carry on. Invoice creation and the pending flag are still in one transaction.
  • PUT /eis_billing/e_invoice_response returns a JSON 404 for an unknown invoice.

Not changed

  • GetInvoiceNumber and AddDeposits still run inside the top-up transaction. They are fast because the EveryPay link is generated locally. Moving them out means changing issue_prepayment_invoice for all callers.
  • No billing-side check against sending the same e-invoice twice. Skipping invoices with sent_at_omniva would block auction's manual "send e-invoice" button. The registry already skips invoices that were sent, through do_not_send_e_invoice?.
  • No HTTP retries or custom timeouts for billing POSTs. Retrying a non-idempotent POST is what creates duplicates.

Tests

  • test/tasks/registrars/reload_balance_test.rb:
    • an e-invoice timeout doesn't roll back the invoice, and the job is enqueued;
    • a billing error for one registrar doesn't roll back another registrar's invoice.
  • test/integration/eis_billing/e_invoice_response_test.rb: an unknown invoice gets a JSON 404.
  • All three new tests fail on the old code.
  • Related suites pass: registrar, invoice, add_credit, admin invoices, process_payments, invoice_email_cancellation and send_e_invoice_legacy (91 runs, 0 failures).

Closes #2948

registrars:reload_balance sent the e-invoice synchronously inside a DB
transaction wrapping all registrars. A 60s billing response caused
Net::ReadTimeout, rolled back the registry invoice while billing kept it,
and the next run issued a duplicate invoice.

- Enqueue SendEInvoiceJob instead of performing it inline
- SendEInvoiceJob skips invoices that no longer exist
- Drop the outer transaction; a failing registrar no longer rolls back others
- e_invoice_response returns JSON 404 for unknown invoices instead of a 500 HTML page

Refs #2948
@github-actions

Copy link
Copy Markdown
Contributor

🚀 Deploy Complete!

Property Value
App registry
Slot 1
URL https://registry1-dev.cloud.tld.ee
Namespace registry1-dev

(Environment ready for testing)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Autotopup invoice failed to be added to the registry

1 participant