Joomla 3.10 rescue: 8,000 members, broken VAT, dead SSO

An 8,000-member professional association handed us the keys to their member portal at 4pm on a Friday. The board had spent three months trying to find someone to take over from the previous developer, who had quietly stopped answering email in late 2024. The treasurer had one question, and she asked it before we had even logged in: where are the VAT invoices from last year?

The site was Joomla 3.10.12. Members signed in with a custom SSO module that talked to an LDAP server the association also owned. Payments went through a third-party plugin that wrote receipts to a folder on disk and emailed them to members. Somewhere in 2023, that email had stopped firing, and nobody had noticed until the bookkeeper asked for the year-end export.

This is the story of the first two weeks. It's not a clean victory. It's the order we did things in, and the things we wish we had done first.

The first hour: read-only, no heroics

Before touching anything, we made a full snapshot. mysqldump of the database, a tar of the webroot, and a copy of the LDAP slapd config. We pulled them down over SFTP to a local encrypted volume and verified the dump with a checksum. Then we asked the host for read-only credentials and rotated every other password we had been given.

ssh association-prod \
  'mysqldump --single-transaction --quick --routines --triggers \
     -u backup_ro assoc_joomla | gzip -9' \
  > assoc_joomla-$(date +%Y%m%d).sql.gz

rsync -aH --numeric-ids \
  association-prod:/var/www/joomla/ ./webroot-snapshot/

We did this before reading a single line of PHP. Joomla 3.10 went end-of-life in August 2023, which means every undiscovered bug on that server is now a permanent feature unless we move it. A clean snapshot is the only thing standing between a rescue and a catastrophe.

We also stood up a staging copy on a separate host that same evening, behind HTTP basic auth, restored from the dump and pointed at a throwaway MySQL instance. Every change we proposed over the next two weeks was rehearsed there first, with the same PHP version, the same OpenSSL build, and the same time zone. Three of our planned interventions failed in staging in ways they would not have failed in development, including a Joomla cache rebuild that hung for forty minutes because the staging disk was on slow rotational storage. We would rather find that on staging than at midnight on prod.

Mapping the SSO before changing anything

The custom SSO was a Joomla authentication plugin in plugins/authentication/assocldap/. It bound to LDAP as a service account, searched for the user by an attribute called memberNumber, and on success wrote a Joomla session and a long-lived cookie called assoc_sso. That cookie was also honoured by a separate Drupal-based events site on a subdomain, which is how single sign-on worked across the two properties.

The cookie itself looked like this when we unwrapped one from a test login:

assoc_sso = base64(memberNumber|expiry_unix) . '|' . hmac_sha1(SECRET, base64_payload)

We spent half a day drawing this on paper before we touched it. Two things jumped out.

First, the cookie was signed with HMAC-SHA1 using a secret hardcoded in the plugin file. Anyone who had ever cloned the repo for local dev had that secret. There were no Git tags, no rotation history, and no record of who had pulled the codebase since 2017. Second, the plugin disabled Joomla's built-in two-factor for any user who came in through SSO, with a comment that read // TODO: re-enable once LDAP TOTP works. That TODO was dated 2019.

There was a parallel conversation to have with the volunteer who maintained the Drupal events subdomain. Their site verified the same cookie using a copy of the same secret, pasted into a sites/default/settings.php override years earlier. Any rotation had to happen on both servers inside the same maintenance window, or members would be logged out of one site and silently still logged into the other. We coordinated over a shared notes document and scheduled the swap for a Sunday morning when the events calendar was quiet.

We did not rip the plugin out. We wrote a one-page document that described what it did, where the secret lived, and which user classes bypassed 2FA. Then we shared it with the board before proposing any change. A legacy auth system is a load-bearing wall. You map it before you cut.

The missing VAT invoices

The payment plugin was a fork of a commercial Joomla extension whose vendor had gone dark in 2022. The fork lived in plugins/system/assocpay/ and handled SEPA direct debits for membership dues. It generated PDF invoices using FPDF and emailed them via Joomla's JMail wrapper.

We grepped the logs for the period the treasurer flagged. The PDFs were still being written to disk. The database row marking each invoice as sent = 1 was still being updated. But the mail.log on the server showed no outbound traffic for the invoice address from March 2023 onward.

The bug was three lines deep. The plugin used JMail::isHtml(true), then attached the PDF, then called send(). In a Joomla 3.10 point release from early 2023, the order of attachment processing changed inside the PHPMailer bundle. The plugin's call to setBody() happened after the attachment was registered, and the new PHPMailer silently rejected the body as malformed and returned true anyway. The plugin logged success. No mail left the building.

// What the plugin did. Worked until early 2023, then silently failed.
$mailer = JFactory::getMailer();
$mailer->isHtml(true);
$mailer->addAttachment($pdfPath);   // attachment registered first
$mailer->setBody($htmlBody);        // body set afterwards: rejected internally
$ok = $mailer->Send();              // returns true. nothing actually sent.

if ($ok) {
    $db->setQuery("UPDATE assoc_invoices SET sent = 1 WHERE id = {$id}")
       ->execute();
}

This is the kind of failure you only find by reading the actual PHPMailer source, not the plugin's wrapper. The treasurer had been right to worry. Eighteen months of invoices had been generated, stamped as sent in the database, and never delivered.

What we did about the invoices

We did not try to retroactively email 18 months of invoices in one go. That would have triggered every spam filter in the Netherlands and made a small problem into a public one. Instead:

1. We confirmed with the bookkeeper which invoices were legally required to be re-issued under Dutch VAT rules and which were merely useful to have. The Belastingdienst requirements on invoice retention are specific, and "the PDF exists on a server" is not the same as "the invoice was issued."
2. We wrote a one-off script that re-generated each invoice with a clear Duplicaat watermark and a covering note explaining the delay.
3. We staged the re-send over six weeks, batched by member cohort, with a contact address that went to a human on the board.

The covering note was three short paragraphs. It said that the association's records showed each invoice had been generated on its original date but a technical fault had prevented delivery, that the duplicate carried the original date for VAT purposes, and that any member who needed an adjustment for already-filed bookkeeping should reply to a named address. Out of 8,000 members we received two such replies. Both were handled in one email each. The board signed the note. We did not.

Stabilise, then move

The temptation on a job like this is to immediately propose a full rebuild on a modern stack. Joomla 4 or 5, a clean re-implementation of the SSO against OIDC, a hosted payment provider, the works. That conversation does need to happen. It is not the conversation for week one.

In week one and two we did the smallest set of changes that made the site safe to leave running for another quarter:

• Rotated the LDAP service account password and the SSO cookie secret. Forced a re-login for everyone. Communicated this two days in advance, in Dutch, with a screenshot of what the new login screen looked like.
• Patched the payment plugin's mail order-of-operations bug. Backported the fix into a tagged branch in a fresh Git repository, because the previous developer had been deploying by SFTP and there was no version control. Every file in the webroot got an initial commit dated 2026, with a short note explaining that prior history was unknown.
• Put the site behind a WAF with a rule blocking the known CVE-2023-23752 webservice-info exposure, which the unpatched 3.10 install was vulnerable to until we finished the cleanup. The same rule set blocked the four most common Joomla 3 brute-force user-agents at the edge, which cut admin-login attempts from roughly 9,000 a day to under 200.
• Wrote a runbook the board could read. Plain Dutch, no jargon, one page. It listed who to call when the site went down, how to put it in maintenance mode from the host's control panel without a developer, and where the latest snapshot lived. That single page closed about six months of recurring agenda items.

Only then did we open the conversation about a migration target. Joomla 5, a separate identity provider, and a payment provider with a real API. That work is now underway. It will take about four months. It would have failed in week one.

What this kind of work actually looks like

None of the above was glamorous. It was four days of reading other people's PHP, two days on the phone with a treasurer and a bookkeeper, one long evening with a packet capture watching LDAP binds, and a lot of careful note-taking. We bill it the same way we bill everything else, and we tell clients up front that the first two weeks are diagnosis, not delivery. The contract for those weeks names exactly that: a snapshot, an inventory of every plugin with its last known good version, a written map of the auth flow, and a remediation plan with rough effort estimates. No new features. No redesign. The client agreeing to that contract is the single highest-leverage decision in a rescue.

When we did the same exercise on a members' platform for a Dutch trade body last year, the failure mode was almost identical: a payment integration that had drifted out of compliance with a vendor update, and a custom auth layer that nobody was willing to touch. We solved it the same way: snapshot, map, stabilise, then propose the legacy migration. The rebuild is the easy part. The hard part is earning the right to do it.

If you've inherited one of these

Spend an hour today doing one thing. Open your payment plugin's mail-sending code and grep for the last time anyone changed the order of setBody, addAttachment, and send. Then check your mail server's outbound log for the date that file was last touched. If those two dates don't tell a consistent story, you have the same problem we did, and you have it right now.