Joomla 1.5 to Laravel 12: a 19-year migration playbook

The phone rings at 16:42 on a Friday. A container of refrigerated tuna is sitting at ECT Delta and the AGS declaration has to be filed before the closing window. The dossier clerk types the EORI into the CRM search. The page hangs. She refreshes. It hangs again. She opens phpMyAdmin in a second tab and runs the SELECT herself.

This is the working condition of about thirty Dutch and Belgian customs brokers we have talked to in the last two years. The site is a Joomla 1.5 install from 2008. The CRM is a custom module bolted on by a developer who emigrated to Spain. The dossier table is MyISAM, 187,000 rows, latin1_swedish_ci, and a single FULLTEXT index that nobody knows how to rebuild. Everything works until it does not.

This post is the playbook we used at one of those brokers. A 31-person operation in Rotterdam, two offices, around fourteen million euros of annual customs throughput. The brief was small in scope and large in nerves: replace the site and the CRM, keep every historic dossier searchable, do not break the Friday afternoon clearing rhythm. We ran the project over four months, with a parallel-run window and a single four-hour cutover.

What kept Joomla 1.5 alive for 19 years

Joomla 1.5 reached end of general support in September 2012. The Joomla project announced this clearly over a decade ago. The broker who ran the site did not stick with it because they did not know. They stuck with it because the dossier table held thirteen years of price history, AGS reference numbers, customer-side credit limits, and notes from staff who had since retired. The site was a database with a UI on top.

The site itself was three screens of value: a public marketing front, a customer login, and an internal CRM the staff actually lived in. Nobody minded that the marketing front looked like 2009. Everyone hated the CRM, and nobody dared touch it.

That is the pattern. Old PHP stays alive in industries where data outlives software, where the audit trail is regulated (customs, healthcare, accounting), and where the cost of a bad migration shows up as a fine rather than a refund. If you are going to replace it, the burden of proof is on you, not on the platform.

Mapping the dossier model before the schema

We did not open a code editor for two weeks.

The first job on a migration like this is to draw the entity model the business actually uses, not the one in the legacy schema. In customs work the entities are: dossier (one declaration), party (importer, exporter, broker), goods line (with HS code), document (BL, invoice, packing list, T1, EUR.1), and event (status changes from arrival to release). The Joomla schema had ten tables. The business used six entities. The mapping was not one-to-one. One of the old tables was a join table that nobody had populated in seven years. Another was a duplicate of the customer table from a 2014 import that everyone had forgotten about.

We did this by sitting in the office for three days. We watched what the dossier clerks actually searched for: an EORI number, a container number, a customer name with a typo, a date range, a vessel name. Every one of those queries goes into the new search index. Anything we never saw used got flagged for cold storage rather than the main index. The customer was relieved when we told them which tables they could leave behind.

Reshaping the database under load

The old database was MyISAM with latin1_swedish_ci. The new one is InnoDB with utf8mb4_0900_ai_ci. That is not a flag flip. Latin1 stores high-bit characters as single bytes that are not valid UTF-8. A direct dump and import gives you mojibake on every Dutch, Belgian, French, and German customer name in the table.

The conversion we used:

mysqldump --default-character-set=latin1 \
  --skip-set-charset --hex-blob \
  legacy_db > raw.sql

# rewrite charset declarations in the dump
perl -pi -e 's/latin1_swedish_ci/utf8mb4_0900_ai_ci/g; \
             s/CHARSET=latin1/CHARSET=utf8mb4/g' raw.sql

mysql --default-character-set=utf8mb4 new_db < raw.sql

The trick is that the bytes in the dump are already valid latin1. Dumping with --default-character-set=latin1 --skip-set-charset tells MySQL to give us the raw bytes without transcoding. Importing into utf8mb4 then reinterprets them correctly. Spot-check by selecting customer names that contain ë, é, ç, and ß. They should look right in the new database when you query it with --default-character-set=utf8mb4.

Search across 187,000 dossiers

A MyISAM FULLTEXT index is not portable to InnoDB without a rebuild, and InnoDB's tokenizer is bad at exact-match container numbers, EORI strings, and HS codes. It splits on punctuation and lower-cases everything, which is fine for prose and useless for codes.

We put Meilisearch in front. 187,000 dossiers fit in 240 MB on disk. A cold index build runs in six minutes. The Laravel side uses Scout with the Meilisearch driver:

// app/Models/Dossier.php
use Laravel\Scout\Searchable;

class Dossier extends Model
{
    use Searchable;

    public function toSearchableArray(): array
    {
        return [
            'id'            => (int) $this->id,
            'reference'     => $this->reference,
            'eori'          => $this->party_importer_eori,
            'container_nos' => $this->containers->pluck('number')->all(),
            'customer_name' => $this->customer?->name,
            'opened_at'     => $this->opened_at?->timestamp,
            'status'        => $this->status,
        ];
    }
}

The reason we ship opened_at as a Unix timestamp rather than a date string is so Meilisearch can filter and sort it natively. The reason we flatten container_nos into the document rather than joining at search time is so a clerk typing TGHU2851004 hits a single index lookup instead of three round trips through Eloquent.

Result: clerk types six characters, hits enter, sees results in 40 to 90 milliseconds on a twelve-euro-per-month Hetzner box. The old Joomla search took between two and eleven seconds depending on how many users were on the system.

EORI lookups under 400ms

An EORI is the EU's importer and exporter identifier. The validation reference is the EU's EORI validation service, available as both a web form and a SOAP endpoint. SOAP from a Dutch data centre to the Brussels endpoint is rarely under 300 milliseconds on a good day, and the service has been slow or unavailable for hours at a time.

So we never call it on the hot path.

The Laravel app keeps a local table of every EORI it has ever seen, with the validation result and a refresh timestamp. A scheduled job revalidates each one on a rolling window: seven days for active customers, ninety days for dormant ones. The clerk's lookup hits the local database, not Brussels. Cold-path validation for an EORI we have never seen is queued and shown as "checking" in the UI, with a banner the moment the answer comes back.

// app/Services/EoriLookup.php
public function lookup(string $eori): EoriResult
{
    $cached = EoriRecord::find($eori);

    if ($cached && $cached->fresh_enough()) {
        return EoriResult::fromCache($cached);
    }

    if (! $cached) {
        EoriRecord::create([
            'eori'      => $eori,
            'status'    => 'pending',
            'queued_at' => now(),
        ]);
        ValidateEoriJob::dispatch($eori);
    }

    return EoriResult::pending();
}

Median lookup latency on the cached path measured 38 milliseconds on production. The 400 millisecond budget the client gave us is comfortable, with headroom for the customer page that joins on three EORIs at once.

The email-agent layer

The CRM was the bigger half of the project. The email layer was the cheaper half that paid for itself in three weeks.

A customs broker's inbox is mostly the same five emails: a Bill of Lading from the carrier, a commercial invoice from the shipper, a packing list, a release request from the customer, and a question about ETA. Each one needs to be matched to a dossier and have its attachment filed.

The email agent runs on a dedicated mailbox that all carriers and customers BCC. It does three jobs and stops:

1. Match the incoming message to a dossier, by BL number, container number, or the reply-chain In-Reply-To header.
2. Classify the attachment and file it on the dossier under the right document type.
3. If the message contains a question the agent can answer from the dossier (a customer asking when a container arrives at ECT), draft a reply and put it in a pending-review queue. Never auto-send.

The "never auto-send" rule is not technical. It is contractual. A customs broker who auto-sends a wrong ETA to a customer is liable for the demurrage. We have not yet met a broker who wants to argue with their professional indemnity insurer about this.

This is also where we agree with the recent reminder that any CEO who thinks AI is replacing the employee is doing the math wrong. The dossier clerk who used to spend forty minutes a morning sorting attachments now spends six. The forty minutes did not become headcount savings. They became time she spends on the calls only she can make: the customer with a credit problem, the container with damaged seals, the rejected EUR.1 certificate.

The freeze, the cutover, the rollback

The hardest part of a migration like this is not technical.

We ran the old and the new system in parallel for two weeks. Writes went to both. Reads went to the old one. The dossier clerks were told to ignore the new UI for week one, and to use it as a read-only second screen in week two. By the end of week two, three of the five clerks had started using the new UI as their primary screen without being asked. That was the signal we waited for.

The cutover window was a Sunday morning at 06:00, four hours, no port traffic. The last delta sync ran from 05:00 to 05:40. DNS flipped at 06:00. The rollback plan was a documented dig command, a documented git revert, and a printed copy of the old database root password kept in the office safe. We never needed it. The Monday morning team meeting was, in the operations manager's words, "boring, which is what we paid for".

The old Joomla 1.5 site stayed online at legacy.[clientdomain].nl as a read-only archive for six months, then went down. Nobody noticed.

What we would do differently

Two things, both about scope.

First, the marketing front. We rebuilt it inside the same Laravel app on the assumption that the client would want to edit copy themselves. They did not. They asked us to handle copy edits as part of an annual retainer. We could have left the public site on the legacy host for another six months and saved two weeks of work, and the client's customers would not have known the difference.

Second, the document storage. We moved historic PDFs from local disk into S3 during the cutover. With hindsight we would have left them on local disk for the first month and migrated them in a second pass. Bundling them into the cutover added two hours to the freeze window and gave us nothing the business cared about on day one.

When we built the email-agent for this Rotterdam broker, the thing we ran into was that carrier emails come in seventeen different layouts and the BL number is rarely in the same place twice. We solved it by training the classifier on three years of the broker's own archived mail rather than on a generic shipping corpus, which is the kind of judgement that a legacy migration almost always rests on.

If you run something that started life as Joomla 1.5, ezPublish, or hand-rolled PHP 5, the cheapest five minutes you can spend today is to dump your top ten search queries from your access log, count the unique customer codes or EORIs they hit, and ask whether those queries would survive in the next system. The map of what the business actually does lives in that list, and the migration plan writes itself around it.