Drupal 7 to Payload: a four-week shadow-traffic cutover

In late February a board member of a 22-person professional association in Leiden sent us a screenshot of their members' portal. The Drupal admin bar at the top was orange. The footer still said Powered by Drupal 7. The CMS that ran the site had been end-of-life since 5 January 2025, and the PHP underneath it had been unsupported since 2020. The site had not crashed. It also had not been patched in fifteen months. Their accountant had just flagged the SEPA pipeline in the annual audit. Six thousand two hundred active machtigingen lived in a custom sepa_mandates MySQL table, and the auditor wanted to know who was on the hook if a row went missing.

This is the playbook we ran to move it. Four weeks, shadow traffic, no downtime longer than a DNS TTL.

The shape of seventeen years of data

The first week was not code. It was a spreadsheet.

A Drupal 7 site that has run since 2009 (originally a D6 install, migrated to D7 in 2014, then left to age) is never just a CMS. By the time we opened phpMyAdmin the schema had 312 tables. Roughly 40 were Drupal core, 80 belonged to contributed modules the team had stopped using, and the rest were custom: members_extra, sepa_mandates, sepa_batches, committee_membership, event_registration_v2 (the v2 is doing work). The portal also wrote into GroupOffice over a homegrown webhook bus: a PHP cron every five minutes scraped the watchdog table for keyword rows, then POSTed them to a GroupOffice endpoint. Nobody on the current board had built that bus, and the original developer had moved to Australia.

We started by reading every table. Not querying, reading. The shape of the data tells you what the application actually does, which is usually 30% of what the docs say it does. If a legacy MySQL has a _v2, _old, or _backup table, grep the codebase for its name before assuming it is dead. We have twice found production reads against tables that were nominally archived.

The stack we picked, and why

The brief from the board was short: keep it boring, keep it Dutch-hostable, and keep us out of vendor lock-in. The new stack was Payload CMS on Node 22 for the admin and content model, Hono for the SEPA and webhook endpoints, and Postgres 16 for storage.

Payload's collection model is a JavaScript object you check into git, which fixes the single most painful thing about Drupal 7 migrations: content types living in the database. Hono runs on plain Node, on Bun, or on a worker, so we left ourselves a door even though we deployed to Node for now. Postgres replaced MySQL because the SEPA and audit logic wanted real transactions and proper enums; the MySQL 5.7 the old box ran did not even enforce CHECK constraints. We did not pick Strapi (admin too rigid for the committee logic), Directus (the team had no appetite for another admin SPA), or Laravel (a fair option, but the two in-house developers were already a Node shop).

The shadow-traffic plan

The risk in any migration that touches money is that you discover the bug after the customers have. We borrowed a pattern from the API world: shadow traffic. For two weeks before cutover, every request that reached the old Drupal site was also forwarded, asynchronously, to the new Payload and Hono stack. Both wrote to their own databases. Both produced their own SEPA exports. We diffed the exports nightly and surfaced any divergence on a Slack channel called #sepa-diff.

The forwarder was a small Caddy block in front of the existing nginx, with a mirror worker doing the duplication:

leden.example.nl {
    reverse_proxy old-portal:80
    # Mirror worker reads the access log tail and replays each
    # request body against new-portal:3000, fire-and-forget.
}

leden-shadow.internal {
    reverse_proxy new-portal:3000
}

The mirror itself was a Hono worker that copied the request body, replayed it against the new stack, and logged the response code, latency, and body hash to a Postgres table. Two weeks of mirror traffic gave us roughly 14,000 paired requests to diff. About 80 of them disagreed on day one. On day fourteen, the count was zero.

Week one: schema and read-side parity

Week one was about getting the data into Postgres and the read side of the portal to render. We used pgloader to do the bulk move from MySQL to Postgres, then layered a hand-written transform on top:

pgloader \
  --with "quote identifiers" \
  --with "data only" \
  --with "preserve index names" \
  mysql://reader:***@old-db/ledenportaal \
  postgresql://migrator:***@new-db/ledenportaal_raw

The raw schema went into a legacy Postgres schema. Nothing in legacy was touched after that. The new Payload-shaped tables lived in public, and a set of versioned SQL scripts (001_members.sql, 002_mandates.sql, and so on) read from legacy and wrote into public. We could rerun the whole pipeline in 40 seconds, which meant we could afford to be wrong fifty times.

The Payload collection for members ended up looking like this:

import type { CollectionConfig } from 'payload'

export const Members: CollectionConfig = {
  slug: 'members',
  admin: { useAsTitle: 'fullName' },
  fields: [
    { name: 'legacyId', type: 'number', unique: true, index: true },
    { name: 'fullName', type: 'text', required: true },
    { name: 'email', type: 'email', required: true, unique: true },
    { name: 'memberSince', type: 'date' },
    { name: 'committees', type: 'relationship', relationTo: 'committees', hasMany: true },
    { name: 'mandateRef', type: 'text', index: true }, // SEPA UMR
  ],
}

The legacyId field is the cheap insurance policy of every migration we have run. Keep the old primary key, indexed and unique, on every row. When something goes wrong six months later and a board member asks why their committee chair badge is missing, you can join back to the legacy schema in one query.

Week two: the SEPA gate

The 6,200 machtigingen were the part of the project that did not sleep well. A SEPA Direct Debit mandate is a legal artefact. It has a UMR (Unique Mandate Reference), a date of signature, an IBAN, and a status. If you lose the UMR or change the signature date, the bank will reject the next collection and the association will not get paid.

Two things saved us. First, we never let the new system write a mandate during the migration window. Mandates flowed one way: old portal into new database. Second, we added a verifier that compared every nightly SEPA pain.008 export between the two stacks and refused to allow cutover if any row diverged.

// hono-app/src/sepa/verify.ts
import { Hono } from 'hono'
import { compareExports } from './diff'

const app = new Hono()

app.get('/sepa/verify-tonight', async (c) => {
  const legacy = await fetchLegacyExport()   // .xml from the old PHP cron
  const fresh  = await buildFreshExport()    // generated from Postgres

  const diff = compareExports(legacy, fresh)
  if (diff.length === 0) {
    return c.json({ ok: true, mandates: legacy.count })
  }
  await postSlack('#sepa-diff', diff)
  return c.json({ ok: false, diff }, 409)
})

export default app

On day three of week two, the diff caught a real bug. The legacy PHP code stripped leading zeros from the BIC field on export. Our new export preserved them. The bank had been receiving stripped BICs for years, accepting them anyway, and the legacy was technically wrong. We matched the legacy behaviour for the migration window and filed a ticket to clean it up the week after.

Week three: the GroupOffice webhook bus

The PHP cron that scraped watchdog and POSTed to GroupOffice was the kind of code that makes you want to call its author. Ten files, no tests, magic numbers everywhere. We did not rewrite it. We wrapped it.

The new Hono app exposed exactly the same endpoint shape that the old cron POSTed to. For week three, the cron still ran, but it sent its payloads to a Hono route that did two things: forwarded the call to GroupOffice unchanged, and recorded the payload in Postgres. After five days we had a full replay log of every CRM event the portal had ever fired. We then wrote a new TypeScript event emitter that produced the same payloads from Payload's afterChange hooks, and replayed three days of live events through it against a staging GroupOffice. When the diffs matched, we shut off the PHP cron.

// Payload collection hook
hooks: {
  afterChange: [
    async ({ doc, operation, req }) => {
      if (operation !== 'create' && operation !== 'update') return
      await req.payload.create({
        collection: 'crm-events',
        data: {
          type: `member.${operation}`,
          payload: doc,
          dispatchedAt: null,
        },
      })
    },
  ],
}

A separate Hono worker drained the crm-events collection every 30 seconds and POSTed to GroupOffice, with retry, dead-letter, and the same magic numbers the old PHP had used. Boring is the goal.

Week four: the cutover hour

Cutover was 06:00 on a Sunday. The DNS TTL on leden.example.nl had been dropped to 60 seconds the previous Wednesday. The old portal went into read-only mode at 05:55. At 06:00 we flipped Caddy to send 100% of traffic to the new stack instead of mirroring it. The mirror kept running for 48 hours, in the opposite direction: every write that hit the new portal was replayed against the old one as well, in case we needed to roll back.

We did not roll back. Two members reported that their password did not work, which we expected: the old portal had stored passwords in a half-bcrypt half-MD5 hybrid that nobody could explain. We had pre-warned every member by email and added a reset-password notice on the login page. Both members reset in under a minute and never wrote again.

Three things we did the night before that we will keep doing forever:

1. Print the runbook on paper. The Wi-Fi in the association's office is from 2011 and chooses its own moments.
2. Lock the old database to read-only at the MySQL user level, not the application level. Application-level read-only is a setting. Database-level read-only is a fact.
3. Keep the old box running for 90 days after cutover. The cost of keeping a virtual machine alive is twelve euros a month. The cost of needing it and not having it is a board meeting.

Two regrets from the seam

We spent four days early on trying to reverse engineer the legacy SEPA export format from the PHP. We should have just diffed two real exports byte by byte from the start; we got there in week two and saved a week the next time we ran a migration like this. And we underestimated how much board energy a portal migration consumes. Twenty-two people is small enough that every member knows the people on the board, which means every login bug becomes a personal phone call. Communicating the cutover schedule turned out to be a third of the work.

The smallest thing you can do today, if you run a legacy Drupal 7 site: run SHOW TABLES on your database, then grep your codebase for each table name. Anything that returns zero hits is either dead or doing something nobody on your team knows about. Both are worth knowing.

When we built the new ledenportaal, the part that scared us was the SEPA pipeline. We solved it with a one-way mirror and a nightly diff against the legacy export, and we use the same shadow-traffic pattern on every legacy migration we take on. The pattern is older than we are. It just works.