Drupal 9 to Payload + Astro: an 11-week marketing rebuild

The publish button took fourteen minutes. Anouk, the marketing lead at a 34-person HR-tech SaaS in Rotterdam, stared at the progress bar with her tea going cold. A typo on the pricing page. One word. The Drupal admin spun, cache warmed, CDN flushed, varnish purged. By the time the page was live, she had answered eight Slack messages and forgotten what she had been doing.

That fourteen minutes was the symptom. The cause sat one layer down: a Drupal 9.4 install that had been good enough in 2022 and accumulated forty-one contrib modules since. Twelve of them were doing what the team's frontend now did natively. Six were abandoned. The site rendered fine. Publishing did not.

We were brought in to migrate it. Eleven weeks later the same edit happens in nine seconds, the 1,420 landing pages that drive their organic search still rank, and the HubSpot lead routing nobody dares touch still routes. This is how that went.

What the audit actually surfaced

The team had assumed they would land on Next.js with a headless WordPress or Sanity behind it. That was the request when they emailed us. We did not start with the stack. We started with two spreadsheets.

The first one listed every URL Google had indexed in the last six months. There were 1,420 of them. About 380 were programmatic location and role combinations ("hr software for accounting firms in Utrecht") that pulled 62% of their non-branded organic traffic. They had been generated by a custom Drupal module two engineers wrote in 2022, both of whom had left.

The second spreadsheet listed every webhook the Drupal site fired. Forty-three. Most pointed at HubSpot. The lead-routing flow used eleven of them. If a form on the careers page submitted at 04:00 UTC on a Tuesday it landed with the recruiter for technical roles in EMEA. Get that wrong and a senior backend candidate ends up in the inbox of someone selling to dental clinics in the US.

The audit told us what could not change.

• The 1,420 URLs had to keep returning 200 at the same paths.
• The eleven HubSpot webhooks had to fire with identical payloads.
• The programmatic landing pages had to keep generating from a data table, not from hand-edited files.
• Anouk and one other editor had to be able to publish a typo fix in under ten seconds.

Everything else was negotiable.

Why Payload and Astro, not Next.js and Sanity

We do a lot of Next.js work. It was the obvious choice. We did not pick it for this.

The marketing site does not need server-side rendering on every request. It needs to ship fast HTML, send a tiny amount of JavaScript, and rebuild quickly when an editor saves. Astro's islands architecture ships zero JavaScript by default and only hydrates the components that need it. The pricing calculator hydrates. The 1,200-word case study page does not. On the old Drupal site, the average page shipped 280KB of JS for a contact form. On Astro, the same page ships 11KB.

For the CMS we wanted something self-hosted, code-first, and Postgres-backed. Sanity would have worked, but their content lives in a managed cluster we cannot back up with pg_dump. Payload CMS is a Node app that writes to your own Postgres. The team's ops engineer already ran three Node services in their fly.io account. One more was tolerable.

The third reason was migration. Payload collections map cleanly to Drupal content types. Astro's content collections can pull from any source at build time, including a programmatic data table. We could move the 380 location pages without rewriting the generator.

The 1,420-URL problem

The cardinal rule of marketing-site migrations is that ranking URLs do not move. They keep their path, their canonical, and their meta description. Anything else is throwing organic traffic away.

We exported every indexed URL from Search Console, joined it with the Drupal database, and produced a three-column CSV: old path, new path, status. For 1,308 of them, old and new were identical. For 89 the path changed (mostly a /blog/ prefix the team had added in 2023 and regretted). For 23 the page had been deleted years ago but was still indexed. Those went to 410 Gone, not 301.

Astro handles redirects in astro.config.mjs. We generated the config from the CSV.

// scripts/build-redirects.mjs
import { readFileSync, writeFileSync } from 'node:fs'
import { parse } from 'csv-parse/sync'

const rows = parse(readFileSync('migration/url-map.csv'), { columns: true })

const redirects = {}
for (const row of rows) {
  if (row.status === 'moved' && row.old_path !== row.new_path) {
    redirects[row.old_path] = {
      destination: row.new_path,
      status: 301,
    }
  }
}

writeFileSync(
  'src/redirects.generated.json',
  JSON.stringify(redirects, null, 2)
)
console.log(`Wrote ${Object.keys(redirects).length} redirects`)

The 23 deleted-but-indexed URLs live in a separate file the Astro middleware reads and returns 410 for. Google de-indexes 410s faster than 404s, which matters when you have a year of clean-up debt to clear.

The HubSpot webhook bridge

This was the part the team was nervous about. The Drupal site had a custom module that did three things on every form submission: enrich the lead with company data, route it to one of eleven HubSpot owners based on a fifty-line decision tree, and fire a Slack message into the channel for that owner's team.

Touching that module was out. It had been written by an engineer who had left, was undocumented, and was load-bearing for the sales team's quota.

So we did not touch the logic. We extracted the decision tree into a single Cloudflare Worker that read the same Clearbit API and called the same HubSpot Contacts API to produce an identical payload. Then we dual-fired for two weeks: every form submission went to both the old Drupal module and the new Worker. A cron diffed the resulting HubSpot contact records every hour. After fourteen days with zero diffs, we cut Drupal off.

Content migration from Drupal nodes to Payload collections

Drupal stores content in a node table with field tables for every field. There were 47 field tables. We did not write a SQL-to-SQL migration. We wrote a script that called Drupal's JSON:API endpoint, normalized each node into the Payload schema, and POSTed it to Payload's REST API.

// migration/import-articles.mjs
import { getPayloadClient } from './payload-client.mjs'

const DRUPAL = 'https://old.example.com/jsonapi/node/article'

async function* fetchAllArticles() {
  let url = `${DRUPAL}?page[limit]=50`
  while (url) {
    const res = await fetch(url, {
      headers: { Accept: 'application/vnd.api+json' },
    })
    const json = await res.json()
    for (const node of json.data) yield node
    url = json.links?.next?.href ?? null
  }
}

const payload = await getPayloadClient()

for await (const node of fetchAllArticles()) {
  await payload.create({
    collection: 'articles',
    data: {
      title: node.attributes.title,
      slug: node.attributes.path.alias.replace(/^\//, ''),
      body: node.attributes.body.processed,
      publishedAt: node.attributes.created,
      legacyNid: node.attributes.drupal_internal__nid,
    },
  })
}

We kept legacyNid on every record. Three weeks after launch we still use it to debug which Drupal node a given Payload entry came from. The team asked us to drop it before launch. We refused. It costs nothing and it has paid off twice.

Once the migration ran clean, we needed to retire the old Drupal database. There was a moment, around week eight, when we realized we were going to leave behind seventeen tables of node revisions, six gigabytes of file references, and a flag_count table that had been collecting toggles since 2019. We did not delete rows. We took a final snapshot, dropped the schema, archived the dump, moved on. Surgical deletes on Postgres tables that size are slow and risky. Clearing the schema in one shot is fast, obvious, and impossible to half-finish.

The 9-second publish, explained

Fourteen minutes was Drupal cache warming, Cloudflare purge, varnish flush, and a CDN edge that took its time. Nine seconds is Astro's incremental build, a Vercel deploy hook, and an edge cache invalidation on the changed paths only.

The breakdown looks like this:

• Editor saves in Payload: 200ms (DB write plus a revalidation webhook).
• Vercel deploy hook fires: 100ms.
• Astro incremental build of the changed routes: 6.2s on average.
• Edge cache purge for the affected URLs: 1.8s.
• Buffer and network: about 700ms.

The trick is that Astro only rebuilds what changed. A typo fix on the pricing page rebuilds two routes (the page and the sitemap), not the entire 1,420-page site. Full builds, which we run nightly, take four minutes forty seconds. Editors do not see those.

What happened to organic rankings

The first three weeks after switchover are when a marketing-site migration either survives or does not. We watched four numbers in Search Console at 09:00 every morning: total clicks, total impressions, average position on the top-fifty queries, and the count of URLs returning 200 versus 3xx versus 4xx versus 5xx.

Total clicks dipped 4% in week one, recovered in week two, and were up 7% by week six. The dip lined up almost exactly with Google re-crawling the 89 moved paths and the 23 410s. Once the new sitemap finished propagating, traffic stabilized at a slightly higher baseline than the Drupal site had carried for the previous quarter. We did not buy that recovery. We earned it by not breaking the URL structure.

Average position moved less than half a place on the top fifty queries. The pages that did lose ground were three blog posts from 2021 that had never ranked well to begin with and whose internal links we had pruned on purpose. The pages that gained were the programmatic location pages, which now loaded in under 400ms instead of two seconds. Core Web Vitals reward that more than people realize.

The number we watched most carefully was the count of URLs Google still saw as 200. It started at 1,397 (we had returned 410 for the 23 dead pages from day one), held steady through week three, and ticked up by week six as the index finished settling. No surprises, which on a migration is the only outcome worth wanting.

Three things we would change

We did not get everything right.

We underestimated the image migration. Drupal stored 11,400 images with derived styles in seventeen sizes. Payload's media collection wanted originals and re-derived on demand. The first pass tried to migrate everything. We killed it on day three and migrated only the 2,800 images referenced by live content. The other 8,600 sit in a bucket nobody reads.

We should have written a smaller initial decision tree for the HubSpot routing and grown it back from the diff log, instead of reproducing all fifty lines on day one. Half of those lines existed for a 2022 campaign that ended in 2023.

We let the team keep four Drupal contrib module behaviors as Astro components when we should have asked harder whether they earned the complexity. One of them, a "related insights" carousel on every blog post, gets clicked 0.3% of the time. The team has now removed it.

The smallest thing you could do today

If you have a Drupal or WordPress site of any size, open Search Console, export the URLs that have received at least one click in the last 90 days, and put them in a spreadsheet. That list is what cannot move. Everything else is a candidate to retire. Most migrations fail because nobody made that list before they started.

When we built this legacy migration for the Rotterdam team, the thing we kept running into was that the parts of Drupal everyone complained about (the publish wait, the editor UX) were not the parts that were load-bearing for the business. The load-bearing parts were the URL structure and the webhook payloads. We rebuilt the loud problems and preserved the quiet ones. That was the whole job.