Automated decision traps: GDPR 22 and AI Act 14 cheatsheet

It is 08:00 in Utrecht. The underwriter at a mid-size lender has the agent pulled up on her second screen. Next to her, the compliance officer is holding a printed checklist with fourteen items. Launch is Friday. Nothing ships until every box is green.

We have walked into some version of that room on every automated decision project we have done for a Dutch insurer or lender. The pattern is consistent enough that we keep the list pinned in the project channel. Half of the fourteen close with a UI tweak the compliance officer can request directly from product. Half force a backend engineer to add a human-in-the-loop write gate before the agent is allowed to touch the system of record.

This is that cheatsheet, in the order we work through it.

The two articles that frame the work

GDPR Article 22 says a data subject has the right not to be subject to a decision based solely on automated processing, including profiling, where that decision produces legal or similarly significant effects. Refusing a loan or pricing a premium counts. So does flagging a claim as fraudulent.

AI Act Article 14 sits on top of that. It says high-risk AI systems need effective human oversight: a natural person who can intervene, override, or stop the system. Credit scoring and insurance pricing are explicitly on the high-risk list in Annex III.

Read together, the rules are not subtle. If the agent writes a decline to the system of record without a human reviewer's sign-off, the company is on the wrong side of both. The interesting work is in the layered defences before that ever happens.

The official texts are short enough to keep open in a tab: GDPR and the AI Act. The Dutch data-protection authority publishes its own reading at the Autoriteit Persoonsgegevens; it is worth a quarterly skim if you ship in this market.

Group A: traps the compliance officer closes with a UI change

These seven are policy, copy, or form-flow issues. They do not need a backend engineer. They do need someone in product who can ship a banner, a link, or a paragraph by end of day.

1. The privacy notice does not describe the logic

Article 22(3) and recital 71 expect "meaningful information about the logic involved." A sentence saying "we use automated systems" is not it. The fix is one paragraph naming the features that drive the decision (income, payment history, existing debt, geo risk) and the high-level mechanism (statistical model, threshold rules). You do not have to publish the weights.

2. No 'ask for a human' affordance on the decline letter

The right to obtain human intervention has to be exercisable. A footnote pointing at info@ is not exercisable. The fix is a single CTA on the decline letter and the user portal: "Request review by a human officer." Wire it to a ticket queue with a 5-business-day SLA.

3. Consent banner conflates analytics with decision-making

We see this constantly. The cookie banner says "we use cookies to improve your experience and personalise offers." That sentence is doing two jobs and one of them needs Article 22 framing. Split the banner. Analytics on one toggle, automated profiling on another, and make the second one opt-in.

4. The customer-service script has no escalation path

When a rejected applicant calls, the agent on the phone needs a documented route to a human reviewer. We have seen scripts that say "the system has decided, there is no appeal." That sentence will lose a case. Update the script. Train the team.

5. No explanation request form in the portal

Beyond intervention, a data subject can ask for an explanation of a specific decision. The portal needs a form that captures the decision ID, the requester's identity, and a free-text question. The response gets drafted by the reviewer who handled the case, not by the model.

6. Consequences of automated processing buried in the T&Cs

The consequences disclosure has to be findable. We move it out of section 14 of the terms and put it on its own page, linked from the application form. One screen, plain Dutch, no nesting.

7. The cookie wall blocks the opt-out

A profiling opt-out that only appears if the user accepts cookies is not an opt-out. The fix is to render the privacy controls without requiring consent, and to gate the rest of the page behind whatever cookie strategy you want.

Those seven close in days, not weeks. They get the compliance officer to amber. Now the engineering work.

Group B: traps that need a backend write gate

These seven are the ones that delay launches. They are not bad copy. They are decisions about who can write what to the system of record, when, and with what trail.

8. The agent writes a final decision before a human signs

This is the original sin. The model scores, the agent updates the application status to "rejected" in the core system, and a letter goes out. A human looks at it the next morning. By then Article 22 has already been broken. The fix is to split the score step from the write step. The agent writes a proposal to a queue. A reviewer writes the decision.

The pattern we use, stripped down:

async function propose(application) {
  const result = await model.score(application);

  const proposal = {
    application_id: application.id,
    model_version: model.version,
    score: result.score,
    top_features: result.shap_top_5,
    proposed_outcome: result.proposed_outcome,
    confidence: result.confidence,
    status: 'awaiting_review',
    created_at: new Date().toISOString(),
  };

  await db.proposals.insert(proposal);
  await queue.enqueue('review', proposal.id);

  // No customer-facing change. No status update in the core system.
  return { proposal_id: proposal.id };
}

async function commitDecision(proposalId, reviewerId, outcome, reason) {
  const proposal = await db.proposals.findById(proposalId);
  if (proposal.status !== 'awaiting_review') {
    throw new Error('proposal already resolved');
  }
  await db.transaction(async (tx) => {
    await tx.proposals.update(proposalId, {
      status: 'reviewed',
      reviewer_id: reviewerId,
      reviewer_outcome: outcome,
      reviewer_reason: reason,
      reviewed_at: new Date().toISOString(),
    });
    await tx.applications.update(proposal.application_id, {
      status: outcome,
    });
    await tx.audit.insert({
      type: 'decision_committed',
      proposal_id: proposalId,
      reviewer_id: reviewerId,
      model_version: proposal.model_version,
    });
  });
}

The reviewer can copy the model's proposal verbatim. That is fine. What matters is that a human user ID is attached to every write, every time.

9. Reviewer dashboard shows a score but not feature contributions

Article 14 requires effective oversight. A reviewer staring at a 0.83 risk score with no context is not exercising oversight, they are rubber-stamping. The dashboard needs the top features that drove the score, with direction and magnitude. SHAP values, partial dependencies, whatever the model supports. Without that, the audit trail is meaningless.

10. No audit row tying decision to reviewer identity

Every committed decision needs a row in an immutable log with reviewer ID, model version, input snapshot hash, and timestamp. We have seen audit tables that only log the model's output. When the regulator asks who approved a specific decline three months ago, the answer cannot be "the system."

11. Input features are not persisted

If the model reads from a feature store that overwrites yesterday's values, the explanation for a decision made last Tuesday cannot be reconstructed. The fix is to snapshot the feature vector at decision time and store it with the proposal. Cheap. Boring. Saves you when the request for explanation arrives in month nine.

12. No threshold routing for edge cases

The agent should not be making borderline calls in autonomous mode at all. Establish a confidence threshold (we usually start at 0.90 and tune from there). Above it, fast-lane review with reviewer-approves-by-default. Below it, full review with reviewer-decides-from-scratch. Below a lower threshold (say 0.60), human-only with no model recommendation shown.

13. No kill switch

A reviewer or the head of underwriting needs a way to disable the agent in production within seconds. Not a deploy. A flag in a config table, checked on every call. We have walked into one shop where stopping the agent meant emailing the vendor. That is not effective oversight under Article 14.

14. The retraining loop ingests overrides without a flag

When a reviewer overrides the model, that override is a training signal. If you feed it back into the next training run unflagged, you are teaching the model that humans were right because humans were right, and the model converges on whatever the reviewers are already doing. Tag overrides. Hold them out of the next training run. Let the data-science team decide how to incorporate them deliberately.

Running the cheatsheet in practice

On a kickoff call, we send the list to the compliance officer and the engineering lead at the same time. Group A goes to product. Group B goes to backend. The two streams run in parallel for two weeks. We do not consider a pipeline shippable until both columns are green, and we structure the launch so autonomous mode is a flag we can flip back off in a single config change.

One detail worth flagging: the EDPB's guidelines on automated decision-making (WP251rev.01, still the working reference) treat "human review" as substantive only when the reviewer has authority and competence to change the outcome. A reviewer who can only rubber-stamp a model output is not a reviewer in the regulatory sense. That is why trap 9 matters as much as trap 8.

When we built the underwriting agent for a Dutch consumer lender last quarter, the thing we ran into was trap 11: the upstream feature store overwrote daily, and the first explanation request from a declined applicant arrived a month after the snapshot was already gone. We solved it by writing a frozen feature vector into the proposals table at decision time. Boring fix, important fix. Most of our AI agent work for regulated clients now ships with that snapshot from day one.

If you are about to ship an automated decision pipeline this week, do one thing today: open the code path that writes the application status and check who calls it. If the model can call it without a reviewer ID attached to the write, you have your first ticket.