Skip to content

fix(sharding): skip events with NULL sharding keys - #470

Merged
driv3r merged 1 commit into
mainfrom
fix/null-sharding-key-events
Oct 28, 2026
Merged

driv3r merged 1 commit into
mainfrom
fix/null-sharding-key-events

Conversation

@grodowski

Copy link
Copy Markdown
Member

Problem

Deleting rows with a NULL sharding key aborts an active move with parsing old sharding key: invalid type %!t(<nil>).

These rows should be ignored, consistent with the initial row copy.

Changes

  • Treat NULL sharding keys as nonmatching instead of invalid.
  • Check row-image presence when detecting updates entering or leaving the migrating shard. Transitions to or from NULL still fail safely.
  • Preserve errors for unexpected key types.
  • Add unit coverage for NULL inserts, deletes, updates, shard-boundary transitions, and sharding value zero.
  • Add an integration test that deletes a NULL-key row after the initial copy and verifies a subsequent write reaches the destination.

Validation

  • Full sharding suite passed against MySQL 8.0.
  • Root unit tests and copy-filter tests passed with the race detector.
  • The new integration test reproduces the original NULL-key error against the unmodified filter and passes with the fix.
  • An additional repeated integration run timed out during database seeding before Ghostferry initialized; local MySQL subsequently became unreachable.
  • Existing unrelated vet warnings remain.

Scope

Ghostferry still requires FULL binlog row images. This change does not add support for partial row images or relax cutover locking requirements.

Ignore NULL-key row events that do not belong to the migrating shard
instead of aborting the move. Preserve errors for updates entering or
leaving the shard, including transitions to or from NULL.

Add unit coverage and an integration test confirming binlog streaming
continues after deleting a NULL-key row.
@grodowski
grodowski requested a review from driv3r October 28, 2026 15:44
@driv3r
driv3r merged commit 26f131b into main Oct 28, 2026
19 checks passed
@driv3r
driv3r deleted the fix/null-sharding-key-events branch October 28, 2026 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants