Skip to content

JSON: skip the whole inlined value entry in large-format documents - #163

Open
cyppe wants to merge 3 commits into
krowinski:masterfrom
cyppe:fix/json-large-inline-slot
Open

cyppe wants to merge 3 commits into
krowinski:masterfrom
cyppe:fix/json-large-inline-slot

Conversation

@cyppe

@cyppe cyppe commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

In the binary JSON format, value entries have a fixed width: 1 type byte plus 2 bytes in the small format, or 4 bytes in the large format, which is used for documents over 64 KiB (sql-common/json_binary.cc). A literal or a 16-bit integer is inlined in its entry and is shorter than a large entry.

parseArrayOrObject() skipped the unused bytes only for INT16/UINT16. An inlined true, false or null in a large document left 2 bytes unread, so every later entry was read from the wrong position. On MySQL 8.4, '{"a":true, ..., "f":"<70 KB>"}' throws LengthException: The offset for the value in the JSON binary document is 132096, which is larger than the binary form of the JSON document. Other layouts can decode wrong values instead.

The fix measures how far the scalar moved the cursor and skips the rest of the entry, for every inlined type.

Tests:

  • Unit: JsonBinaryDecoderServiceTest::testShouldSkipTheWholeInlinedSlotInLargeArrays and ...InLargeObjects decode a hand-built large array [true, 7, null] and object {"a":true,"b":7}.
  • Integration: TypesTest::testShouldDecodeLiteralsAndShortIntsInsideLargeJsonDocuments decodes a large object and a large array with literals and small integers.

Both fail on master and pass with the fix. The full suite passes locally against MySQL 8.4.13 and MariaDB 11.4 (PHP 8.4). The PHP 8.2 CI cells fail on master too (PHPUnit 12 needs PHP 8.3, see #159).

This is the third of a few small, independent decoder fixes, after #161 and #162; it touches only the JSON decoder.

Value entries are fixed width: 1 type byte plus 2 bytes (small) or 4
bytes (large, documents > 64 KiB). Only INT16/UINT16 skipped the unused
slot bytes; an inlined literal (true/false/null) in a large document left
2 bytes unread, so every following entry was read off by two. On MySQL
8.4.8 '{"a":true, ... 70 KB ...}' threw a LengthException, and other
layouts can decode wrong values. Skip whatever the scalar left of the
slot, for every inlined type.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant