Skip to content

ENH/DOC: Add advance_step and warn against splitting PCG streams by powers of two - #505

Merged
bashtage merged 7 commits into
mainfrom
wip-doc-pcg-advance-powers-of-two
Nov 4, 2026
Merged

bashtage merged 7 commits into
mainfrom
wip-doc-pcg-advance-powers-of-two

Conversation

@bashtage

@bashtage bashtage commented Nov 4, 2026 •

Copy link
Copy Markdown
Owner

closes #450

Documents the problem Sebastiano Vigna reported in #450. Splitting a single PCG stream with advance by a large power of two gives strongly related streams. Also adds advance_step(max_draws), which returns a safe step for splitting a stream.

An LCG modulo 2**128 (or 2**64 for PCG32) has lowest e bits that repeat every 2**e steps, so advancing by 2**e leaves those bits unchanged. After advance(2**127), the two states differ only in the top bit at every step.

Changes

  • New static method advance_step(max_draws) on PCG64 (inherited by PCG64DXSM), LCG128Mix and PCG32:
    • It returns the odd integer closest to 2**XX / phi for the smallest XX with 2**XX / phi > max_draws. For example, PCG64.advance_step(10**10) is 0x278dde6e5 (XX = 34).
    • It uses only integer arithmetic. The integer part of 2**XX / phi is (isqrt(5 * 4**XX) - 2**XX) // 2, which is exact because sqrt(5) * 2**XX is irrational. XX is either max_draws.bit_length() or one more, so at most two isqrt calls are needed.
    • The step is never smaller than max_draws, so streams advanced by i * step don't overlap while n * step <= 2**128 (2**64 for PCG32).
    • max_draws must be between 1 and the integer part of 2**128 / phi (2**64 / phi for PCG32), or a ValueError is raised. Non-integers raise TypeError. NumPy integers are accepted.
    • The shared code is a cdef function, golden_advance_step, in common.pyx. Type stubs are updated.
    • .. versionadded:: 2.6.0.
    • Each advance docstring points to it three times: in the Warnings box, in a new See Also section, and in the Notes, whose example now uses it. It is listed under "Parallel generation" on the API pages of all four generators, and parallel.rst mentions it and uses it in its example.
  • The overlap limit on the number of streams:
    • Streams advanced by i * step are only guaranteed not to overlap when n * step is at most the period, because the starts i * step are taken modulo the period.

    • Every advance docstring states this limit in its Warnings box and in its Notes, together with the rule that the step must be at least the number of draws per stream.

    • The advance_step Notes say it does not check the limit, because it doesn't know n. I left it that way on purpose: there is no num_streams argument.

    • For PCG64, PCG64DXSM and LCG128Mix the docs note that the limit is not practical: about 2.5e26 streams of 10**12 draws.

    • PCG32's period is only 2**64, so the PCG32.advance and PCG32.advance_step Notes include a table of the largest number of streams. A summary appears in its Warnings box and class docstring, and in parallel.rst.

      Draws per stream Streams
      10**6 about 1.4e13
      10**9 about 1.4e10
      10**12 about 1.4e7
      10**15 13,254
      10**18 12

      They also give the simpler rule that n streams of m draws always fit when n * m <= 2**63, because the step is less than 2 * max_draws. They recommend PCG64, PCG64DXSM or LCG128Mix when many streams or many draws per stream are needed.

    • The "Parallel Features" sections of the PCG64, PCG64DXSM and PCG32 class docstrings now point to advance_step for splitting a stream. They previously named a fixed step.

  • Release notes (change-log.rst, 2.6.0):
  • PCG64.advance (which PCG64DXSM inherits), LCG128Mix.advance and PCG32.advance get a numpydoc "Warnings" section. Napoleon renders it as a warning box. Each one says what happens to that generator's output functions:
    • "xsl-rr": the 32-bit halves are swapped and one bit is flipped. More generally, 2**(128 - t) gives rotations with at most t bits flipped.
    • "dxsm", "dxsm-128" and PCG64DXSM: the outputs are never congruent modulo 4, and PractRand fails the interleaved streams at 1 MB.
    • LCG128Mix:
      • "upper": identical except for the top bit. In general, the low 64 - t bits are shared.
      • "lower": identical, as after any advance by a multiple of 2**64.
      • "murmur3": no visible pattern.
    • PCG32 with 2**63: the 16-bit halves are swapped and one bit is flipped.
    • Each section ends with the size of the effect after advance(2**64) (2**32 for PCG32) and points to the Notes.
  • The Notes of the same three methods explain how to choose a step for splitting a stream: advance stream i by i * step with an odd step.
    • Any odd step makes the states of two streams differ as if at random, whatever its size, apart from the lowest few bits, which repeat every few steps in any LCG. About half the bits differ, with a binomial spread.

    • The size of the step only has to exceed the number of draws per stream, so that stream i doesn't run into stream i + 1.

    • A convenient step is the odd integer closest to 2**XX / phi, with XX chosen to fit the draws per stream. The examples given:

      • XX = 48: 0x9e3779b97f4b, about 1.7e14 draws per stream.
      • XX = 64: 0x9e3779b97f4a7c15, about 1.1e19.
      • XX = 128: NumPy's PCG64.jumped step.
      • XX = 32 for PCG32: 0x9e3779b9, about 2.7e9.

      Each has a short advance(i * step) example.

    • Streams i and j share the lowest v state bits, where 2**v is the largest power of two dividing j - i. With fewer than 2**20 streams that is at most 19 bits. NumPy's "Upgrading PCG64 with PCG64DXSM" page reports that XSL-RR streams sharing their lowest 58 bits (and the increment) fail PractRand when interleaved.

    • jumped(iter) multiplies its golden-ratio step by iter, so iter should not be a large power of two.

  • The "Parallel Features" examples of PCG64, PCG64DXSM and PCG32 now say that each instance uses its own inc, and that a single stream should be split with jumped (see advance).
  • In doc/source/parallel.rst, the advancing section gets a short warning with the recommended step and an example, and now lists PCG64DXSM among the PCG generators that support advance.
  • Fixed the return type in the LCG128Mix.advance docstring, which said PCG64.

Checked

  • I reproduced every relation listed above on this branch, using the installed package:
    • xsl-rr, upper, lower and PCG32: checked directly. For upper and xsl-rr, the general 2**(128 - t) statements were checked for several t.
    • dxsm: no draw was congruent modulo 4 to its partner in 100,000 draws.
    • The PractRand result is quoted from advance() by large powers of two yields correlated streams #450; I did not re-run it.
  • I measured how the state bits of two streams differ over 50,000 steps, for each LCG: both PCG64 multipliers, LCG128Mix and PCG32.
    • Steps tested: odd 2**XX / phi for XX = 16, 24, 32, 40, 48, 56, 64, 96 and 128 (8 to 64 for PCG32), and odd steps 1, 3 and 999.
    • Bit 0 always differs.
    • Bits 1 to about 5 follow fixed patterns.
    • Bits 16 and up differ at a rate of 0.493 to 0.507, with no correlation between neighbouring bits (largest |corr| about 0.01). Their number of differing bits passes a chi-squared test against a binomial, with p-values from 0.02 to 0.99.
    • Overall, about 64.5 of 128 bits differ with a spread of about 5.6 (32.5 of 64 and 4.0 for PCG32), as for random flips.
    • Steps that fail:
      • 2**127 gives 1 differing bit.
      • 2**64 gives 32, and the lowest 64 never differ.
      • 2**32 gives 48, and the lowest 32 never differ.
      • The old even jumped step (...834) shares its lowest 2 bits.
    • Size doesn't matter: steps 1, 3 and 999 look just as random. That is why the docs say the size matters only for overlap.
  • NumPy's docs:
    • "Parallel Applications" covers only jumped, whose golden-ratio step "spreads out the states around the period optimally", and the need not to reuse jump indices.
    • PCG64.advance has no warning about step sizes.
    • "Upgrading PCG64 with PCG64DXSM" says advance(2**N) leaves the lowest N bits identical, and gives the 58-bit PractRand result quoted above.
  • advance_step:
    • The integer formula agrees with 200-digit decimals on 1,172 cases: every XX boundary (the integer part of 2**XX / phi, and one either side) and powers of two, for 64 and 128 bits.
    • New tests in test_direct.py:
      • Every boundary case against a decimal reference.
      • Known values for all four generators, called on the class, an instance and a NumPy integer.
      • advance_step(2**127) is NumPy's PCG64.jumped step.
      • The error cases.
      • Streams split with advance_step(1000) match consecutive 1000-draw blocks of one stream, so they don't overlap.
      • The step is always less than 2 * max_draws. Separately, 200,000 random max_draws values gave a largest ratio of 1.99999.
      • Every stream limit quoted in the docstrings matches 2**64 // step (2**128 // step for the 128-bit generators).
  • The eight advance and advance_step docstrings go through Napoleon and docutils, each advance producing one warning box. All 28 docstring examples pass under doctest, including the hex(step) outputs. parallel.rst parses cleanly and its example runs.
  • A minimal nitpicky Sphinx 9.1.1 build of the four generator pages, parallel.rst and change-log.rst produces each advance_step page.
    • The See Also links resolve both ways for every generator, PCG64DXSM's inherited docstrings included.
    • The parallel.rst and change-log links resolve.
    • The PCG32 pages render the stream table, and their links to PCG64, PCG64DXSM and LCG128Mix resolve.
    • No new warnings. The only ones about advance are the existing delta : integer, positive type warnings, plus change-log references to pages this minimal build leaves out.
  • WSL, Python 3.12, after merging main: the branch builds, and the full suite gives 8754 passed, 158 skipped.

🤖 Generated with Claude Code

bashtage and others added 2 commits November 4, 2026 08:39
Advancing an LCG modulo 2**128 (2**64 for PCG32) by 2**e leaves the
lowest e bits of the state unchanged, so splitting one stream with
advance(2**127) gives two streams whose states differ only in the top
bit, and whose outputs are strongly related (issue #450).

Add a Warnings section to PCG64.advance (also used by PCG64DXSM),
LCG128Mix.advance and PCG32.advance that describes the effect on each
output function, point the Parallel Features examples to it, and add
a warning to the advancing section of parallel.rst. Also list
PCG64DXSM there and fix the return type in the LCG128Mix.advance
docstring.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Suggest advancing by multiples of the odd integer closest to
2**128 / phi (2**64 / phi for PCG32). With an odd step the states of
two streams always differ in the lowest bit and, on average, in about
half of their bits, while 2**127 changes one bit and 2**64 never
changes the lowest 64 bits. Add an example to parallel.rst.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.81%. Comparing base (8c71290) to head (338711e).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #505      +/-   ##
==========================================
+ Coverage   94.76%   94.81%   +0.04%     
==========================================
  Files          56       56              
  Lines        9837     9923      +86     
  Branches      206      214       +8     
==========================================
+ Hits         9322     9408      +86     
  Misses        495      495              
  Partials       20       20              
Flag Coverage Δ
adder 94.74% <100.00%> (+0.04%) ⬆️
subtractor 94.74% <100.00%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@bashtage
bashtage marked this pull request as ready for review November 4, 2026 10:37
bashtage and others added 2 commits November 4, 2026 10:44
Any odd step makes the states of two streams differ as if at random,
apart from their lowest few bits, whatever its size: about half of the
bits differ, with a binomial spread. The size only has to exceed the
number of draws per stream, so that the streams do not overlap. Say so
in the Notes of the PCG advance methods, give odd golden-ratio steps
2**XX / phi for smaller XX (48 and 64 for the 128-bit generators, 32 for
PCG32), say how many low bits streams i and j share, and cite NumPy's
58-bit result for XSL-RR. Keep the Warnings to the problem itself, and
use the 2**64 / phi step in the parallel.rst example.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PCG64, PCG64DXSM, LCG128Mix and PCG32 gain a static method
advance_step(max_draws). It returns the odd integer closest to
2**XX / phi for the smallest XX with 2**XX / phi > max_draws, using
only integer arithmetic: the integer part of 2**XX / phi is
(isqrt(5 * 4**XX) - 2**XX) // 2. The step is never smaller than
max_draws, so streams advanced by i * step do not overlap.

The advance docstrings point to it from the Warnings, a See Also
section and the Notes, whose examples now use it, and parallel.rst
does the same. The method is listed on each generator's API page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bashtage bashtage changed the title DOC: Warn against advancing PCG generators by large powers of two ENH/DOC: Add advance_step and warn against splitting PCG streams by powers of two Nov 4, 2026
bashtage and others added 3 commits November 4, 2026 13:28
…owers-of-two

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
List advance_step under the new features, and mention it in the
entry for the advance warnings (#450).

With the odd jumped steps from #506 now merged, the advance Notes
say that the XX = 128 step (XX = 64 for PCG32) is also the step of
jumped, and a test checks that jumped(i) equals advance(i * step)
for that step.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Streams advanced by i * step are only guaranteed not to overlap when
n * step is at most the period. Every advance docstring now says so in
its Warnings and Notes, with the requirement that the step be at least
the number of draws per stream, and the advance_step Notes say that it
does not check the limit because it does not know n.

The period of PCG32 is only 2**64, so for PCG32 the docs of advance,
advance_step and the class give the largest number of streams for
1e6 to 1e18 draws per stream, note that n streams of m draws fit when
n * m <= 2**63, and recommend PCG64, PCG64DXSM or LCG128Mix when many
streams or many draws per stream are needed. parallel.rst does the
same. The class docs of PCG64, PCG64DXSM and PCG32 now point to
advance_step for splitting a stream.

Tests check the documented limits and that the step is less than
2 * max_draws.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bashtage
bashtage merged commit 0a3fcf3 into main Nov 4, 2026
44 checks passed
@bashtage
bashtage deleted the wip-doc-pcg-advance-powers-of-two branch November 4, 2026 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

advance() by large powers of two yields correlated streams

1 participant