Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/bit_generators/lcg128mix.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ Parallel generation
:toctree: generated/

~LCG128Mix.advance
~LCG128Mix.advance_step
~LCG128Mix.jumped

Extending
Expand Down
1 change: 1 addition & 0 deletions doc/source/bit_generators/pcg32.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ Parallel generation
:toctree: generated/

~PCG32.advance
~PCG32.advance_step
~PCG32.jumped

Extending
Expand Down
1 change: 1 addition & 0 deletions doc/source/bit_generators/pcg64.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ Parallel generation
:toctree: generated/

~PCG64.advance
~PCG64.advance_step
~PCG64.jumped

Extending
Expand Down
1 change: 1 addition & 0 deletions doc/source/bit_generators/pcg64dxsm.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ Parallel generation
:toctree: generated/

~PCG64DXSM.advance
~PCG64DXSM.advance_step
~PCG64DXSM.jumped

Extending
Expand Down
15 changes: 14 additions & 1 deletion doc/source/change-log.rst
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,18 @@ New features
They used to set a flag shared by the whole process, and creating or
reseeding any instance turned SIMD back on. The output is the same with
and without SIMD.
- Added ``advance_step(max_draws)`` to :class:`~randomgen.pcg64.PCG64`,
:class:`~randomgen.pcg64.PCG64DXSM`, :class:`~randomgen.pcg64.LCG128Mix`
and :class:`~randomgen.pcg32.PCG32`
(:meth:`~randomgen.pcg64.PCG64.advance_step`). It returns a step for
splitting one stream into several, where stream ``i`` is advanced by
``i * step`` and makes at most ``max_draws`` draws. The step is the odd
integer closest to :math:`2^{XX}/\phi`, where :math:`\phi` is the golden
ratio, for the smallest :math:`XX` with :math:`2^{XX}/\phi` larger than
``max_draws``, and is computed with integer arithmetic only. Because it is
odd, the states of the streams differ as if at random, apart from their
lowest few bits. Because it is at least ``max_draws``, ``n`` streams do not
overlap as long as ``n * step`` is no larger than the period.

.. _stream-changes-2-6-0:

Expand Down Expand Up @@ -160,7 +172,8 @@ Other fixes
- A failed aligned allocation raises ``MemoryError`` instead of crashing,
and HC-128 no longer relies on unaligned pointer casts.
- The ``advance`` methods of the PCG generators warn against splitting one
stream with a large power of two, and suggest an odd step
stream with a large power of two, and explain how to choose an odd step,
which ``advance_step`` computes
(`#450 <https://github.com/bashtage/randomgen/issues/450>`_).
- The documentation of ``random(dtype=np.longdouble)`` explains why it gives
different values across platforms.
Expand Down
46 changes: 45 additions & 1 deletion doc/source/parallel.rst
Original file line number Diff line number Diff line change
Expand Up @@ -109,10 +109,54 @@ required to ensure exact reproducibility.

In addition to the cryptographic PRNGs,
the PCG-based generators also support ``advance``: :class:`~randomgen.pcg64.PCG64`,
:class:`~randomgen.pcg64.LCG128Mix`, and :class:`~randomgen.pcg32.PCG32`.
:class:`~randomgen.pcg64.PCG64DXSM`, :class:`~randomgen.pcg64.LCG128Mix`, and
:class:`~randomgen.pcg32.PCG32`.
Note that :class:`~randomgen.hc128.HC128` is based on a stream cipher and so
does not support advancing a counter.

.. warning::

Do not split a single stream of a PCG-based generator by advancing it by a
large power of two, such as :math:`2^{127}` (:math:`2^{63}` for
:class:`~randomgen.pcg32.PCG32`). These generators are built on an LCG
modulo a power of two, whose lowest :math:`e` bits repeat every
:math:`2^e` steps, so the streams share the lowest bits of their states and
their outputs are strongly related. For example, after advancing
:class:`~randomgen.pcg64.PCG64` by :math:`2^{127}`, each output is the
other with its 32-bit halves swapped and one bit flipped. The ``advance``
method of each generator describes the effect on its output functions.

Instead, advance stream ``i`` by ``i * step`` with an odd ``step``. With
any odd step the states of two streams differ as if at random, apart from
their lowest few bits. The size of the step matters for overlap: it must be
at least the number of draws that each stream makes, and ``n`` streams are
only guaranteed not to overlap when ``n * step`` is no larger than the
period. A convenient step is the odd integer closest to
:math:`2^{XX}/\phi`, where :math:`\phi` is the golden ratio, for example
``0x9e3779b97f4a7c15`` (:math:`XX = 64`, about :math:`1.1 \times 10^{19}`
draws per stream) or, for :class:`~randomgen.pcg32.PCG32`,
``0x9e3779b9`` (:math:`XX = 32`, about :math:`2.7 \times 10^{9}` draws per
stream). Each generator's ``advance_step`` method, such as
:meth:`PCG64.advance_step <randomgen.pcg64.PCG64.advance_step>`, takes the
largest number of draws that any stream makes and returns this step for the
smallest :math:`XX` that works. ``jumped`` uses a golden-ratio step and is
also a good choice.

The period of :class:`~randomgen.pcg32.PCG32` is only :math:`2^{64}`, which
limits it to, for example, about :math:`1.4 \times 10^{7}` streams of
:math:`10^{12}` draws. Use :class:`~randomgen.pcg64.PCG64`, or the related
:class:`~randomgen.pcg64.PCG64DXSM` or :class:`~randomgen.pcg64.LCG128Mix`,
whose period is :math:`2^{128}`, when many streams or many draws per stream
are needed.

.. code-block:: python

from randomgen import PCG64

# Each stream makes at most 10**12 draws
STEP = PCG64.advance_step(10**12)
streams = [PCG64(1234).advance(i * STEP) for i in range(16)]

.. _jumping:

Jumping the PRNG state
Expand Down
1 change: 1 addition & 0 deletions randomgen/common.pxd
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ cdef object random_raw(bitgen_t *bitgen, object lock, object size, object output
cdef object prepare_cffi(bitgen_t *bitgen)
cdef object prepare_ctypes(bitgen_t *bitgen)
cdef object wrap_int(object val, object bits)
cdef object golden_advance_step(object max_draws, object bits)
cdef object check_state_array(object arr, np.npy_intp required_len,
int required_bits, object name)
cdef object check_state_int(object value, str name, object lower, object upper)
Expand Down
30 changes: 30 additions & 0 deletions randomgen/common.pyx
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from collections import namedtuple
import math
import operator
import sys
from threading import RLock
Expand Down Expand Up @@ -420,6 +421,35 @@ cdef object wrap_int(object val, object bits):
val = val - upper * divisor
return val

cdef object golden_advance_step(object max_draws, object bits):
"""
Odd integer closest to 2**xx / phi for the smallest xx with 2**xx / phi > max_draws

Only integer arithmetic is used. 2**xx / phi = (sqrt(5) - 1) * 2**xx / 2 and
sqrt(5) * 2**xx is irrational, so with r = isqrt(5 * 4**xx), the integer part
of 2**xx / phi is (r - 2**xx) // 2. As 2**xx / phi is irrational, it is larger
than an integer n exactly when its integer part is at least n.
"""
n = operator.index(max_draws)
# 2**xx / phi > n needs 2**xx > n, so xx >= n.bit_length(). Since 1 / phi > 1 / 2,
# 2**xx >= 2 * n is enough, so xx is either n.bit_length() or one more.
if n >= 1:
for xx in (n.bit_length(), n.bit_length() + 1):
if xx > bits:
break
p = 1 << xx
floor = (math.isqrt(5 * p * p) - p) // 2
if floor >= n:
# 2**xx / phi lies strictly between floor and floor + 1, so the
# closest odd integer is floor if it is odd and floor + 1 otherwise
return floor if floor & 1 else floor + 1
p = 1 << bits
upper = (math.isqrt(5 * p * p) - p) // 2
raise ValueError(
f"max_draws must be between 1 and {upper}, the integer part of "
f"2**{bits} / phi, where phi is the golden ratio."
)

cdef int check_state_config(object bit_generator, str name, object current,
object new) except -1:
"""
Expand Down
7 changes: 6 additions & 1 deletion randomgen/pcg32.pxd
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
cimport numpy as np
from libc.stdint cimport uint32_t, uint64_t

from randomgen.common cimport BitGenerator, fully_qualified_name, wrap_int
from randomgen.common cimport (
BitGenerator,
fully_qualified_name,
golden_advance_step,
wrap_int,
)


cdef extern from "src/pcg32/pcg32.h":
Expand Down
2 changes: 2 additions & 0 deletions randomgen/pcg32.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,6 @@ class PCG32(BitGenerator):
@state.setter
def state(self, value: dict[str, str | dict[str, int]]) -> None: ...
def advance(self, delta: int) -> PCG32: ...
@staticmethod
def advance_step(max_draws: int) -> int: ...
def jumped(self, iter: int = ...) -> PCG32: ...
178 changes: 178 additions & 0 deletions randomgen/pcg32.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,18 @@ cdef class PCG32(BitGenerator):
>>> for i in range(10):
... rg[i].bit_generator.advance(i * 2**32)

Each instance above uses a different increment ``inc``. To split a single
stream, advance stream ``i`` by ``i * step``, where ``step`` is
``advance_step(max_draws)`` and ``max_draws`` is the most draws that any
stream makes, or use ``jumped``. Do not advance by a large power of two
such as ``2**63``, which gives strongly related streams (see
``advance``). The period of ``PCG32`` is only ``2**64``, which limits how
many streams fit without overlapping, for example to about
``1.4 * 10**7`` streams of ``10**12`` draws. Use
:class:`~randomgen.pcg64.PCG64`, or the related
:class:`~randomgen.pcg64.PCG64DXSM` or :class:`~randomgen.pcg64.LCG128Mix`,
when many streams or many draws per stream are needed.

**Compatibility Guarantee**

``PCG32`` makes a guarantee that a fixed seed and will always produce
Expand Down Expand Up @@ -184,6 +196,36 @@ cdef class PCG32(BitGenerator):
self : PCG32
RNG advanced delta steps

Warnings
--------
Do not use ``advance`` with a large power of two, such as ``2**63``
(half of the period), to split a single stream (the same seed and
``inc``) into several. The state is an LCG modulo ``2**64``, whose
lowest ``e`` bits repeat every ``2**e`` steps, so advancing by
``2**e`` leaves them unchanged. After ``advance(2**63)``, the two
states differ only in their top bit at every step, and each output
is the other with its 16-bit halves swapped and one bit flipped.

After ``advance(2**32)``, the states differ in about 16 of their 64
bits and never in the lowest 32.

The size of the step also matters. Stream ``i``, advanced by
``i * step``, can make at most ``step`` draws before it reaches the
start of stream ``i + 1``, and ``n`` streams are only guaranteed not
to overlap when ``n * step <= 2**64``. The period of ``PCG32`` is
short enough for this to limit real applications, for example to
about ``1.4 * 10**7`` streams of ``10**12`` draws. Use
:class:`~randomgen.pcg64.PCG64`, or the related
:class:`~randomgen.pcg64.PCG64DXSM` or
:class:`~randomgen.pcg64.LCG128Mix`, when many streams or many draws
per stream are needed. ``advance_step`` returns an odd step for a
given number of draws per stream, and the Notes explain how to choose
one.

See Also
--------
advance_step : Step for splitting one stream into several.

Notes
-----
Advancing a RNG updates the underlying RNG state as-if a given
Expand All @@ -199,12 +241,148 @@ cdef class PCG32(BitGenerator):
differs from the number of bits generated by the underlying
RNG. For example, two 16-bit integer values can be simulated
from a single draw of a 32-bit RNG.

To split one stream into several, advance stream ``i`` by ``i * step``
with an odd ``step``. With an odd step, the states of two streams
differ in their lowest bit at every step and their other bits differ as
if at random: about 32 of the 64 bits differ, with the spread of a
binomial distribution. Only the lowest few bits follow a fixed pattern,
since in any LCG they repeat every few steps. This holds for an odd
step of any size. The size sets how far apart the streams are: stream
``i`` reaches the start of stream ``i + 1`` after ``step`` draws, so
``step`` must be at least the number of draws that any stream makes.
The starts ``i * step`` are taken modulo the period, ``2**64``, so
``n`` streams are only guaranteed not to overlap when
``n * step <= 2**64``.

A convenient step is the odd integer closest to ``2**XX / phi``, where
phi is the golden ratio and ``XX`` is chosen so that the step is at
least the number of draws per stream. ``advance_step(max_draws)``
returns this step for the smallest ``XX`` that works, using only
integer arithmetic. For example, ``XX = 32`` gives ``0x9e3779b9``,
about ``2.7 * 10**9`` draws per stream, and ``XX = 64`` gives
``0x9e3779b97f4a7c15``, the step of ``jumped``. With ``XX = 64`` the
multiples of the step wrap around the period, and the golden ratio
spreads the streams evenly around it.

>>> step = PCG32.advance_step(10**10)
>>> hex(step)
'0x278dde6e5'
>>> streams = [PCG32(1234).advance(i * step) for i in range(8)]

The period of ``PCG32``, ``2**64``, is short enough for
``n * step <= 2**64`` to limit the number of streams in practice. With
the step from ``advance_step``, the largest number of streams that are
guaranteed not to overlap is

================= ==================
Draws per stream Streams
================= ==================
``10**6`` ``1.4 * 10**13``
``10**9`` ``1.4 * 10**10``
``10**12`` ``1.4 * 10**7``
``10**15`` 13,254
``10**18`` 12
================= ==================

The step from ``advance_step`` is less than twice the number of draws
per stream, so ``n`` streams of ``m`` draws always fit when
``n * m <= 2**63``. When many streams or many draws per stream are
needed, use :class:`~randomgen.pcg64.PCG64`, or the related
:class:`~randomgen.pcg64.PCG64DXSM` or
:class:`~randomgen.pcg64.LCG128Mix`. Their period of ``2**128`` allows
about ``2.5 * 10**26`` streams of ``10**12`` draws.

Streams ``i`` and ``j`` are ``(j - i) * step`` draws apart, so they
share the lowest ``v`` bits of their states, where ``2**v`` is the
largest power of two that divides ``j - i``. ``jumped(iter)`` advances
by ``iter`` times a golden-ratio step, so it is also a good choice when
``iter`` is not a large power of two.
"""
delta = wrap_int(delta, 64)
with self.lock:
pcg32_advance_state(&self.rng_state, <uint64_t>delta)
return self

@staticmethod
def advance_step(max_draws):
"""
advance_step(max_draws)

Step for splitting one stream into streams of up to max_draws draws

.. versionadded:: 2.6.0

Parameters
----------
max_draws : int
The largest number of draws that any stream makes. Must be at
least 1 and at most the integer part of ``2**64 / phi``, about
``1.1 * 10**19``, where phi is the golden ratio.

Returns
-------
step : int
The odd integer closest to ``2**XX / phi``, where ``XX`` is the
smallest integer for which ``2**XX / phi`` is larger than
``max_draws``. The step is never smaller than ``max_draws``.

Raises
------
ValueError
If ``max_draws`` is outside the range above.
TypeError
If ``max_draws`` is not an integer.

See Also
--------
advance : Advance the state as-if delta draws have occurred.

Notes
-----
Advance stream ``i`` by ``i * step`` to split one stream into several.
The step is odd, so the states of two streams differ as if at random,
apart from their lowest few bits. The step is at least ``max_draws``,
so a stream that makes at most ``max_draws`` draws does not reach the
start of the next stream. The Notes of ``advance`` explain this choice
of step.

The starts ``i * step`` are taken modulo the period, ``2**64``, so
streams ``0`` to ``n - 1`` are only guaranteed not to overlap when
``n * step <= 2**64``. ``advance_step`` does not check this, since it
does not know ``n``, and the period of ``PCG32`` is short enough for
the limit to matter. The largest ``n`` is ``2**64 // step``:

================= ==================
``max_draws`` Largest ``n``
================= ==================
``10**6`` ``1.4 * 10**13``
``10**9`` ``1.4 * 10**10``
``10**12`` ``1.4 * 10**7``
``10**15`` 13,254
``10**18`` 12
================= ==================

The step is less than ``2 * max_draws``, so ``n`` streams always fit
when ``n * max_draws <= 2**63``. When many streams or many draws per
stream are needed, use :class:`~randomgen.pcg64.PCG64`, or the related
:class:`~randomgen.pcg64.PCG64DXSM` or
:class:`~randomgen.pcg64.LCG128Mix`. Their period of ``2**128`` allows
about ``2.5 * 10**26`` streams of ``10**12`` draws.

The step is computed with integer arithmetic only, so it is exact for
any ``max_draws``.

Examples
--------
>>> from randomgen import PCG32
>>> step = PCG32.advance_step(10**10)
>>> hex(step)
'0x278dde6e5'
>>> streams = [PCG32(1234).advance(i * step) for i in range(8)]
"""
return golden_advance_step(max_draws, 64)

cdef jump_inplace(self, object iter):
"""
Jump state in-place
Expand Down
Loading
Loading