Skip to content

Fix enable_url_table session sharing - #1815

Open
dajiaohuang wants to merge 1 commit into
apache:mainfrom
dajiaohuang:r4-enable-url-table-shared-session
Open

dajiaohuang wants to merge 1 commit into
apache:mainfrom
dajiaohuang:r4-enable-url-table-shared-session

Conversation

@dajiaohuang

@dajiaohuang dajiaohuang commented Oct 27, 2026 •

Copy link
Copy Markdown

Closes #1708

Rationale

enable_url_table() previously returned a handle backed by a separate SessionContext allocation while retaining the session ID. This allowed live handles to diverge in state and could invalidate weak FFI task-context providers when callers reassigned the returned handle.

Changes

  • Rebuild the URL-table catalog through the existing session's state_ref() and return a handle sharing the same Arc<SessionContext>.
  • Preserve the session ID and make repeated calls idempotent.
  • Add Python state-sharing and FFI-provider regression coverage, and update the user guide, extension/contributor documentation, and FFI capsule protocol guidance.

User-facing change

Both the original handle and the returned handle now observe URL-table enablement and subsequent state changes. Reassigning the result remains valid. This is a behavioral API change; the api change label is required. I attempted to apply the label, but GitHub denied AddLabelsToLabelable for this account, so the label remains unapplied.

Validation

  • cargo check --features substrait --package datafusion-python — passed.
  • Focused Python and FFI regression tests — 2 passed.
  • pytest python/ on Windows — 1,626 passed, 11 skipped, 2 failed. The failures are in the read_arrow and register_arrow doctests: TemporaryDirectory cleanup raises WinError 32 while data.arrow is still held open.
  • Selected pre-commit hooks passed: Ruff, Ruff format, ty, Rust fmt, and codespell.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

enable_url_table returns a handle on a different session, unlike every other with_* derivation

1 participant