Skip to content

[GH-3020] Add ST_H3Compact - #3480

Merged
jiayuasu merged 1 commit into
apache:masterfrom
simenojensen:codex/gh-3020-h3-compact
Oct 19, 2026
Merged

jiayuasu merged 1 commit into
apache:masterfrom
simenojensen:codex/gh-3020-h3-compact

Conversation

@simenojensen

Copy link
Copy Markdown
Contributor

Did you read the Contributor Guide?

Yes, I have read the Contributor Rules and Contributor Development Guide.

Is this PR related to a ticket?

Related to #3020. Adds ST_H3Compact following ST_H3ToParent (#3310), ST_H3ToChildren (#3470), and ST_H3Uncompact (#3474). This PR leaves #3020 open for the remaining hierarchy function.

What changes were proposed in this PR?

Add ST_H3Compact(cells), backed by the existing H3 compactCells API. It recursively replaces complete sets of child cells with their parents and retains cells that cannot be compacted. The resulting array may contain cells at different resolutions. Empty arrays return empty arrays, and null array arguments return null in Spark and Flink.

Expose the function through Spark SQL, the Scala and Python DataFrame APIs, and Flink SQL. Add Spark and Flink API documentation and catalog entries for v2.0.0. Input cells must be unique and at the same resolution, following H3's contract; H3 errors propagate without deduplicating the input.

Tests cover recursive compaction, incomplete child sets, retained cells alongside compacted parents, pentagons, compact/uncompact round trips, singletons at resolutions 0 and 15, empty arrays, literal and column null arguments, native error propagation, and the Scala/Python wrapper overloads.

How was this patch tested?

Validated locally with Java 17, Spark 3.5.0 / Scala 2.12, Flink 1.19.0, and Python 3.11:

  • mvn -o spotless:apply and mvn -o spotless:check — passed.
  • mvn -o -pl common -am '-Dtest=org.apache.sedona.common.FunctionsTest#h3Compact*' -Dsurefire.failIfNoSpecifiedTests=false -Dmaven.javadoc.skip=true install — all 6 tests passed.
  • mvn -o -pl spark/common -DskipTests test-compile — production and test sources compiled.
  • mvn -o -pl spark/common -Dsuites=org.apache.sedona.sql.functions.STH3Functions,org.apache.sedona.sql.dataFrameAPITestScala org.scalatest:scalatest-maven-plugin:2.2.0:test — all 282 tests passed (17 H3 tests and 265 Scala DataFrame API tests).
  • mvn -o -pl flink -Dtest=FunctionTest#testH3Compact test — passed, including partial sets and empty arrays supplied through typed table columns.
  • python -m pytest tests/sql/test_function.py tests/sql/test_dataframe_api.py -k test_st_h3_compact -q — both tests passed against the newly compiled JVM classes and Maven dependency classpath supplied through SPARK_DIST_CLASSPATH.
  • Black checks for all three modified Python files and git diff --check — passed.

Did this PR include necessary documentation updates?

Yes. Added Spark SQL and Flink function pages and updated both API catalogs, using the current SNAPSHOT version v2.0.0.

@jiayuasu jiayuasu added this to the sedona-2.0.0 milestone Oct 19, 2026
@jiayuasu
jiayuasu merged commit 99a3cf3 into apache:master Oct 19, 2026
43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants