Skip to content

[Hotfix][Build] Backport Docker pipeline fix and Hudi Hadoop exclusion to 0.9.0 (#4485) - #4495

Merged
czy006 merged 2 commits into
0.9.0from
backport-090-docker-hudi
Nov 13, 2026
Merged

czy006 merged 2 commits into
0.9.0from
backport-090-docker-hudi

Conversation

@czy006

@czy006 czy006 commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Backports the two fixes needed to rebuild and publish the official 0.9.0-incubating Docker images, addressing #4485.

What's included

  1. Docker pipeline fix (from [Build] Fix Docker image publishing: vendor docker/* actions as submodules, drop legacy Flink 1.14.6 #4491, master): vendor the five docker/* GitHub Actions as pinned git submodules under .github/actions/ (the repository Actions allowlist rejects third-party actions at workflow startup), add the workflow_dispatch trigger with the dry-run switch, and update the Flink optimizer matrix (1.14.6 → 1.18.1). Also carries master's workflow shape for the image jobs (single default Hadoop 3 image — Hadoop 2 support ends with 0.9.0 per [AMORO-4451][Build] Discontinue Hadoop 2 support #4452, so no Hadoop 2 image variant is built for this release).

  2. Hudi Hadoop exclusion (from [Hudi] Exclude the transitive Hadoop 2.10 stack from hudi-java-client #4494, master): exclude org.apache.hadoop:* from hudi-java-client so Hudi's transitive hbase-client:2.4.9 no longer drags a full Hadoop 2.10 stack into -Psupport-all-formats distributions — the KerberosName shadowing that breaks the AMS Terminal ([Bug]: Terminal fails with NoSuchMethodError in the apache/amoro Docker image because Hudi brings in hadoop-auth 2.10.0 #4485).

Why a release-branch PR

The v0.9.0-incubating tag cannot be moved; images for the release are rebuilt by dispatching the fixed workflow on the 0.9.0 branch (dry-run=false), which builds from the same source as the released tag plus exactly these two fixes. No jar artifacts are re-published; only Docker Hub images change.

Verification plan

…modules

Same change as #4491 on master: the repository Actions allowlist rejects
the third-party docker/* actions at workflow startup, so the image
publishing pipeline has been failing since the policy was tightened.
Vendor the five docker actions as pinned submodules under
.github/actions/ and reference them as local actions; adds the
workflow_dispatch trigger with a dry-run switch and the Flink optimizer
matrix cleanup (1.14.6 -> 1.18.1).
…#4494)

hudi-java-client pulls hudi-common -> hbase-client 2.4.9, which drags in a
full Hadoop 2.10 stack (hadoop-auth, hadoop-common, hadoop-hdfs,
hadoop-hdfs-client, yarn client/api/common, mapreduce-client-core,
hadoop-distcp, hadoop-annotations - ten artifacts). In distributions that
bundle the Hudi format (e.g. the Docker image built with
-Psupport-all-formats) these shadow the managed hadoop-client-api/runtime
3.4.3: KerberosName gets loaded from hadoop-auth 2.10.0 while
HadoopKerberosName from hadoop-client-api 3.4.3 calls the missing
setRuleMechanism, breaking the AMS Terminal with NoSuchMethodError (issue
#4485).

Exclude org.apache.hadoop:* from the Hudi dependency so the host supplies
the single managed Hadoop client - the same boundary the AMS module already
applies to its Spark dependencies.

Verified: dependency:tree for amoro-format-hudi goes from 10 Hadoop-2.10.0
artifacts to 0 (only hadoop-client-api/runtime 3.4.3 remain, provided by
amoro-common), and the module still compiles.
@czy006

czy006 commented Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

Why this PR also carries the workflow fix (and not only the Hudi exclusion)

GitHub Actions reads the workflow file from the ref being built, not from master. The only way to rebuild the 0.9.0-incubating images is to dispatch the workflow on this 0.9.0 branch (the v0.9.0-incubating tag is immutable and must not be re-pushed), and the workflow copy currently on this branch is the pre-#4491 version with two blockers:

  1. It references the third-party docker/* actions directly, which the repository Actions allowlist rejects at validation time — every dispatch would fail with startup_failure, exactly like every run since 2026-10-26.
  2. It has no workflow_dispatch trigger, so it cannot be started manually at all.

Neither fix alone is sufficient: without the workflow fix nothing can be built; without the Hudi exclusion the built image would still carry the #4485 Terminal bug. The two commits together are the minimal set for publishing a healthy 0.9.0 image.

Dispatching from master is not an option either: master has already moved to 1.0.0-SNAPSHOT (#4493) and would build the wrong version.

@czy006
czy006 merged commit 894e7b4 into 0.9.0 Nov 13, 2026
8 checks passed
@czy006
czy006 deleted the backport-090-docker-hudi branch November 13, 2026 11:17
czy006 added a commit that referenced this pull request Nov 13, 2026
The v0.9.0-incubating tag is immutable, so the 0.9.0-incubating Docker
images are rebuilt from this branch (per #4495). The semver/latest tag
rules only fire on tag pushes, and a branch build would otherwise tag
images as 0.9.0-snapshot. Add explicit raw rules that produce the
release tags (0.9.0-incubating, -hadoop3, optimizer -flink1.18/-1.20/
-spark3.5 variants, latest) when building from refs/heads/0.9.0, and
scope the -snapshot rules to master builds.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant