Skip to content

feat(release): vendor prebuilt runtimes on release branches only - #407

Merged
Peefy merged 1 commit into
mainfrom
refactor/vendor-libs-release-branch-only
Oct 13, 2026
Merged

Peefy merged 1 commit into
mainfrom
refactor/vendor-libs-release-branch-only

Conversation

@Peefy

@Peefy Peefy commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #207

Summary

go/lib/<platform>/ and wasm/kcl.wasm are ~370MB of compiled artifacts
and ~89% of the tracked tree, and every one of them is reproducible from
the kcl-lang/kcl release of the same
version. They are now tracked on release/* branches and release tags
only; development branches drop them and fetch them on demand.

A fresh clone goes from 394MB to 21MB.

Same shape as wasmtime-go,
which keeps its build/ directory empty on main and fills it in from CI,
with the artifacts committed onto the release. The mechanism is not new
here either — swift-release.yaml already git add -fs the universal
libkcl_lib_c.a onto its tag.

What changes for consumers

before after
release tag vendored vendored (unchanged)
development branch vendored hack/download-libs.sh
type-check / vet, no binaries n/a go build -tags no_libembed ./...

Only the development-branch row is a behaviour change.

How it works

  • hack/download-libs.sh pulls one platform's runtime from the matching
    kcl-lang/kcl release. The two iOS static archives are cross-compiled from
    this repo's c/ crate instead, because they are kcl-lib-c and have no
    published asset. Nine CI jobs call it.
  • vendor-libs.yaml runs on a push to release/v* and commits every
    artifact onto that branch.
  • //go:embed is a compile-time dependency, so the six kcl_lib_*.go
    files (Go's implicit _GOOS_GOARCH filename constraint picks one) take a
    !no_libembed opt-out, and kcl_lib_noembed.go covers -tags no_libembed.
    install now rejects an empty CliLib with an actionable error rather than
    writing a zero-byte library that fails later at dlopen.

Why a branch, not a force-moved tag

swift-release.yaml force-updates its tag after the fact. That does not work
here: wasm-test.yaml publishes @kcl-lib/wasm to npm on a tag push, and
kcl.wasm is in that package's files array. A workflow that rewrites the
tag afterwards races the publish, and the failure is silent — npm publishes
successfully with no runtime inside it. On a release branch the binaries are
committed before the tag exists, so there is no ordering to get wrong.

Consequence: release tags must be cut from release/*, not from main.
The procedure is in docs/lib-vendoring.md.

windows/arm64 is not vendored yet

vendor-libs.yaml derives the version from the branch name and downloads
kcl-lib-<tag>-windows-arm64.zip, which no kcl release has shipped.
kcl-lang/kcl#2219 adds that target
to the release matrix, but only the first kcl tag cut after it lands carries the
asset — so vendoring that one platform fails until then.

That also broke dart-test.yaml's windows-11-arm leg, which used to pass
only because go/lib/windows-arm64/kcl.dll was committed here. The leg is now
commented out with a pointer back to kcl#2219; re-enable it once a release
carrying the asset is out and KCL_VERSION points at it. No other workflow
builds for windows/arm64, so this is the only coverage affected.

The existing v0.12.x tags still carry their binaries and are untouched. History
is deliberately not rewritten — git filter-repo would strip them from those
tags and break anyone pinning one.

Test plan

  • fresh clone of the resulting tree is 21MB, down from 394MB
  • go build ./... on darwin/arm64 without the binary fails with the
    expected //go:embed message (unchanged, and self-explanatory)
  • go build -tags no_libembed ./... and go vet pass on all six
    GOOS/GOARCH pairs
  • -tags no_libembed + InstallKcl returns the actionable error and
    writes no library file
  • full go test ./... passes with the binaries present
  • hack/download-libs.sh exercised end to end against stubbed archives:
    download, unpack, stage, unknown platform, missing asset, --wasm,
    bash 3.2 empty-array path, and the Windows zip fallback
  • every one of the nine call sites resolves against its job's
    working-directory
  • shellcheck -S style clean; parses under the bash 3.2 that ships with
    macOS
  • CI green on the real download paths across all 7 go-test legs, 7 dart
    legs, the 4 AST-diff bindings and the wasm/npm job — the local sandbox
    could not reach release-assets.githubusercontent.com, so these runs are
    what actually exercise the download

Follow-ups this does not do

wasm/kcl.wasm is now a network fetch on every wasm CI run, and the wasm
tests carry hardcoded expectations about what that specific build can and
cannot do (wasm/tests/consistency.test.ts, generate_api.test.ts). They
pass today because the version is pinned by KCL_VERSION; if that pin is
ever loosened to "latest", those skip lists will start failing. Worth a
follow-up issue.

🤖 Generated with Claude Code

@Peefy
Peefy force-pushed the refactor/vendor-libs-release-branch-only branch 2 times, most recently from d3f91ad to c742aa7 Compare October 13, 2026 12:57
go/lib/<platform>/ and wasm/kcl.wasm are ~370MB of compiled artifacts and
~89% of the tracked tree, and every one of them is reproducible from the
kcl-lang/kcl release of the same version. They are now tracked on
release/* branches and release tags only; development branches drop them
and fetch them on demand, which takes a fresh clone from 394MB to 21MB.

Same shape as wasmtime-go: an empty build/ directory on main plus a CI
script that fills it in, with the artifacts committed onto the release.

- hack/download-libs.sh fetches the platform's runtime from the matching
  kcl-lang/kcl release, cross-compiling the two iOS static archives from
  c/ since those have no published asset. Nine CI jobs now call it.
- vendor-libs.yaml vendors everything onto a pushed release/* branch.
  It commits to the branch rather than force-updating a tag, because
  wasm-test.yaml publishes @kcl-lib/wasm to npm on a tag push and
  kcl.wasm is in that package's files: a workflow that rewrites the tag
  afterwards races the publish and silently ships a package with no
  runtime in it.
- go/lib embeds the platform binary with //go:embed, which is a compile
  time dependency, so the six embed files take a !no_libembed opt-out
  and a new kcl_lib_noembed.go covers -tags no_libembed. install now
  rejects an empty CliLib with an actionable error instead of writing a
  zero-byte library that fails later at dlopen.

Breaking for anyone building from a development branch: run
hack/download-libs.sh, or build with -tags no_libembed to type-check.
Release tags are unaffected. History is left alone on purpose --
filter-repo would strip the binaries from the existing v0.12.x tags.

The dart-test windows-11-arm leg is commented out for now: it passed only
because go/lib/windows-arm64/kcl.dll was committed here, and no kcl release
publishes that asset yet. kcl-lang/kcl#2219 adds the target to that repo's
release matrix; re-enable the leg once a release carrying it is out.

Signed-off-by: Peefy <xpf6677@163.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Peefy
Peefy force-pushed the refactor/vendor-libs-release-branch-only branch from c742aa7 to db127fd Compare October 13, 2026 13:12
@Peefy
Peefy merged commit f2a00ef into main Oct 13, 2026
48 of 49 checks passed
@Peefy
Peefy deleted the refactor/vendor-libs-release-branch-only branch October 13, 2026 13:34
Peefy added a commit that referenced this pull request Oct 16, 2026
Pushing release/v0.13.1 exposed three defects, all of which #407 shipped
because the workflow had never actually executed.

`Resolve versions` built the lib version as `v${branch#release/}`, but the
branch already reads `release/v0.13.1`, so the expansion yielded `vv0.13.1`
and the guard rejected it against KCL_VERSION. Every push failed at the
first step:

    lib version:     vv0.13.1
    SDK KCL_VERSION: v0.13.1
    ##[error]go/install/install.go KCL_VERSION (v0.13.1) does not match the
    branch version (vv0.13.1).

`download-artifact` without a `name` downloads each artifact into its own
staged/<artifact-name>/ subdirectory, so the merge step looked for
staged/go/lib and would have failed on `test -d`. merge-multiple puts both
artifacts into one repository-rooted tree, which is what the step assumes.

The iOS upload listed the two archives as bare paths, and upload-artifact
roots an archive at the least common ancestor of what it uploads — here
go/lib, so the archive would have contained ios-arm64/libkcl.a rather than
go/lib/ios-arm64/libkcl.a. Anchoring the search at the repository root with
`path: .` + `include:` keeps both artifacts rooted identically regardless of
which platforms a job builds.

Signed-off-by: Peefy <xpf6677@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhancement & Go Lib] Only in the release branch vendor KCL dynamic library

1 participant