From 2310687c790a0b010a144b0c7e3056653920a300 Mon Sep 17 00:00:00 2001 From: Ed Burns Date: Sun, 16 Aug 2026 12:49:51 -0700 Subject: [PATCH 1/3] Gate Java native packaging by host Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2ea248b2-112a-4548-865e-922721479b0c --- ...ncorrect-os-arch-artifacts-are-produced.md | 224 ++++++++++++++++++ .../20260816-notes-to-linux-x64-agent.md | 137 +++++++++++ java/README.md | 20 ++ java/copilot-native/pom.xml | 98 +++++--- .../adr/adr-007-native-bundling-strategy.md | 5 +- 5 files changed, 450 insertions(+), 34 deletions(-) create mode 100644 2323-java-windows-x64-embed-rust-remove-before-merge/20260815-make-it-so-no-incorrect-os-arch-artifacts-are-produced.md create mode 100644 2323-java-windows-x64-embed-rust-remove-before-merge/20260816-notes-to-linux-x64-agent.md diff --git a/2323-java-windows-x64-embed-rust-remove-before-merge/20260815-make-it-so-no-incorrect-os-arch-artifacts-are-produced.md b/2323-java-windows-x64-embed-rust-remove-before-merge/20260815-make-it-so-no-incorrect-os-arch-artifacts-are-produced.md new file mode 100644 index 000000000..f374b6c2f --- /dev/null +++ b/2323-java-windows-x64-embed-rust-remove-before-merge/20260815-make-it-so-no-incorrect-os-arch-artifacts-are-produced.md @@ -0,0 +1,224 @@ +# Prevent incorrect native classifier artifacts on unsupported build hosts + +## Goal + +Change the Java Maven build so a normal reactor build produces a platform-classified native JAR only when the build host matches a currently implemented native platform. For the current implementation: + +- Linux x64 builds produce `copilot-sdk-java-runtime--linux-x64.jar`. +- macOS, Windows, Linux ARM64, and other unsupported hosts do not fetch Linux native files, stage `native/linux-x64`, run Linux-native structural checks, or produce a `*-linux-x64.jar`. +- The OS-neutral `copilot-sdk-java-runtime-.jar`, sources JAR, and Javadoc JAR remain. ADR-007 requires the primary placeholder artifact for Maven Central; these artifacts make no OS/architecture claim. +- Adding a supported platform later requires one small host profile with platform properties, while the common fetch, package, and verification logic remains shared. + +Do not add macOS or Windows native support in this change. + +## Execution boundary and serial handoff + +This plan has two serial phases: + +1. The current agent performs the implementation and macOS-only validation described below. It is prohibited from exercising any Linux x64 code. It must not use Docker, a VM, emulation, forced Maven profile activation, cross-compilation, or any other mechanism to run or validate the Linux x64 native path. +2. Only after the macOS work is complete and merged to the topic branch, a separate Linux x64 agent checks out the resulting topic-branch `HEAD` and performs the Linux x64 validation and any Linux-only follow-up fixes. + +The current agent must not perform phase 2. Instead, as its final implementation step, it must write a self-contained, `copilot --yolo`-ready prompt for the Linux x64 agent to: + +```text +2323-java-windows-x64-embed-rust-remove-before-merge/20260816-notes-to-linux-x64-agent.md +``` + +Do not create that handoff prompt before the macOS implementation and validation are complete. The prompt must describe the repository state as already containing the merged macOS work and must instruct the Linux agent to work from the checked-out `HEAD`, not to repeat or revert the macOS phase. + +## Existing problem + +PR #2301 introduced `java/copilot-native` as an unconditional reactor module. Its POM currently: + +1. Defines `copilot.native.classifier` globally as `linux-x64`. +2. Binds `fetch-native-linux-x64` to `generate-resources` on every host. +3. Binds `jar-linux-x64` and `verify-native-jars` to `package` on every host. + +Consequently, a macOS build downloads and packages Linux files and leaves artifacts such as: + +```text +java/copilot-native/target/copilot-sdk-java-runtime--linux-x64.jar +java/copilot-native/target/native-staging/linux-x64/... +``` + +This contradicts ADR-007's per-platform classifier model: classifiers identify the runtime platform and must not merely reflect a hard-coded build default. + +## Design + +Keep `copilot-native` in the reactor on every host so Maven still: + +- cleans its `target` directory; +- builds the required OS-neutral Maven Central placeholder artifacts; +- runs the platform-independent `fetch-native.test.mjs` tests. + +Make the native-specific executions dormant in the base build by assigning them the non-lifecycle phase `none`. Add a host-activated `native-linux-x64` profile that: + +- activates only when Maven reports operating system `Linux` and architecture `amd64`; +- does not activate when `copilot.native.skip.download` is set; +- defines `copilot.native.classifier=linux-x64`; +- defines `copilot.native.cli.filename=copilot`; +- rebinds the shared native fetch, fetch-script test, classifier-JAR, and structural-verification executions to their real lifecycle phases. + +Use generic execution IDs so future profiles can rebind the same implementation: + +| Execution | Dormant phase | Linux x64 phase | +| --- | --- | --- | +| `fetch-native` | `none` | `generate-resources` | +| `test-fetch-native` | `none` | `test` | +| `jar-native` | `none` | `package` | +| `verify-native-jars` | `none` | `package` | + +Maven merges profile executions with base executions by plugin coordinates and execution ID. The profile therefore needs to repeat only the execution IDs and phases, not the fetch/package/verification configuration. + +Do not move `copilot-native` into a host-specific reactor profile. Excluding the module would prevent `mvn clean` on an unsupported host from removing native artifacts left by an earlier supported-host build, and it would unnecessarily remove the OS-neutral placeholder artifact from the reactor. + +## Implementation + +### 1. Refactor `java/copilot-native/pom.xml` + +1. Remove the global hard-coded `copilot.native.classifier` property and its Linux-only comment. +2. Rename native-specific execution IDs: + - `fetch-native-linux-x64` to `fetch-native`; + - `jar-linux-x64` to `jar-native`. +3. Set the base phase of `fetch-native`, `test-fetch-native`, `jar-native`, and `verify-native-jars` to `none`. +4. Keep all existing goals and configuration on those base executions. Continue passing `${copilot.native.classifier}` to `fetch-native.mjs` and using it for staging paths, classifier names, and resource paths. +5. Configure `test-fetch-native` with `${skipTests}` so `-DskipTests` consistently skips the exec-based script tests. +6. Replace the structural check's hard-coded `copilot` resource name with `${copilot.native.cli.filename}`. This preserves Linux behavior and lets a future Windows profile select `copilot.exe` without copying the verification block. +7. Keep the placeholder resource and the OS-neutral sources/Javadoc JAR executions bound exactly as they are. +8. Add a `native-linux-x64` profile: + - activate with `Linuxamd64`; + - add a negated property activation for `copilot.native.skip.download`, so `-Dcopilot.native.skip.download=true` prevents the profile from activating; + - set `copilot.native.classifier` to `linux-x64`; + - set `copilot.native.cli.filename` to `copilot`; + - re-declare the four generic execution IDs with phases `generate-resources`, `test`, `package`, and `package`, respectively. +9. Simplify `skip-native-download`: + - preserve its existing property activation and its skip of `exec-maven-plugin`, including the script test; + - remove overrides for the old `jar-linux-x64` and `verify-native-jars` execution IDs because the native host profile will not activate when the skip property is present. +10. Update comments and the module description only where needed to distinguish the OS-neutral placeholder build from host-matched classifier packaging. + +Do not add `os-maven-plugin`. Maven's built-in OS activation is sufficient for the one currently implemented build platform, avoids a new build extension, and keeps later support explicit per classifier. + +### 2. Update ADR-007 + +Edit `java/docs/adr/adr-007-native-bundling-strategy.md` without changing the chosen per-platform-classifier decision: + +1. In **Current platform scope**, state that the Maven build binds native packaging only on a host matching an implemented classifier. Currently that is Linux x64; unsupported hosts build only the OS-neutral placeholder artifacts. +2. In **Consequences**, qualify the fetch/package statement so it says a supported host profile fetches and packages its matching `@github/copilot-` package. +3. State that additional platform implementation consists of adding a host profile that supplies the classifier and platform CLI filename and binds the shared executions. + +The existing Java README remains accurate: in-process support is still experimental and Linux x64 only. Do not change runtime platform detection or consumer dependency examples. + +## Future platform extension pattern + +When a new classifier is implemented, add a sibling profile in `java/copilot-native/pom.xml` with: + +1. Exact Maven host OS/architecture activation. +2. `copilot.native.classifier` set to the ADR classifier, for example `darwin-arm64` or `win32-x64`. +3. `copilot.native.cli.filename` set to `copilot` or `copilot.exe`. +4. The same four generic execution IDs rebound to their lifecycle phases. + +Before enabling a new profile, ensure `fetch-native.mjs`, its tests, the lockfile package, structural assertions, and CI all support that classifier. Do not activate a profile merely because `PlatformDetector` recognizes the platform at runtime. + +## Validation and handoff + +Run all Maven commands from `java`, with the required Java environment, `pipefail`, and tee-to-log convention. + +### Phase 1: macOS regression check + +Start from the current macOS host and run: + +```sh +cd java +export IDEA_HOME="/Applications/IntelliJ IDEA CE.app/Contents/MacOS" +export APPCAT_HOME=/Users/edburns/.appcat +export JAVA_HOME="/Library/Java/JavaVirtualMachines/microsoft-25.jdk/Contents/Home" +export ANT_HOME="${HOME}/Downloads/apache-ant-1.10.13" +export M2_HOME="${HOME}/Downloads/apache-maven-3.9.8" +export PATH="${APPCAT_HOME}:${M2_HOME}/bin:${ANT_HOME}/bin:${JAVA_HOME}/bin:${IDEA_HOME}:${PATH}" +set -o pipefail +LOG="$(date +%Y%m%d-%H%M)-job-logs.txt" +mvn clean package -DskipTests 2>&1 | tee "$LOG" +``` + +Inspect that exact log and output tree. Acceptance criteria: + +- Maven succeeds. +- The reactor still includes `copilot-sdk-java-runtime`. +- The log does not run `fetch-native` or `verify-native-jars`. +- `java/copilot-native/target` contains the OS-neutral primary, sources, and Javadoc JARs. +- No filename under `java/copilot-native/target` contains `linux-x64`. +- `java/copilot-native/target/native-staging` does not exist. + +### Phase 1: macOS effective-profile check + +On macOS, run `mvn -pl copilot-native help:active-profiles` and confirm `native-linux-x64` is absent. Do not force or otherwise activate that profile. + +The current agent must stop its executable validation here. A normal macOS `test` or `verify` lifecycle must not invoke `test-fetch-native` because that execution is bound only by the Linux x64 profile. The current agent must not invoke `fetch-native.test.mjs` directly or force the Linux profile active. It must not attempt to approximate any of the following Linux checks on macOS. + +### Phase 1: write the Linux x64 agent prompt + +After all implementation and macOS validation are complete, write `2323-java-windows-x64-embed-rust-remove-before-merge/20260816-notes-to-linux-x64-agent.md`. Make it directly executable as a prompt to `copilot --yolo`, with no dependence on this plan or prior conversation. + +The prompt must tell the Linux x64 agent: + +1. It is running serially after the macOS work has been completed and merged to the topic branch. +2. The checked-out `HEAD` is the authoritative starting point containing that work. It must inspect the existing changes and must not repeat, replace, or revert the macOS implementation. +3. Its scope is to validate the Linux x64 path retained by the new host-activated Maven profile, diagnose any failures, make only the Java or Java-related GitHub Actions changes needed to preserve Linux x64 behavior, and re-run the relevant validation. +4. It must read this implementation plan, ADR-007, `java/pom.xml`, and `java/copilot-native/pom.xml` before changing code. +5. It must start every Maven command from `java`, use the required Java environment, enable `pipefail`, pipe stdout and stderr through `tee` to a timestamped filename containing the literal `job-logs`, and inspect that exact log. +6. It must not add macOS or Windows native support, broaden the native platform matrix, add `os-maven-plugin`, or weaken the macOS unsupported-host guarantees. +7. It must preserve unrelated work already present at `HEAD` and make surgical fixes only if Linux validation exposes a problem. +8. It must report the exact files changed, exact Maven commands and log filenames, produced artifact names, and whether every Linux acceptance criterion passed. + +Embed the following Linux x64 validation and acceptance criteria in that prompt. + +### Phase 2: Linux x64 work delegated to the later agent + +On the native Linux x64 host, the later agent must run `mvn -pl copilot-native help:active-profiles` and confirm `native-linux-x64` is active. It must then run: + +```sh +LOG="$(date +%Y%m%d-%H%M)-job-logs.txt" +mvn -pl copilot-native test 2>&1 | tee "$LOG" +``` + +Acceptance criterion: all `fetch-native.test.mjs` tests pass without downloading a real native package or producing a classifier JAR. + +It must then run: + +```sh +LOG="$(date +%Y%m%d-%H%M)-job-logs.txt" +mvn clean verify 2>&1 | tee "$LOG" +``` + +Linux x64 acceptance criteria: + +- `fetch-native` runs during `generate-resources`. +- Exactly one platform classifier JAR is produced, ending in `-linux-x64.jar`. +- The existing structural verification confirms `runtime.node`, `platform.properties`, and `copilot`. +- The OS-neutral placeholder JAR does not contain native binaries. +- The full Java reactor remains green. + +The later Linux x64 agent must then run: + +```sh +LOG="$(date +%Y%m%d-%H%M)-job-logs.txt" +mvn clean package -pl copilot-native -DskipTests -Dcopilot.native.skip.download=true 2>&1 | tee "$LOG" +``` + +Acceptance criteria: + +- No native download, staging directory, classifier JAR, or native structural verification occurs. +- The OS-neutral placeholder, sources, and Javadoc JARs are still produced. + +If either Linux command fails or an acceptance criterion is not met, the Linux x64 agent must diagnose and fix the Linux-specific regression, then repeat the smallest relevant Maven validation and the full `mvn clean verify` before concluding. + +## Completion criteria + +- A clean macOS reactor build cannot produce a Linux-classified artifact. +- Linux x64 retains the classifier artifact and all existing integrity/structure checks. +- `copilot.native.skip.download=true` retains its placeholder-only behavior. +- Common native build logic exists once; platform profiles contain only activation, platform properties, and phase bindings. +- ADR-007 accurately describes host-matched native packaging and the extension pattern. +- The macOS agent has not exercised Linux x64 code. +- The macOS agent has written the required `20260816-notes-to-linux-x64-agent.md` only after its implementation and macOS validation completed. +- The handoff prompt is self-contained, assumes the merged topic-branch `HEAD`, and delegates all Linux x64 execution and Linux-only remediation to the later Linux x64 agent. diff --git a/2323-java-windows-x64-embed-rust-remove-before-merge/20260816-notes-to-linux-x64-agent.md b/2323-java-windows-x64-embed-rust-remove-before-merge/20260816-notes-to-linux-x64-agent.md new file mode 100644 index 000000000..604b15ca2 --- /dev/null +++ b/2323-java-windows-x64-embed-rust-remove-before-merge/20260816-notes-to-linux-x64-agent.md @@ -0,0 +1,137 @@ +# Validate host-gated Java native packaging on Linux x64 + +Work autonomously in the current `copilot-sdk` checkout. This work runs serially after the macOS implementation was completed and merged to the topic branch. The currently checked-out `HEAD` is the authoritative starting point and already contains that implementation. Inspect it before making changes. Do not repeat, replace, or revert the macOS work. + +## Objective + +Validate that the Java native-runtime Maven module still fetches, packages, and verifies the `linux-x64` classifier on a native Linux x64 host after native packaging was changed to use a host-activated profile. Diagnose any failures, make only the Java or Java-related GitHub Actions changes required to preserve Linux x64 behavior, and repeat validation until all acceptance criteria pass. + +The implemented design is: + +* Native-specific Maven executions are dormant in the base `java/copilot-native/pom.xml`. +* The `native-linux-x64` profile activates only on Linux `amd64` when `copilot.native.skip.download` is absent. +* That profile supplies `copilot.native.classifier=linux-x64` and `copilot.native.cli.filename=copilot`. +* The profile binds the shared `fetch-native`, `test-fetch-native`, `jar-native`, and `verify-native-jars` executions to the Maven lifecycle. +* Unsupported hosts still build the OS-neutral primary, sources, and Javadoc JARs without producing native-looking artifacts. + +## Required preparation + +Before changing code: + +1. Read the repository's current Java instructions. +1. Read `2323-java-windows-x64-embed-rust-remove-before-merge/20260815-make-it-so-no-incorrect-os-arch-artifacts-are-produced.md`. +1. Read `java/docs/adr/adr-007-native-bundling-strategy.md`. +1. Read the **Development Setup for native embedding** section in `java/README.md`. +1. Read `java/pom.xml` and `java/copilot-native/pom.xml`. +1. Inspect `git status` and the relevant `HEAD` diff/history so you understand the merged macOS implementation and preserve unrelated work. +1. Confirm the host is native Linux x64 before invoking Maven. Do not force the profile active on another operating system or architecture. + +Start every Maven command from `java`. Configure the Java and Maven environment required by the current repository instructions and Linux host. Enable `pipefail`, pipe both stdout and stderr through `tee`, and use a unique local-time filename matching `YYYYMMDD-HHMM--job-logs.txt`. Echo and retain each exact filename, then inspect that exact file rather than locating it with a glob or modification-time sorting. + +## Scope constraints + +* Do not add macOS or Windows native support. +* Do not broaden the native platform matrix. +* Do not add `os-maven-plugin`. +* Do not weaken the unsupported-host guarantees implemented by the macOS work. +* Preserve the common native build logic; platform profiles must contain only activation, platform properties, and lifecycle phase bindings. +* Preserve unrelated changes already present at `HEAD`. +* Make surgical fixes only when Linux validation exposes a defect. +* Do not change files outside `java` or Java-related GitHub Actions. +* Do not commit, push, or open a pull request unless explicitly requested. + +## Validation + +### Confirm profile activation + +From `java`, run: + +```sh +set -o pipefail +LOG="$(date +%Y%m%d-%H%M)-active-profiles-job-logs.txt" +echo "LOG_FILE=$LOG" +mvn -pl copilot-native help:active-profiles 2>&1 | tee "$LOG" +``` + +Inspect that exact log and confirm `native-linux-x64` is active. + +### Run native fetch-script tests + +From `java`, run: + +```sh +set -o pipefail +LOG="$(date +%Y%m%d-%H%M)-native-tests-job-logs.txt" +echo "LOG_FILE=$LOG" +mvn -pl copilot-native test 2>&1 | tee "$LOG" +``` + +Acceptance criterion: all `fetch-native.test.mjs` tests pass without downloading a real native package or producing a classifier JAR. + +### Run the full Java reactor + +From `java`, run: + +```sh +set -o pipefail +LOG="$(date +%Y%m%d-%H%M)-verify-job-logs.txt" +echo "LOG_FILE=$LOG" +mvn clean verify 2>&1 | tee "$LOG" +``` + +Inspect the exact log and output tree. All of these criteria must pass: + +* `fetch-native` runs during `generate-resources`. +* Exactly one platform classifier JAR is produced, ending in `-linux-x64.jar`. +* Structural verification confirms the classifier JAR contains `native/linux-x64/runtime.node`, `native/linux-x64/platform.properties`, and `native/linux-x64/copilot`. +* The OS-neutral placeholder JAR contains no native binaries. +* The full Java reactor succeeds. + +### Validate placeholder-only behavior + +From `java`, run: + +```sh +set -o pipefail +LOG="$(date +%Y%m%d-%H%M)-skip-native-job-logs.txt" +echo "LOG_FILE=$LOG" +mvn clean package -pl copilot-native -DskipTests -Dcopilot.native.skip.download=true 2>&1 | tee "$LOG" +``` + +Inspect the exact log and output tree. All of these criteria must pass: + +* No native download occurs. +* No `native-staging` directory is produced. +* No platform classifier JAR is produced. +* Native structural verification does not run. +* The OS-neutral primary, sources, and Javadoc JARs are produced. + +## Failure handling + +If a command fails or an acceptance criterion is not met: + +1. Diagnose the root cause from the exact Maven log and effective profile/build configuration. +1. Fix only the Linux-specific regression or shared profile wiring responsible for the failure. +1. Preserve the macOS unsupported-host behavior and the extension pattern described above. +1. Run the smallest relevant Maven check after each fix. +1. Before concluding, repeat `mvn clean verify` and every validation step affected by the fix. + +## Documentation update + +During this work, update the **Development Setup for native embedding** section in `java/README.md` so it reflects the Linux x64 behavior actually observed and any fixes made: + +* Replace the pending Linux x64 validation statement with the verified result. +* Keep the required JDK, Maven, Node.js, npm authentication, and run-from-`java` prerequisites accurate. +* Keep the unsupported-host and placeholder-only instructions accurate. +* Correct commands or expected artifacts if Linux validation shows that the current text is incomplete or inaccurate. +* Do not document macOS, Windows, or another classifier as natively supported. + +## Final response + +Report: + +* Exact files changed, including the required `java/README.md` update. +* Exact Maven commands and exact `job-logs` filenames. +* Produced native and OS-neutral artifact filenames. +* Whether each acceptance criterion passed. +* Any remaining blocker, stated plainly. diff --git a/java/README.md b/java/README.md index 57700a1e9..c216d0d19 100644 --- a/java/README.md +++ b/java/README.md @@ -487,6 +487,26 @@ mvn verify -Dskip.test.harness=true mvn jacoco:prepare-agent@wire-up-coverage-instrumentation antrun:run@print-test-jdk-banner surefire:test failsafe:integration-test failsafe:verify jacoco:report@build-coverage-report-from-tests -Denforcer.skip=true ``` +#### Development Setup for native embedding + +Run native-runtime Maven commands from the `java` directory. Native packaging requires Node.js and npm in addition to JDK 25 and Maven because `copilot-native/scripts/fetch-native.mjs` retrieves the pinned npm runtime package. + +The build currently supports native packaging only on Linux x64. Maven activates the `native-linux-x64` profile automatically on a Linux `amd64` host. That profile runs the native fetch-script tests, fetches the pinned `@github/copilot-linux-x64` package, builds the `linux-x64` classifier JAR, and verifies its native contents. Ensure npm can authenticate to the package registry before running the build. + +On macOS, Windows, Linux ARM64, and other unsupported hosts, do not force the Linux profile. A normal build produces only the OS-neutral primary, sources, and Javadoc JARs; it does not run the Linux x64 fetch-script tests, download or stage Linux native files, or produce a `linux-x64` classifier JAR. Use this command to validate that behavior: + +```bash +mvn -pl copilot-native clean verify +``` + +To build only the OS-neutral artifacts on a supported host, disable native download and packaging: + +```bash +mvn -pl copilot-native clean package -DskipTests -Dcopilot.native.skip.download=true +``` + +Linux x64 validation is pending. The follow-up work must confirm automatic profile activation, run the fetch-script tests and full Java reactor, verify the classifier JAR contents, and confirm the placeholder-only command above suppresses all native output. Update this section with the validated Linux x64 behavior and any required command changes after that work is complete. + ## License MIT — see [LICENSE](sdk/LICENSE) for details. diff --git a/java/copilot-native/pom.xml b/java/copilot-native/pom.xml index 7c36801a5..8715357d2 100644 --- a/java/copilot-native/pom.xml +++ b/java/copilot-native/pom.xml @@ -17,7 +17,7 @@ jar GitHub Copilot SDK :: Java :: Native Runtime - Native runtime binaries for the GitHub Copilot Java SDK, published as per-platform classifier JARs + Native runtime artifacts for the GitHub Copilot Java SDK, with host-matched platform classifier JARs https://github.com/github/copilot-sdk @@ -35,12 +35,6 @@ its SHA-512 integrity hash. --> ${project.basedir}/../.. - - linux-x64 ${project.build.directory}/native-staging org.codehaus.mojo exec-maven-plugin - fetch-native-linux-x64 - generate-resources + fetch-native + none exec @@ -93,11 +88,12 @@ test-fetch-native - test + none exec + ${skipTests} node --test @@ -118,8 +114,8 @@ native//platform.properties. --> - jar-linux-x64 - package + jar-native + none jar @@ -173,7 +169,7 @@ verify-native-jars - package + none run @@ -193,10 +189,10 @@ - + - + @@ -222,33 +218,47 @@ - skip-native-download + native-linux-x64 + + Linux + amd64 + - copilot.native.skip.download - true + !copilot.native.skip.download + + linux-x64 + copilot + org.codehaus.mojo exec-maven-plugin - - true - + + + fetch-native + generate-resources + + + test-fetch-native + test + + org.apache.maven.plugins maven-jar-plugin - jar-linux-x64 - none + jar-native + package @@ -258,12 +268,36 @@ verify-native-jars - none + package + + + skip-native-download + + + copilot.native.skip.download + true + + + + + + org.codehaus.mojo + exec-maven-plugin + + true + + + + + diff --git a/java/docs/adr/adr-007-native-bundling-strategy.md b/java/docs/adr/adr-007-native-bundling-strategy.md index f561842fe..89f2a54d3 100644 --- a/java/docs/adr/adr-007-native-bundling-strategy.md +++ b/java/docs/adr/adr-007-native-bundling-strategy.md @@ -208,7 +208,7 @@ If none succeeds, startup fails. The PATH fallback does not claim to support eve ### Current platform scope -The platform detector recognizes the 8 classifiers listed in this ADR. The current Maven packaging and documented experimental support publish only `linux-x64`. Additional classifier artifacts remain follow-up work. +The platform detector recognizes the 8 classifiers listed in this ADR. The Maven build binds native packaging only when the build host matches an implemented classifier. Currently, only Linux x64 hosts fetch and package the `linux-x64` classifier. Unsupported hosts build only the OS-neutral placeholder, sources, and Javadoc artifacts. Additional classifier artifacts remain follow-up work. ## Binding technology: JNA over Panama FFM @@ -359,8 +359,9 @@ The pattern follows DJL's `LibUtils.loadLibrary()` approach: detect the platform 2. Locates the matching `runtime.node` binary on the classpath (via `getResourceAsStream` from the classifier JAR). 3. Extracts `runtime.node` and the transitional CLI entrypoint into `~/.copilot/runtime-cache/` if valid cached files are not already present. 4. Loads it via [JNA](#references) using the C ABI entry points, per the [binding technology decision](#binding-technology-jna-over-panama-ffm) above. The JNA-specific code is confined behind an internal binding interface to preserve a future FFM migration path. -* The Java build fetches the pinned `@github/copilot-` npm package, verifies its SHA-512 integrity from `nodejs/package-lock.json`, and packages the version-matched runtime and CLI files. +* A supported host profile fetches the pinned matching `@github/copilot-` npm package, verifies its SHA-512 integrity from `nodejs/package-lock.json`, and packages the version-matched runtime and CLI files. * The current release work publishes the `linux-x64` classifier. The planned classifier set expands to the other detected platforms. +* Adding an implemented platform requires a host profile that supplies the classifier and platform CLI filename and binds the shared fetch, fetch-script test, package, and verification executions to the Maven lifecycle. * `cli-native.node` is not bundled. It provides terminal UI features that are irrelevant to the Java SDK's programmatic API surface. ## Related work items From 7e41e942613113f91db575577ecb27ec63cd5d81 Mon Sep 17 00:00:00 2001 From: Ed Burns Date: Sun, 16 Aug 2026 22:34:13 +0000 Subject: [PATCH 2/3] docs(java): validate Linux native packaging Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: de80978a-085e-476a-97a2-d2c5858446f6 --- java/README.md | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/java/README.md b/java/README.md index c216d0d19..91b8f7124 100644 --- a/java/README.md +++ b/java/README.md @@ -491,7 +491,7 @@ mvn jacoco:prepare-agent@wire-up-coverage-instrumentation antrun:run@print-test- Run native-runtime Maven commands from the `java` directory. Native packaging requires Node.js and npm in addition to JDK 25 and Maven because `copilot-native/scripts/fetch-native.mjs` retrieves the pinned npm runtime package. -The build currently supports native packaging only on Linux x64. Maven activates the `native-linux-x64` profile automatically on a Linux `amd64` host. That profile runs the native fetch-script tests, fetches the pinned `@github/copilot-linux-x64` package, builds the `linux-x64` classifier JAR, and verifies its native contents. Ensure npm can authenticate to the package registry before running the build. +Validated on a native Linux x64 host: Maven activates the `native-linux-x64` profile automatically on Linux `amd64` when `copilot.native.skip.download` is not set. That profile runs the native fetch-script tests, fetches the pinned `@github/copilot-linux-x64` package during `generate-resources`, packages the `linux-x64` classifier JAR during `package`, and verifies its native contents. Ensure npm can authenticate to the package registry before running the build. On macOS, Windows, Linux ARM64, and other unsupported hosts, do not force the Linux profile. A normal build produces only the OS-neutral primary, sources, and Javadoc JARs; it does not run the Linux x64 fetch-script tests, download or stage Linux native files, or produce a `linux-x64` classifier JAR. Use this command to validate that behavior: @@ -499,13 +499,22 @@ On macOS, Windows, Linux ARM64, and other unsupported hosts, do not force the Li mvn -pl copilot-native clean verify ``` -To build only the OS-neutral artifacts on a supported host, disable native download and packaging: +To build only the OS-neutral artifacts on any host, disable native download and packaging: ```bash mvn -pl copilot-native clean package -DskipTests -Dcopilot.native.skip.download=true ``` -Linux x64 validation is pending. The follow-up work must confirm automatic profile activation, run the fetch-script tests and full Java reactor, verify the classifier JAR contents, and confirm the placeholder-only command above suppresses all native output. Update this section with the validated Linux x64 behavior and any required command changes after that work is complete. +The verified Linux x64 checks are: + +```bash +mvn -pl copilot-native help:active-profiles +mvn -pl copilot-native test +mvn clean verify +mvn clean package -pl copilot-native -DskipTests -Dcopilot.native.skip.download=true +``` + +On a supported Linux x64 host, the classifier JAR contains `native/linux-x64/runtime.node`, `native/linux-x64/platform.properties`, and `native/linux-x64/copilot`. The placeholder JAR remains OS-neutral and contains no native binaries. Unsupported hosts retain the placeholder-only behavior without producing a `-linux-x64.jar`. ## License From 84a429bc19967b3cb9b09a78628d81a83af95445 Mon Sep 17 00:00:00 2001 From: Ed Burns Date: Mon, 17 Aug 2026 01:13:31 +0000 Subject: [PATCH 3/3] fix(java): guard native packaging by libc Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: de80978a-085e-476a-97a2-d2c5858446f6 --- .github/workflows/java-sdk-tests.yml | 9 +- java/README.md | 28 +++-- java/copilot-native/pom.xml | 111 +++++++++++++++++- .../scripts/validate-native-host.mjs | 55 +++++++++ .../scripts/validate-native-host.test.mjs | 67 +++++++++++ .../adr/adr-007-native-bundling-strategy.md | 6 +- 6 files changed, 261 insertions(+), 15 deletions(-) create mode 100644 java/copilot-native/scripts/validate-native-host.mjs create mode 100644 java/copilot-native/scripts/validate-native-host.test.mjs diff --git a/.github/workflows/java-sdk-tests.yml b/.github/workflows/java-sdk-tests.yml index bd0a34bd2..918e6ce10 100644 --- a/.github/workflows/java-sdk-tests.yml +++ b/.github/workflows/java-sdk-tests.yml @@ -40,10 +40,13 @@ jobs: with: node-version: 22 + - name: Validate Linux glibc native host + run: node copilot-native/scripts/validate-native-host.mjs linux-x64 + - name: Run Java SDK tests (InProcess) env: CI: "true" - run: mvn clean verify -Pinprocess + run: mvn clean verify -Pinprocess -Dcopilot.native.skip.download=false - name: Generate Test Report Summary if: always() @@ -125,7 +128,9 @@ jobs: if: matrix.test-jdk == '25' env: CI: "true" - run: mvn verify -Dskip.test.harness=true + run: | + node copilot-native/scripts/validate-native-host.mjs linux-x64 + mvn verify -Dskip.test.harness=true -Dcopilot.native.libc=glibc -Dcopilot.native.skip.download=false - name: Switch to JDK 17 if: matrix.test-jdk == '17' diff --git a/java/README.md b/java/README.md index 91b8f7124..22a3e7cba 100644 --- a/java/README.md +++ b/java/README.md @@ -491,27 +491,37 @@ mvn jacoco:prepare-agent@wire-up-coverage-instrumentation antrun:run@print-test- Run native-runtime Maven commands from the `java` directory. Native packaging requires Node.js and npm in addition to JDK 25 and Maven because `copilot-native/scripts/fetch-native.mjs` retrieves the pinned npm runtime package. -Validated on a native Linux x64 host: Maven activates the `native-linux-x64` profile automatically on Linux `amd64` when `copilot.native.skip.download` is not set. That profile runs the native fetch-script tests, fetches the pinned `@github/copilot-linux-x64` package during `generate-resources`, packages the `linux-x64` classifier JAR during `package`, and verifies its native contents. Ensure npm can authenticate to the package registry before running the build. +Validated on a native Linux x64 glibc host: Maven activates the `native-linux-x64` profile on Linux `amd64` when `copilot.native.libc=glibc` is set. The build validates the host before downloading or packaging native files. The profile runs the native script tests, fetches the pinned `@github/copilot-linux-x64` package during `generate-resources`, packages the `linux-x64` classifier JAR during `package`, and verifies its native contents. An absent or explicitly false `copilot.native.skip.download` value preserves normal native packaging. Ensure npm can authenticate to the package registry before running the build. -On macOS, Windows, Linux ARM64, and other unsupported hosts, do not force the Linux profile. A normal build produces only the OS-neutral primary, sources, and Javadoc JARs; it does not run the Linux x64 fetch-script tests, download or stage Linux native files, or produce a `linux-x64` classifier JAR. Use this command to validate that behavior: +Before opting in, validate that Node.js reports glibc for the build host: ```bash -mvn -pl copilot-native clean verify +node copilot-native/scripts/validate-native-host.mjs linux-x64 +mvn -pl copilot-native clean verify -Dcopilot.native.libc=glibc ``` -To build only the OS-neutral artifacts on any host, disable native download and packaging: +The `inprocess` test profile performs the same validation and native packaging automatically, so the full in-process test command remains: ```bash -mvn -pl copilot-native clean package -DskipTests -Dcopilot.native.skip.download=true +mvn -Pinprocess clean verify +``` + +On macOS, Windows, Linux ARM64, Linux x64 musl, and other unsupported hosts, do not set `copilot.native.libc=glibc`. A normal build produces only the OS-neutral primary, sources, and Javadoc JARs; it does not run the Linux x64 native script tests, download or stage Linux native files, or produce a `linux-x64` classifier JAR. + +To build only the OS-neutral artifacts on any host, or override the glibc opt-in, disable native download and packaging: + +```bash +mvn -pl copilot-native clean package -DskipTests -Dcopilot.native.libc=glibc -Dcopilot.native.skip.download=true ``` The verified Linux x64 checks are: ```bash -mvn -pl copilot-native help:active-profiles -mvn -pl copilot-native test -mvn clean verify -mvn clean package -pl copilot-native -DskipTests -Dcopilot.native.skip.download=true +node --test copilot-native/scripts/fetch-native.test.mjs copilot-native/scripts/validate-native-host.test.mjs +mvn -pl copilot-native help:active-profiles -Dcopilot.native.libc=glibc -Dcopilot.native.skip.download=false +mvn -pl copilot-native test -Dcopilot.native.libc=glibc +mvn clean verify -Dcopilot.native.libc=glibc +mvn clean package -pl copilot-native -DskipTests -Dcopilot.native.libc=glibc -Dcopilot.native.skip.download=true ``` On a supported Linux x64 host, the classifier JAR contains `native/linux-x64/runtime.node`, `native/linux-x64/platform.properties`, and `native/linux-x64/copilot`. The placeholder JAR remains OS-neutral and contains no native binaries. Unsupported hosts retain the placeholder-only behavior without producing a `-linux-x64.jar`. diff --git a/java/copilot-native/pom.xml b/java/copilot-native/pom.xml index 8715357d2..698ce4eae 100644 --- a/java/copilot-native/pom.xml +++ b/java/copilot-native/pom.xml @@ -70,6 +70,20 @@ org.codehaus.mojo exec-maven-plugin + + validate-native-host + none + + exec + + + node + + ${project.basedir}/scripts/validate-native-host.mjs + ${copilot.native.classifier} + + + fetch-native none @@ -98,6 +112,7 @@ --test ${project.basedir}/scripts/fetch-native.test.mjs + ${project.basedir}/scripts/validate-native-host.test.mjs @@ -229,7 +244,8 @@ amd64 - !copilot.native.skip.download + copilot.native.libc + glibc @@ -242,6 +258,65 @@ org.codehaus.mojo exec-maven-plugin + + validate-native-host + validate + + + fetch-native + generate-resources + + + test-fetch-native + test + + + + + org.apache.maven.plugins + maven-jar-plugin + + + jar-native + package + + + + + org.apache.maven.plugins + maven-antrun-plugin + + + verify-native-jars + package + + + + + + + + + inprocess + + linux-x64 + copilot + + + + + org.codehaus.mojo + exec-maven-plugin + + + validate-native-host + validate + fetch-native generate-resources @@ -295,6 +370,40 @@ true + + + validate-native-host + none + + + fetch-native + none + + + test-fetch-native + none + + + + + org.apache.maven.plugins + maven-jar-plugin + + + jar-native + none + + + + + org.apache.maven.plugins + maven-antrun-plugin + + + verify-native-jars + none + + diff --git a/java/copilot-native/scripts/validate-native-host.mjs b/java/copilot-native/scripts/validate-native-host.mjs new file mode 100644 index 000000000..10b19530e --- /dev/null +++ b/java/copilot-native/scripts/validate-native-host.mjs @@ -0,0 +1,55 @@ +/*--------------------------------------------------------------------------------------------- + * Copyright (c) Microsoft Corporation. All rights reserved. + *--------------------------------------------------------------------------------------------*/ + +import { pathToFileURL } from "node:url"; + +export function validateNativeHost(classifier, host) { + if (classifier !== "linux-x64") { + throw new Error(`Unsupported native build classifier: ${classifier}`); + } + if (host.platform !== "linux" || host.arch !== "x64") { + throw new Error( + `Native ${classifier} packaging requires Linux x64; detected ${host.platform}-${host.arch}`, + ); + } + if (!host.glibcVersionRuntime) { + throw new Error( + `Native ${classifier} packaging requires glibc; musl and unknown libc hosts are unsupported`, + ); + } + + return `Validated native build host: ${classifier} (glibc ${host.glibcVersionRuntime})`; +} + +export function detectNativeHost() { + const report = process.report?.getReport(); + return { + platform: process.platform, + arch: process.arch, + glibcVersionRuntime: report?.header?.glibcVersionRuntime, + }; +} + +function main() { + const [classifier] = process.argv.slice(2); + if (!classifier) { + console.error("Usage: node validate-native-host.mjs "); + process.exitCode = 1; + return; + } + + try { + console.log(validateNativeHost(classifier, detectNativeHost())); + } catch (error) { + console.error(error.message); + process.exitCode = 1; + } +} + +if ( + process.argv[1] && + import.meta.url === pathToFileURL(process.argv[1]).href +) { + main(); +} diff --git a/java/copilot-native/scripts/validate-native-host.test.mjs b/java/copilot-native/scripts/validate-native-host.test.mjs new file mode 100644 index 000000000..cbad530c0 --- /dev/null +++ b/java/copilot-native/scripts/validate-native-host.test.mjs @@ -0,0 +1,67 @@ +/*--------------------------------------------------------------------------------------------- + * Copyright (c) Microsoft Corporation. All rights reserved. + *--------------------------------------------------------------------------------------------*/ + +import assert from "node:assert/strict"; +import test from "node:test"; + +import { validateNativeHost } from "./validate-native-host.mjs"; + +test("accepts Linux x64 with glibc", () => { + assert.equal( + validateNativeHost("linux-x64", { + platform: "linux", + arch: "x64", + glibcVersionRuntime: "2.39", + }), + "Validated native build host: linux-x64 (glibc 2.39)", + ); +}); + +test("rejects Linux x64 with musl or unknown libc", () => { + assert.throws( + () => + validateNativeHost("linux-x64", { + platform: "linux", + arch: "x64", + glibcVersionRuntime: undefined, + }), + /requires glibc/, + ); +}); + +test("rejects a non-Linux host", () => { + assert.throws( + () => + validateNativeHost("linux-x64", { + platform: "darwin", + arch: "x64", + glibcVersionRuntime: undefined, + }), + /requires Linux x64/, + ); +}); + +test("rejects a non-x64 host", () => { + assert.throws( + () => + validateNativeHost("linux-x64", { + platform: "linux", + arch: "arm64", + glibcVersionRuntime: "2.39", + }), + /requires Linux x64/, + ); +}); + +test("rejects an unimplemented classifier", () => { + assert.throws( + () => + validateNativeHost("linuxmusl-x64", { + platform: "linux", + arch: "x64", + glibcVersionRuntime: undefined, + }), + /Unsupported native build classifier/, + ); +}); diff --git a/java/docs/adr/adr-007-native-bundling-strategy.md b/java/docs/adr/adr-007-native-bundling-strategy.md index 89f2a54d3..bed843a06 100644 --- a/java/docs/adr/adr-007-native-bundling-strategy.md +++ b/java/docs/adr/adr-007-native-bundling-strategy.md @@ -208,7 +208,7 @@ If none succeeds, startup fails. The PATH fallback does not claim to support eve ### Current platform scope -The platform detector recognizes the 8 classifiers listed in this ADR. The Maven build binds native packaging only when the build host matches an implemented classifier. Currently, only Linux x64 hosts fetch and package the `linux-x64` classifier. Unsupported hosts build only the OS-neutral placeholder, sources, and Javadoc artifacts. Additional classifier artifacts remain follow-up work. +The platform detector recognizes the 8 classifiers listed in this ADR. The Maven build binds native packaging only for an explicitly selected implemented classifier. Currently, Linux x64 glibc hosts can opt in with `copilot.native.libc=glibc`, while the `inprocess` test profile selects the required `linux-x64` classifier automatically. Both paths validate the host before downloading or packaging native files. Linux x64 musl and other unsupported hosts build only the OS-neutral placeholder, sources, and Javadoc artifacts unless they explicitly request in-process tests, which fail during host validation. Additional classifier artifacts remain follow-up work. ## Binding technology: JNA over Panama FFM @@ -359,9 +359,9 @@ The pattern follows DJL's `LibUtils.loadLibrary()` approach: detect the platform 2. Locates the matching `runtime.node` binary on the classpath (via `getResourceAsStream` from the classifier JAR). 3. Extracts `runtime.node` and the transitional CLI entrypoint into `~/.copilot/runtime-cache/` if valid cached files are not already present. 4. Loads it via [JNA](#references) using the C ABI entry points, per the [binding technology decision](#binding-technology-jna-over-panama-ffm) above. The JNA-specific code is confined behind an internal binding interface to preserve a future FFM migration path. -* A supported host profile fetches the pinned matching `@github/copilot-` npm package, verifies its SHA-512 integrity from `nodejs/package-lock.json`, and packages the version-matched runtime and CLI files. +* A validated supported-host profile fetches the pinned matching `@github/copilot-` npm package, verifies its SHA-512 integrity from `nodejs/package-lock.json`, and packages the version-matched runtime and CLI files. * The current release work publishes the `linux-x64` classifier. The planned classifier set expands to the other detected platforms. -* Adding an implemented platform requires a host profile that supplies the classifier and platform CLI filename and binds the shared fetch, fetch-script test, package, and verification executions to the Maven lifecycle. +* Adding an implemented platform requires validated host activation, a profile that supplies the classifier and platform CLI filename, and lifecycle bindings for the shared host validation, fetch, script test, package, and verification executions. * `cli-native.node` is not bundled. It provides terminal UI features that are irrelevant to the Java SDK's programmatic API surface. ## Related work items