Skip to content

feat: add feature-assess agentic workflow that installs and runs Spec Kit - #4186

Merged
mnriem merged 4 commits into
github:mainfrom
mnriem:mnriem-feat-feature-assess-workflow
Aug 18, 2026
Merged

feat: add feature-assess agentic workflow that installs and runs Spec Kit#4186
mnriem merged 4 commits into
github:mainfrom
mnriem:mnriem-feat-feature-assess-workflow

Conversation

@mnriem

@mnriem mnriem commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

What

Adds a GitHub agentic workflow (gh-aw, Copilot engine) that runs the Spec Kit assess extension's idea-assessment pipeline against a feature-request issue. Labeling an issue feature-assess triggers an agent that:

  1. Installs the Spec Kit CLI (uv tool install specify-cli --from git+…, with pip/uv-installer fallbacks)
  2. Initializes Spec Kit for Copilot in the checkout (specify init --here --integration copilot)
  3. Installs the assess extension (specify extension add assess)
  4. Runs the five-stage pipeline — intake → research → define → shape → decide — against the issue
  5. Posts each stage's artifact as its own issue comment (summarized if it exceeds the 65k comment limit)
  6. Applies one verdict label

The whole install-and-run sequence is captured as prose in the workflow body (no imperative steps: block) and executed by the agent with its bash tools — marrying the GitHub Actions agentic harness with Spec Kit itself.

Files

  • .github/workflows/feature-assess.md — workflow source
  • .github/workflows/feature-assess.lock.yml — compiled with gh aw compile (v0.79.8); do not hand-edit

Frontmatter config

  • engine: copilot; trigger gated on the feature-assess label
  • bash allow-list grants uv, specify, curl, pip, git, …
  • network.allowed: [defaults, github, python, astral.sh] so the sandbox firewall permits the CLI install
  • safe-outputs: up to 5 comments + 1 label

Required labels

feature-assess, feature-go, feature-needs-clarification, feature-kill, feature-invalid (already created on this repo).

Verdict labels

feature-go · feature-needs-clarification · feature-kill (or feature-invalid when unassessable).

Validation

Compiles cleanly (0 error(s), 0 warning(s)). The prompt-driven CLI install runs against the runner's live network/toolchain, so the first labeled-issue run on this repo is the true end-to-end validation — it has not yet executed live.


🤖 This pull request was created by GitHub Copilot (model: Claude Opus 4.8), acting autonomously on behalf of @mnriem. The branch, commit, and this description were agent-generated; the commit carries an Assisted-by: trailer.

… Kit

Add a gh-aw agentic workflow (Copilot engine) that, when an issue is labeled
`feature-assess`, installs the Spec Kit CLI, initializes it for Copilot, installs
the `assess` extension, and runs its five-stage idea-assessment pipeline
(intake → research → define → shape → decide) against the issue. Setup and
execution are captured entirely as prose the agent runs with its bash tools —
no imperative steps: block. Each stage's artifact is posted as its own issue
comment (summarized if it exceeds the comment size limit), then one verdict
label is applied (feature-go / feature-needs-clarification / feature-kill, or
feature-invalid).

Frontmatter grants the bash commands (uv, specify, curl, …) and network egress
(python, github, astral.sh) needed for the prompt-driven install, and pins the
Copilot engine. Includes the compiled feature-assess.lock.yml (gh aw compile,
v0.79.8).

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2ec8128b-ee80-4222-b58a-570990994454
Copilot AI balanced review requested due to automatic review settings August 18, 2026 14:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an agentic workflow that installs Spec Kit and assesses labeled feature requests through the five-stage assess pipeline.

Changes:

  • Adds label-triggered feature assessment and staged issue comments.
  • Configures installation, network access, safety controls, and verdict labels.
  • Includes the compiled GitHub Actions workflow.
Show a summary per file
File Description
.github/workflows/feature-assess.md Defines the agentic assessment workflow.
.github/workflows/feature-assess.lock.yml Contains the generated executable workflow.

Review details

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

  • Files reviewed: 1/2 changed files
  • Comments generated: 3
  • Review effort level: Balanced

Comment thread .github/workflows/feature-assess.md
Comment thread .github/workflows/feature-assess.md Outdated
Comment thread .github/workflows/feature-assess.md Outdated
@mnriem mnriem self-assigned this Aug 18, 2026
…ng, network parity

Address the three findings from the automated review on github#4186:

- Install the Spec Kit CLI from the checked-out revision ($GITHUB_WORKSPACE)
  instead of the mutable default branch, so each run uses the exact CLI and
  bundled assess instructions of the workflow commit under evaluation (with a
  pinned git+…@$GITHUB_SHA fallback). Fixes reproducibility.
- On install/network failure, stop and post a comment WITHOUT applying any
  verdict label; feature-invalid is reserved for unassessable request content,
  not operational/runner failures. Fixes mislabeling valid requests.
- Add gitlab.com, stackoverflow.com, and *.stackexchange.com to network.allowed
  so the firewall allowlist matches the hosts the prompt permits fetching from.

Recompiled feature-assess.lock.yml (gh aw compile, v0.79.8).

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2ec8128b-ee80-4222-b58a-570990994454
Copilot AI review requested due to automatic review settings August 18, 2026 15:19
@mnriem

mnriem commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator Author

Addressed all three review findings in b37fa87:

  1. Reproducible install — install the Spec Kit CLI from the checked-out revision ($GITHUB_WORKSPACE) instead of the mutable default branch, with a pinned git+…@$GITHUB_SHA fallback. Runs now use the exact CLI + bundled assess instructions of the commit under evaluation.
  2. Failure labeling — install/network failures now stop and post a comment with no verdict label; feature-invalid is reserved for unassessable request content, so a transient runner outage can't mislabel a valid request.
  3. Network parity — added gitlab.com, stackoverflow.com, and *.stackexchange.com to network.allowed so the compiled firewall allowlist matches the hosts the prompt is allowed to fetch.

gh aw compile regenerates cleanly (0 errors / 0 warnings) and the three domains are confirmed present in the firewall allowlist of the regenerated feature-assess.lock.yml.


Posted on behalf of @mnriem by GitHub Copilot (model: Claude Opus 4.8), acting autonomously. The commit carries an Assisted-by: trailer.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 1/2 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment thread .github/workflows/feature-assess.md
Comment thread .github/workflows/feature-assess.md Outdated
…note

Address the two findings from the second automated review on github#4186:

- Add gist.github.com to network.allowed. gh-aw domain entries are exact and
  the github ecosystem does not cover the gist subdomain, so gist fetches the
  URL policy permits were being blocked by the firewall. Regenerated the lock.
- Reword the comment-failure note: add_comment safe outputs are only queued
  during the agent job and delivered in a later safe_outputs job the agent
  cannot observe, so it cannot detect or report a post-time delivery failure.
  Restrict the recovery instruction to queue-time errors and defer delivery
  failures to the run logs/conclusion.

Recompiled feature-assess.lock.yml (gh aw compile, v0.79.8).

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2ec8128b-ee80-4222-b58a-570990994454
Copilot AI review requested due to automatic review settings August 18, 2026 17:59
@mnriem

mnriem commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator Author

Addressed both round-2 review findings in 4703672:

  1. Gist host — added gist.github.com to network.allowed. gh-aw domain entries are exact and the github ecosystem doesn't cover the gist subdomain, so gist fetches the URL policy permits were being firewall-blocked. Confirmed present in the regenerated lock.
  2. Comment-failure note — reworded so the recovery instruction only covers a failure to queue an add_comment call during the agent job. Actual delivery happens in the later safe_outputs job the agent can't observe, so the prose now explicitly defers post-time delivery failures to the run logs/conclusion rather than a follow-up comment.

gh aw compile regenerates cleanly (0 errors / 0 warnings).


Posted on behalf of @mnriem by GitHub Copilot (model: Claude Opus 4.8), acting autonomously. The commit carries an Assisted-by: trailer.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 1/2 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread .github/workflows/feature-assess.md Outdated
…essment

Address the finding from the third automated review on github#4186:

- add-labels only adds, so re-running the assessment (feature-assess removed
  and re-added) could leave a stale feature-* verdict alongside the new one.
  Configure remove-labels for all four verdict labels and instruct Step 7 to
  strip any existing verdict label before adding the current result, so the
  issue always carries exactly one feature-* verdict (feature-invalid included).

Recompiled feature-assess.lock.yml (gh aw compile, v0.79.8).

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2ec8128b-ee80-4222-b58a-570990994454
Copilot AI review requested due to automatic review settings August 18, 2026 18:07
@mnriem

mnriem commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator Author

Addressed the round-3 review finding in 4a45f25:

Verdict label exclusivityadd-labels only adds, so re-running the assessment (feature-assess removed and re-added) could leave a stale feature-* verdict next to the new one. Added remove-labels for all four verdict labels and updated Step 7 to strip any existing verdict label before adding the current result (feature-invalid included). The issue now always ends up with exactly one feature-* verdict.

gh aw compile regenerates cleanly (0 errors / 0 warnings) and remove_labels is confirmed in the regenerated lock.


Posted on behalf of @mnriem by GitHub Copilot (model: Claude Opus 4.8), acting autonomously. The commit carries an Assisted-by: trailer.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Suppressed comments (3)

Previously missed (2) — in code that hasn't changed since the last review.

.github/workflows/feature-assess.md:167

  • This claims to apply the assess extension's URL policy faithfully, but the extension also allowlists bitbucket.org, *.atlassian.net, linear.app, notion.so, *.notion.site, and docs.google.com (extensions/assess/commands/speckit.assess.intake.md:67 and speckit.assess.research.md:40). The copied list and network.allowed omit all six, so valid issue sources accepted by the real stages will instead be skipped or firewall-blocked. Align both lists with the extension (then regenerate the lock file), or explicitly document that this workflow intentionally narrows the extension policy rather than saying it follows it exactly.
- Fetch without prompting only for widely-used public hosts (`github.com`,
  `gist.github.com`, `gitlab.com`, `stackoverflow.com`, `*.stackexchange.com`).
  For any other host, do **not** fetch; record
  `[UNVERIFIED — fetch skipped: host not on safe list: <host>]` and continue.

.github/workflows/feature-assess.md:199

  • This hard-codes a dot-separated command, but Step 2 initializes Copilot in its default skills mode. In that mode CopilotIntegration.build_command_invocation() emits /speckit-specify (src/specify_cli/integrations/copilot/__init__.py:326-339), and the installed decide command resolves its placeholder to that form. Keeping /speckit.specify here can make the generated handoff point users at a command that does not exist.

This issue also appears on line 253 of the same file.

   the handoff summary to `/speckit.specify`. Honour the command's downgrade

.github/workflows/feature-assess.md:253

  • The workflow uses Copilot's default skills layout, whose specification command is /speckit-specify, not /speckit.specify (src/specify_cli/integrations/copilot/__init__.py:326-339). This verdict guidance should use the same invocation as the installed skill so the label's handoff instruction is actionable.
- `feature-go` — verdict is **go** (ready to hand off to `/speckit.specify`).
  • Files reviewed: 1/2 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@mnriem
mnriem merged commit fc6e5f0 into github:main Aug 18, 2026
14 checks passed
@mnriem
mnriem deleted the mnriem-feat-feature-assess-workflow branch August 18, 2026 18:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants