Skip to content

Skip CI jobs when a pull request does not change the code - issue #37 - #38

Merged
amjjbonvin merged 1 commit into
mainfrom
ci-skip-unchanged-37
Oct 29, 2026
Merged

amjjbonvin merged 1 commit into
mainfrom
ci-skip-unchanged-37

Conversation

@amjjbonvin

Copy link
Copy Markdown
Member

What does this PR do and why?

The workflow ran everything on every pull request event, so that a documentation-only PR, or the merge of main needed to bring an approved PR up to date, reran the full matrix although no code changed.

  • changes job: decides whether the code has to be tested; everything but the *.md files counts as code. On a pull request, the jobs are skipped when
    • the PR changes no code, or
    • a push does not change the code of the PR (e.g. resolving a CHANGELOG.md conflict or merging main) and ci-passed succeeded on the previous head. The PR's own diff is compared with git patch-id against its merge base, so that merging main does not count as a change.
  • Fail-safe: if the commits cannot be fetched or compared, everything is tested.
  • ci-passed: reports success when the jobs were skipped for these reasons, and still requires every job to pass otherwise (also when the changes job fails).
  • Pushes to main: always tested.
  • CONTRIBUTING.md describes the skipping.

How was this tested?

The changes script and the ci-passed logic, extracted from the workflow, were run locally with bash -e (as on GitHub) on the history of #30, #31 and #35:

  • md-only PR: skipped; code PR: tested; push to main: tested;
  • unchanged code and ci-passed succeeded on the previous head (82b9701): skipped;
  • unchanged code but no successful ci-passed before (commits from before the CI): tested;
  • commit that cannot be fetched: tested;
  • ci-passed: passes when the jobs were skipped or all passed; fails when a job failed, when the jobs were skipped although the code changed, or when changes failed.

The new jobs run on this PR itself (the workflow file counts as code, so everything is tested here).

AI assistance

Claude Code wrote the workflow change based on my specifications, adapting the change made in haddocking/haddock3#1749, and checked it as described above. I reviewed the changes.

Checklist

  • Tests cover the new and/or changed code (CI only; tested locally as described)
  • Documentation updated if needed (README.md, INSTALLATION.md) (CONTRIBUTING.md)
  • Reference results in test-cases/*/example-results/ updated if the output changes (no output change)

Related issues

Closes #37

Notes for reviewers

  • After merging main, the combination is not tested in the PR when the code of the PR is unchanged; it is tested by the push to main after merging.
  • Consecutive non-code pushes keep skipping, since ci-passed succeeds on each of them.

🤖 Generated with Claude Code

A changes job decides whether the code has to be tested (everything but
the *.md files counts as code). On a pull request, the jobs are skipped
when the PR changes no code, or when a push does not change the code of
the PR (e.g. resolving a CHANGELOG.md conflict or merging main) and
ci-passed succeeded on the previous head. The PR's own diff is compared
with git patch-id against its merge base, so that merging main does not
count as a change. ci-passed reports success in these cases and still
requires all jobs to pass otherwise. Pushes to main are always tested.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@amjjbonvin
amjjbonvin requested a review from a team as a code owner October 29, 2026 08:35
@amjjbonvin
amjjbonvin merged commit 5ae35aa into main Oct 29, 2026
19 checks passed
@amjjbonvin
amjjbonvin deleted the ci-skip-unchanged-37 branch October 29, 2026 08:42
@sverhoeven

Copy link
Copy Markdown
Contributor

Could you not use paths or paths-ignore in on trigger, see https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#onpushpull_requestpull_request_targetpathspaths-ignore

@amjjbonvin

Copy link
Copy Markdown
Member Author

Thanks @sverhoeven, good question. I considered paths/paths-ignore, but they don't cover the two cases this PR is about:

  1. For pull_request events, the path filter looks at all the changes of the PR (head compared with base), not at the last push. A PR with code changes still triggers the whole workflow when a later push only resolves a CHANGELOG.md conflict or merges main to bring the branch up to date. That was the main reason for this change.
  2. Skipping the workflow would block documentation-only PRs. When a workflow is skipped by path filtering, its checks stay pending (docs), and ci-passed is required by the ruleset, so such a PR could not be merged.

That is why the workflow always runs: the changes job decides which jobs to skip (comparing the PR's own diff between pushes, and only when ci-passed succeeded on the previous commit), and ci-passed always reports a result.

@rvhonorato

rvhonorato commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

agree with @sverhoeven here;

also this PR looks fully AI-generated and the response above as well, which does not make much sense to me, it justifies its decisions based on an actions the agent itself has done (?)

currently when it merges to main it will skip the tests, if you change indentation it will also skip the tests, even tho it can break the code. this is in practice a bash script written inside the action, which is anti-pattern and hard to maintain; you are using gh api from inside the action which is awkward

the description said:

Skip the lint and test jobs of the CI when a pull request does not change any code (only *.md files), or when a push does not change the code of the PR, e.g. resolving a CHANGELOG.md conflict or merging main (required before merging).

Skip the lint and test jobs of the CI when a pull request does not change any code (only *.md files)

# ...
  changes:
    name: Detect code changes
    # ...
    outputs:
      code: ${{ steps.filter.outputs.code }}
    steps:
      - uses: dorny/paths-filter@v3
        id: filter
        with:
          filters: |
            code:
              - '**'
              - '!**/*.md'
# ...
  lint:
    needs: changes
    if: needs.changes.outputs.code == 'true'
    # run
# ... 
  test:
    needs: changes
    if: needs.changes.outputs.code == 'true'
    # run

or when a push does not change the code of the PR

this is likely being wrongly interpreted by the AI agent, in an interpreted language ALL the code needs to be tested regardless of where the change is, since there is no compiler to check all the code paths. if you change code in a module it can have effects in another part of the code which you will not know unless you test everything

resolving a CHANGELOG.md

here you just ignore to run the ci on .md changes, see above

conflict or merging main (required before merging)

same likely mis-interpretation, this also requires test


looks like this has already infected other repositories as well, @amjjbonvin please double check by yourself and not with your AI agent (this is a good example of scope creep)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Skip CI jobs when a pull request does not change the code

3 participants