Restore latest skills for CLI 1.22+

.agents/skills/bump-cli-compat/SKILL.md

@@ -64,7 +64,7 @@ If any tag doesn't exist, report the error and stop.
 
 ### Step 3: Read current manifest
 
-Read `internal/build/cli-compat.json`. Note the current versions and the list of versioned entries.
+Read `internal/build/cli-compat.json` and `AgentSkillsFallbackVersion` in `internal/build/clicompat.go`. Note the current versions and the list of versioned entries. The newest CLI range uses `"skills": "latest"`; the compiled constant supplies the tested fallback. Keep older released ranges pinned. Routine skills releases need no manifest bump. CLIs before `1.6.0` reject `"latest"` and use a valid cached or embedded manifest without returning a parse error; losing remote AppKit and skills compatibility updates for these CLIs is accepted.
 
 ### Step 4: Determine update type
 
@@ -81,24 +81,24 @@ Ask the user:
 
 ### Step 4a: No breaking changes (update in-place)
 
-Update the **highest versioned entry** to the new appkit and skills versions. Do NOT add new versioned keys. The manifest is range-based: updating the highest entry automatically covers all CLI versions in that range.
+Update the **highest versioned entry** to the new `appkit` version, keeping `"skills": "latest"`. Update `AgentSkillsFallbackVersion` in `internal/build/clicompat.go` to the tested skills version. Do NOT add new versioned keys. The manifest is range-based: updating the highest entry automatically covers all CLI versions in that range.
 
-Write the updated `internal/build/cli-compat.json`.
+Write the updated `internal/build/cli-compat.json` and `internal/build/clicompat.go`.
 
 ### Step 4b: Breaking changes (add new entry)
 
 Ask the user for the **minimum CLI version** that supports the new features.
 
-Add a new entry keyed to that CLI version with the new appkit and skills versions. Keep older entries unchanged so older CLI binaries stay compatible.
+Add a new entry keyed to that CLI version with the new `appkit` and `"skills": "latest"`. Update `AgentSkillsFallbackVersion` in `internal/build/clicompat.go` to the tested skills version. If skills compatibility changed, replace `"latest"` in the affected older ranges with the last compatible release. Preserve unrelated AppKit and skills compatibility boundaries.
 
-Write the updated `internal/build/cli-compat.json`.
+Write the updated `internal/build/cli-compat.json` and `internal/build/clicompat.go`.
 
 ### Step 5: Validate
 
 Run the Go tests to ensure the manifest is well-formed:
 
 ```bash
-go test ./libs/clicompat/... -run TestEmbeddedManifest -v
+go test ./libs/clicompat/... -run 'Test(EmbeddedManifest|ResolveEmbeddedAgentSkillsVersion)' -v
 ```
 
 If validation fails, show the errors and fix them before proceeding.
@@ -107,25 +107,25 @@ If validation fails, show the errors and fix them before proceeding.
 
 ```bash
 # Create a new branch from the current branch (or main)
-git checkout -b bump-cli-compat-appkit-{appkit_version}-skills-{skills_version}
+git checkout -b bump-cli-compat-appkit-{appkit_version}-skills-fallback-{skills_version}
 
 # Stage and commit
-git add internal/build/cli-compat.json
-git commit -s -m "Bump cli-compat to appkit {appkit_version}, skills {skills_version}"
+git add internal/build/cli-compat.json internal/build/clicompat.go
+git commit -s -m "Bump cli-compat to appkit {appkit_version}, skills fallback {skills_version}"
 
 # Push and create PR
 git push -u origin HEAD
 gh pr create \
-  --title "Bump cli-compat to appkit {appkit_version}, skills {skills_version}" \
+  --title "Bump cli-compat to appkit {appkit_version}, skills fallback {skills_version}" \
   --body "$(cat <<'EOF'
 ## Summary
 Bump `cli-compat.json` to use:
 - AppKit `{appkit_version}`
-- Agent Skills `{skills_version}`
+- Agent Skills track latest, with compiled fallback `{skills_version}`
 
 ## Checklist
 - [ ] Evals passed with no regressions
-- [ ] `go test ./libs/clicompat/... -run TestEmbeddedManifest` passes
+- [ ] `go test ./libs/clicompat/... -run 'Test(EmbeddedManifest|ResolveEmbeddedAgentSkillsVersion)'` passes
 EOF
 )"
 ```

.nextchanges/cli/aitools-latest-skills.md

@@ -0,0 +1 @@
+* Fix `databricks aitools install` overwriting newer raw skills with an older pinned release. CLI 1.22.0 and newer now install the latest published skills and prevent automatic downgrades. ([#7069](https://github.com/databricks/cli/pull/7069))

internal/build/README.md

@@ -2,23 +2,37 @@
 
 `cli-compat.json` maps Databricks CLI versions to compatible AppKit and Agent Skills versions. The CLI uses this manifest to determine which template version to use for `apps init` and which skills version to use for `aitools install`.
 
+Raw skills track the latest published skills release for the newest CLI range. Plugin installs use the agent's marketplace commands without a version pin; the manifest's skills settings only control raw-file delivery.
+
 ## Manifest format
 
 ```json
 {
-  "0.296.0": { "appkit": "0.27.0", "skills": "0.1.5" },
-  "0.290.0": { "appkit": "0.24.0", "skills": "0.1.4" },
-  "0.280.0": { "appkit": "0.20.0", "skills": "0.1.0" }
+  "1.22.0": { "appkit": "0.84.0", "skills": "latest" },
+  "1.20.0": { "appkit": "0.84.0", "skills": "0.2.30" },
+  "1.0.0": { "appkit": "0.76.1", "skills": "0.2.10" },
+  "0.299.2": { "appkit": "0.24.0", "skills": "0.1.5" }
 }
 ```
 
 Each key is a CLI version in semver format. Each entry defines a **range floor**: it applies to that CLI version and all versions above it, up to (but not including) the next entry. The manifest should be **sparse** — not every CLI version needs its own entry. Only add a new entry when a compatibility boundary changes.
 
+`skills` is an exact pin for its CLI range or `"latest"` to install the skills repository's latest published GitHub release. When the latest-release lookup fails or the selected skills manifest is missing, ranges tracking latest use `AgentSkillsFallbackVersion` from `internal/build/clicompat.go`. Pinned ranges use their embedded manifest's skills version as the fallback. Changing either fallback takes effect in the next CLI build. Released binaries keep the fallback they shipped with.
+
+This skills fallback only applies to raw-file delivery. Version checks omit update availability when latest is unknown, and plugin operations don't substitute the fallback version.
+
+Raw-skills updates and reinstalls reject automatic downgrades, including with `--force`: the fallback must be at least as new as the installed release. Existing installs without a comparable release require a successful latest-release fetch. Explicit `DATABRICKS_SKILLS_REF` overrides and compatibility pins can still select an older release.
+
+CLIs before `1.6.0` reject the whole remote manifest when any entry uses `"latest"`, even an entry they never resolve to. They use a valid cached manifest or their embedded manifest without returning a parse error. These CLIs stop receiving remote compatibility updates for both AppKit and skills.
+
+Latest tracking starts at the CLI `1.22.0` range. CLI `1.20.x` and `1.21.x` stay pinned.
+
 For example, with the manifest above:
-- CLI `0.285.0` → uses `0.280.0` entry (appkit `0.20.0`)
-- CLI `0.293.0` → uses `0.290.0` entry (appkit `0.24.0`)
-- CLI `0.300.0` → uses `0.296.0` entry (appkit `0.27.0`, highest versioned)
-- CLI `0.0.0-dev+abc` → uses `0.296.0` entry (dev builds use the highest versioned entry)
+- CLI `0.299.5` → uses `0.299.2` entry (appkit `0.24.0`)
+- CLI `1.19.0` → uses `1.0.0` entry (appkit `0.76.1`)
+- CLI `1.21.0` → uses `1.20.0` entry (appkit `0.84.0`, skills `0.2.30`)
+- CLI `1.22.0` → uses `1.22.0` entry (appkit `0.84.0`, latest skills with compiled fallback `0.2.30`)
+- CLI `0.0.0-dev+abc` → uses `1.22.0` entry (dev builds use the highest versioned entry)
 
 ## How the CLI resolves versions
 
@@ -41,12 +55,14 @@ Set `DATABRICKS_FORCE_EMBEDDED_COMPAT=true` to skip all tiers and use only the e
 
 ## When to update
 
-The goal is to **keep the manifest sparse** — only add entries at compatibility boundaries. After each AppKit or Agent Skills release:
+The goal is to **keep the manifest sparse** — only add entries at compatibility boundaries. Routine skills releases do not require a manifest update: keep `"skills": "latest"` on the newest range.
+
+When updating AppKit or recording a tested skills fallback:
 
 1. **Run evals** on the new AppKit version. If there is no regression, proceed.
 2. **Open a PR** to update `cli-compat.json`. The change depends on the type of release:
-   - **No breaking changes** (the new AppKit/skills version works with all existing CLI versions): update the existing highest versioned entry's appkit/skills values in-place. Do NOT add a new versioned key. All CLI versions in that range automatically pick up the new versions.
-   - **Breaking changes** (the new AppKit templates require specific `apps init` features, or the new skills version requires CLI commands that older CLIs lack): add a new entry keyed to the **minimum CLI version** that supports the new features. Older entries keep their previous appkit/skills values so older CLI binaries stay compatible.
+   - **No breaking changes**: update the highest entry's `appkit` in-place, keeping `"skills": "latest"`. To record a tested skills fallback, update `AgentSkillsFallbackVersion` in `internal/build/clicompat.go`. Do NOT add a new versioned key.
+   - **Breaking changes** (the new AppKit templates require specific `apps init` features, or the new skills version requires CLI commands that older CLIs lack): add a new entry keyed to the **minimum CLI version** that supports the new features. Set `"skills": "latest"` on the new entry and update the compiled skills fallback to a tested release. If skills compatibility changed, replace `"latest"` in the affected older ranges with the last compatible release. Preserve unrelated AppKit and skills compatibility boundaries.
 
 This process is manual for now but can be automated as part of the release workflow in the future. Use the `/bump-cli-compat` Claude Code skill to automate the update and PR creation.
 
@@ -58,7 +74,7 @@ The manifest is validated by Go tests in `libs/clicompat/`:
 go test ./libs/clicompat/... -run TestEmbeddedManifest -v
 ```
 
-This checks: valid JSON, at least one entry, valid semver keys, valid semver entry values, and ascending key order.
+This checks: valid JSON, at least one entry, valid semver keys and AppKit versions, concrete skills versions or `"latest"`, a concrete compiled skills fallback, and latest tracking on the newest CLI range.
 
 ## Pruning policy
 

internal/build/cli-compat.json

@@ -1,4 +1,5 @@
 {
+  "1.22.0": { "appkit": "0.84.0", "skills": "latest" },
   "1.20.0": { "appkit": "0.84.0", "skills": "0.2.30" },
   "1.0.0": { "appkit": "0.76.1", "skills": "0.2.10" },
   "0.299.2": { "appkit": "0.24.0", "skills": "0.1.5" }

internal/build/clicompat.go

@@ -2,6 +2,10 @@ package build
 
 import _ "embed"
 
+// AgentSkillsFallbackVersion is the tested release used when fetching the latest
+// skills release fails. Updating it requires a new CLI build.
+const AgentSkillsFallbackVersion = "0.2.30"
+
 // CLICompatManifestJSON is the cli-compat.json manifest embedded at compile time.
 // Used as the last-resort fallback when both remote fetch and local cache fail.
 //

libs/aitools/installer/dump.go

@@ -16,7 +16,7 @@ import (
 // the cli-compat version pin, --skills cherry-picks, and --experimental exactly
 // as the agent install path does, since it reuses resolveSkills.
 func DumpSkillsToPath(ctx context.Context, src ManifestSource, destDir string, opts InstallOptions) (int, error) {
-	ref, explicit, err := GetSkillsRef(ctx)
+	ref, explicit, err := getSkillsRef(ctx, true)
 	if err != nil {
 		return 0, err
 	}

libs/aitools/installer/dump_test.go

@@ -1,6 +1,7 @@
 package installer
 
 import (
+	"errors"
 	"io/fs"
 	"os"
 	"path/filepath"
@@ -50,3 +51,18 @@ func TestDumpSkillsToPathCherryPick(t *testing.T) {
 	_, err = os.Stat(filepath.Join(dest, "databricks-jobs"))
 	assert.ErrorIs(t, err, fs.ErrNotExist)
 }
+
+func TestDumpSkillsToPathFallsBackWhenLatestUnknown(t *testing.T) {
+	ctx, stderr := cmdio.NewTestContextWithStderr(t.Context())
+	setupFetchMock(t)
+	src, fallbackRef := setupSkillsFallback(t, errors.New("network down"))
+	dest := filepath.Join(t.TempDir(), "out")
+
+	n, err := DumpSkillsToPath(ctx, src, dest, InstallOptions{})
+	require.NoError(t, err)
+	assert.Equal(t, 2, n)
+	assert.Contains(t, stderr.String(), "Using skills version "+DisplaySkillsVersion(fallbackRef))
+	for _, name := range []string{"databricks-sql", "databricks-jobs"} {
+		assert.FileExists(t, filepath.Join(dest, name, "SKILL.md"))
+	}
+}

libs/aitools/installer/installer.go

@@ -92,35 +92,42 @@ var latestSkillsReleaseRefFn = fetchLatestSkillsReleaseRef
 const skillsLatestSentinel = clicompat.AgentSkillsLatest
 
 // GetSkillsRef returns the skills repo ref to use and whether it was explicitly
-// pinned (as opposed to tracking latest).
+// pinned (as opposed to tracking latest). An unresolved latest release is an error.
 //
 // By default we track the latest skills release tag, not the skills repo's main
-// branch. cli-compat.json is the remote control for this: it normally reports
-// "latest", but if a future change makes a skills release incompatible with
-// older CLIs, it can be edited remotely to pin those CLIs to the last compatible
-// version with no CLI release. DATABRICKS_SKILLS_REF overrides everything for
-// eval/reproducibility. (AppKit version resolution is unaffected; cli-compat
-// still manages it via its own field.)
+// branch. cli-compat.json is the remote control for this: its newest range
+// normally tracks latest, but if a future change makes a skills release
+// incompatible with older CLIs, it can be edited remotely to pin those CLIs to
+// the last compatible version with no CLI release. DATABRICKS_SKILLS_REF
+// overrides everything for eval/reproducibility. (AppKit version resolution is
+// unaffected; cli-compat still manages it via its own field.)
 func GetSkillsRef(ctx context.Context) (ref string, explicit bool, err error) {
+	return getSkillsRef(ctx, false)
+}
+
+func getSkillsRef(ctx context.Context, allowFallback bool) (ref string, explicit bool, err error) {
 	if ref := env.Get(ctx, "DATABRICKS_SKILLS_REF"); ref != "" {
 		return ref, true, nil
 	}
 	v, err := clicompat.ResolveAgentSkillsVersion(ctx)
 	if err != nil {
 		log.Debugf(ctx, "could not resolve skills compatibility, resolving latest release: %v", err)
-		return resolveLatestSkillsReleaseRef(ctx)
+		return resolveLatestSkillsReleaseRef(ctx, allowFallback)
 	}
 	if v == "" || v == skillsLatestSentinel {
-		return resolveLatestSkillsReleaseRef(ctx)
+		return resolveLatestSkillsReleaseRef(ctx, allowFallback)
 	}
 	return "v" + v, true, nil
 }
 
-func resolveLatestSkillsReleaseRef(ctx context.Context) (ref string, explicit bool, err error) {
+func resolveLatestSkillsReleaseRef(ctx context.Context, allowFallback bool) (ref string, explicit bool, err error) {
 	ref, err = latestSkillsReleaseRefFn(ctx)
 	if err == nil {
 		return ref, false, nil
 	}
+	if !allowFallback {
+		return "", false, fmt.Errorf("resolve latest skills release: %w", err)
+	}
 
 	embedded, embeddedErr := clicompat.ResolveEmbeddedAgentSkillsVersion()
 	if embeddedErr == nil && embedded != "" && embedded != skillsLatestSentinel {
@@ -267,25 +274,56 @@ func FetchSkillsManifestWithFallback(ctx context.Context, src ManifestSource, re
 	return manifest, ref, err
 }
 
-// InstallSkillsForAgents fetches the manifest and installs skills for the given agents.
-// This is the core installation function. Callers are responsible for agent detection,
-// prompting, and printing the "Installing..." header.
-func InstallSkillsForAgents(ctx context.Context, src ManifestSource, targetAgents []*agents.Agent, opts InstallOptions) error {
-	ref, explicit, err := GetSkillsRef(ctx)
-	if err != nil {
-		return err
+// installedSkillsRelease returns the newest recorded raw-skills release and
+// whether fallback is safe to consider. Branch refs cannot be ordered, so they
+// require a successful latest-release fetch instead of an embedded fallback.
+func installedSkillsRelease(state *InstallState) (ref string, allowFallback bool) {
+	if state == nil || len(state.Skills) == 0 {
+		return "", true
+	}
+	ref = semver.Canonical("v" + DisplaySkillsVersion(state.Release))
+	allowFallback = ref != ""
+	// Plugin installs can overwrite state.Release, and selective skill installs
+	// can leave files from a newer release. Keep their provenance in the check.
+	knownSkills := make(map[string]bool, len(state.Skills))
+	for path, record := range state.Files {
+		if record.Origin == "" {
+			continue
+		}
+		origin := semver.Canonical("v" + DisplaySkillsVersion(record.Origin))
+		if origin == "" {
+			allowFallback = false
+		} else {
+			name, _, _ := strings.Cut(path, "/")
+			knownSkills[name] = true
+		}
+		if semver.Compare(origin, ref) > 0 {
+			ref = origin
+		}
 	}
-	cmdio.LogProgress(ctx, "Using skills version "+DisplaySkillsVersion(ref))
-	cmdio.LogProgress(ctx, "Fetching skills manifest...")
-	manifest, ref, err := FetchSkillsManifestWithFallback(ctx, src, ref, !explicit)
-	if err != nil {
-		return err
+	// Without raw-file provenance, a mixed legacy/plugin install has no
+	// reliable raw-skills release to compare with the fallback.
+	if len(state.Plugins) > 0 {
+		for name := range state.Skills {
+			if !knownSkills[name] {
+				allowFallback = false
+			}
+		}
 	}
+	return ref, allowFallback
+}
 
-	if opts.IncludeExperimental && !manifestHasExperimental(manifest) {
-		log.Warnf(ctx, "--experimental was set but the manifest at %s exposes no experimental skills. Set DATABRICKS_SKILLS_REF to a release or ref that includes them.", ref)
+func checkSkillsDowngrade(installedRef, ref string, explicit bool) error {
+	if !explicit && semver.Compare(ref, installedRef) < 0 {
+		return fmt.Errorf("refusing to downgrade skills from %s to %s; retry when the latest release is available or set DATABRICKS_SKILLS_REF=%s to select this version explicitly", DisplaySkillsVersion(installedRef), DisplaySkillsVersion(ref), ref)
 	}
+	return nil
+}
 
+// InstallSkillsForAgents fetches the manifest and installs skills for the given agents.
+// This is the core installation function. Callers are responsible for agent detection,
+// prompting, and printing the "Installing..." header.
+func InstallSkillsForAgents(ctx context.Context, src ManifestSource, targetAgents []*agents.Agent, opts InstallOptions) error {
 	scope := opts.Scope
 	if scope == "" {
 		scope = ScopeGlobal
@@ -310,7 +348,7 @@ func InstallSkillsForAgents(ctx context.Context, src ManifestSource, targetAgent
 		}
 	}
 
-	// Load existing state for idempotency checks.
+	// Load existing state for idempotency and downgrade checks.
 	state, err := LoadState(baseDir)
 	if err != nil {
 		return fmt.Errorf("failed to load install state: %w", err)
@@ -323,10 +361,31 @@ func InstallSkillsForAgents(ctx context.Context, src ManifestSource, targetAgent
 	// full install first, which rebuilds complete state. Global scope only, since
 	// legacy installs only ever wrote to the global dir.
 	// hasLegacyInstall lives in update.go (shared with the update/uninstall paths).
-	if state == nil && scope == ScopeGlobal && len(opts.SpecificSkills) > 0 && hasLegacyInstall(ctx, baseDir) {
+	legacyInstall := state == nil && scope == ScopeGlobal && hasLegacyInstall(ctx, baseDir)
+	if legacyInstall && len(opts.SpecificSkills) > 0 {
 		return errors.New("legacy install detected without state tracking; run 'databricks aitools install' (without a skill name) first to rebuild state")
 	}
 
+	installedRef, allowFallback := installedSkillsRelease(state)
+	allowFallback = allowFallback && !legacyInstall
+	ref, explicit, err := getSkillsRef(ctx, allowFallback)
+	if err != nil {
+		return err
+	}
+	cmdio.LogProgress(ctx, "Using skills version "+DisplaySkillsVersion(ref))
+	cmdio.LogProgress(ctx, "Fetching skills manifest...")
+	manifest, ref, err := FetchSkillsManifestWithFallback(ctx, src, ref, allowFallback && !explicit)
+	if err != nil {
+		return err
+	}
+	if err := checkSkillsDowngrade(installedRef, ref, explicit); err != nil {
+		return err
+	}
+
+	if opts.IncludeExperimental && !manifestHasExperimental(manifest) {
+		log.Warnf(ctx, "--experimental was set but the manifest at %s exposes no experimental skills. Set DATABRICKS_SKILLS_REF to a release or ref that includes them.", ref)
+	}
+
 	// Filter skills based on options, experimental flag, and CLI version.
 	targetSkills, err := resolveSkills(ctx, manifest.Skills, opts)
 	if err != nil {