bundle/direct/dstate: read-only feature-flag gate for direct deployment state by shreyas-goenka · Pull Request #5660 · databricks/cli

shreyas-goenka · 2026-06-19T08:42:51Z

Why

Forward-compat scaffolding so this CLI reads a future "feature flags" deployment state correctly, without writing one yet. A later release can add a real feature and flip the baseline; until then this change bakes in the read behavior with minimal friction.

currentStateVersion (2) is what this CLI writes.
featureStateVersion (3) is the version a future CLI will write once it records Header.Features.
supportedStateVersion is the highest version this CLI can read. It equals featureStateVersion during this two-phase bump and tracks currentStateVersion otherwise.

What

On read (migrateState):

version 2 passes;
version 3 with an empty features list is accepted as-is — migrateState returns early, leaving the on-disk version at 3 (it is not flipped down to 2, so a subsequent deploy keeps writing version 3);
version 3 recording any feature is refused, naming the offending keys and pointing at the docs:

the deployment state requires features this CLI does not support: future_feature; upgrade to the latest CLI version and see https://docs.databricks.com/aws/en/dev-tools/bundles/state-features#state-features for more information

a version newer than supportedStateVersion is rejected as too new.

The version-3 special case is scaffolding, pinned by a forcing-function unit test (TestEmptyFeatureStateAcceptedWithoutFlippingVersion) so it is removed when the baseline is really bumped to 3. The feature_flags acceptance test deploys an empty-features v3 state and asserts state_version stays 3 on disk.

This pull request and its description were written by Isaac.

eng-dev-ecosystem-bot · 2026-06-19T09:10:31Z

Integration test report

Commit: 6f1292a

Run: 29319742708

	Env	🔄flaky	💚RECOVERED	🙈SKIP	✅pass	🙈skip	Time
💚	aws linux		4	4	230	1085	5:20
💚	aws windows		4	4	232	1083	7:31
💚	aws-ucws linux		4	4	316	1002	5:21
💚	aws-ucws windows		4	4	318	1000	7:21
💚	azure linux		4	4	230	1084	5:12
🔄	azure windows	2	2	4	232	1082	9:35
💚	azure-ucws linux		4	4	318	999	6:00
💚	azure-ucws windows		4	4	320	997	8:08
💚	gcp linux		4	4	229	1086	5:00
💚	gcp windows		4	4	231	1084	7:24

8 interesting tests: 4 SKIP, 2 RECOVERED, 2 flaky

	Test Name	aws linux	aws windows	aws-ucws linux	aws-ucws windows	azure linux	azure windows	azure-ucws linux	azure-ucws windows	gcp linux	gcp windows
💚	TestAccept	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R
🙈	TestAccept/bundle/invariant/no_drift	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/ssh/connection	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
💚	TestFetchRepositoryInfoAPI_FromRepo	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R
🔄	TestFetchRepositoryInfoAPI_FromRepo/root	💚R	💚R	💚R	💚R	💚R	🔄f	💚R	💚R	💚R	💚R
🔄	TestFetchRepositoryInfoAPI_FromRepo/subdir	💚R	💚R	💚R	💚R	💚R	🔄f	💚R	💚R	💚R	💚R

Top 10 slowest tests (at least 2 minutes):

duration	env	testname
7:02	azure-ucws windows	TestAccept
6:46	azure windows	TestAccept
6:19	aws windows	TestAccept
6:19	gcp windows	TestAccept
6:12	aws-ucws windows	TestAccept
2:57	gcp linux	TestAccept
2:50	azure linux	TestAccept
2:49	azure-ucws linux	TestAccept
2:46	aws linux	TestAccept
2:42	aws-ucws linux	TestAccept

shreyas-goenka · 2026-06-29T12:49:29Z

+	for name := range db.Features {
+		names = append(names, name)
+	}
+	slices.Sort(names)


makes tests with multiple features deterministic.

shreyas-goenka · 2026-06-29T13:41:22Z

+// RecordFeature marks the state as depending on the named feature, stamping the
+// minimum CLI version required to read it (from featureMinCLIVersion). It must be
+// called before the WAL is started (UpgradeToWrite) so the change is persisted.
+func (db *DeploymentState) RecordFeature(name string) {


Get rid of this function - make this decision at open.

shreyas-goenka · 2026-06-29T13:41:50Z

+{
+ "state_version": 3,
+ "features": {
+  "dummy": "1.2.0"


talk to Julia, if aligned get rid of versions from state and consider hosting every feature on our docs page.

denik · 2026-07-13T14:31:55Z

-	if db.StateVersion > currentStateVersion {
-		return fmt.Errorf("state version %d is newer than supported version %d; upgrade the CLI", db.StateVersion, currentStateVersion)
+	if db.StateVersion > featureStateVersion {
+		return fmt.Errorf("state version %d is newer than supported version %d; upgrade the CLI", db.StateVersion, featureStateVersion)


should we allocate constant supportedStateVersion? Currently it's 3, the same as featureStateVersion but in the future the constant will remain but it'll be equal to currentStateVersion most likely unless we execute another 2-phase bump like this one.

Agree with this.

We would just cleanup featureStateVersion constant in the future and be left again with just the current version. I'm not sure I followed exactly what the recommendation was here.

I added supportedStateVersion in any case.

denik · 2026-07-13T14:35:50Z

+
+Exit code: 1
+
+=== a version-3 state with an empty features map is accepted (treated as version 2)


We should have another test with non-empty deploy to ensure that state is not bumped.

shreyas-goenka · 2026-07-13T16:21:46Z

+	// CLI lacks, so refuse it and tell the user to upgrade.
+	if db.StateVersion == featureStateVersion {
+		if len(db.Features) == 0 {
+			db.StateVersion = currentStateVersion


This will flip flop the version between 2/3. That's fine though because that only happens when there are no features. If that happens then a newer client will upgrade it back to 3.

We should not flip the version on disk; can we keep at 3? just skip the migrations below.

Agree, we can return here. No other checks needed.

pietern · 2026-07-15T12:09:12Z

-	if db.StateVersion > currentStateVersion {
-		return fmt.Errorf("state version %d is newer than supported version %d; upgrade the CLI", db.StateVersion, currentStateVersion)
+	if db.StateVersion > featureStateVersion {
+		return fmt.Errorf("state version %d is newer than supported version %d; upgrade the CLI", db.StateVersion, featureStateVersion)


Agree with this.

pietern · 2026-07-15T12:11:24Z

+	// CLI lacks, so refuse it and tell the user to upgrade.
+	if db.StateVersion == featureStateVersion {
+		if len(db.Features) == 0 {
+			db.StateVersion = currentStateVersion


Agree, we can return here. No other checks needed.

…nt state Add forward-compat scaffolding so this CLI reads a future "feature flags" deployment state correctly. featureStateVersion (3) is the version a future CLI will write once it records Header.Features; this CLI writes no features and always stamps currentStateVersion (2). On read: currentStateVersion passes; featureStateVersion passes iff its features list is empty (normalized to version 2); a featureStateVersion state recording any feature is refused with the offending feature keys and a link to the docs, telling the user to upgrade. The v3-empty == v2 equivalence is special-cased to version 3 only and pinned by a forcing-function unit test so it is removed when the baseline is really bumped to 3. Co-authored-by: Isaac

Address review feedback (Denis, Pieter): the "state too new" check compared against featureStateVersion, conflating "the version a future CLI writes" with "the highest version this CLI can read". Introduce supportedStateVersion for the read ceiling. It equals featureStateVersion during this two-phase bump and tracks currentStateVersion the rest of the time; the check now reads against it. Co-authored-by: Isaac

Address review feedback (Denis, Pieter): a featureStateVersion state with no features is accepted by returning early from migrateState without running the downstream migrations, leaving the on-disk version at featureStateVersion. The previous code flipped it down to currentStateVersion, so a deploy rewrote a v3 state as v2. Acceptance test now deploys and asserts state_version stays 3. Co-authored-by: Isaac

shreyas-goenka mentioned this pull request Jun 19, 2026

bundle/direct: add a feature-flag list to the direct deployment state #5403

Closed

shreyas-goenka temporarily deployed to test-trigger-is June 19, 2026 08:43 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/direct-state-feature-flags branch 2 times, most recently from 663ad08 to a1b0613 Compare June 29, 2026 09:30

shreyas-goenka temporarily deployed to test-trigger-is June 29, 2026 09:30 — with GitHub Actions Inactive

shreyas-goenka commented Jun 29, 2026

View reviewed changes

shreyas-goenka force-pushed the shreyas-goenka/direct-state-feature-flags branch from a1b0613 to a550105 Compare June 29, 2026 12:55

shreyas-goenka temporarily deployed to test-trigger-is June 29, 2026 12:56 — with GitHub Actions Inactive

shreyas-goenka marked this pull request as ready for review June 29, 2026 12:57

shreyas-goenka requested review from andrewnester, denik and pietern June 29, 2026 12:58

shreyas-goenka commented Jun 29, 2026

View reviewed changes

shreyas-goenka force-pushed the shreyas-goenka/direct-state-feature-flags branch from a550105 to 2d62964 Compare July 6, 2026 23:04

shreyas-goenka temporarily deployed to test-trigger-is July 6, 2026 23:04 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/direct-state-feature-flags branch from 2d62964 to e559fca Compare July 7, 2026 00:26

shreyas-goenka temporarily deployed to test-trigger-is July 7, 2026 00:26 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/direct-state-feature-flags branch from e559fca to c62c873 Compare July 7, 2026 08:59

shreyas-goenka temporarily deployed to test-trigger-is July 7, 2026 08:59 — with GitHub Actions Inactive

shreyas-goenka force-pushed the shreyas-goenka/direct-state-feature-flags branch from c62c873 to 1a7c392 Compare July 7, 2026 09:01

shreyas-goenka temporarily deployed to test-trigger-is July 7, 2026 09:01 — with GitHub Actions Inactive

shreyas-goenka temporarily deployed to test-trigger-is July 8, 2026 15:30 — with GitHub Actions Inactive

denik reviewed Jul 13, 2026

View reviewed changes

shreyas-goenka temporarily deployed to test-trigger-is July 13, 2026 14:36 — with GitHub Actions Inactive

shreyas-goenka commented Jul 13, 2026

View reviewed changes

Comment thread bundle/direct/dstate/state.go

shreyas-goenka temporarily deployed to test-trigger-is July 13, 2026 16:25 — with GitHub Actions Inactive

shreyas-goenka requested a review from denik July 13, 2026 16:28

shreyas-goenka force-pushed the shreyas-goenka/direct-state-feature-flags branch from c724a74 to 6f1292a Compare July 14, 2026 08:52

shreyas-goenka changed the title ~~bundle/direct: add a feature-flag list to the direct deployment state~~ bundle/direct/dstate: read-only feature-flag gate for direct deployment state Jul 14, 2026

shreyas-goenka temporarily deployed to test-trigger-is July 14, 2026 08:53 — with GitHub Actions Inactive

pietern reviewed Jul 15, 2026

View reviewed changes

shreyas-goenka temporarily deployed to test-trigger-is July 15, 2026 12:22 — with GitHub Actions Inactive

shreyas-goenka added 3 commits July 15, 2026 14:38

shreyas-goenka force-pushed the shreyas-goenka/direct-state-feature-flags branch from 9efcb33 to be41023 Compare July 15, 2026 12:39

shreyas-goenka temporarily deployed to test-trigger-is July 15, 2026 12:40 — with GitHub Actions Inactive

pietern approved these changes Jul 15, 2026

View reviewed changes

minimize diff

2625500

shreyas-goenka temporarily deployed to test-trigger-is July 15, 2026 13:17 — with GitHub Actions Inactive

shreyas-goenka added this pull request to the merge queue Jul 15, 2026

Merged via the queue into main with commit 93be415 Jul 15, 2026
23 checks passed

shreyas-goenka deleted the shreyas-goenka/direct-state-feature-flags branch July 15, 2026 14:20


		Exit code: 1

		=== a version-3 state with an empty features map is accepted (treated as version 2)

Uh oh!

Conversation

shreyas-goenka commented Jun 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What

Uh oh!

eng-dev-ecosystem-bot commented Jun 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Integration test report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

shreyas-goenka commented Jun 19, 2026 •

edited

Loading

eng-dev-ecosystem-bot commented Jun 19, 2026 •

edited

Loading