🌱 Add e2e test for operator workload persistence after catalog deletion by camilamacedo86 · Pull Request #3737 · operator-framework/operator-lifecycle-manager

camilamacedo86 · 2026-01-13T14:35:20Z

This PR adds a comprehensive e2e test to verify that installed operators continue functioning normally after their source catalog is deleted. This ensures catalogs are only needed for installation and upgrades, not for runtime operation.

Complete Test Flow

Phase 1: Setup & Installation (WITH Catalog)

Create CRD and CSV with permissions (ServiceAccount, Role, RoleBinding)
Create catalog source
Create subscription to install operator from catalog
Verify CSV reaches Succeeded state
Verify all OLM-managed resources exist:
- Deployment (with correct replicas)
- ServiceAccount
- Role
- RoleBinding
Record UIDs of all resources

Phase 2: Catalog Deletion

Delete catalog source
Verify catalog source and its pod are completely removed

Phase 3: Passive Health Checks (WITHOUT Catalog)

Consistently verify for 3 minutes:
- CSV remains in Succeeded phase
- Deployment stays healthy with same UID (not recreated)
- Replica counts remain correct

Phase 4: Active OLM Management Tests (WITHOUT Catalog)

Config Management Test:
- Update subscription to add environment variable
- Verify OLM propagates it to deployment
- Proves: OLM actively manages deployment based on subscription config
Reconciliation Test:
- Delete all OLM-managed resources:
- Deployment
  - Wait for deletion confirmation
  - Verify OLM recreates ALL of them with new UIDs
  - Verify recreated deployment still has subscription config (env var)
  - Proves: OLM actively reconciles CSV state and recreates the deployment only

Phase 5: Final State Validation

Verify subscription still tracks the installed CSV
Verify CRD still exists and is functional

What This Proves

The test validates that after catalog deletion, OLM still:

✅ Keeps CSV in healthy state
✅ Manages deployment configuration via subscription
✅ Reconciles and recreates deleted OLM-managed resources (Deployment ONLY)
✅ Maintains subscription tracking
✅ Preserves all installed CRDs

PS.: ServiceAccount, Role, RoleBinding has not been re-created

anik120

Curious to know, what motivated this test?

camilamacedo86 · 2026-01-16T14:55:01Z

Curious to know, what motivated this test?

We will remove the marketplace index.
The motivation is to ensure that if any catalog is removed, everything continues to work correctly (except for upgrades that explicitly require that catalog).

Additionally, we should ensure that the installed solution works even if we temporarily cannot reach the catalog—for example, if the registry is unavailable for any reason.

test/e2e/catalog_e2e_test.go

ankitathomas

Looks good!

ankitathomas · 2026-01-19T14:45:24Z

test/e2e/catalog_e2e_test.go

+			return nil
+		}, pollDuration, pollInterval).Should(Succeed())
+
+		By("Verify CSV remains in succeeded state after catalog deletion")


Should we also check Subscription behavior here for missing catalogsource?

That is a great idea 🎉
Adding that thank you

perdasilva · 2026-01-20T08:19:35Z

test/e2e/catalog_e2e_test.go

+
+		defer func() {
+			Eventually(func() error {
+				return ctx.Ctx().KubeClient().ApiextensionsInterface().ApiextensionsV1().CustomResourceDefinitions().Delete(context.Background(), crd.GetName(), metav1.DeleteOptions{})


Why not use client.IgnoreNotFound here?

perdasilva · 2026-01-20T08:21:32Z

/approve

openshift-ci · 2026-01-20T08:21:39Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: perdasilva

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [perdasilva]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

tmshort · 2026-01-20T16:09:41Z

Looks good to me, but might want to address Per's comment.

Assisted-by: Cursor/CLAUDE

tmshort · 2026-01-20T19:57:39Z

/lgtm

openshift-ci bot requested review from dtfranz and grokspawn January 13, 2026 14:35

camilamacedo86 changed the title ~~🌱 Add e2e test for operator workload persistence after catalog deletion~~ WIP 🌱 Add e2e test for operator workload persistence after catalog deletion Jan 13, 2026

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 13, 2026

camilamacedo86 force-pushed the add-e2e-test-res branch 3 times, most recently from 221472c to defba84 Compare January 13, 2026 19:14

camilamacedo86 changed the title ~~WIP 🌱 Add e2e test for operator workload persistence after catalog deletion~~ 🌱 Add e2e test for operator workload persistence after catalog deletion Jan 13, 2026

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 13, 2026

camilamacedo86 requested review from anik120, perdasilva and tmshort January 13, 2026 21:48

anik120 reviewed Jan 16, 2026

View reviewed changes

perdasilva reviewed Jan 19, 2026

View reviewed changes

test/e2e/catalog_e2e_test.go Outdated Show resolved Hide resolved

camilamacedo86 force-pushed the add-e2e-test-res branch 3 times, most recently from 9bf9eaf to d70e585 Compare January 19, 2026 14:24

ankitathomas previously approved these changes Jan 19, 2026

View reviewed changes

openshift-ci bot assigned ankitathomas Jan 19, 2026

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jan 19, 2026

camilamacedo86 dismissed ankitathomas’s stale review via e71ddf5 January 19, 2026 14:53

camilamacedo86 force-pushed the add-e2e-test-res branch from d70e585 to e71ddf5 Compare January 19, 2026 14:53

openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Jan 19, 2026

camilamacedo86 force-pushed the add-e2e-test-res branch 3 times, most recently from a5e3b9a to f1a4927 Compare January 19, 2026 17:56

camilamacedo86 requested review from anik120, ankitathomas and perdasilva January 19, 2026 18:49

perdasilva reviewed Jan 20, 2026

View reviewed changes

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 20, 2026

Add e2e test for operator workload persistence after catalog deletion

449588d

Assisted-by: Cursor/CLAUDE

camilamacedo86 force-pushed the add-e2e-test-res branch from f1a4927 to 449588d Compare January 20, 2026 17:10

camilamacedo86 requested a review from perdasilva January 20, 2026 19:18

openshift-ci bot assigned tmshort Jan 20, 2026

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jan 20, 2026

openshift-merge-bot bot merged commit afb9e80 into operator-framework:master Jan 20, 2026
19 of 21 checks passed

camilamacedo86 deleted the add-e2e-test-res branch January 20, 2026 20:18

tmshort mentioned this pull request Jan 22, 2026

Mark 'operator workload continues running after catalog source is deleted' test as flaky #3746

Closed

camilamacedo86 mentioned this pull request Jan 22, 2026

🌱 (e2e): Enhance e2e test to check workiload resilience when catalogs goes away #3747

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

🌱 Add e2e test for operator workload persistence after catalog deletion#3737

🌱 Add e2e test for operator workload persistence after catalog deletion#3737
openshift-merge-bot[bot] merged 1 commit intooperator-framework:masterfrom
camilamacedo86:add-e2e-test-res

camilamacedo86 commented Jan 13, 2026 •

edited

Loading

Uh oh!

anik120 left a comment

Uh oh!

camilamacedo86 commented Jan 16, 2026 •

edited

Loading

Uh oh!

Uh oh!

ankitathomas left a comment

Uh oh!

ankitathomas Jan 19, 2026

Uh oh!

camilamacedo86 Jan 19, 2026

Uh oh!

perdasilva Jan 20, 2026

Uh oh!

camilamacedo86 Jan 20, 2026

Uh oh!

perdasilva commented Jan 20, 2026

Uh oh!

openshift-ci bot commented Jan 20, 2026

Uh oh!

tmshort commented Jan 20, 2026

Uh oh!

tmshort commented Jan 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Comments

Conversation

camilamacedo86 commented Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

anik120 left a comment

Choose a reason for hiding this comment

Uh oh!

camilamacedo86 commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

ankitathomas left a comment

Choose a reason for hiding this comment

Uh oh!

ankitathomas Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

camilamacedo86 Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

perdasilva Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

camilamacedo86 Jan 20, 2026

Choose a reason for hiding this comment

Uh oh!

perdasilva commented Jan 20, 2026

Uh oh!

openshift-ci bot commented Jan 20, 2026

Uh oh!

tmshort commented Jan 20, 2026

Uh oh!

tmshort commented Jan 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

camilamacedo86 commented Jan 13, 2026 •

edited

Loading

camilamacedo86 commented Jan 16, 2026 •

edited

Loading