feat: add optional pipeline-created snapshot processing#1259
Draft
arewm wants to merge 2 commits intokonflux-ci:mainfrom
Draft
feat: add optional pipeline-created snapshot processing#1259arewm wants to merge 2 commits intokonflux-ci:mainfrom
arewm wants to merge 2 commits intokonflux-ci:mainfrom
Conversation
d0a1c2f to
ea47199
Compare
Implement support for build pipelines to create Snapshots directly rather than having Integration Service create them. This enables handling of complex use cases like bundle images, multiple artifacts, and single component mode abstraction while maintaining full backward compatibility. Key changes: - Add GetSnapshotName() function to parse SNAPSHOT results from PipelineRuns - Enhance buildpipeline adapter to process existing snapshots when SNAPSHOT result present - Add pending annotation handling to skip ITS processing until snapshot is ready - Include comprehensive test coverage for both traditional and new workflows - Maintain 100% backward compatibility with existing snapshot creation workflow The feature is opt-in based on presence of the SNAPSHOT result in the PipelineRun. When absent, the traditional snapshot creation workflow continues unchanged. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: arewm <arewm@users.noreply.github.com> rh-pre-commit.version: 2.3.2 rh-pre-commit.check-secrets: ENABLED
ea47199 to
9b78921
Compare
Based on offline review feedback 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: arewm <arewm@users.noreply.github.com> rh-pre-commit.version: 2.3.2 rh-pre-commit.check-secrets: ENABLED
|
@arewm: The following test has Failed, say /retest to rerun failed tests.
Inspecting Test ArtifactsTo inspect your test artifacts, follow these steps:
mkdir -p oras-artifacts
cd oras-artifacts
oras pull quay.io/konflux-test-storage/konflux-team/integration-service:konflux-e2e-jszf6Test results analysis🚨 Failed to provision a cluster, see the log for more details: Click to view logs�[37mDEBU�[0m running 'mapt aws kind create'
�[37mDEBU�[0m context initialized for mapt97d04a9d
�[37mDEBU�[0m checking stack spotOption-kind-konflux-e2e-jszf6
�[37mDEBU�[0m managing stack spotOption-kind-konflux-e2e-jszf6
�[36mINFO�[0m Updating (spotOption-kind-konflux-e2e-jszf6):
�[36mINFO�[0m
�[36mINFO�[0m + pulumi:pulumi:Stack kind-konflux-e2e-jszf6-spotOption-kind-konflux-e2e-jszf6 creating (0s)
�[37mDEBU�[0m Based on prices for instance types [c5a.8xlarge c6a.8xlarge c7i-flex.8xlarge c7i.8xlarge c6i.8xlarge c5ad.8xlarge c7a.8xlarge c6id.8xlarge] is az us-east-2b, current price is 0.63 with a score of 9
�[37mDEBU�[0m Spot data: {[c5a.8xlarge c6a.8xlarge c7i-flex.8xlarge c7i.8xlarge c6i.8xlarge c5ad.8xlarge c7a.8xlarge c6id.8xlarge] 0.75828 us-east-2 us-east-2b 0}
�[36mINFO�[0m @ updating......
�[36mINFO�[0m + rh:qe:aws:bso bso-bso creating (0s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + rh:qe:aws:bso bso-bso created
�[36mINFO�[0m + pulumi:pulumi:Stack kind-konflux-e2e-jszf6-spotOption-kind-konflux-e2e-jszf6 created (4s)
�[36mINFO�[0m Outputs:
�[36mINFO�[0m az : "us-east-2b"
�[36mINFO�[0m instancetypes: [
�[36mINFO�[0m [0]: "c5a.8xlarge"
�[36mINFO�[0m [1]: "c6a.8xlarge"
�[36mINFO�[0m [2]: "c7i-flex.8xlarge"
�[36mINFO�[0m [3]: "c7i.8xlarge"
�[36mINFO�[0m [4]: "c6i.8xlarge"
�[36mINFO�[0m [5]: "c5ad.8xlarge"
�[36mINFO�[0m [6]: "c7a.8xlarge"
�[36mINFO�[0m [7]: "c6id.8xlarge"
�[36mINFO�[0m ]
�[36mINFO�[0m max : 0.75828
�[36mINFO�[0m region : "us-east-2"
�[36mINFO�[0m score : 0
�[36mINFO�[0m
�[36mINFO�[0m Resources:
�[36mINFO�[0m + 2 created
�[36mINFO�[0m
�[36mINFO�[0m Duration: 5s
�[36mINFO�[0m
�[37mDEBU�[0m managing stack stackKind-kind-konflux-e2e-jszf6
�[36mINFO�[0m Updating (stackKind-kind-konflux-e2e-jszf6):
�[36mINFO�[0m
�[36mINFO�[0m + pulumi:pulumi:Stack kind-konflux-e2e-jszf6-stackKind-kind-konflux-e2e-jszf6 creating (0s)
�[36mINFO�[0m @ updating...........
�[36mINFO�[0m + aws:ec2:Vpc vpc-main-akd-net creating (0s)
�[36mINFO�[0m + aws:ec2:Eip main-akd-lbeip creating (0s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:ec2:Eip main-akd-lbeip created (1s)
�[36mINFO�[0m + tls:index:PrivateKey main-akd-pk creating (0s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:ec2:Vpc vpc-main-akd-net created (1s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:lb:TargetGroup main-akd-tg-8888 creating (0s)
�[36mINFO�[0m + aws:ec2:Subnet subnet-publicmain-akd-net0 creating (0s)
�[36mINFO�[0m + aws:lb:TargetGroup main-akd-tg-6443 creating (0s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:lb:TargetGroup main-akd-tg-8888 created (1s)
�[36mINFO�[0m + aws:ec2:Subnet subnet-publicmain-akd-net0 created (1s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:lb:TargetGroup main-akd-tg-22 creating (0s)
�[36mINFO�[0m + aws:lb:TargetGroup main-akd-tg-6443 created (2s)
�[36mINFO�[0m + tls:index:PrivateKey main-akd-pk created (3s)
�[36mINFO�[0m + aws:ec2:SecurityGroup default-main-akd-net-main-akd-net creating (0s)
�[37mDEBU�[0m Requesting a spot instance of types: c5a.8xlarge, c6a.8xlarge, c7i-flex.8xlarge, c7i.8xlarge, c6i.8xlarge, c5ad.8xlarge, c7a.8xlarge, c6id.8xlarge at us-east-2b paying: 0.758280
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:lb:TargetGroup main-akd-tg-22 created (1s)
�[36mINFO�[0m + aws:ec2:InternetGateway igw-main-akd-net creating (0s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:lb:TargetGroup main-akd-tg-9443 creating (0s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:ec2:SecurityGroup main-akd-sg creating (0s)
�[36mINFO�[0m + aws:ec2:InternetGateway igw-main-akd-net created (2s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:lb:TargetGroup main-akd-tg-9443 created (2s)
�[36mINFO�[0m + aws:lb:TargetGroup main-akd-tg-8180 creating (0s)
�[36mINFO�[0m + aws:ec2:SecurityGroup default-main-akd-net-main-akd-net created (4s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:lb:LoadBalancer main-akd-lb creating (0s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:ec2:KeyPair main-akd-pk creating (0s)
�[36mINFO�[0m + aws:lb:TargetGroup main-akd-tg-8180 created (1s)
�[36mINFO�[0m + aws:ec2:SecurityGroup main-akd-sg created (3s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:ec2:KeyPair main-akd-pk created (1s)
�[36mINFO�[0m + aws:ec2:RouteTable routeTable-publicmain-akd-net0 creating (0s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:ec2:LaunchTemplate main-akd-lt creating (0s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:ec2:RouteTable routeTable-publicmain-akd-net0 created (2s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:ec2:RouteTableAssociation routeTableAssociation-publicmain-akd-net0 creating (0s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:ec2:RouteTableAssociation routeTableAssociation-publicmain-akd-net0 created (0.88s)
�[36mINFO�[0m @ updating.......
�[36mINFO�[0m + aws:ec2:LaunchTemplate main-akd-lt created (6s)
�[36mINFO�[0m + aws:autoscaling:Group main-akd-asg creating (0s)
�[36mINFO�[0m @ updating....................
�[36mINFO�[0m + aws:autoscaling:Group main-akd-asg created (16s)
�[36mINFO�[0m @ updating......................................................................................................................
�[36mINFO�[0m + aws:lb:LoadBalancer main-akd-lb created (142s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:lb:Listener main-akd-listener-22 creating (0s)
�[36mINFO�[0m + aws:lb:Listener main-akd-listener-6443 creating (0s)
�[36mINFO�[0m + aws:lb:Listener main-akd-listener-8888 creating (0s)
�[36mINFO�[0m + aws:lb:Listener main-akd-listener-9443 creating (0s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:lb:Listener main-akd-listener-22 created (0.81s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:lb:Listener main-akd-listener-6443 created (1s)
�[36mINFO�[0m + aws:lb:Listener main-akd-listener-8888 created (1s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + aws:lb:Listener main-akd-listener-9443 created (2s)
�[36mINFO�[0m + aws:lb:Listener main-akd-listener-8180 creating (0s)
�[36mINFO�[0m @ updating......
�[36mINFO�[0m + aws:lb:Listener main-akd-listener-8180 created (2s)
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + command:remote:Command main-kind-readiness-akd-cmd creating (0s)
�[36mINFO�[0m @ updating..................
�[36mINFO�[0m + command:remote:Command main-kind-readiness-akd-cmd creating (15s) Dial 1/inf failed: retrying
�[36mINFO�[0m @ updating..................
�[36mINFO�[0m + command:remote:Command main-kind-readiness-akd-cmd creating (30s) Dial 2/inf failed: retrying
�[36mINFO�[0m @ updating...................
�[36mINFO�[0m + command:remote:Command main-kind-readiness-akd-cmd creating (45s) Dial 3/inf failed: retrying
�[36mINFO�[0m @ updating..................
�[36mINFO�[0m + command:remote:Command main-kind-readiness-akd-cmd creating (61s) Dial 4/inf failed: retrying
�[36mINFO�[0m @ updating..................
�[36mINFO�[0m + command:remote:Command main-kind-readiness-akd-cmd creating (76s) Dial 5/inf failed: retrying
�[36mINFO�[0m @ updating........
�[36mINFO�[0m + command:remote:Command main-kind-readiness-akd-cmd creating (80s)
�[36mINFO�[0m + command:remote:Command main-kind-readiness-akd-cmd creating (80s) status: done
�[36mINFO�[0m + command:remote:Command main-kind-readiness-akd-cmd creating (80s) extended_status: done
�[36mINFO�[0m + command:remote:Command main-kind-readiness-akd-cmd creating (80s) boot_status_code: enabled-by-generator
�[36mINFO�[0m + command:remote:Command main-kind-readiness-akd-cmd creating (80s) last_update: Thu, 01 Jan 1970 00:00:55 +0000
�[36mINFO�[0m + command:remote:Command main-kind-readiness-akd-cmd creating (80s) detail: DataSourceEc2Local
�[36mINFO�[0m + command:remote:Command main-kind-readiness-akd-cmd creating (80s) errors: []
�[36mINFO�[0m + command:remote:Command main-kind-readiness-akd-cmd creating (80s) recoverable_errors: {}
�[36mINFO�[0m + command:remote:Command main-kind-readiness-akd-cmd created (80s) recoverable_errors: {}
�[36mINFO�[0m @ updating....
�[36mINFO�[0m + command:remote:Command main... the content is too long - please download the artifact to see the full content
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implement support for build pipelines to create Snapshots directly rather than having Integration Service create them. This enables handling of complex use cases like bundle images, multiple artifacts, and single component mode abstraction while maintaining full backward compatibility.
Key changes:
The feature is opt-in based on presence of SNAPSHOT result in PipelineRun. When absent, the traditional snapshot creation workflow continues unchanged.
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com
Signed-off-by: arewm arewm@users.noreply.github.com
rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
Maintainers will complete the following section