Skip to content

fix(webhooks): eliminate redundant DB queries from webhook execution path#3523

Merged
waleedlatif1 merged 4 commits intostagingfrom
fix/dequeue
Mar 11, 2026
Merged

fix(webhooks): eliminate redundant DB queries from webhook execution path#3523
waleedlatif1 merged 4 commits intostagingfrom
fix/dequeue

Conversation

@waleedlatif1
Copy link
Collaborator

Summary

  • Eliminate ~10-12 redundant DB queries from webhook API route + background job
  • Parallelize auth verification with preprocessing in API route via Promise.all
  • Pass pre-fetched workflowRecord and actorUserId through to avoid re-resolving
  • Replace manual init in webhook-execution.ts with preprocessExecution (matching schedule/workflow execution pattern)
  • Defer credential account resolution from hot path to background job
  • Parallelize loadDeployedWorkflowState, webhook query, and credential resolution in background job
  • Add workspaceId to job payload to skip internal lookup in loadDeployedWorkflowState

Type of Change

  • Bug fix

Testing

Tested manually

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel
Copy link

vercel bot commented Mar 11, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Mar 11, 2026 9:51pm

Request Review

@cursor
Copy link

cursor bot commented Mar 11, 2026

PR Summary

Medium Risk
Refactors the webhook trigger path and background execution setup, including billing actor propagation and timeout/workflow record resolution; mistakes could cause webhook executions to be rejected or attributed to the wrong user/workspace. Behavior should be mostly equivalent, but it touches core webhook ingestion and execution orchestration.

Overview
Speeds up the webhook trigger hot path by moving the reachability test ahead of auth, simplifying preprocessing handling, and threading actorUserId from checkWebhookPreprocessing into queueWebhookExecution instead of re-resolving billing in the queue step.

Unifies the background webhook-execution job initialization around preprocessExecution, passes workspaceId through the job payload, parallelizes loading deployed workflow state + webhook record + credential user resolution, and defers/relocates credential account lookup out of the API route. Tests were updated to reflect the new checkWebhookPreprocessing return shape ({ error, actorUserId }).

Written by Cursor Bugbot for commit 4b4e71c. Configure here.

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Mar 11, 2026

Greptile Summary

This PR eliminates ~10–12 redundant DB queries from the webhook trigger hot path by threading a pre-fetched workflowRecord and actorUserId from preprocessExecution through to queueWebhookExecution, and by parallelising loadDeployedWorkflowState, the webhook DB lookup, and credential resolution in the background job via Promise.all.

Key changes:

  • checkWebhookPreprocessing now returns { error, actorUserId } instead of NextResponse | null, allowing the billing actor resolved at preprocessing time to be reused downstream without a second DB round-trip.
  • queueWebhookExecution in processor.ts drops its own credential resolution and workspaceId guard, relying on preprocessExecution to have already validated these.
  • webhook-execution.ts replaces manual getHighestPrioritySubscription + getWorkflowById calls with preprocessExecution, aligning it with the schedule/workflow execution pattern.
  • Behavioral change to flag: handleProviderReachabilityTest was moved before verifyProviderAuth in route.ts. This means Grain reachability probes (empty-body POSTs) now succeed without credentials, creating a minor path-enumeration surface. The previous ordering required auth first.
  • credentialAccountUserId is no longer populated in the queued job payload; the field still exists in the type and is guarded in the background job, but the else branch of the ternary is now dead code.

Confidence Score: 3/5

  • Mostly safe, but the auth-ordering change in route.ts bypasses auth for Grain reachability probes and should be explicitly justified before merging.
  • The DB-query optimisations are sound and the preprocessExecution unification is a clean refactor. The main risk is the reachability-test-before-auth reordering, which is a behavioural change that was not called out in the PR description and introduces a path-enumeration surface. All other changes (dead-code ternary, workspaceId guard removal) are minor.
  • apps/sim/app/api/webhooks/trigger/[path]/route.ts — auth/reachability ordering change needs explicit justification.

Important Files Changed

Filename Overview
apps/sim/app/api/webhooks/trigger/[path]/route.ts Auth ordering changed: reachability test now runs before verifyProviderAuth, which bypasses auth for Grain empty-body probes. Also removes the try/catch wrapper around preprocessing and simplifies error handling using the new WebhookPreprocessingResult type.
apps/sim/background/webhook-execution.ts Background job refactored to use preprocessExecution (matching schedule/workflow patterns), parallel Promise.all for workflow state + webhook record + credential resolution, and workspaceId from payload. Dead-code branch in credential resolution condition.
apps/sim/lib/webhooks/processor.ts checkWebhookPreprocessing return type changed from NextResponse
apps/sim/app/api/webhooks/trigger/[path]/route.test.ts Test mock updated to match new WebhookPreprocessingResult shape ({ error: null, actorUserId }) — minimal, targeted change.

Sequence Diagram

sequenceDiagram
    participant C as Webhook Caller
    participant R as route.ts (POST)
    participant P as processor.ts
    participant BG as webhook-execution.ts (BG job)
    participant DB as Database

    C->>R: POST /webhooks/trigger/:path
    R->>P: findAllWebhooksForPath()
    P->>DB: SELECT webhook JOIN workflow
    DB-->>P: webhooks[]
    P-->>R: foundWebhook, foundWorkflow

    loop For each webhook
        R->>P: handleProviderReachabilityTest() ← NEW: before auth
        alt Grain empty-body probe
            P-->>R: 200 OK (no auth needed)
            R-->>C: 200 OK
        end

        R->>P: verifyProviderAuth()
        alt Auth fails
            P-->>R: 401
            R-->>C: 401
        end

        R->>P: checkWebhookPreprocessing()
        Note over P: preprocessExecution()<br/>resolves actorUserId,<br/>subscription, timeout<br/>(workflowRecord pre-fetched)
        P-->>R: { error: null, actorUserId }

        R->>P: queueWebhookExecution(actorUserId)
        Note over P: No credential resolution<br/>No workspaceId guard<br/>(handled by preprocess)
        P->>DB: Enqueue BG job (with workspaceId)
        P-->>R: 200 Queued
    end

    R-->>C: 200 OK

    BG->>BG: preprocessExecution()<br/>(checkRateLimit:false, checkDeployment:false)
    par Parallelised in BG job
        BG->>DB: loadDeployedWorkflowState(workspaceId)
        BG->>DB: SELECT webhook record
        BG->>DB: resolveCredentialAccountUserId (if needed)
    end
    BG->>BG: executeWorkflowCore()
Loading

Last reviewed commit: 4b4e71c

…ion warning

- Revert parallel auth+preprocessing to sequential auth→preprocessing
  to prevent rate-limit exhaustion via unauthenticated requests
- Add warning log when credential account resolution fails in background job
@waleedlatif1
Copy link
Collaborator Author

@greptile

@waleedlatif1
Copy link
Collaborator Author

@cursor review

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

…ead credentialAccountUserId field

- Move reachability test back after auth to prevent path enumeration
- Remove dead credentialAccountUserId from WebhookExecutionPayload
- Simplify credential resolution condition in background job
@waleedlatif1 waleedlatif1 merged commit 19ef526 into staging Mar 11, 2026
6 checks passed
@waleedlatif1 waleedlatif1 deleted the fix/dequeue branch March 11, 2026 21:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant