Skip to content

fix(grain): update to stable version of API#3556

Merged
icecrasher321 merged 4 commits intostagingfrom
fix/grain-api-updates
Mar 13, 2026
Merged

fix(grain): update to stable version of API#3556
icecrasher321 merged 4 commits intostagingfrom
fix/grain-api-updates

Conversation

@icecrasher321
Copy link
Collaborator

Summary

Update to stable version of grain api as in (https://grainhq.notion.site/Grain-Personal-API-877184aa82b54c77a875083c1b560de9)

Type of Change

  • Bug fix
  • Breaking change

Testing

Tested manually

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@cursor
Copy link

cursor bot commented Mar 13, 2026

PR Summary

Medium Risk
Medium risk because it changes external webhook subscription creation/deletion semantics (new Grain API endpoints + required viewId) and adds new pre-lookup verification responses that affect webhook trigger routing during deploy/setup.

Overview
Updates the Grain integration to the stable Personal API. Grain webhook subscription create/list/delete now hit the new /_/public-api/* endpoints, require a view_id (plus optional actions) instead of the old hook_type/filter/include fields, and the app exposes a new grain_list_views tool and UI wiring to fetch view IDs.

Adds a pending webhook verification flow for providers that probe URL reachability before DB rows exist. Deployment registers short-lived “pending verification” entries (Redis-backed with in-memory fallback) and the webhook trigger route returns a temporary 200 for matching GET/HEAD/empty-POST probes on those paths; generic webhooks can opt in via a new verifyTestEvents trigger setting. Tests were added/updated to cover the new verification behavior.

Written by Cursor Bugbot for commit cac054a. Configure here.

@vercel
Copy link

vercel bot commented Mar 13, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Mar 13, 2026 5:43am

Request Review

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Mar 13, 2026

Greptile Summary

This PR migrates the Grain integration from an unstable versioned API (v2 URL path + Public-Api-Version header) to the stable Grain Personal API. The change introduces a view_id-based subscription model (replacing the old hook_type/filter/include approach), adds a new grain_list_views tool so users can discover their view IDs, and implements a PendingWebhookVerificationTracker to answer Grain's URL-verification probe before the webhook DB record exists — the probe now correctly receives a 200 OK without needing a pre-existing webhook row.

Key changes:

  • All Grain API URLs updated from /_/public-api/v2/… to /_/public-api/…; Public-Api-Version header removed; version: 2 moved into the request body.
  • view_id is now required for all Grain webhook triggers (recording, highlight, story, and generic), replacing the removed hook_type/filters.
  • New pending-verification.ts module stores short-lived (120 s) entries in Redis (with an in-memory fallback) to enable pre-creation URL verification; the in-memory fallback is unreliable across multiple server instances when Redis is unavailable.
  • New GET route handler added so Grain's HTTP GET verification probe is handled correctly.
  • data.views || data || [] in list_views.ts could silently assign a non-array object to views if the API returns an unexpected successful response body.

Confidence Score: 4/5

  • Safe to merge with minor improvements; the functional changes are well-scoped and the new verification flow is correctly guarded by TTL.
  • The core API migration is clean and the new pending-verification infrastructure is well-tested. The main risks are: (1) the in-memory fallback for PendingWebhookVerificationTracker is silently broken in multi-instance deployments without Redis — mitigated by Redis in production, but could cause confusion in staging/dev; (2) data.views || data || [] in list_views.ts can produce a non-array views field on an unexpected 200 response; (3) the finally-less clearAll() pattern in deploy.ts is fragile for future maintainers. None of these are blocking issues.
  • apps/sim/lib/webhooks/pending-verification.ts (in-memory multi-instance caveat), apps/sim/tools/grain/list_views.ts (response fallback), apps/sim/lib/webhooks/deploy.ts (clearAll placement)

Important Files Changed

Filename Overview
apps/sim/lib/webhooks/provider-subscriptions.ts Migrates Grain webhook creation/deletion to stable API: removes v2 versioned URL + Public-Api-Version header, adds version: 2 in request body, switches to view_id/actions model, adds a hard guard on the returned webhook id. The grain_webhook carve-out in the triggerId validation condition is a minor asymmetry noted in previous threads.
apps/sim/lib/webhooks/pending-verification.ts New module that tracks webhook paths awaiting external verification (before the DB record is created). Uses Redis as primary store with in-memory fallback; the in-memory path is silently unreliable in multi-instance/serverless deployments without Redis.
apps/sim/lib/webhooks/deploy.ts Integrates PendingWebhookVerificationTracker into the deploy flow: registers each webhook path before calling the Grain API (so verification probes can be answered), and clears registrations on success or failure. clearAll() is guarded by individual catch blocks but not a finally, which is fragile if new code paths are introduced.
apps/sim/tools/grain/list_views.ts New tool to list Grain views for webhook subscriptions. typeFilter is forwarded URL-encoded. The fallback `data.views
apps/sim/tools/grain/create_hook.ts Migrated to stable API: replaced hookType/filters/includes with viewId/actions, updated URL, removed Public-Api-Version header, added version: 2 in body, added explicit !data?.id guard before returning. Clean and intentional.
apps/sim/lib/webhooks/processor.ts Adds handlePreLookupWebhookVerification for responding to verification probes before a webhook DB record exists. Also refactors handlePreDeploymentVerification to use requiresPendingWebhookVerification. Logic is correct.
apps/sim/blocks/blocks/grain.ts Adds grain_list_views operation and a viewId field for grain_create_hook. Removes the legacy filter/include fields from grain_create_hook. Changes are consistent with the stable API migration.
apps/sim/app/api/webhooks/trigger/[path]/route.ts Adds a GET handler that delegates to handlePreLookupWebhookVerification before returning 405. The `
apps/sim/triggers/grain/webhook.ts Adds required viewId field to the generic Grain webhook trigger config. Description updated to reflect view-scoped behavior.

Sequence Diagram

sequenceDiagram
    participant User as User / UI
    participant SimApp as Sim App (deploy.ts)
    participant PVStore as PendingVerification Store (Redis / in-memory)
    participant GrainAPI as Grain API
    participant RouteHandler as Webhook Route Handler

    User->>SimApp: Deploy workflow with Grain trigger (viewId)
    SimApp->>PVStore: register(path, provider='grain', TTL=120s)
    SimApp->>GrainAPI: POST /_/public-api/hooks {version:2, hook_url, view_id, actions?}
    GrainAPI->>RouteHandler: GET /api/webhooks/trigger/{path} (verification probe)
    RouteHandler->>PVStore: getPendingWebhookVerification(path)
    PVStore-->>RouteHandler: entry (not expired)
    RouteHandler->>RouteHandler: matchesPendingWebhookVerificationProbe(entry, {method:'GET'})
    RouteHandler-->>GrainAPI: 200 OK { status: 'ok', message: 'Webhook endpoint verified' }
    GrainAPI-->>SimApp: { id: 'webhook-uuid', hook_url, view_id, ... }
    SimApp->>SimApp: Validate responseBody.id exists
    SimApp->>SimApp: db.transaction — insert webhook record (externalId = id)
    SimApp->>PVStore: clearAll()
    SimApp-->>User: Deploy success
Loading

Comments Outside Diff (1)

  1. apps/sim/lib/webhooks/pending-verification.ts, line 541 (link)

    In-memory fallback breaks in multi-instance deployments

    inMemoryPendingVerificationStore is a module-level singleton. When Redis is unavailable (e.g., local dev or a Redis outage), the pending verification entry is stored in the memory of the specific server instance that handled the deploy request. If the Grain verification probe is then routed to a different instance (as happens with Vercel or any horizontally-scaled setup), getPendingWebhookVerification returns null on that instance, and the probe gets a 405 Method Not Allowed instead of 200 OK, causing webhook creation to fail.

    This is mitigated by Redis when it's available, but the in-memory path is silently broken across instances. Consider adding a log warning when falling back to in-memory storage, so operators know the behaviour is degraded:

    } else {
      logger.warn('Redis unavailable; using in-memory pending verification store. This will not work across multiple instances.', { path: registration.path })
      inMemoryPendingVerificationStore.set(registration.path, entry)
    }

Last reviewed commit: cac054a

@icecrasher321
Copy link
Collaborator Author

bugbot run

@icecrasher321
Copy link
Collaborator Author

@greptile

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

@icecrasher321
Copy link
Collaborator Author

bugbot run

@icecrasher321
Copy link
Collaborator Author

@greptile

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

@icecrasher321 icecrasher321 merged commit d90f828 into staging Mar 13, 2026
12 checks passed
return {
success: true,
output: {
views: data.views || data || [],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

views may silently receive a non-array value

data.views || data || [] looks safe on the happy path, but if the Grain stable API ever returns a plain array (no wrapping object), data.views is undefined (falsy) and data — an array — is returned directly. That part is fine. However, if data is a truthy non-array object (e.g., { message: "ok" } from an unexpected 200 body), it gets assigned to views, violating the GrainView[] contract at runtime without any error being thrown.

Prefer an explicit check to guard against this:

Suggested change
views: data.views || data || [],
views: Array.isArray(data.views) ? data.views : Array.isArray(data) ? data : [],

}
})

await pendingVerificationTracker.clearAll()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clearAll() called before configurePollingIfNeeded return paths

pendingVerificationTracker.clearAll() is called at line 680 (inside the try block, after the DB transaction but before the polling loop). If configurePollingIfNeeded triggers the early-return path (return { success: false, error: pollingError }), the function exits without a finally-guarded cleanup — but since clearAll() already ran at this point, the pending entries are correctly cleared.

The concern is the catch branch lower in the function also calls clearAll() as a separate statement. If a new early-return path is ever introduced between line 680 and the end of the try block (e.g., a new pollingError cleanup that throws instead of returning), the double-call is harmless but the absence of a finally block means the intent isn't self-documenting. Consider wrapping the tracker cleanup in a finally block for robustness:

try {
  await db.transaction(...)
  for (const sub of createdSubscriptions) { ... }
} catch (error: any) {
  // cleanup
} finally {
  await pendingVerificationTracker.clearAll()
}

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@waleedlatif1 waleedlatif1 deleted the fix/grain-api-updates branch March 13, 2026 05:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant