OpenAI integration: DurableRunner with LLM retry opts and lazy session flushing#164
Merged
OpenAI integration: DurableRunner with LLM retry opts and lazy session flushing#164
Conversation
…ing model retry opts, set parallel_tool_calls to false
…ing model retry opts, set parallel_tool_calls to false
igalshilman
reviewed
Dec 15, 2025
igalshilman
reviewed
Dec 15, 2025
| if limit is not None: | ||
| return current_items[-limit:] | ||
| return current_items | ||
| return self._items[-limit:] |
Contributor
There was a problem hiding this comment.
are you sure about this?
this seems to take the last limit items (the negative sign) ?
igalshilman
reviewed
Dec 15, 2025
| for handoff in agent.handoffs: | ||
| # recursively wrap tools in handoff agents | ||
| handoffs_with_wrapped_tools.append(sequentialize_and_wrap_tools(handoff, disable_tool_autowrapping)) # type: ignore | ||
| runner = DEFAULT_AGENT_RUNNER or AgentRunner() |
Contributor
There was a problem hiding this comment.
what is happening here? should we just delegate to the OpenAI runner?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request makes significant improvements to the Restate OpenAI integration, focusing on enhanced LLM call retry configuration, session management, and simplifying the agent runner interface. The most important changes are grouped below.
LLM Retry and Durable Model Calls:
LlmRetryOptsdataclass to allow fine-grained configuration of retry behavior for LLM calls, including maximum attempts, duration, intervals, and backoff factor. TheDurableModelCallsandRestateModelWrapperclasses were updated to use this new configuration, replacing the previous simple retry count. ([[1]](https://github.com/restatedev/sdk-python/pull/164/files#diff-bcfe297c913d38628b0f9bee4137165993514de2182421c3fb86f6f4df0e2f86L15-R69),[[2]](https://github.com/restatedev/sdk-python/pull/164/files#diff-bcfe297c913d38628b0f9bee4137165993514de2182421c3fb86f6f4df0e2f86L74-R112),[[3]](https://github.com/restatedev/sdk-python/pull/164/files#diff-bcfe297c913d38628b0f9bee4137165993514de2182421c3fb86f6f4df0e2f86L105-R137))Session Management Improvements:
RestateSessionclass to cache session items in memory, reducing redundant context fetches. Added aflushmethod to persist changes and updatedadd_items,get_items, andpop_itemto use the cache. Now, session state is explicitly flushed after agent runs. ([[1]](https://github.com/restatedev/sdk-python/pull/164/files#diff-bcfe297c913d38628b0f9bee4137165993514de2182421c3fb86f6f4df0e2f86R152-R188),[[2]](https://github.com/restatedev/sdk-python/pull/164/files#diff-bcfe297c913d38628b0f9bee4137165993514de2182421c3fb86f6f4df0e2f86L216-R284))Runner and Agent Execution Simplification:
Runnerclass withDurableRunner, simplifying the agent run interface. Therunmethod now takesinputandllm_retry_optsdirectly, configures the model provider and disables parallel tool calls viaModelSettings, and automatically flushes session state. The previous tool wrapping logic was removed in favor of built-in sequentialization. ([[1]](https://github.com/restatedev/sdk-python/pull/164/files#diff-bcfe297c913d38628b0f9bee4137165993514de2182421c3fb86f6f4df0e2f86L192-R234),[[2]](https://github.com/restatedev/sdk-python/pull/164/files#diff-bcfe297c913d38628b0f9bee4137165993514de2182421c3fb86f6f4df0e2f86L204-R246),[[3]](https://github.com/restatedev/sdk-python/pull/164/files#diff-bcfe297c913d38628b0f9bee4137165993514de2182421c3fb86f6f4df0e2f86L216-R284))Public API and Utility Functions:
__init__.pyto exportDurableRunner,RestateSession,LlmRetryOpts, and new utility functionsrestate_object_contextandrestate_contextfor easier context access. ([python/restate/ext/openai/__init__.pyL15-L21](https://github.com/restatedev/sdk-python/pull/164/files#diff-484be77d523f4d783c48e3a57394969cb6ae3a6c383b69c5a11fd25162c806a9L15-L21))These changes collectively make the OpenAI integration for Restate more robust, configurable, and easier to use.