perf: replace enum construction with frozenset lookup

[ryan-crabbe]

CallTypes(call_type) was constructing an enum from string on every call, taking ~4.6µs/call (69.6% of total function time). Replace with a frozenset membership test for ~0.8µs/call (8.3x faster).

Before

After

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Type

Performance
🧹 Refactoring

Changes

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 2, 2026 11:01pm

[greptile-apps]

Greptile Overview

Greptile Summary

This PR optimizes the _is_streaming_request function in litellm/utils.py by replacing expensive enum construction with a fast frozenset membership test, achieving an 8.3x speedup (from ~4.6µs to ~0.8µs per call).

Key changes:

• Introduced _STREAMING_CALL_TYPES frozenset containing both enum members and their string values for generate_content_stream and agenerate_content_stream
• Replaced try-except enum construction logic with simple in operator for O(1) lookup
• Added comprehensive test suite (10 test cases) covering all input types and edge cases

Performance impact:

The function is called frequently in the request processing path (4 call sites in utils.py alone). This optimization reduces overhead in a hot path, particularly beneficial for high-throughput scenarios. The change aligns with the custom instruction to avoid creating unnecessary objects in the critical request path.

Confidence Score: 5/5

• This PR is safe to merge with minimal risk - it's a well-tested performance optimization with equivalent behavior
• The implementation is correct, maintains behavioral equivalence with the original code, includes comprehensive test coverage, and delivers significant performance improvements. The frozenset correctly handles both string and enum inputs, and the logic properly preserves the original function's semantics.
• No files require special attention

Important Files Changed

Filename	Overview
litellm/utils.py	Replaced enum construction with frozenset lookup for 8.3x performance improvement in _is_streaming_request function
tests/test_litellm/test_utils.py	Added comprehensive test coverage for _is_streaming_request with 10 test cases covering string/enum inputs and edge cases

Sequence Diagram

sequenceDiagram
    participant Caller
    participant _is_streaming_request
    participant _STREAMING_CALL_TYPES as frozenset
    
    Note over Caller,_STREAMING_CALL_TYPES: Before: Enum Construction (4.6µs)
    Caller->>_is_streaming_request: kwargs, call_type (str or enum)
    _is_streaming_request->>_is_streaming_request: Check if stream=True in kwargs
    alt stream=True
        _is_streaming_request-->>Caller: True
    else stream not True
        _is_streaming_request->>_is_streaming_request: OLD: Try CallTypes(call_type) construction
        _is_streaming_request->>_is_streaming_request: OLD: Compare with enum members
        _is_streaming_request-->>Caller: True/False (slow)
    end
    
    Note over Caller,_STREAMING_CALL_TYPES: After: Frozenset Lookup (0.8µs)
    Caller->>_is_streaming_request: kwargs, call_type (str or enum)
    _is_streaming_request->>_is_streaming_request: Check if stream=True in kwargs
    alt stream=True
        _is_streaming_request-->>Caller: True
    else stream not True
        _is_streaming_request->>_STREAMING_CALL_TYPES: NEW: call_type in frozenset?
        _STREAMING_CALL_TYPES-->>_is_streaming_request: True/False (fast O(1) lookup)
        _is_streaming_request-->>Caller: True/False
    end

Loading

[greptile-apps]

_{2 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}