Skip to content

Cancel for streaming Responses #2643

@maininformer

Description

@maininformer

Confirm this is a feature request for the Python library and not the underlying OpenAI API.

  • This is a feature request for the Python library

Describe the feature or improvement you're requesting

Allow for cancellation of streaming/synchronous responses.

Additional context

Currently, responses.cancel is only supported for background responses

For streaming responses one can sever the connection and start a new run from a new response_id node.

However, this could have cost implications. I.e. if a user cancels a run, we do not want to pay for extra compute on the the next ResponseCompletedEvent; this is especially notable if the LLM wants to do multiple tool calls.

Is this, canceling a streaming response, something that is already happening by default when a an OpenAI user disconnects? if not could we please have this feature?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions