-
Notifications
You must be signed in to change notification settings - Fork 4.3k
Open
Description
Confirm this is a feature request for the Python library and not the underlying OpenAI API.
- This is a feature request for the Python library
Describe the feature or improvement you're requesting
Allow for cancellation of streaming/synchronous responses.
Additional context
Currently, responses.cancel
is only supported for background responses
For streaming responses one can sever the connection and start a new run from a new response_id
node.
However, this could have cost implications. I.e. if a user cancels a run, we do not want to pay for extra compute on the the next ResponseCompletedEvent
; this is especially notable if the LLM wants to do multiple tool calls.
Is this, canceling a streaming response, something that is already happening by default when a an OpenAI user disconnects? if not could we please have this feature?
hayescode
Metadata
Metadata
Assignees
Labels
No labels