Skip to content

Horizontal Scalability for stateful streamable HTTP #843

@fschoenberger

Description

@fschoenberger

Is your feature request related to a problem? Please describe.
I would like to be able to horizontally scale my MCP server while using streamable HTTP. Right now, it seems like there is a "pick-your poison" scenario: As a MCP server author I can either

  • Use stateful mode, but give up on horizontal scalability,
  • or use stateless mode and give up on sampling, elicitation, and progress reporting.

Describe the solution you'd like
I would like to be able to horizontally scale my MCP server while using stateful streamable HTTP. For that, I would envision something like socket.io's Redis backend , where all state is stored, such that everything "just works".

Describe alternatives you've considered
Stateless streamable HTTP doesn't work for me, because I would need to give up on features such as solicitation, sampling and progress reporting.

Additional context
Horizontal Scalability is important because it increases the resiliency and availability of the system - as demand grows, so does the number of replicas of the service. Unhealthy replicas are killed and replaced by healthy ones. In front of the service sits a load balancer that is aware of the replicas and routes requests to them.

Some applications have internal state that needs to be persisted across requests (e.g. PHP's $_SESSION state). For that, most people use a solution like Redis or ValKey, such that state is efficiently stored in a cache, as opposed to in-memory for each replica. For examplesocket.io, a WebSocket abstraction library, that manages potentially long-living connections, also supports using redis as a session state backend.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions