Skip to content

500 error encountered on flows/<flow_id>/runs requests due to size of payload #322

@RobBlumberg

Description

@RobBlumberg

I have a flow running in production on a 2min schedule for which artifacts / metadata are being stored. I am trying to interact with those artifacts with the client API. However, requests to /flows/<flow_id>/runs (via instantiating Flow("FlowName")) are now failing with

Metadata request: (/flows/<flow_id>/runs) failed (code 500): {"message": "Internal server error"}

Request to other endpoints, like flows/<flow_id>/runs/<run_id>, are going through just fine. After taking a closer look on API Gateway, I generated the same request through the console there, and get

Execution failed due to configuration error: Integration response of reported length 28729085 is larger than allowed maximum of 10485760 bytes.
Tue May 31 22:49:46 UTC 2022 : Method completed with status: 500

Basically the response payload is larger than the non-configurable 10MB limit on API Gateway.

I can get around this by requesting individual runs directly, but it would be great to still be able to use the flow apis to interact with the child runs of the flow via the client. (Perhaps adding the ability to pass filtering params in the requests, like fetching the last n runs?). Also curious if this is something that others have run into when deploying flows to production where many runs are produced and stored, and if there are any workarounds or things I am missing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions