Skip to content

Allow Parse Job to be launched with more than 16Mb of data in params #8341

@codeKonami

Description

@codeKonami

New Feature / Enhancement Checklist

Current Limitation

When running a Parse Job, the params is saved in the _JobStatus collection by default in MongoDB thus limiting the size of the params to the MongoDB BSON limit of 16Mb : https://www.mongodb.com/docs/manual/reference/limits/#:~:text=The%20maximum%20BSON%20document%20size,MongoDB%20provides%20the%20GridFS%20API.

Feature / Enhancement Description

We would need to modify this code :

return jobHandler.setRunning(jobName, params).then(jobStatus => {

Two ways of doing it :

  • We could pass a context while launching the Parse Job telling that we should not save the params.
  • We could measure the size of the params and therefore truncate the params inside the JobStatus document.

Example Use Case

For example I want to launch a job with 32Mb of data to analyse, I will launch the job but explicitly telling Parse to not save the params in database.

Alternatives / Workarounds

No easy workaround

3rd Party References

POSTing more than 16Mb of data is quite common everywhere :)

Metadata

Metadata

Assignees

No one assigned

    Labels

    state:releasedReleased as stable versionstate:released-alphaReleased as alpha versiontype:featureNew feature or improvement of existing feature

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions