-
-
Notifications
You must be signed in to change notification settings - Fork 4.8k
Closed
Labels
state:releasedReleased as stable versionReleased as stable versionstate:released-alphaReleased as alpha versionReleased as alpha versiontype:featureNew feature or improvement of existing featureNew feature or improvement of existing feature
Description
New Feature / Enhancement Checklist
- I am not disclosing a vulnerability.
- I am not just asking a question.
- I have searched through existing issues.
Current Limitation
When running a Parse Job, the params is saved in the _JobStatus collection by default in MongoDB thus limiting the size of the params to the MongoDB BSON limit of 16Mb : https://www.mongodb.com/docs/manual/reference/limits/#:~:text=The%20maximum%20BSON%20document%20size,MongoDB%20provides%20the%20GridFS%20API.
Feature / Enhancement Description
We would need to modify this code :
return jobHandler.setRunning(jobName, params).then(jobStatus => { |
Two ways of doing it :
- We could pass a context while launching the Parse Job telling that we should not save the params.
- We could measure the size of the params and therefore truncate the params inside the JobStatus document.
Example Use Case
For example I want to launch a job with 32Mb of data to analyse, I will launch the job but explicitly telling Parse to not save the params in database.
Alternatives / Workarounds
No easy workaround
3rd Party References
POSTing more than 16Mb of data is quite common everywhere :)
Metadata
Metadata
Assignees
Labels
state:releasedReleased as stable versionReleased as stable versionstate:released-alphaReleased as alpha versionReleased as alpha versiontype:featureNew feature or improvement of existing featureNew feature or improvement of existing feature