Craft Cloud post_max_size and max_input_vars PHP values are too low for Formie form integrations

[jamesmacwhite]

Hi,

The Craft Cloud environment currently sets a very conservative post_max_size value of 6M and max_input_vars of 1000, these current values will break the Formie plugin when using integration classes which provide large field mappings which leads to the payload size being increased.

I have been burnt by this before on Fortrabbit and after some debugging with @engram-design, it was due to default lower values for these two PHP environment variables being implemented.
verbb/formie#1212 (comment)

Under Craft Cloud currently certain Formie forms can be broken, because once a form payload exceeds the post_max_size or max_input_vars, any changes or amends are going to be silently dropped and lost. No warning, no error, it is a breaking scenario. Furthermore, the most dangerous scenario is a form exceeding this can actually end up deleting form fields without warning as well, because the data is discarded causing Formie to think the fields aren't there.

The post_max_size default is likely too low with larger Formie integrations enabled. max_input_vars would also needs increasing from 1000. In some cases it might be the max_input_vars value rather than the post size. However local development environments like DDEV set post_max_size as 100M and max_input_vars as 5000, in contrast to Craft Cloud these values are massively different, potentially leading to silent issues when deploying. Fortunately (or unfortunately as I was two years ago), I had the pleasure of finding this out the hard way between dev and staging/production. However changing the PHP vars on the environment affected was easy enough to resolve at the time, two PHP ini config changes for weeks of debugging and worry was very stressful but a simple fix in the end.

Given the serverless architecture on Craft Cloud, I can't just modify the PHP vars like other providers, therefore I feel this needs highlighting as it's a incompatibility with a well-used Craft CMS plugin currently and has production breaking consequences which aren't immediately obvious, until you dig into it.

To clarify the issue isn't Formie itself, but integrations which have large payloads (usually field mappings), making Formie actions like saving a form, editing etc suddenly become larger requests, due to the integration and payload size. There is perhaps better handling Formie could do, but some API integrations are large by their design and there's no getting away from that currently.

It would be great if these PHP ini vars could be reviewed urgently, giving the breaking nature of the current values.

[jamesmacwhite]

Logged under the Formie GitHub as well for visibility:

verbb/formie#2114

[jamesmacwhite]

An update for anyone who comes across this. First of all Craft Cloud support are great, they've been very helpful. 👌

Formie has made updates to reduce integration payload sizes when there are empty fields. The main issue is actually max_input_vars being set to 1000 by default on Craft Cloud. However when a new release of Formie is made, payloads from form integrations will be smaller in the future.

However there are concerns that this still might be a too low setting for some scenarios, particularly for larger integrations, but there is an undocumented way of changing PHP config vars on Craft Cloud through your own .ini file if it is located in the path php/conf.d at the root of a project deployed on Craft Cloud. You can have the file called anything providing it is something .ini in that path. I don't think this is technically "secret" just probably not something many are needing to do.

post_max_size is set at 6M due to AWS Lambda, given the serverless nature of Craft Cloud. This cannot be changed given its an AWS limit. I've had some chats and explored scenarios of where this could be a problem. The general view is the main area would be front-end uploads, having 6MB of textual POST data would be uncommon, so it was considered to be OK for most cases and in this scenario, it is not the POST size but the amount of vars in a request being the issue, which is solvable by the the undocumented PHP var change above.

Craft Cloud are going to publish documentation on how front end uploading can use the S3 layer rather than AWS Lambda, by using S3 it would not be subject to the 6MB limitation and bypass that restriction entirely. This would be reliant on plugin developers supporting Craft Cloud's method, but there are ways to detect a Craft Cloud hosted site vs typical Craft CMS compute hosted site.

[timkelty]

Related: #51

[jamesmacwhite]

I believe Craft Cloud sets max_input_vars to 4000 during the building of the image now during a deployment which solves part of this issue when it was originally raised. This was added partly due to this issue a while back, it can be set to an alternative value i.e. higher by the undocumented way to modify some PHP vars during the build process, so it becomes less of an issue.

The post_max_size cannot be changed due to AWS Lambda. Although to hit 6 MB of pure POST data as in text was aptly illustrated by @angrybrad as a rare scenario, given the size of the sample text file and the amount of data it would need to contain to reach it.

#51 is indeed the more relevant area because one of the main scenarios the POST size would be bloated is file uploads. Official documentation on this area is very welcome, as you'll basically hit the 6 MB very quickly without using S3 for handling files uploaded via forms. I'd imagine Craft Cloud environments using plugins like Formie will very quickly get a shock, unless specific handling has been added, either a Craft Cloud specific detection or a file upload field which can leverage an S3 bucket, rather than standard web form POST data.