File uploads - locally and on s3

[kbadova]

Overview

This adds a files Django app, which enables both DIRECT and PASS-THRU upload scenarios, where the files can be uploaded both LOCAL and to S3.

The following combinations are possible:

• Django admin + DIRECT + LOCAL
• Django admin + DIRECT + S3
• API + DIRECT + LOCAL
• API + DIRECT + S3
• API + PASS-THRU + LOCAL
• API + PASS-THRU + S3

For the PASS-THRU, the API interface for the frontend remains the same - meaning, you don't need to write different code to handle LOCAL vs S3 on the frontend.

We can mix and match, depending on needs:

1. Have Django admin uploading directly to S3.
2. While having a pass-thru API for the frontend.
3. And everything works with the same model / codebase.

For frontend testing, there's a separate PR here - https://github.com/HackSoftware/Example-React-Project/pull/1

Settings

This PR introduces few new settings.

The file upload style & storage are controlled via:

# Example .env file

FILE_UPLOAD_STRATEGY="pass-thru"
FILE_UPLOAD_STORAGE="s3"

Additionally, there are a bunch of AWS related settings, so django-storages and boto3 can do their job. They are located in config/settings/files_and_storages.py

Details

For S3 bucket configuration, you'll need the following CORS JSON:

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "POST"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": []
    }
]

Or if you know what you're doing with your bucket - configure it yourself 👌

Presigned data returned from S3 looks like this:

{'fields': {'Content-Type': 'image/png',
            'acl': 'private',
            'key': 'files/bafdccb665a447468e237781154883b5.png',
            'policy': 'eyJleHBpcmF0aW9uIjogIjIwMjItMDQtMDVUMTE6NDk6MjJaIiwgImNvbmRpdGlvbnMiOiBbeyJhY2wiOiAicHJpdmF0ZSJ9LCB7IkNvbnRlbnQtVHlwZSI6ICJpbWFnZS9wbmcifSwgeyJidWNrZXQiOiAiZGphbmdvLXN0eWxlZ3VpZGUtZXhhbXBsZSJ9LCB7ImtleSI6ICJmaWxlcy9iYWZkY2NiNjY1YTQ0NzQ2OGUyMzc3ODExNTQ4ODNiNS5wbmcifSwgeyJ4LWFtei1hbGdvcml0aG0iOiAiQVdTNC1ITUFDLVNIQTI1NiJ9LCB7IngtYW16LWNyZWRlbnRpYWwiOiAiQUtJQVNPWkxaSTVGSkRKNlhUU1ovMjAyMjA0MDUvZXUtY2VudHJhbC0xL3MzL2F3czRfcmVxdWVzdCJ9LCB7IngtYW16LWRhdGUiOiAiMjAyMjA0MDVUMTE0OTEyWiJ9XX0=',
            'x-amz-algorithm': 'AWS4-HMAC-SHA256',
            'x-amz-credential': 'AKIASOZLZI5FJDJ6XTSZ/20220405/eu-central-1/s3/aws4_request',
            'x-amz-date': '20220405T114912Z',
            'x-amz-signature': '7d8be89aabec12b781d44b5b3f099d07be319b9a41d9a9c804bd1075e1ef5735'},
 'url': 'https://django-styleguide-example.s3.amazonaws.com/'}

Resources

• https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html
• https://devcenter.heroku.com/articles/s3-upload-python
• https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-presigned-urls.html#generating-a-presigned-url-to-upload-a-file
• https://docs.aws.amazon.com/AmazonS3/latest/userguide/acl-overview.html#canned-acl

[RadoRado]

👀