Achieving storage parity in Django with local object storage

Updated

In modern Django projects, uploaded files are often stored in S3-compatible object storage. Using different storage classes for local and production can complicate the codebase and cause subtle production-only issues. A practical solution is to use object storage locally too.

The open source community, through django-storages, has greatly simplified interactions with a wide range of storage backends. The main issue arises when more advanced or fine-grained file operations are required: at a lower level, the django-storages API and Django's native File Storage API are not specified in a uniform way.

And yes -- I'm looking at you: paths, signed URLs, ACLs, overwrites. Both the bane and the boon.

A local object storage is a great workaround for achieving behavioral parity among the environments and for writing reliable automated tests. Specifically, we're going to see how to set up a dockerized local RustFS instance.

This article used MinIO until September 2026, when MinIO stopped publishing free images: first on Docker Hub, then on quay.io. RustFS is an S3-compatible server that still publishes an official image.

Adding RustFS to your Compose file

First, let's add the RustFS service to our Compose file with its health check:

services:

  ...

  rustfs:
    environment:
      - RUSTFS_ACCESS_KEY=rustfs-admin
      - RUSTFS_SECRET_KEY=rustfs-admin
    healthcheck:
      test: ["CMD", "curl", "-fs", "http://localhost:9000/health"]
      interval: 3s
      timeout: 3s
      retries: 30
    image: rustfs/rustfs:latest
    ports:
      - 9000:9000
      - 9001:9001
    volumes:
      - rustfs_data:/data

If your Django application does not run inside Docker, then you will need to expose the RustFS service through a custom network to make it reachable from outside Docker.

Once the service is running, the RustFS web console will be available at http://localhost:9001/rustfs/console/, while the S3 API will be exposed on port 9000.

Integrating RustFS with Django

Next, even out the functional gap by making django-storages your primary storage backend. In your Django settings, specify the following parameters:

  • AWS_ACCESS_KEY_ID -> RUSTFS_ACCESS_KEY

  • AWS_SECRET_ACCESS_KEY -> RUSTFS_SECRET_KEY

  • AWS_STORAGE_BUCKET_NAME -> your bucket, e.g. my-bucket

  • AWS_S3_ENDPOINT_URL -> http://rustfs:9000

  • AWS_S3_REGION_NAME -> your preferred region

  • AWS_LOCATION -> optional subfolder within the bucket

Creating the bucket

RustFS creates no bucket and sets no policy on its own, and its client ships as release archives rather than as an image. There is no need for a client, though: django-storages already holds a boto3 connection to the bucket, and the S3 API can do both. Save the following as scripts/bucket.py:

import json

from django.core.files.storage import default_storage

bucket = default_storage.bucket
if not bucket.creation_date:
    bucket.create()
bucket.Policy().put(
    Policy=json.dumps(
        {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Principal": "*",
                    "Action": "s3:GetObject",
                    "Resource": f"arn:aws:s3:::{bucket.name}/*",
                }
            ],
        }
    )
)

Then run it once the service is healthy, for instance from the entrypoint of your Django container:

python manage.py shell < scripts/bucket.py

With that, we create the bucket if it is missing and configure its permissions to be public read-only. Running it again changes nothing, so it can run on every start.

This configuration ensures that your local RustFS setup behaves consistently with your production storage, enabling reliable development and testing.

0