MinIO Was Easy, SeaweedFS Made Me Think

Why I Tried SeaweedFS After MinIO

This is a SeaweedFS vs MinIO guide disguised as the follow-up to my first self-hosted object storage migration. The app was still FastAPI. The client still did not want AWS S3. The code still used boto3. But the storage decision changed because the workload looked more media-heavy, more file-heavy, and more likely to outgrow a simple single-node object store.

After I got one app working on MinIO, I thought I understood the self-hosted storage problem.

That confidence lasted exactly until the next deployment.

The new constraint looked similar: no AWS S3, no managed object storage, media-heavy app, private/generated files, FastAPI backend, browser-facing URLs, and enough user uploads that “just put it in Postgres” would be a crime.

MinIO was the obvious first thought because it had already worked. It is S3-compatible, easy to run, and the boto3 migration story is basically one endpoint_url. But by this point I was also thinking about the enterprise side of the decision. MinIO is still open source, but serious enterprise packaging, support, and newer commercial product positioning live behind MinIO’s paid AIStor/subscription world. For a client-owned deployment where the whole premise was “we want control and predictable cost”, I wanted to at least look at the other serious self-hosted option.

That is how SeaweedFS entered the chat.

Note — SeaweedFS: a distributed storage system originally designed around fast file lookup and efficient small-file storage. It can expose a filesystem-like filer, volume servers, and an S3-compatible API. That last part is what made it usable for my app without rewriting every upload/download path.

MinIO felt like “S3, but on my server.”

SeaweedFS felt like “a storage system that happens to speak S3.”

That difference matters.

What Is SeaweedFS?

SeaweedFS is a distributed storage system that can expose multiple interfaces: volume servers for file content, a filer for metadata and directory-like access, and an S3-compatible API for clients like boto3 and AWS CLI. That means a Python or FastAPI app can keep using S3-style upload, download, and pre-signed URL code while SeaweedFS handles storage behind the scenes.

The important mental shift is this: MinIO is easiest to think about as an AWS S3-compatible object store. SeaweedFS is easier to understand as a broader storage system with an S3 gateway on top.

SeaweedFS Terms for S3-Compatible Storage

If I come back to this in six months, this is the section I want near the top.

Object storage: storage for blobs addressed by keys, not rows or normal filesystem paths. Example key: users/42/gallery/image.webp. The slash is part of the name; it is not necessarily a real folder.

Bucket: a top-level container for objects. S3 calls it a bucket. S3-compatible systems usually keep the same word.

S3-compatible: not AWS S3, but speaks enough of the same API that tools like boto3, AWS CLI, and pre-signed URLs can work.

boto3: the Python AWS SDK. In this project it is not “AWS code” anymore; it is just the S3 protocol client.

Endpoint URL: the base URL boto3 talks to. For AWS it is implicit. For MinIO/SeaweedFS it is explicit, like http://seaweed:8333.

SigV4: AWS Signature Version 4, the signing algorithm used for authenticated S3 requests and pre-signed URLs.

Pre-signed URL: a temporary URL that contains a signature. Anyone with the URL can access that one object until it expires.

Path-style URL: https://media.example.com/bucket/key.

Virtual-hosted-style URL: https://bucket.media.example.com/key.

Filer: SeaweedFS’s metadata/file layer. It gives a directory-like view over files whose bytes live on volume servers.

Volume server: SeaweedFS component that stores actual file content.

Master: SeaweedFS coordinator that tracks volumes and where file ids live.

S3 gateway/API: the SeaweedFS-facing door that lets normal S3 clients upload and download objects.

That is the whole mental model: my app talks S3, SeaweedFS translates that into its own filer/volume world, and the browser receives either public URLs or pre-signed URLs.

Why Not Just Stay on MinIO?

I still like MinIO. For a single machine, simple object storage, and a codebase that already uses boto3, it is almost unfairly convenient. One binary, one console, one bucket, done.

But this app had a slightly different smell:

Lots of generated media. Images, audio, and possibly video. That means many files, many reads, and a lot of cache behavior.

A need for public and internal paths. The backend should talk to storage privately. The browser should see a clean HTTPS media domain.

A future where “single disk with object API” might stop being enough. I did not need a giant cluster on day one, but I wanted a path that did not force another migration the moment storage became more serious.

SeaweedFS looked attractive because it is not only trying to mimic S3. Its architecture has masters, volume servers, filers, and an S3 gateway. That is more moving parts, yes. But it also means the S3 API is not the whole product.

Note — filer: in SeaweedFS, the filer gives you a directory/file abstraction and metadata layer. The actual bytes live on volume servers. The S3 API sits in front of that world so S3 clients can still PUT and GET objects.

The official SeaweedFS README talks a lot about fast small-file access, volume-id lookup, and using existing metadata stores for filer metadata. That is not something I cared about in the first hour. I cared about “does boto3 upload work?” But later, those design choices started making the system feel more powerful than MinIO for the kind of messy media workload I was dealing with.

SeaweedFS vs MinIO

This is not a benchmark. This is the practical decision table I wish I had while choosing.

MinIO:
  best when      I want S3-compatible object storage quickly
  mental model   server -> bucket -> object
  setup feel     simple
  operations     easier to explain
  downside       feels narrower if I later need a bigger storage architecture

SeaweedFS:
  best when      I expect lots of files and want room to grow storage topology
  mental model   master -> filer -> volume -> optional S3 API
  setup feel     more nouns, more config, more thinking
  operations     more powerful but less beginner-friendly
  downside       I need to understand the pieces before debugging production

Enter fullscreen mode Exit fullscreen mode

If the app only needs “store some files and give me S3 URLs”, MinIO is the clean answer.

If the app is media-heavy and might eventually need multiple storage nodes, filer behavior, public/internal endpoints, and more control over how files are organized, SeaweedFS becomes interesting.

I did not pick SeaweedFS because it was easier. I picked it because the future shape of the app looked less tiny than the first deployment.

Here is the same comparison in a more search-friendly form:

Feature MinIO SeaweedFS Main feel S3-compatible object store Distributed storage system with S3 API Setup complexity Lower Higher boto3 support Yes Yes, through S3 gateway Pre-signed URLs Yes Yes, with careful public endpoint config Best for Simple self-hosted object storage Lots of files, media workloads, flexible topology Mental model Bucket and object Master, filer, volume, S3 gateway Operational burden Easier to explain More moving parts to understand

Configuring boto3 for SeaweedFS S3 API

The pleasant surprise: the app code still stayed S3-shaped.

The client factory looked almost the same as MinIO. The important bits were endpoint_url, SigV4, and path-style addressing:

# FastAPI SeaweedFS boto3 configuration for S3-compatible storage
import boto3
from botocore.config import Config

async def get_s3_client():
    endpoint_url = settings.AWS_S3_ENDPOINT_URL

    if not endpoint_url:
        raise ValueError("AWS_S3_ENDPOINT_URL is not configured")

    config = Config(
        signature_version="s3v4",
        s3={"addressing_style": "path"},
    )

    return boto3.client(
        "s3",
        aws_access_key_id=settings.AWS_ACCESS_KEY_ID,
        aws_secret_access_key=settings.AWS_SECRET_ACCESS_KEY,
        region_name=settings.AWS_REGION,
        endpoint_url=endpoint_url,
        config=config,
    )

Enter fullscreen mode Exit fullscreen mode

Note — path-style addressing: https://storage.example.com/bucket/key instead of https://bucket.storage.example.com/key. Self-hosted S3-compatible systems usually want path-style unless you have wildcard DNS and TLS set up for every bucket hostname.

Uploads stayed boring, which is exactly what I wanted:

async def upload_to_s3_file(file_obj, s3_key, content_type, bucket_name, s3_client=None):
    if s3_client is None:
        s3_client = await get_s3_client()

    def _upload():
        s3_client.upload_fileobj(
            Fileobj=file_obj,
            Bucket=bucket_name,
            Key=s3_key,
            ExtraArgs={"ContentType": content_type},
        )

    await asyncio.to_thread(_upload)
    presigned_url = await generate_presigned_url(s3_key, bucket_name=bucket_name)
    return s3_key, presigned_url

Enter fullscreen mode Exit fullscreen mode

That asyncio.to_thread is doing real work. boto3 is blocking. FastAPI is async. If you call boto3 directly inside the request handler, one slow upload/download can block the event loop and make unrelated requests feel haunted. Offloading the blocking storage call into a thread keeps the async server breathing.

Designing a FastAPI Storage Module

The smartest thing in this migration was not SeaweedFS. It was keeping storage behind one module.

The rest of the app should not know whether the backend is AWS S3, MinIO, SeaweedFS, or a directory behind nginx. Most code should only need verbs like:

await upload_to_s3_file(file_obj, key, content_type, bucket)
url = await generate_presigned_url(key)
public_url = await generate_public_s3_url(key)
await delete_s3_object(key)

Enter fullscreen mode Exit fullscreen mode

That sounds boring, but boring boundaries save migrations.

The module owns:

credentials
endpoint_url
public_url
bucket name
path-style addressing
presigned URL generation
blocking boto3 calls
Redis caching
ContentType
delete behavior

Enter fullscreen mode Exit fullscreen mode

The feature code owns:

what file is being uploaded
who is allowed to see it
what key/name it should have
what DB row points to it

Enter fullscreen mode Exit fullscreen mode

When those boundaries blur, every storage migration becomes a treasure hunt. I prefer boring now. Boring is portable.

Generating SeaweedFS Pre-Signed URLs with Public and Internal Endpoints

With MinIO, I had already learned that pre-signed URLs are fragile. The host is part of the SigV4 signature. Change the host after signing and you get SignatureDoesNotMatch.

SeaweedFS made that lesson more explicit because I wanted two endpoints:

AWS_S3_ENDPOINT_URL=http://seaweed:8333
AWS_S3_PUBLIC_URL=https://media.example.com
AWS_BUCKET_NAME=app-media

Enter fullscreen mode Exit fullscreen mode

The backend container should use the internal endpoint:

http://seaweed:8333

Enter fullscreen mode Exit fullscreen mode

The browser should use the public HTTPS endpoint:

https://media.example.com

Enter fullscreen mode Exit fullscreen mode

My first instinct was to generate the URL internally, then rewrite the host to the public domain. That is exactly the trap from the MinIO migration. It looks like a harmless string operation, but it breaks the signature.

So I split the clients.

One client is for backend storage operations:

async def get_s3_client():
    return boto3.client(
        "s3",
        endpoint_url=settings.AWS_S3_ENDPOINT_URL,
        aws_access_key_id=settings.AWS_ACCESS_KEY_ID,
        aws_secret_access_key=settings.AWS_SECRET_ACCESS_KEY,
        region_name=settings.AWS_REGION,
        config=Config(signature_version="s3v4", s3={"addressing_style": "path"}),
    )

Enter fullscreen mode Exit fullscreen mode

Another client is specifically for pre-signed URLs:

async def get_s3_presign_client():
    endpoint_url = (
        settings.AWS_S3_PUBLIC_URL
        or settings.AWS_S3_ENDPOINT_URL
    ).rstrip("/")

    return boto3.client(
        "s3",
        endpoint_url=endpoint_url,
        aws_access_key_id=settings.AWS_ACCESS_KEY_ID,
        aws_secret_access_key=settings.AWS_SECRET_ACCESS_KEY,
        region_name=settings.AWS_REGION,
        config=Config(signature_version="s3v4", s3={"addressing_style": "path"}),
    )

Enter fullscreen mode Exit fullscreen mode

That is the clean version of what I learned the painful way:

Sign the URL with the hostname the browser will actually request.

If the browser requests https://media.example.com/app-media/file.webp, the pre-signing client should also use https://media.example.com. Not http://seaweed:8333. Not an IP. Not a Docker service name. The public host.

Why SeaweedFS Hostnames Break Pre-Signed URLs

This is the thing I keep needing to re-explain to myself.

A pre-signed URL is not just “a URL with a token”. It is a signed representation of a specific HTTP request. The signature is calculated from pieces like:

HTTP method
path
query params
headers
host
expiry
secret key

Enter fullscreen mode Exit fullscreen mode

So if boto3 signs this:

GET http://seaweed:8333/app-media/image.webp

Enter fullscreen mode Exit fullscreen mode

and the browser actually requests this:

GET https://media.example.com/app-media/image.webp

Enter fullscreen mode Exit fullscreen mode

SeaweedFS receives a request whose host does not match the signed host. The storage server recalculates the signature from the request it received, compares it to the signature in the URL, and says no.

This is why the fix is not “replace the hostname in the string”.

The fix is:

Use internal endpoint for backend upload/download.
Use public endpoint for presigned URL generation.
Make nginx/proxy route public endpoint to the storage service without changing path/query.

Enter fullscreen mode Exit fullscreen mode

That is the shape.

SeaweedFS Public URL Helper

Not every file needs a pre-signed URL. Some generated assets can be public, and for those I wanted a fast path that did not sign anything:

import urllib.parse

async def generate_public_s3_url(s3_key: str) -> str | None:
    if not s3_key:
        return None

    bucket_name = settings.AWS_BUCKET_NAME
    encoded_key = urllib.parse.quote(s3_key, safe="/")
    public_endpoint = (
        settings.AWS_S3_PUBLIC_URL
        or settings.AWS_S3_ENDPOINT_URL
    ).rstrip("/")

    return f"{public_endpoint}/{bucket_name}/{encoded_key}"

Enter fullscreen mode Exit fullscreen mode

This is not a security feature. This is the opposite: it assumes the object is public or the endpoint is allowed to serve it without a signature. But it matters because not all media in an app has the same privacy level.

Private generated content gets pre-signed URLs.

Public thumbnails, static generated samples, and assets meant to be indexed can use clean public URLs.

One storage backend. Two access patterns.

Caching SeaweedFS Pre-Signed URLs in Redis

Pre-signing is cheap, but not free. Also, signing on every list endpoint is wasteful if you are rendering dozens of media items. So I cached pre-signed URLs in Redis, always with a TTL shorter than the actual expiry:

async def generate_presigned_url(s3_key, expires_in=None, bucket_name=None):
    if not s3_key:
        return None

    expires_in = expires_in or settings.S3_PRESIGNED_EXPIRES

    cached = await redis_cache.get_presigned(s3_key)
    if cached:
        return cached

    s3_client = await get_s3_presign_client()
    bucket = bucket_name or settings.AWS_BUCKET_NAME

    url = s3_client.generate_presigned_url(
        ClientMethod="get_object",
        Params={"Bucket": bucket, "Key": s3_key},
        ExpiresIn=expires_in,
    )

    cache_ttl = min(max(expires_in - 60, 0), 86400)
    await redis_cache.set_presigned(s3_key, url, ttl=cache_ttl)
    return url

Enter fullscreen mode Exit fullscreen mode

The expires_in - 60 part is the boring bit that prevents edge-of-expiry bugs. You do not want to hand the frontend a URL that technically has three seconds left because Redis still had it. Shave a minute off and sleep better.

I also learned from the MinIO migration that cache keys should be treated as infrastructure-coupled. If you change storage hostnames, old pre-signed URLs can survive in Redis and point at the wrong world. The fix is either clear the cache during migration or store enough metadata to reject stale URLs.

SeaweedFS Smoke Tests Before Production

I do not trust storage because config “looks right” anymore. I trust storage after a script puts bytes in and gets the same bytes out.

def smoke_storage():
    client = get_s3_client()
    bucket = os.environ["AWS_BUCKET_NAME"]
    key = "smoke/test.txt"
    data = b"hello-storage"

    client.put_object(
        Bucket=bucket,
        Key=key,
        Body=data,
        ContentType="text/plain",
    )

    got = client.get_object(Bucket=bucket, Key=key)["Body"].read()
    assert got == data

    url = client.generate_presigned_url(
        ClientMethod="get_object",
        Params={"Bucket": bucket, "Key": key},
        ExpiresIn=300,
    )
    print(url)

    client.delete_object(Bucket=bucket, Key=key)

Enter fullscreen mode Exit fullscreen mode

Then I test the public URL path too:

curl -I "https://media.example.com/app-media/smoke/test.txt"

Enter fullscreen mode Exit fullscreen mode

And if it is pre-signed:

curl -I "$PRESIGNED_URL"

Enter fullscreen mode Exit fullscreen mode

The failure messages tell you where to look:

AccessDenied              credentials or bucket policy
NoSuchBucket              bucket was never created or wrong bucket env
NoSuchKey                 object key mismatch
SignatureDoesNotMatch     host/path/query changed after signing
Connection refused        endpoint/port/proxy problem
SSL error                 public URL/TLS/proxy certificate problem

Enter fullscreen mode Exit fullscreen mode

This little table alone would have saved me hours.

SeaweedFS Environment Variables for FastAPI

The env vars that mattered were basically these:

AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...
AWS_BUCKET_NAME=app-media
AWS_REGION=us-east-1

# Internal service URL. Backend uses this for direct storage operations.
AWS_S3_ENDPOINT_URL=http://seaweed:8333

# Public browser URL. Presigned URLs should be generated against this host.
AWS_S3_PUBLIC_URL=https://media.example.com

S3_PRESIGNED_EXPIRES=3600
REDIS_URL=redis://redis:6379/0

Enter fullscreen mode Exit fullscreen mode

The names still say AWS_, which is a little funny because AWS is exactly what I was not using. I kept them because boto3 and old code already spoke that language. Sometimes consistency beats semantic purity.

Future-me note: if public files are broken, inspect AWS_S3_PUBLIC_URL. If backend uploads are broken, inspect AWS_S3_ENDPOINT_URL. If pre-signed URLs generate but fail in browser, inspect the host in the generated URL before touching the upload code.

Why SeaweedFS Felt Harder Than MinIO

SeaweedFS was harder because it has more nouns.

MinIO gives you server, bucket, object.

SeaweedFS gives you master, volume, filer, S3 gateway, bucket, collection, replication, filer metadata, public endpoint, internal endpoint, and sometimes IAM depending on how you deploy it.

That does not make it bad. It makes it less fake-simple.

With MinIO, I mostly thought about S3 compatibility.

With SeaweedFS, I had to think about how the storage system itself was shaped. Where does metadata live? Which endpoint should the app use? Which endpoint should the browser use? What happens when I add another volume server? Am I using the S3 layer as the main interface, or do I eventually want filer-level access too?

This is where SeaweedFS started feeling more powerful. It was not trying to be only a replacement for S3. It was giving me a storage architecture I could grow into.

Note — volume server: the SeaweedFS component that stores actual file content. The master knows which volume has which file id. This is part of why SeaweedFS can make file lookup efficient: once you know the volume id, finding the bytes is direct and cacheable.

The cost of that power is operational complexity. You have to understand more before you can debug confidently. When something fails, “is the S3 API up?” is not the only question. You may need to ask whether the filer can talk to volumes, whether the public endpoint is signing correctly, whether the bucket exists, whether the credentials map to the right permissions, and whether your reverse proxy is preserving the host the signature expects.

SeaweedFS Troubleshooting Checklist

If media suddenly breaks, I would debug from the outside inward.

1. Is the browser URL sane?

Does it use https?
Does it contain the expected bucket?
Does the hostname match AWS_S3_PUBLIC_URL?
Does it still point to an old MinIO/S3 host?

Enter fullscreen mode Exit fullscreen mode

2. Does the public endpoint answer?

curl -I https://media.example.com

Enter fullscreen mode Exit fullscreen mode

3. Can the backend talk to internal storage?

curl -I http://seaweed:8333

Enter fullscreen mode Exit fullscreen mode

4. Can boto3 list buckets from inside the backend environment?

client = await get_s3_client()
print(client.list_buckets())

Enter fullscreen mode Exit fullscreen mode

5. Is Redis handing out stale URLs?

Clear presigned URL cache after endpoint changes.
Never trust old cached URLs after storage migration.

Enter fullscreen mode Exit fullscreen mode

6. Is nginx preserving path and query string?

Pre-signed URLs need the exact path and query params. A proxy that drops query strings, normalizes paths, or redirects to a different host can break signatures.

This debugging order matters because storage failures can look like frontend bugs. The image tag fails. The video player spins. The gallery looks empty. But the real bug might be four layers down in a signature host mismatch.

Should You Use SeaweedFS Instead of MinIO?

Yes, but not automatically.

For a small app on one VM where the only requirement is “replace S3 quickly”, I would still reach for MinIO first. It is simpler, friendlier, and easier to explain to the next developer.

For a media-heavy app with lots of generated files, a future need for scaling storage horizontally, and a client who wants self-hosted infrastructure without buying into a commercial object-store subscription, I would seriously consider SeaweedFS again.

It is lower-level in the best and worst ways.

The best way: it gives you more control over the storage architecture than “one S3-compatible box with a console”.

The worst way: it makes you responsible for understanding that architecture.

The thing I liked most is that my application code did not have to become SeaweedFS-specific. It still spoke S3 through boto3:

s3_client.upload_fileobj(
    Fileobj=file_obj,
    Bucket=bucket_name,
    Key=s3_key,
    ExtraArgs={"ContentType": content_type},
)

Enter fullscreen mode Exit fullscreen mode

That is the trick I keep coming back to. The app should know as little as possible about the storage backend. Put the weirdness in one module. Make the rest of the code say “upload object”, “generate URL”, “delete object”, and move on with its life.

MinIO taught me that S3-compatible storage can save a migration.

SeaweedFS taught me that S3 compatibility is just the front door. Sometimes the building behind it is much bigger.

Frequently Asked Questions

Is SeaweedFS compatible with AWS S3?

SeaweedFS provides an S3-compatible API through its S3 gateway. That lets tools like boto3, AWS CLI, and S3-style pre-signed URL flows work without rewriting most application storage code.

Can FastAPI use SeaweedFS?

Yes. A FastAPI app can use SeaweedFS through boto3 by setting endpoint_url to the SeaweedFS S3 endpoint, using SigV4, and keeping path-style addressing enabled.

Is SeaweedFS better than MinIO?

SeaweedFS is not automatically better than MinIO. MinIO is usually simpler for single-server self-hosted object storage. SeaweedFS becomes more interesting when the app has lots of files, media-heavy workloads, public/internal endpoint needs, or a future path toward a more distributed storage topology.

Why do SeaweedFS pre-signed URLs fail with SignatureDoesNotMatch?

The most common reason is signing the URL with one hostname and requesting it through another. Sign pre-signed URLs using the public hostname the browser will actually request, such as https://media.example.com, not the internal Docker hostname like http://seaweed:8333.

What is AWS_S3_PUBLIC_URL used for?

AWS_S3_PUBLIC_URL separates the browser-facing storage endpoint from the backend’s internal storage endpoint. The backend can upload to AWS_S3_ENDPOINT_URL, while pre-signed URLs are generated against AWS_S3_PUBLIC_URL so the signature matches the public request.

Should I use SeaweedFS for self-hosted media storage?

SeaweedFS is worth considering for self-hosted media storage when you expect many generated files, want S3-compatible access from Python, and may need more flexible storage topology later. For a small app that only needs a simple S3-compatible bucket, MinIO is usually easier.