Show HN: I built open source file sharing solution using AWS S3

https://s3-file-share-for-free-35n2u.kinsta.app/

By rohitghumare at 4:37 AM

I created a 100% Open source Company-wide Self-hosted File Sharing Solution for Teams

Recently, I wanted to share HD images and video files with my graphic designer. She’s exceptional at her craft but isn’t familiar with AWS S3

So, I got an idea and built this.

Github Repo: https://github.com/rohitg00/s3-file-share-for-free

Detailed Guide: https://ghumare64.medium.com/i-built-a-company-wide-self-hos...

billev2k | 1 comment | 3 days ago

I'm pretty leery of making the "access key" and "secret key" so public (like typing them into a web page, or setting them in environment variables). Of course it adds significant friction to set up an IAM identity for every user, and "low friction" is one of the key requirements here.

ryanianian | 1 comment | 3 days ago

A "correct" implementation would give you a temporary IAM role or something (STS) based on a JWT or other authn mechanism.

This is not that difficult if you're already invested in an identity ecosystem, but a right pain without something to bootstrap it.

On the plus side, AWS creds can be made to be temporary and limited in scope to just the nouns/verbs required. Creating and vending those tokens is an exercise for the reader.

420official | 1 comment | 3 days ago

It really isn't that challenging to get going with JWT auth in AWS. Gitlab has pretty good documentation for how to use Gitlab ID tokens to assume roles that includes everything other than how to generate a JWT here: https://docs.gitlab.com/ee/ci/cloud_services/aws/

And of course generating OIDC PKI JWTs is pretty easy and well documented elsewhere.

The harder parts in my mind are:

  - Updating this OSS project to serve a JWK from OIDC .well-known
  - Convincing people that this method of authn is safe and that those keys are securely stored

rohitghumare | 0 comments | 3 days ago

I completely Agree on this point. I have this in mind for implementation. For now, I'm focusing on bringing more cloud providers.

jatins | 1 comment | 3 days ago

might be fine for internal company use but pasting access/secret key on a third party website will get you a call from security...or worse, won't

lizzas | 1 comment | 3 days ago

My initial though. At least create an IAM user per file :-). Maybe that defeats the convenience.

Cloning and installing is also an option.

rohitghumare | 1 comment | 3 days ago

I can add this option with just a few changes in a code, but how can it be easy to use for any team?

lizzas | 0 comments | 3 days ago

I am not an IAM expert but maybe the app should have an admin login that sets the IAM user with full permissions on any s3 bucket(s) needed for the app to work.

There should be instructions on how to set that IAM user up (dont make it the root! It just needs full access to a single bucket ideally).

bhawks | 2 comments | 3 days ago

Magic Wormhole gives you secure file transmission for free.

Give one of the implementations a try: https://github.com/psanford/wormhole-william

zikduruqe | 1 comment | 3 days ago

Why not S3 presigned URLs? It's already baked into the service anyways.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/ShareO...

gregw2 | 0 comments | 3 days ago

I agree with other posters that long lived non-autorotating IAM/S3 secret keys are not a good idea. A common alternate approach is presigned URLs. And not just on S3.

Google Cloud, Backblaze, Digital Ocean, Cloudflare, Azure all have this presigned URLs functionality too (I checked for the degree of lockin before I started using S3's presigned URLs in a set of bulk-data APIs at one place I've worked.)

GCP: https://cloud.google.com/storage/docs/access-control/signed-...

Digital Ocean: https://docs.digitalocean.com/products/spaces/how-to/set-fil...

Hetzner: https://docs.hetzner.com/storage/object-storage/faq/buckets-...

Backblaze: https://backblaze-prod.us.document360.io/apidocs/b2-get-down...

Cloudflare R2: https://developers.cloudflare.com/r2/api/s3/presigned-urls/

Azure: https://learn.microsoft.com/en-us/rest/api/storageservices/d...

etc.

Also, presigned URLs can be used not just for downloading files with a temporary URL but can also be created for uploading files with a temporary URL.

ryanianian | 1 comment | 3 days ago

Magic wormhole is great for live, peer-to-peer transfers.

But it is not great if you want to distribute a file multiple times, asynchronously, or with other functionality gained from centralized storage. This is where people typically use email, dropbox, or perhaps the tool from TFA.

nijave | 0 comments | 3 days ago

I think you can seed torrents via HTTP but you'd want to add a layer of file encryption in case someone discovers your torrent

Non-free Resilio Sync basically automated this

up2isomorphism | 1 comment | 3 days ago

You guys waste too much time building “open” stuff on S3.

nobodywillobsrv | 0 comments | 3 days ago

This is what I was wondering ... why start with s3? Why not "simply" build on IPFS or something like that as a start. Even a shared syncthing key would be MVP.

rohitghumare | 0 comments | 3 days ago

Quick Update: I didn't expect so many people to show interest in this project. As a few suggested, I plan to add more storage solutions in the next few days.

Testing Other Cloud Providers:

Backblaze B2 ($5/TB/month)

Wasabi ($6/TB/month)

Google Cloud Storage ($20/TB/month) - I am ready to deploy to production.

DigitalOcean Spaces ($5/TB/month)

Cloudflare R2 ($15/TB/month)

Hetzner Storage ($3/TB/month)

scarface_74 | 1 comment | 3 days ago

This seems like such a horrible, insecure idea that would never pass muster at any company I’ve ever worked at

woranl | 1 comment | 3 days ago

If you have a dedicated bucket just for this and the authentication is just for that bucket, then is it still considered insecure?

scarface_74 | 0 comments | 3 days ago

Even having long lived access keys and secret keys anywhere is insecure.

There really isn’t a need for them. If you are running the application on any AWS compute - EC2, Lambda, ECS, EKS, etc, there is an IAM role attached to the VM that gives code permission to run.

On the client facing side, it should be connected to your Orgs SSO solution so when a person leaves the company, you deactivate the user in one place.

Besides, I can’t think of any organization of even 2 people that isn’t already using Office365/OneDrive or Google/GSuite with plenty of shared storage. The cost per seat for either is $6 - $25 per user.

perching_aix | 1 comment | 3 days ago

I guess this is for smaller organizations with no MS365 subscription (and thus access to SharePoint)?

justmarc | 1 comment | 3 days ago

Does MS365 cover all potential use cases, needs and scenarios?

perching_aix | 0 comments | 3 days ago

All in the world? I suppose I haven't tried it for recipes, but I guess Copilot could help with that too.

As far as sharing files goes though, yes. I mean, it allows you to... share files, and do so in a controlled manner. Even edit them in-app as long as they're of a supported format.

iJohnDoe | 1 comment | 3 days ago

I obviously wouldn’t put my key into a third-party site. However, is there any concern with the self-hosted solution? Looks pretty cool and wouldn’t mind using it. Just make a key specifically for this?

rohitghumare | 0 comments | 2 days ago

Yes, feel free to try open-source version available on Github.

Just published a new release to support multiple cloud storage options.

Thanks

rohitghumare | 0 comments | 2 days ago

New Release is out now

> We support 7 cloud providers now. > Beta storage providers are still being tested.

Stable: Amazon S3 ($23/TB/month) Google Cloud Storage ($20/TB/month) Cloudflare R2 ($15/TB/month)

Beta/Testing: Wasabi ($6.99/TB/month) Backblaze B2 ($6/TB/month) DigitalOcean Spaces ($5/TB/month) Hetzner Storage ($4/TB/month)

mickael-kerjean | 3 comments | 3 days ago

Hi! I’m the author of another open-source project in the same space that I’ve been working on for a while: Filestash [1]

Were you familiar with Filestash before starting this? If so, was there something specific you felt was missing that inspired you to create your own solution? Would love to hear your thoughts.

github: https://github.com/mickael-kerjean/filestash

demo on s3: https://demo.filestash.app/login?type=s3&access_key_id=Q3AM3...

TwoNineFive | 0 comments | 3 days ago

Your reply comes off as arrogant and hijacking someone's announcement like this is tacky. "Were you aware I had already done this better. Why would you do something I already did? Did I mention I already did this?"

rohitghumare | 0 comments | 3 days ago

Never came across this, Interesting project. Thanks for sharing

lomkju | 1 comment | 3 days ago

Nice!

Could you tell me why Google Drive didn’t work for you?

Google Drive seems to cheaper and has better UX than S3 - $8 for 2 TB (India) - Supports file versioning - 750GB bandwidth per day

folmar | 0 comments | 3 days ago

If you do any less typical files Google Drive will balk at you at all times, at least for the other users who use the browser UI.

What works bad: * folder with lots of medium size files * large ZIP * video bigger than a few tens of megabytes

7bit | 1 comment | 4 days ago

Why should I use your solution and not Next cloud, for example?

rohitghumare | 0 comments | 4 days ago

Our S3 File Manager offers a lightweight, zero-configuration solution focused purely on S3 storage management, making it ideal for teams who need simple, cost-effective cloud storage without the overhead of a full collaboration suite like Nextcloud. While Nextcloud excels at comprehensive collaboration, our tool excels at simplicity and AWS integration. But we don't want to limit it to S3. We will try to include all storage by next month.

szszrk | 1 comment | 4 days ago

Hey, looks interesting and practical.

Any plans on making it compatible with other s3 implementation (other cloud vendors, local minio.io etc)?

rohitghumare | 1 comment | 4 days ago

Yes! Will release next version which supports all cloud providers from S3 to Hetzner Cloud storage!!

szszrk | 1 comment | 4 days ago

That is really neat. I had so many tiny use cases in my previous companies that could be solved by just a simple UI like this one. If you combine that with things like s3 static websites, it could be a beast that replaces some long-forgotten CMS solutions.

rohitghumare | 0 comments | 4 days ago

Completely Agree. I would appreciate it if you could add your ideas as an issue here: https://github.com/rohitg00/s3-file-share-for-free/issues