Safely Host Your Own S3 Server at Home: Ditch the Crazy AWS Bills with MinIO on Windows and SSL

MinioSSLDeployment.md

Host Your Own S3 Server at Home: Ditch the Crazy AWS Bills with MinIO on Windows and SSL

I'll show how to set up local S3 server at home that can be accessed via an HTTPS link, along with a guide on how to set it up! The following code is just an example of how it will look in result. Just like a regural AWS S3 endpoint. But hosted on your own machine. For free.

Contents

1. The Genesis of This Article
2. About AWS S3 Costs
3. Set Up Dynamic DNS with NoIP
4. Port 9000 Forwarding
5. Step 3: Installing and Configuring MinIO
6. Securing with SSL
7. Starting Your MinIO Server

The Genesis of This Article

My journey into video processing SaaS in the cloud began with an awareness of the massive data requirements from the outset. I was looking at handling 300-800 GB even during the development phase. Initial calculations of AWS S3 costs were surprisingly high, reinforcing its reputation for being costly.

Another compelling reason to store data locally was to fully utilize my existing home hardware as my project scaled. This includes everything from my RTX3090 GPU-equipped PC to older laptops, Nvidia Jetsons, Raspberry Pis, and more, essentially making them self-sustaining.

It seemed impractical to have users upload videos to the cloud only to download them again for processing. With a self-hosted S3 setup, users could directly feed videos into my LAN-based powerhouse, complete with GPUs.

About AWS S3 Costs

To give you an idea: suppose you want a service allowing 2TB of uploads monthly (equivalent to about 60 hours of 4K 120FPS GoPro footage) and 4TB of downloads (for sharing, different processing options, etc.). Here's the cost breakdown:

Using Amazon S3 for this would entail significant costs. For instance, storing 4TB of data, at $0.023 per GB in the up to 50TB tier, amounts to a monthly cost of $94.21. Download costs at $0.09 per GB add up to $368.64 for 4TB.

The total comes to approximately $462.86 per month , a substantial sum for any business. This led me to consider a self-hosted MinIO solution.

Consider the alternative: a 4TB external hard drive costs around $92 - just 20% of the estimated monthly S3 cost.

MinIO Setup

Let's go through the steps to replace the AWS S3 endpoint with a local MinIO server.

Step 1: Set Up Dynamic DNS with NoIP

First, a dynamic DNS service is essential to keep your server accessible, even if your home IP changes. NoIP offers a free hostname, which is ideal for our setup.

1. Sign up for a free plan at NoIP and create your hostname.
2. Access your router settings (usually at 192.168.0.1 or http://home).
3. Locate the DDNS settings and enter your NoIP credentials and hostname.

Step 2: Port Forwarding

Ensure external requests can reach your server:

1. In your router settings, forward port 9000 to your PC's IP address. This is the port MinIO uses.

Step 3: Installing and Configuring MinIO

Now, let's get MinIO up and running:

1. Download MinIO for Windows 64-bit from MinIO's website or choose a suitable version from their GitHub repository.
2. Install it on your Windows machine.
3. Change the default username and password via Command Prompt (Admin):

setx MINIO_ACCESS_KEY YourNewAccessKey -m
setx MINIO_SECRET_KEY YourNewSecretKey -m

4. Adjust your firewall settings to allow traffic on port 9000 and permit MinIO to use this port. The detailed steps involve opening Windows Firewall Settings, creating a new inbound rule for port 9000, and allowing MinIO through the firewall.

After setting up the firewall, verify its configuration by running the MinIO server and using a Python script on a different machine in your network to create a bucket in MinIO.

import boto3


s3 = boto3.resource('s3',
                   endpoint_url='https://your_noip_hostname:9000',
                   aws_access_key_id='your_minio_access_key',
                   aws_secret_access_key='your_minio_secret_key')


file_name = 'path_to_your_file'
bucket_name = 'your_bucket_name'
object_name = 'your_desired_object_name_in_s3'


# This line uploads the file to the minio on your Windows machine
s3.Bucket(bucket_name).put_object(Key=object_name, Body=open(file_name, 'rb'))

Step 4: Securing with SSL

We'll use Certbot to obtain a free SSL certificate:

1. Download and install Certbot.
2. Temporarily open and forward port 80 on your router for domain validation by Certbot.
3. Run Certbot to obtain your SSL certificates:

certbot certonly - standalone -d yournoipdomain.com -d www.yournoipdomain.com

You can close the port 80 now. 4. Rename the obtained certificates to public.crt and private.key and move them to C:/Users/your_username/.minio/certs. 5. Restart MinIO.

Step 5: Starting Your MinIO Server

Finally, start your MinIO server:

1. Open the terminal and navigate to the directory with your MinIO executable.
2. Start the MinIO server:

minio server testdir

Congratulations! Your Minio server is now operational, and you can use it just like AWS S3.

You can test the set up by uploading some file or creating the bucket using the code above.

Considerations for Self-Hosted S3 with MinIO

There are key considerations to keep in mind. Bandwidth limits from your ISP can affect server performance, especially with large data transfers. Additionally, managing backups manually is crucial as hard drives are susceptible to wear and tear. This approach requires a balance between cost savings and the additional responsibilities of maintaining a self-hosted solution. Cheers.