Hosting services, like Storj, behind firewall/GNAT, using Wireguard on a VPS and DNAT (Port Forwarding) with iptables.

vps.md

DNAT with iptables over wireguard hosted on a VPS to host services behind a firewall/GNAT.

This is a short description of how to host services, using STORJ node as an example, on a host behind GNAT, or otherwise restrictive firewall, by forwarding packets through WireGuard endpoint on a relatively fast nearby VPS. This is not specific to Storj, and can be adopted to hosting other services.

As an example we will use an Oracle Cloud instance. Free tier still provides 10TB of monthly traffic that is sufficient for most node operators. Just make sure to create an account in a closest datacenter to minimize extra latency.

Notes on configuring the cloud instance

1. Create the oracle compute instance (ideally, Ampere, because they are awesome, but if that is not availabe, any other will do too).

2. Pick any OS you prefer, here we'll describe Ubuntu, as a most popular one.

3. Configure public IP address (this is the default), and upload SSH key to access the instance.

4. Then edit the Ingress Rules in the Default Security List in the VCN associated with the instance and rules to allow:

   • Traffic from anywhere 0.0.0.0/0, any port, to destination port 28967, one for udp, one for tcp. This is for storj.

   • UDP to port 51820, for WireGuard. It does not need to be this specific port, any will do, just adjust the rest accordingly. The source network can also be narrowed down to your ISP's address range, if desired.

     Statless	Source	IP Protocol	Source Port Range	Destination Port Range	Type and Code	Allows	Description
     No	0.0.0.0/0	TCP	All	28967		TCP Traffic for port 28967	Storj TCP
     No	0.0.0.0/0	UDP	All	28967		UDP Traffic for port 28967	Storj UDP
     No	0.0.0.0/0	UDP	All	51820		UDP Traffic for port 51820	Wireguard

   That's all that needs to be done in Oracle console.

5. Optionaly, configure the public IP as an A record on your DNS provider, to use DNS name and not an ugly IP address in the subsequent configuration and in your storj node.

Installing and configuring wireguard tunnel

1. ssh to your new instance, update software, and install wireguard:

   sudo apt update && sudo apt upgrade
   sudo reboot
   sudo apt install wireguard -y

2. Configure wireguard tunnel between your node and VPS. There are tons of tutorials, here are the steps for reference:

   On the VPS:

   1. Initialize the config file

      (umask 077 && printf "[Interface]\nPrivateKey= " | sudo tee /etc/wireguard/wg0.conf > /dev/null)
      wg genkey | sudo tee -a /etc/wireguard/wg0.conf | wg pubkey | sudo tee /etc/wireguard/publickey

   2. Add peer information (public key and address) after configuing it below

   3. Enable ipv4 forwarding: in /etc/sysctl.conf uncomment

      # Uncomment the next line to enable packet forwarding for IPv4
      net.ipv4.ip_forward=1   

      and for the change to take effect load it:

      sudo sysctl -p

      Note: it is possible to configure this key in a number of other configuration files, see man sysctl, but in this case either provide path to file to -p argument or simply use sudo sysctl --system, that will parse all configuration files.

   4. Enable and start the wireguard service:

      sudo systemctl enable wg-quick@wg0
      sudo systemctl start wg-quick@wg0

   On the client, assuming it's a TrueNAS, and storj runs in the jail, we would need few things:

   1. In the jail properties tick the allow_tun flag. (e.g. iocage set allow_tun=1 jailname)
   2. On the host under System -> Tunables add LOADER variable if_wg_load with the value YES, to load wireguard kernel module.
   3. Initialize the wireguard config file and create keys just like above, noting that in FreeBSD the default configuration file location is /usr/local/etc/wireguard/wg0.conf
   4. In the jail, in the /etc/rc.conf add

      wireguard_enable="YES"
      wireguard_interfaces="wg0"

   Generally, the config files shall look like so:

   1. On the server: /etc/wireguard/wg0.conf

      [Interface]
      PrivateKey = <server private key>
      
      ListenPort = 51820
      Address = 10.0.60.1
      
      # Allow WireGuard through the firwall
      PreUp = iptables -I INPUT 6 -p udp --dport 51820 -j ACCEPT
      PostDown = iptables -D INPUT -p udp --dport 51820 -j ACCEPT
      
      [Peer]
      PublicKey = <client public key>
      AllowedIPs = 10.0.60.2/32

   2. On the client: /usr/local/etc/wireguard/wg0.conf

      [Interface]
      PrivateKey = <client private key>
      Address = 10.0.60.2
      
      [Peer]
      PublicKey = <server public key>
      AllowedIPs = 10.0.60.1/32
      Endpoint = sub.example.com:51820
      PersistentKeepalive = 25

   3. Start the service on the client:

      service wireguard start

   At this point the client shall be able to ping the server, and the server shall be able to ping the clinet, at 10.0.60.1 and 10.0.60.2 addresses, respectively.

Packet forwarding

Now the very last thing, the meat of this tutorial. In the [Interface] section on the server in the /etc/wireguard/wg0.conf add the following PreUp and PostDown rules (PostDown rules are copies of PreUp rules, but with -A or -I options replaced with -D, to delete the rule):

# Allow WireGuard's own traffic to reach the server.
PreUp = iptables -I INPUT -p udp --dport 51820 -j ACCEPT
PostDown = iptables -D INPUT -p udp --dport 51820 -j ACCEPT

# Allow incoming Storj connections on the public interface BEFORE they are forwarded.
PreUp = iptables -I INPUT -p tcp --dport 51820 -j ACCEPT
PostDown = iptables -D INPUT -p tcp --dport 51820 -j ACCEPT
PreUp = iptables -I INPUT -p udp --dport 51820 -j ACCEPT
PostDown = iptables -D INPUT -p udp --dport 51820 -j ACCEPT

# Port forward incoming Storj traffic to the VPN client.
PreUp = iptables -t nat -I PREROUTING -i ens3 -p tcp --dport 51820 -j DNAT --to-destination 10.0.60.2:28967
PostDown = iptables -t nat -D PREROUTING -i ens3 -p tcp --dport 51820 -j DNAT --to-destination 10.0.60.2:28967
PreUp = iptables -t nat -I PREROUTING -i ens3 -p udp --dport 51820 -j DNAT --to-destination 10.0.60.2:28967
PostDown = iptables -t nat -D PREROUTING -i ens3 -p udp --dport 51820 -j DNAT --to-destination 10.0.60.2:28967

# Allow the now-forwarded traffic to pass from the public interface to the VPN interface.
PreUp = iptables -I FORWARD -i ens3 -o %i -m state --state RELATED,ESTABLISHED -j ACCEPT
PreUp = iptables -I FORWARD -i ens3 -o %i -p tcp -d 10.0.60.2 --dport 28967 -j ACCEPT
PreUp = iptables -I FORWARD -i ens3 -o %i -p udp -d 10.0.60.2 --dport 28967 -j ACCEPT
PostDown = iptables -D FORWARD -i ens3 -o %i -m state --state RELATED,ESTABLISHED -j ACCEPT
PostDown = iptables -D FORWARD -i ens3 -o %i -p tcp -d 10.0.60.2 --dport 28967 -j ACCEPT
PostDown = iptables -D FORWARD -i ens3 -o %i -p udp -d 10.0.60.2 --dport 28967 -j ACCEPT

# Allow outbound traffic from the VPN client out to the internet
PreUp = iptables -I FORWARD -i %i -o ens3 -j ACCEPT
PostDown = iptables -D FORWARD -i %i -o ens3 -j ACCEPT

# Perform NAT for traffic from the VPN client going to the internet
PreUp = iptables -t nat -I POSTROUTING -s 10.0.60.2/32 -o ens3 -j MASQUERADE
PostDown = iptables -t nat -D POSTROUTING -s 10.0.60.2/32 -o ens3 -j MASQUERADE

These achieve few things:

• Allow traffic to Wireguard port, so that your server can connect to establish the tunnel.
• Allow new tcp and udp connections to Storj port (We are inserting the rule in the very top of the chain; it’s enough to make sure it’s before rule 6, which on oracle instances is reject; so use -I, not -A).
• Port forward incoming Storj traffic to the VPN client.
• Allow the now-forwarded traffic to pass from the public interface to the VPN interface
• Allow outbound traffic from the VPN client out to the internet.
• Perform NAT for traffic from the VPN client going to the internet

On the server, restart the wireguard service:

sudo systemctl restart wg-quick@wg0

On the client, restart the wireguard service:

service wireguard restart

In the config.yaml of the storage node modify the external address to point to your vps:

# the public address of the node, useful for nodes behind NAT
contact.external-address: sub.example.com:28967

[Re]start the node, and check the status page. It shall be now happily connected.

Is there a way to do everything on the "Storj client computer" within the Dockerfile or within a docker-compose, so that it is fully modular and does not impact the host OS on the Storj client computer.

Beautiful!! I was able to make it work with storj

any videos on this ?

It would be the most boring video on the planet -- some dude copy pasting commands and clicking buttons :) You can do it from text at your own pace ;) Don't forget to click subscribe and hit the bell icon to get notified about all future videos!

Is there a way to do everything on the "Storj client computer" within the Dockerfile or within a docker-compose, so that it is fully modular and does not impact the host OS on the Storj client computer.

Sure. I guess you could create your own image based on the storagenode image, and add wireguard configuration as a separate layer. Still, someone needs to configure the server part of it, and it's easier to do manually since it only needs to done once.

Thank you for this framework. I recently discovered Technitium DNS (recommended!) and would like to move my Oracle Free tier to docker with WG-Easy and a Technitium docker instance. I have the WG up and running and can access the GUI interface from either the WG or WG Container IP. (first real container project!). Now i am try to do the same with Technitium. It is installed and running. I can get a nslookup reply from the IP listed in the container. But I cannot get to the GUI using an internal IP (either that of the WG or the container) nor can I get it to reply to a DNS query. Do I need pre/post rules to get from the WG IP to the Technitium IP or some type of networking configuration between containers on the VM ?