Steps to configure a transparent NGINX reverse proxy

nginx_reverse_proxy.md

Nginx reverse proxy

Simple insturctions on how to setup a nginx transparent reverse proxy.

• What does it do?
• How does it work?
• How to set it up?
  • Proxied VM configuration
    • HTTP
    • HTTPS
  • Proxy VM configuration
    • VM configuration
    • Nginx configuration
    • Hosts config

What does it do?

It allows you to have a single VM with a static public IP address, which redirects requests to other VMs on the same subnet keeping the original requester's IP address.

How does it work?

The process is very simple:

• create an A domain proxy.foo
• create a C domain for your product awesome.foo pointing to proxy.foo
• on the proxy VM, update /etc/hosts so awesome.foo points to the right VM's static ip address
• nginx receives the requests to awesome.foo, resolves it using /etc/hosts and sends it to the right VM.

How to set it up?

The configuration isn't straightforward at the very beginning, but it's quite doable.

NOTE: each client (aka. VM) behind proxy will need a special configuration as well.

Proxied VM configuration

HTTP

This is to set the real IP address in nginx logs:

set_real_ip_from XXX.XXX.XXX.XXX; # Proxy's IP address on the same subnet
real_ip_header X-Forwarded-For;
real_ip_recursive on;

NOTE: it is important to set that as some crawlers will attempt to fetch via http and ignore the redirect.

HTTPS

This must be present in nginx' configuration or the page won't load at all:

listen 443 proxy_protocol ssl;

set_real_ip_from XXX.XXX.XXX.XXX; # Proxy's IP address on the same subnet
real_ip_header proxy_protocol;

Proxy VM configuration

VM configuration

Many cloud providers like to overwrite /etc/hosts on each reboot, which completely breaks this system. So, let's avoid that.

cat /etc/cloud/cloud.cfg | grep -v update_etc_hosts > /tmp/cloud.cfg
rm /etc/cloud/cloud.cfg
mv /tmp/cloud.cfg /etc/cloud/cloud.cfg
chmod a+r /etc/cloud/cloud.cfg

To make it a little faster I used the already present systemd-resolved, but any similar service will work.

ALSO: to be 100% sure you may want to limit all outgoing traffic from your VM, to avoid it being abused by malicious third parties.

Nginx configuration

Put all of this straight into /etc/nginx/nginx.conf:

### Config
###############################################################################
user              nginx;
worker_processes  1;
pid               /var/run/nginx.pid;
error_log         /dev/null;
load_module /usr/lib/nginx/modules/ngx_stream_module.so;

events
{
  worker_connections 1024;
}





### HTTP
###############################################################################
http
{
  include            /etc/nginx/mime.types;
  default_type       application/octet-stream;
  log_format main    '$remote_addr - $remote_user [$time_local] "$request" '
                       '$status $body_bytes_sent $http_referer '
                       '"$http_user_agent" "$http_x_forwarded_for"';
  sendfile           on;
  keepalive_timeout  65;

  server
  {
    listen 80 default_server;

    access_log /dev/null;
    error_log  /dev/null;

    resolver 127.0.0.53;

    location /
    {
      proxy_bind $binary_remote_addr transparent;
      proxy_pass $scheme://$host:$server_port;
      proxy_set_header Upgrade           $http_upgrade;
      proxy_set_header Connection        "upgrade";
      proxy_set_header X-Real-IP         $remote_addr;
      proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_set_header X-Forwarded-Host  $host;
      proxy_set_header X-Forwarded-Port  $server_port;
    }
  }
}



### HTTPS
###############################################################################
stream
{
  resolver 127.0.0.53;

  server
  {
    listen 443;

    error_log /dev/null;

    ssl_preread on;

    proxy_protocol on;

    proxy_bind $binary_remote_addr transparent;
    proxy_connect_timeout 5s;
    proxy_pass $ssl_preread_server_name:$server_port;
  }
}

Hosts config

Be sure to set the right hostname resolutions in /etc/hosts, for example:

127.0.0.1 localhost
127.0.1.1 localhost.localdomain localhost

# VM containing another project (YYY.YYY.YYY.YYY is its address on the same subnet):
YYY.YYY.YYY.YYY awesome.foo