Setting up Nginx as a reverse proxy for Apache with SSL termination

Reverse Proxy diagram from Wiki Commons

We're currently hosting client sites on a Rackspace server and using their Load Balancer feature as a way to terminate SSL and not have to have multisite certificates.

We only attach one node to the Load Balancer so we're paying for more than we're using.  My proof of concept is to use Nginx to terminate SSL certificates and proxy to the Apache server.  This will save us £ 225 per load balancer, and since we're using ten of them that's quite a significant saving.

My first step was to spin up a free tier EC2 instance running Ubuntu 14.04 LTS.  I guess you can replace this with your favourite cloud or on-the-metal server.

Then I installed my packages. These are the ones I remember so YMMV.

 sudo apt-get install nginx apache2 fail2ban php5-fpm mcrypt php5-mcrypt openssl php5-cli php5 libapache2-mod-php  

My network diagram is slightly different from the picture for this post in that the web server is hosted on the same machine as the proxy.

I decided to run Apache on port 8000 and listen only to connections from localhost. Nginx would listen on port 80 and forward requests to Apache. I decided to let Nginx serve static content because it's pretty quick at doing so and this saves Apache from being overwhelmed by requests.

Configuring Apache

My first port of call was to edit /etc/apache2/ports.conf and make sure that my Listen line looks like this: Listen 127.0.0.1:8000

Then I created two virtual hosts to test with.  Here's a sample:

 <VirtualHost *:8000>  
      ServerName dummy1.mydomain.co.uk  
      ServerAdmin webmaster@localhost  
      DocumentRoot /var/www/dummy3/  
      ErrorLog ${APACHE_LOG_DIR}/dummy3_error.log  
      CustomLog ${APACHE_LOG_DIR}/dummy3_access.log combined  
 </VirtualHost>  

I made a simple index.php file in two new directories /var/www/dummy1 and /var/www/dummy3 which just output two server variables for me to test with. I also copied an image file into those directories so that I could check how static assets would be served.

 <?php  
 echo $_SERVER['SCRIPT_FILENAME'] . '<br>';  
 echo $_SERVER['SERVER_SOFTWARE'] . '<br>';  

Configuring Nginx

I decided to use self-signed certificates for testing and reserve dummy2 for a trial run of a free ssl certificate.  There are quite a few certificate signers who will give you a 30 day certificate to trial.

I created an /etc/nginx/ssl directory because for some reason I don't like to contaminate my conf.d directory and made subdirectories for my sites under that.

I created self-signed certificates (commands are at the top of my host file) and set up the vhosts like this:

	# Read
	# http://wiki.nginx.org/Pitfalls
	# http://wiki.nginx.org/QuickStart#
	# http://tautt.com/best-nginx-configuration-for-security/
	# https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html
	#
	# Generate your key with: openssl dhparam -out /etc/nginx/ssl/dhparam.pem 4096
	# Generate certificate: sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/nginx/ssl/nginx.key -out /etc/nginx/ssl/nginx.crt
	server {
	listen 80;
	listen [::]:80;
	server_name dummy3.yourdomain.co.uk;
	root /var/www/dummy3;
	index index.php;
	access_log /var/log/nginx/dummy3.access.log;
	error_log /var/log/nginx/dummy3.error.log;
	# http://www.gnuterrypratchett.com/#nginx
	add_header X-Clacks-Overhead "GNU Terry Pratchett";
	location / {
	# include /etc/nginx/naxsi.rules;
	# try_files $uri $uri/;
	}
	# Do not log favicon.ico requests
	location = /favicon.ico {
	log_not_found off;
	access_log off;
	}
	# Do not log robots.txt requests
	location = /robots.txt {
	allow all;
	log_not_found off;
	access_log off;
	}
	location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
	expires max;
	proxy_buffering on;
	proxy_cache_valid 200 120m;
	log_not_found off;
	}
	# Redirect 403 errors to 404 error to fool attackers
	error_page 403 = 404;
	# Deny all attempts to access hidden files such as .htaccess, .htpasswd, .DS_Store (Mac).
	# Keep logging the requests to parse later (or to pass to firewall utilities such as fail2ban)
	location ~ /\. {
	deny all;
	}
	location ~ \.php$ {
	proxy_set_header X-Real-IP $remote_addr;
	proxy_set_header X-Forwarded-For $remote_addr;
	proxy_set_header Host $host;
	proxy_pass http://127.0.0.1:8000;
	}
	}
	# =================================================================================================
	server {
	listen 443;
	listen [::]:443;
	ssl on;
	server_name uat.poppyraffle.org.uk;
	root /var/www/mount/uat.poppyraffle.org.uk/public;
	index index.php;
	# config to enable HSTS(HTTP Strict Transport Security) https://developer.mozilla.org/en-US/docs/Security/HTTP_Strict_Transport_Security
	# to avoid ssl stripping https://en.wikipedia.org/wiki/SSL_stripping#SSL_stripping
	add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";
	# SSL Certificate files
	ssl_certificate ssl/dummy3/dummy3.crt;
	ssl_certificate_key ssl/dummy3/dummy3.key;
	ssl_dhparam /etc/ssl/certs/dhparam.pem;
	# General SSL settings
	ssl_protocols TLSv1.2;
	ssl_ciphers HIGH+kEECDH+AESGCM:HIGH+kEECDH:HIGH+kEDH:HIGH:!aNULL;
	ssl_prefer_server_ciphers on;
	keepalive_timeout 60;
	ssl_session_cache shared:SSL:10m;
	ssl_session_timeout 10m;
	# enable OSCP stapling (dummy3 crt must have intermediate key in it)
	ssl_stapling on;
	# ssl_stapling_verify on;
	resolver 8.8.8.8;
	ssl_trusted_certificate ssl/dummy3/dummy3.crt;
	# enable session resumption to improve https performance
	# http://vincent.bernat.im/en/blog/2011-ssl-session-reuse-rfc5077.html
	ssl_session_cache shared:SSL:50m;
	ssl_session_timeout 5m;
	# http://www.gnuterrypratchett.com/#nginx
	add_header X-Clacks-Overhead "GNU Terry Pratchett";
	access_log /var/log/nginx/roy002-raffle.access.log;
	error_log /var/log/nginx/roy002-raffle.error.log;
	# Deny all attempts to access hidden files such as .htaccess, .htpasswd, .DS_Store (Mac).
	# Keep logging the requests to parse later (or to pass to firewall utilities such as fail2ban)
	location ~ /\. {
	deny all;
	}
	location ~* \.(js|css|png|jpg|jpeg|gif|ico|txt)$ {
	expires max;
	proxy_buffering on;
	proxy_cache_valid 200 120m;
	log_not_found off;
	}
	# Do not log favicon.ico requests
	location = /favicon.ico {
	log_not_found off;
	access_log off;
	}
	# Do not log robots.txt requests
	location = /robots.txt {
	allow all;
	log_not_found off;
	access_log off;
	}
	# Try to serve files with Nginx if possible
	location / {
	try_files $uri @proxy;
	}
	# Pass on php to Apache
	location ~* \.php$ {
	fastcgi_split_path_info ^(.+\.php)(/.+)$;
	proxy_pass http://127.0.0.1:8000;
	include /etc/nginx/proxy_params;
	}
	location @proxy {
	proxy_pass http://127.0.0.1:8000;
	include /etc/nginx/proxy_params;
	}
	# Redirect 403 errors to 404 error to fool attackers
	error_page 403 = 404;
	}

view raw dummy3_reverse_proxy.conf hosted with ❤ by GitHub

	proxy_set_header Host $http_host;
	proxy_set_header X-Real-IP $remote_addr;
	proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	proxy_set_header X-Forwarded-Proto $scheme;
	proxy_redirect off;
	client_max_body_size 10m;
	client_body_buffer_size 128k;
	proxy_connect_timeout 90;
	proxy_send_timeout 90;
	proxy_read_timeout 90;
	proxy_buffers 32 4k;

view raw proxy_params hosted with ❤ by GitHub

Now when I hit a static file on HTTP or HTTPS then Nginx serves it up directly. Inspecting the response headers with your favourite browsers debug tool will confirm that images are served by Nginx. Visiting the index file shows that it's loading the correct one and is being handled by Apache. Lastly, checking the certificate will show you that each site is using its own certificate.

That has the potential of saving my company £ 2200, which is a happy thing to be able to do in your first week while your boss is watching :)