Technical description

Setting up Nginx reverse proxy for SSL

Setting up an Nginx web server-based reverse proxy as a frontend for Testlab is easy. This provides more performance and exposes only the webserver to the outside network.

  • How it works

    A reverse proxy is a simple (web) server component which listens to the requests from the internet and forwards the traffic to the actual service backend. With production installations of Testlab, we recommend that you set up an Nginx based reverse proxy that terminates the SSL connections and forwards the HTTP traffic to GlassFish.

  • Installation instructions

    This document gives installation instructions for Debian based systems. The instructions can be easily applied for other server distributions too as most of the configuration work is done on Nginx’s configuration files. In the instructions below, we are setting up the server

    • named “testlab.example.com” with
    • SSL private key in a file named “server.key” and
    • SSL certificate in a file named “server.cer”.

     

    Install Nginx binaries
    # sudo -i
    # apt-get install nginx
    ...
    # /etc/init.d/nginx stop

     

    Configure nginx.conf

    Edit the /etc/nginx/nginx.conf file and add the following:

         log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                         '$status $body_bytes_sent $request_time "$http_referer" '
                         '"$http_user_agent" "$http_x_forwarded_for"'; 
         client_max_body_size 50M; 
         server_tokens off;
         gzip_types text/plain text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;

    .. and remove by commenting out:

         #tcp_nopush on;

     

    Copy SSL key and certificates

    Copy the SSL related server private key file (server.key) and your SSL certificate file (server.cer) to /etc/nginx/ directory. For more information on SSL certificates, see the Nginx documentation.

     

    Create site configuration

    Create a new file /etc/nginx/sites-available/testlabproxy as:

    server {
     listen 80;
     server_name testlab.example.com;
     rewrite ^ https://$host$request_uri? permanent;
    }
    upstream testlab {
     # this must point to your glassfish (running in port 8080 here)
     server 127.0.0.1:8080 max_fails=0;
    }
    
    server {
     listen 443;
     server_name testlab.example.com;
     access_log /var/log/nginx/testlab.example.com.access.log main;
     error_log /var/log/nginx/testlab.example.com.error.log;
     gzip on;
     rewrite ^/$ https://$host/testlab/ permanent;
     rewrite ^/testlab$ https://$host/testlab/ permanent;
    
     ssl on;
     ssl_certificate server.cer;
     ssl_certificate_key server.key;
     ssl_session_cache shared:SSL:10m;
     ssl_session_timeout 10m;
     ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
     ssl_ciphers HIGH:!aNULL:!MD5:!kEDH;
     ssl_prefer_server_ciphers on;
    
     location /testlab/ {
      proxy_pass http://testlab;
      proxy_set_header X-Forwarded-For $remote_addr;
      proxy_set_header X-Forwarded-Ssl on;
      proxy_set_header Host $host;
      location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
       access_log off;
       proxy_pass http://testlab;
       proxy_set_header X-Forwarded-For $remote_addr;
       proxy_set_header X-Forwarded-Ssl on;
       proxy_set_header Host $host;
      }
     }
     location /testlab/server/comm {
      access_log off;
      proxy_pass http://testlab;
      proxy_set_header X-Forwarded-For $remote_addr;
      proxy_set_header X-Forwarded-Ssl on;
      proxy_set_header Host $host;
      proxy_buffering off;
      proxy_connect_timeout 75s;
      proxy_read_timeout 120s;
      proxy_send_timeout 120s;
      proxy_ignore_client_abort on;
     }
     location /api {
      proxy_pass http://testlab;
      proxy_set_header X-Forwarded-For $remote_addr;
      proxy_set_header X-Forwarded-Ssl on;
      proxy_set_header Host $host;
     }
     location /reportweb/ {
      proxy_pass http://testlab;
      proxy_set_header X-Forwarded-For $remote_addr;
      proxy_set_header X-Forwarded-Ssl on;
      proxy_set_header Host $host;
     }
     location /testlab/up {
      access_log off;
      proxy_pass http://testlab;
      proxy_set_header X-Forwarded-For $remote_addr;
      proxy_set_header X-Forwarded-Ssl on;
      proxy_set_header Host $host;
     }
    
     #
     # monitoring
     #
    
     location /testlab/monitoring {
      # add some ip here to allow monitoring endpoint access
      #allow some_ip_here;
      deny all;
      proxy_pass http://testlab;
      proxy_set_header X-Forwarded-For $remote_addr;
      proxy_set_header X-Forwarded-Ssl on;
      proxy_set_header Host $host;
     }
     location /reportweb/monitoring {
      # add some ip here to allow monitoring endpoint access
      #allow some_ip_here;
      deny all;
      proxy_pass http://testlab;
      proxy_set_header X-Forwarded-For $remote_addr;
      proxy_set_header X-Forwarded-Ssl on;
      proxy_set_header Host $host;
     }
    
     # saml-plugin, may be ignored if saml authentication is not in use
     location /saml-serviceprovider-plugin {
      proxy_pass http://testlab/saml-serviceprovider-plugin;
      proxy_set_header X-Forwarded-For $remote_addr;
      proxy_set_header X-Forwarded-Ssl on;
      proxy_set_header Host $host;
      proxy_redirect http://testlab.example.com/ https://testlab.example.com/;
     }
    
    }
    

    As a note: If you are deployed with saml-serviceprovider-plugin, note the last block and the proxy_redirect contained in it. The plugin fails to send redirects correctly if it is deployed behind a proxy. The proxy_redirect clause must be added to fix this by redirecting incorrectly sent “http://…” redirects to point to the proxy on SSL such as “https://testlab.example.com”.

     

    Testlab settings

    To make Testlab application aware of the reverse proxy in front of it, edit the ../domain1/lib/classes/testlab.properties file as follows:

    ...
    server.name=<full server name of your nginx proxy server>
    ...
    server.domain=<leave blank>
    ...
    server.scheme=https
    ...
    server.port.http=80
    ...
    server.port.https=443
    ...
    TokenRedirectServlet.allowed=report https://<nginx proxy server>/reportweb/generator
    ...

    Adjust ports and server names accordingly. The server.name setting should be the full hostname that is accessible by the browser. For example, if your Nginx proxy server is called as nginx.company.com, set the server.name property as “nginx.company.com”. If for some reason you are using non-standard ports (not 80 or not 443), adjust the port values in the above.

    After making changes to testlab.properties file, please restart GlassFish to make sure the settings are taken in effect.

     

    GlassFish HTTP listener setup

    Make sure that the GlassFish running Testlab is configured to listen to the HTTP port the above setup was configured to forward the traffic to. In addition, disable GZIP compression from the endpoint (if enabled) by editing ../domain1/config/domain.xml file:

    Change compression=”off” if enabled in:

    ...
        <http xpowered-by="false" max-post-size-bytes="22020096" 
          default-virtual-server="server" max-connections="250" 
          compression="off" compressable-mime-type="text/html,text/xml,text/plain,text/javascript,application/javascript,text/css">
    ...

    Restart GlassFish if any changes are made.

     

    Activate Nginx site

    Activate the created testlabproxy site and remove the default site with

    # cd /etc/nginx/sites-enabled
    # ln -s /etc/nginx/sites-available/testlabproxy testlabproxy
    # rm default
    # /etc/init.d/nginx start

    Nginx is now listening to 80 and 443 ports and redirects all non-SSL traffic over to SSL. To finalize configuration, make sure to set up a firewall to block all needed ports (other ports than 80 and 443).