Recently I decided to make a number of my services externally available, and so the need arose to put a reverse proxy in place to correctly direct queries to the appropriate server. This guide will present the way I configured this, and attempt to explain some of the design choices along the way. It’s aimed at beginners, so no prior knowledge of these services will be assumed and every effort will be made to explain what each configuration option does.
Design
So, I guess the first place to start is what is a reverse proxy, and why do you need one? In simplest terms, a reverse proxy is a type of proxy server that retrieves a resource on behalf of a client from one or more services. To illustrate this with a practical example, lets assume that I host two services on my network, and I want both to be externally available at the domains cloud.example.com and bitwarden.example.com. Unless I want to specify a port to access at the end of one of these domains, i.e. bitwarden.example.com:4343, both will need to be available on ports 80 and 443. It’s not possible to host two services on the same ports directly, and so this is where the reverse proxy comes in. The reverse proxy is hosted on ports 80 and 443, and it inspects the Host header in each request to determine which service to forward the request on to. This configuration looks like this:
As you can see, a request to the domain name is made from the internet, this is then forwarded by the router to the reverse proxy server, which determines which server the request is to go to. Additionally, this is a good opportunity to introduce SSL termination. This means that the reverse proxy handles all of the certificates for the servers it proxies to, instead of each service managing their own certificate. I’ve found this immensely useful, as it reduces the management load of configuring SSL for every service that I set up. Instead, I obtain a wildcard certificate (*.example.com) and configure it on the proxy server. This way, all hosts with a subdomain of example.com are covered under the certificate and the SSL configurations can be managed in one place.
Now that we know the problem a reverse proxy solves, lets set one up.
Jail Configuration
We’re going to run the reverse proxy in its own jail so that it can be managed easily in isolation from other services. To do this, SSH into your FreeNAS host. If you’re not sure how to do this, you can follow this guide to set it up. Assuming your FreeNAS host is on IP 192.168.0.8:
ssh root@192.168.0.8
If you’re using Windows, you’ll need to use PuTTY or WSL or some other unix emulator. Refer to the above guide for more detail.
Create the jail
Once you’ve established a SSH connection, you can create the jail as follows:
iocage create -n reverse-proxy -r 11.2-RELEASE ip4_addr="vnet0|192.168.0.9/24" defaultrouter="192.168.0.1" vnet="on" allow_raw_sockets="1" boot="on"
To break this down into it’s consituent components:
iocage create: calls on theiocagecommand to create a new iocage jail-n reverse-proxy: gives the jail the name ‘reverse-proxy’-r 11.2-RELEASE: specifies the release of FreeBSD to be installed in the jail.ip4_addr="vnet0|192.168.0.9/24": provides the networking specification; an IP/mask for the jail, and the interface to use,vnet0. This should be something convenient to you on the subnet you wish it to be on. The selection is arbitrary, though if you’re new to this it’s advisable for simplicity to select something on the same subnet as your router.defaultrouter="192.168.0.1": specifies the router for your network, change this as is relevant for youvnet="on": enables the vnet interfaceallow_raw_sockets="1": enables raw sockets, which enables the use of programs such astracerouteandpingwithin the jail, as well as interactions with various network subsystemsboot="on": enables the jail to be auto-started at boot time.
More detail on the parameters that can be used to configure a jail on creation can be found in the man page for iocage
Now to see the status of the newly created jail, execute the following:
iocage list
This will present a print out similar to the following:
+-----+---------------+-------+--------------+----------------+
| JID | NAME | STATE | RELEASE | IP4 |
+=====+===============+=======+==============+================+
| 1 | reverse-proxy | up | 11.2-RELEASE | 192.168.0.9 |
+-----+---------------+-------+--------------+----------------+
Enter the jail by taking note of the JID value and executing the following:
jexec <JID> <SHELL>
For example,
jexec 1 tcsh
Install nginx
Begin the installation process by updating the package manager, and installing nginx (the web server we’re going to use for the reverse proxy) along with the nano text editor and python:
pkg update
pkg install nginx nano python
Enable nginx so that the service begins when the jail is started
sysrc nginx_enable=yes
SSL/TLS Termination
Since the rest of this procedure involves making some decisions about whether or not to use SSL/TLS termination, we’ll discuss it here.
This guide is going to assume that the reverse proxy will be responsible for maintaining the certificates for all of the servers that it proxies to. This does not have to be the case, however. An equally valid configuration would be to have each of the servers handle their own certificates and encryption, or some combination of both. I won’t address these alternatives in this guide, however with a small amount of research the instructions here shouldn’t be too difficult to adapt to your use case.
Additionally, this configuration will use a wildcard certificate. That is, a certificate for the domain *.example.com, which is valid for all subdomains of example.com. This will simplify the process, as only one certificate needs to be obtained and renewed. However, one requirement of obtaining a wildcard certificate from LetsEncrypt is that a DNS-01 challenge is used to verify ownership for the domain. This means that HTTP-01 challenges cannot be used with this method, meaning that you must be using a DNS service that gives you control over your DNS records, or an API plugin to allow for DNS challenges. Certbot have published a list of supported DNS plugins that will enable you to perform a DNS challenge directly. If you’re using one of these providers, I recommend using these. Alternatively, if your DNS provider does not have a plugin, but you have access to edit the DNS records, you can manually configure a TXT record, as described in the certbot documentation. If neither of these alternatives are sufficient for you, acme.sh is a script that has perhaps wider compatability for a range of DNS Providers. Specific compatability is detailed in this community maintained list.
Optionally, you could obtain a certificate for each subdomain that you wish to host and use HTTP-01 challenge validation. This does not require a plugin, and there are a range of ways to do this as described in the LetsEncrypt documentation. There are some basic instructions in this certbot guide, however more research may be required.
To reiterate, this guide will deal only with obtaining a wildcard certificate using a DNS-01 challenge. The DNS provider I use is AWS Route 53, and so this is the plugin I will use.
Certbot installation
Now, lets install certbot. Certbot is free, open source tool for obtaining and maintaining LetsEncrypt certificates. Install it as follows:
pkg install py37-certbot openssl
Additionally, you’ll need to install the appropriate plugin for DNS validation. To show a list of available plugins, execute:
pkg search certbot
At the time of writing, the (relevant) list of results looks like follows:
py37-certbot-1.0.0,1 Let's Encrypt client
py37-certbot-apache-1.0.0 Apache plugin for Certbot
py37-certbot-dns-cloudflare-1.0.0 Cloudflare DNS plugin for Certbot
py37-certbot-dns-cloudxns-1.0.0 CloudXNS DNS Authenticator plugin for Certbot
py37-certbot-dns-digitalocean-1.0.0 DigitalOcean DNS Authenticator plugin for Certbot
py37-certbot-dns-dnsimple-1.0.0 DNSimple DNS Authenticator plugin for Certbot
py37-certbot-dns-dnsmadeeasy-1.0.0 DNS Made Easy DNS Authenticator plugin for Certbot
py37-certbot-dns-gehirn-1.0.0 Gehirn Infrastructure Service DNS Authenticator plugin for Certbot
py37-certbot-dns-google-1.0.0 Google Cloud DNS Authenticator plugin for Certbot
py37-certbot-dns-linode-1.0.0 Linode DNS Authenticator plugin for Certbot
py37-certbot-dns-luadns-1.0.0 LuaDNS Authenticator plugin for Certbot
py37-certbot-dns-nsone-1.0.0 NS1 DNS Authenticator plugin for Certbot
py37-certbot-dns-ovh-1.0.0 OVH DNS Authenticator plugin for Certbot
py37-certbot-dns-rfc2136-1.0.0 RFC 2136 DNS Authenticator plugin for Certbot
py37-certbot-dns-route53-1.0.0 Route53 DNS Authenticator plugin for Certbot
py37-certbot-dns-sakuracloud-1.0.0 Sakura Cloud DNS Authenticator plugin for Certbot
py37-certbot-nginx-1.0.0 NGINX plugin for Certbot
Install the relevant plugin to you. For me, as mentioned, this is Route 53:
pkg install py37-certbot-dns-route53
Configure DNS plugin
To use the DNS plugin, you’re likely going to have to configure it. Consult the documentation for your relevant plugin. The py37-certbot-dns-route53 documentation lists the available methods to configure the Route 53 plugin, however Amazon have conveniently provided us with a CLI tool that will do it for us:
pkg install awscli
Before configuring it, you’ll need to create a Key Pair to provide, and limit, access to your AWS console. Bear in mind that if this server is compromised, the perpetrator will have access to this, so limiting the access this key pair has is advisable. The plugin documentation indicates that the following permissions are required:
route53:ListHostedZonesroute53:GetChangeroute53:ChangeResourceRecordSets
Now, initiate the configuration process:
aws configure
This will prompt you for four pieces of information:
- AWS Access Key ID: From the key pair
- AWS Secret Access Key: From the key pair
- Default Region Name: The region closest to you, i.e. us-west-2. This should be available in your AWS dashboard
- Default output format: text
Now, your configuration should be present in ~/.aws/config, and your credentials should be present in ~/.aws/credentials.
Request a wildcard certificate
To obtain a certificate, simply execute the following command:
certbot certonly --dns-route53 -d '*.example.com'
This will undertake a DNS-01 challenge to verify access to the domain you substitute for example.com using the credentials in the plugin that you set up previously.
Configure certificate auto-renewal
LetsEncrypt certificates are only valid for 90 days. To prevent these expiring, and having to manually repeat renew it, we can automate the renewal process. To do this, we’re going to add a cron job, which is essentially a command that runs at a specified interval. Set your default editor to nano and open up the crontab, where cron jobs are registered:
setenv EDITOR nano
crontab -e
Add the following line:
0 0,12 * * * /usr/local/bin/python -c 'import random; import time; time.sleep(random.random() * 3600)' && /usr/local/bin/certbot renew --quiet --deploy-hook "/usr/sbin/service nginx reload"
Save and Exit (Ctrl + X), and the cron job should be configured. This command will attempt to renew the certificate at midnight and noon every day.
One problem that I’ve had is that I’ve been able to get certificates to renew, however the certificate of the site still expires because the web server configuration hasn’t been reloaded. The --deploy-hook flag solves this issue for us, by reloading the web server when the certificate has been successfully updated.
Now we have our certificate to enable HTTPS, lets move on to configuring nginx.
Configure nginx
Before getting into specific configurations, it might be useful to outline the approach here. Because there is likely to be a number of duplications in the configuration files, some common snippets will be broken out into their own files to ease configuration management. The final list of configuration files we’ll end up with will be:
/usr/local/etc/nginx/nginx.conf
/usr/local/etc/nginx/vdomains/subdomain1.example.com.conf
/usr/local/etc/nginx/vdomains/subdomain2.example.com.conf
/usr/local/etc/nginx/snippets/example.com.cert.conf
/usr/local/etc/nginx/snippets/ssl-params.conf
/usr/local/etc/nginx/snippets/proxy-params.conf
/usr/local/etc/nginx/snippets/internal-access-rules.conf
Certificate configuration
To begin, we’ll start with the snippets:
cd /usr/local/etc/nginx
mkdir snippets
nano snippets/example.com.cert.conf
This file details the SSL/TLS certificate directives identifying the location of your certificates. Paste the following:
# certs sent to the client in SERVER HELLO are concatenated in ssl_certificate
ssl_certificate /usr/local/etc/letsencrypt/live/example.com/fullchain.pem;
ssl_certificate_key /usr/local/etc/letsencrypt/live/example.com/privkey.pem;
# verify chain of trust of OCSP response using Root CA and Intermediate certs
ssl_trusted_certificate /usr/local/etc/letsencrypt/live/example.com/chain.pem;
Remember to replace example.com with your domain, as requested when obtaining a wildcard certificate earlier. Save and Exit (Ctrl + X).
SSL configuration
Use the configuration generator at https://ssl-config.mozilla.org/ to generate a SSL configuration. I’d recommend only using either Intermediate or Modern. I’ve used Intermediate here because at the time of writing I had issues establishing a TLSv1.3 connection, whereas TLSv1.2 was consistently successful, however this compatability comes at the expense of security. The modern configuration is much more secure than the old configuration, for example.
nano snippets/ssl-params.conf
This is the contents of my file:
ssl_session_timeout 1d;
ssl_session_cache shared:MozSSL:10m; # about 40000 sessions
ssl_session_tickets off;
# curl https://ssl-config.mozilla.org/ffdhe2048.txt > /usr/local/etc/ssl/dhparam.pem
ssl_dhparam /usr/local/etc/ssl/dhparam.pem;
# intermediate configuration
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
ssl_prefer_server_ciphers off;
# HSTS (ngx_http_headers_module is required) (63072000 seconds)
add_header Strict-Transport-Security "max-age=63072000" always;
# OCSP stapling
ssl_stapling on;
ssl_stapling_verify on;
# replace with the IP address of your resolver
resolver 192.168.0.1;
Replace the IP address of your resolver as directed, and then Save and Exit (Ctrl + X). If required by your desired configuration, you may also need to download the dhparam.pem certificate:
curl https://ssl-config.mozilla.org/ffdhe2048.txt > /usr/local/etc/ssl/dhparam.pem
Note that at the time of writing, the Modern configuration did not require this, but the Intermediate configuration did.
Proxy header configuration
nano snippets/proxy-params.conf
Paste the following:
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header X-Forwarded-Ssl on;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_http_version 1.1;
Save and Exit (Ctrl + X)
Access policy configuration
This is the policy that we’ll apply to services that you don’t want to be externally available, but still want to access it using HTTPS on your LAN.
nano snippets/internal-access-rules.conf
Populate it with the following:
allow 192.168.0.0/24;
deny all;
Replace the network with the subdomain relevant to you, and Save and Exit (Ctrl + X).
Virtual domain configuration
Create a new directory for virtual domains:
mkdir vdomains
This directory will contain the configurations for each of the subdomains you wish to proxy to. You need to create one configuration file for each subdomain.
Externally available subdomain
As an example, lets assume you have a Nextcloud server you want to proxy to such that it’s externally available outside your network. Create a configuration file for it:
nano vdomains/cloud.example.com.conf
Populate it as follows:
server {
listen 443 ssl http2;
server_name cloud.example.com;
access_log /var/log/nginx/cloud.access.log;
error_log /var/log/nginx/cloud.error.log;
include snippets/example.com.cert.conf;
include snippets/ssl-params.conf;
location / {
include snippets/proxy-params.conf;
proxy_pass http://192.168.0.10;
}
}
Then Save and Exit (Ctrl + X). Lets break this down so you understand what’s happening here. Each server can be handled within a server block. nginx iterates over the server blocks within it’s configuration in order until it finds one that matches the conditions of a request, and if no condition is matched, the server block marked as default_server is used.
The first statements:
listen 443 ssl http2;
server_name cloud.example.com;
access_log /var/log/nginx/cloud.access.log;
error_log /var/log/nginx/cloud.error.log;
This means that this server directive listens on port 443 for a HTTPS connection and enables HTTP/2 compatability. If a HTTPS request is made on port 443, and the Host header in the request matches the server_name directive, then this server block is matched and the directives are executed.
The access_log and error_log directives specify the location of these logs specifically for this server.
include snippets/example.com.cert.conf;
include snippets/ssl-params.conf;
These statement import the directives contained in the files we created earlier, specifically the certificate locations and the SSL parameters.
location / {
include snippets/proxy-params.conf;
proxy_pass http://192.168.0.10;
}
The location block is specific to the requested URI. In this case, the URI in question is /, the root. This means, that when the URL https://cloud.example.com is requested, this location directive is what’s executed. The include statement does the same thing as the snippets above; imports the directives contained in /usr/local/etc/nginx/snippets/proxy-params.conf that we created earlier. The proxy_pass statement is what redirects the request to the subdomain server. In this case, this is where the IP of the Nextcloud jail would go.
Internally available subdomain
If you don’t want this subdomain to be accessible outside of your local network, then you simply need to include the snippets/internal-access-rules.conf file we created earlier. Assuming you have a Heimdall server for example, your configuration file may be created as follows:
nano vdomains/heimdall.example.com.conf
And, assuming that the server is located at http://192.168.0.12, populate it as follows:
server {
listen 443 ssl http2;
server_name heimdall.example.com;
access_log /var/log/nginx/heimdall.access.log;
error_log /var/log/nginx/heimdall.error.log;
include snippets/example.com.cert.conf;
include snippets/ssl-params.conf;
location / {
include snippets/proxy-params.conf;
include snippets/internal-access-rules.conf;
proxy_pass http://192.168.0.12;
}
}
nginx.conf
Now, nginx only looks at /usr/local/etc/nginx/nginx.conf when inspecting configuration, so we have to tie everything we’ve just done in there. Open the file:
nano nginx.conf
The first thing you’ll need to do is disable the default configuration. You can do this by renaming it to nginx.conf.bak as follows:
mv nginx.conf nginx.conf.bak
Then create a new nginx.conf file for our new configuration:
nano nginx.conf
And populate it as follows:
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
# Redirect all HTTP traffic to HTTPS
server {
listen 80 default_server;
listen [::]:80 default_server;
return 301 https://$host$request_uri;
}
# Import server blocks for all subdomains
include "vdomains/*.conf";
}
Save and Exit (Ctrl + X). The important parts of this are the server block listening on port 80, and the include statement. The server block redirects all HTTP traffic to HTTPS to ensure that the SSL/TLS configuration we set up is being used, and the include statement imports the server blocks from all of the virtual domain configuration files. Now we need to start the service:
service nginx start
If it has already started, just reload it. This is the step you’ll have to take after all configuration changes:
service nginx reload
Router configuration
Set up a NAT Port Forward to redirect all traffic received on port 80 at the WAN address to port 80 on the reverse proxy jail, and likewise for port 443. In pfSense (Firewall -> NAT), this looks like the following:
This will ensure that all requests to these addresses will pass through the reverse proxy.
DNS Configuration
In order to make these subdomains accessible both internally, and externally, you’ll need to add entries to a DNS resolver. To do this internally, you’ll need to add an entry for a Host Override, or whatever your router’s equivelant is. In pfSense, navigate to Service -> DNS Resolver -> Host Overrides. Assuming the subdomains proxy.example.com, cloud.example.com and heimdall.example.com, this would look like the following:
As can be seen, all subdomains are being resolved for the reverse proxy jail IP address of 192.168.0.9. For access to these services outside your network, you need to have a valid A record with your DNS provider. As an example, a valid A record would have the name cloud.example.com and the value would be your public IP address.
Certificate Authority Authorization (CAA) Records
If you have a DNS provider that supports it, it might be a good idea to add a CAA Record. A CAA record essentially allows you to declare which certificate authorities you actually use, and forbids other certificate authorities from issueing certificates for your domain. You can read more about these at SSLMate. SSLMate also provide a configuration tool to help you auto-generate your CAA record configuration.
And that’s it! You should be good to go. If you have any questions or need any clarification, leave a comment down below and i’ll try to help where I can. Also, if you notice any errors, please let me know so I can update the guide.
Hey excellent writeup on this topic.
Just a quick question. What’s the difference between using nginx as the reverse proxy vs using HA proxy? I know HA proxy is a load balancer, however just wondering if you could use the HA proxy module within FreeNAS to achieve the same ends as an alternative to setting up a freenas jail.
Since SSL terminates at the reverse proxy, with any webservers running behind the proxy I assume you just configure them to run on port 80?
Hey Kev, I’ve never used HAproxy so I’m not sure I can provide any good commentary on the differences. I used nginx primarily because it’s touted as pretty high performance for reverse proxying, and because it’s so ubiquitous as a web server it was a good excuse for me to learn about its configuration. I also found the configuration of nginx itself relatively straight forward; the complicated part to me seemed to be obtaining a certificate using certbot, especially with the DNS challenge. With that said, load balancing and reverse proxying are different things. From some quick research it looks like HAproxy is capable of reverse proxying, so it could be a viable alternative.
Re: your second question, correct. You would just configure the proxied services to serve HTTP (port 80 not necessarily required, just specify the port in the
proxy_passdirective, i.e.proxy_pass http://192.168.0.10:4567), and the reverse proxy will upgrade the connection to HTTPS.Hello Samuel,
again an excellent guide. Thank you very much.
However, because of your nextcloud guide I’m currently a little bit ahead on the nextcloud behind nginx reverse proxy jail configuration.
I suggest to add “proxy_hide_header” lines before adding individual add_header lines,
e.g. like
proxy_hide_header Strict-Transport-Security
add_header Strict-Transport-Security “max-age=63072000” always
Otherwise for a certain security header option both nextcloud and nginx values are provided which rises comments in the SSL Labs test.
Hope this helps. And now I will try to mimic your “snippets” in order to have a better overview of my config.
Best regards, Markus
Thanks for the suggestion Markus! I’ll definitely have a closer look at putting that in the guide. I got the same result with SSL Labs re: invalid HSTS configuration; I assumed it was because my Nextcloud instance is still looking after its own certificates and SSL policy. I plan on removing this when I upgrade to the latest version and update the nextcloud guide shortly, so I guess I’ll see for sure then, though I suspect this will solve the issue 🙂 proxy_hide_header might be a good intermediate solution. Cheers!
Hello Samuel! Another great guide. I’m working my way through it.
I created the new nginx.conf file and pasted in the new contents, I got the following error:
Performing sanity check on nginx configuration:
nginx: [emerg] “server” directive is not allowed here in /usr/local/etc/nginx/snippets/ssl-params.conf:2
nginx: configuration file /usr/local/etc/nginx/nginx.conf test failed
Starting nginx.
nginx: [emerg] “server” directive is not allowed here in /usr/local/etc/nginx/snippets/ssl-params.conf:2
/usr/local/etc/rc.d/nginx: WARNING: failed to start nginx
What did I miss?
Hi Phil, looks like the error is saying you have a server directive in snippets/ssl-params.conf – remove it; you just want the bare statements. This file is included in each of the vdomain conf files, so if you also have a server directive in ssl params we end up with something like the following:
Which is not what we want. Make sure that your file is exactly as shown in the guide and reload nginx to see if it works 🙂 Let me know if you have any more issues.
Cheers.
Yes! That did it. Success! Thank you Samuel. Since I now have the wildcard certs in place with the reverse proxy, how do i remove the cert I originally created using your nextcloud guide?
I actually haven’t found the time to go through this process myself yet. You could try going through the SSL instructions in reverse and undoing each command? IIRC there’s a revocation command for certbot that allows you to revoke a certificate and remove any lasting traces of it; have a look at the certbot documentation and see what it says. You would also need to revert your vhost settings in apache to provide content over HTTP instead over redirecting to HTTPS – so removing the
<VirtualHost *:443>directive and the redirect in the<VirtualHost *:80>directive. Hope this helps 🙂 CheersWill do Samuel. Thank you!
Have you ever thought about putting the proxy and web server in a different VLAN? I’m not sure if this is applicable to your host however its just another form of isolation from your other network.
I’ve thought about it, but haven’t found the time to work out how appropriate it is. I actually bought a managed switch a while ago to play around with VLANs, but haven’t got around to it yet. Is this how you’ve set your network up?
Yes I recently upgraded my switch hardware (using mostly Unifi switches however I do have a few DLink Managed switches as well). I couple these devices with pfsense similar to yours. I’m aware many of your servers are running on AWS which I haven’t yet made a migration to except for a few on Digital Ocean. Any public facing servers I’m putting in their own separate VLAN(s) along with IoT devices for home. Adding VLANs however does complicate a few things however particularly with certificate management and distribution.
Ah that’s cool 🙂 VLANS could definitely be a good way to go; I’m looking forward to researching them more
Any details how you set up your bitwarden server? I’m intereted in doing the same exact thing with the method you discussed above with nginx reverse proxy in front of the bitwarden server.
Also in general I have a questions about the reverse proxies and termination. What part about your configuration makes nginx the termination point for SSL? I ask this because you have the following in your setup:
location / {
include snippets/proxy-params.conf;
proxy_pass https://192.168.0.10;
}
So you are receiving SSL encrypted traffic into the proxy, and then ??re-encrypting it to the internal server at 192.168.0.10? Is the proxy acting as a MIM in this case?
Hi Kev, thanks for pointing this out, you’re right it should be a
proxy_passto HTTP rather than HTTPS. It was something I had in my configuration for my cloud domain (as it still manages its own SSL until I find time to reconfigure it), but slipped through the cracks for getting updated in the guide. I’ve fixed it now. FWIW if you’re reading this and wondering how to continue letting a service behind the reverse proxy continue to manage its own certificate; this is how.proxy_passto the HTTPS address, and add theproxy_hide_headerdirective to the relevant vdomain conf file to use the headers as passed from the endpoint, and not the reverse proxy.RE: Bitwarden, I was thinking about doing a guide on this but honestly the official instructions are pretty good. I just spun up a debian vm with bhyve and used docker to install it, then followed the prompts for installation. The official distribution is a bit heavy because it uses Microsoft SQL Server underneath, but I didn’t particularly feel like hacking around with any of the alternatives, which have their pro’s and cons (i.e. smaller/faster distribution, but IIRC they only reimplement the API, so don’t ship with the web vault, though I know there are instructions out there on how to get this – just seemed more trouble than it was worth). The examples I’m referring to are rubywarden and bitwarden_rs if you want to go and check them out. There’s a (rough) installation guide for Rubywarden here
Hi Samuel. So I’m hung up on the DNS Configuration section. I don’t have a pfsense box yet. I’m planning on putting one together soon. Right now I have an edgerouter 4. Do you or anyone else have any experience getting this set up with this box?
I do have the hairpin option set up and I can access my nextcloud internally and download files, but I get an unknown error when I try to upload a file.
Do you think the issues are related?
I was actually able to fix this issue by adding the client_max_body_size statement.
Phil, glad you got the upload issue sorted. With regard to DNS configuration, I’ve never used an edge router so I have no idea how to do it. Having said that, some quick research indicates that it might be possible by customising your DNS Forwarding Options. Specifically, it looks like the following command line setting may be roughly equivalent to pfSense’s Host Override (I’m assuming this is what you’re having trouble with and not the port forwarding?):
Where
cloud.domain.comis the address you want to redirect and192.168.0.10is the address of your reverse proxy jail. Hope this helps!Edit: I found this video that looks like it goes through how to set your edge router up to assign static DNS host names using the web interface on the edge router 4
Wow, thank you, this was very useful!
I wish I had found such a comprehensive tutorial a long time ago!
My setup is almost identical to yours, except that:
– I find it more convenient to keep all nginx settings in one file instead of using includes.
– To access proxied hosts from the LAN by entering https://proxiedhost.mydomain.com, I set up NAT Reflection on pfSense (System > Advanced > Firewall & NAT) instead of Host Overrides. See https://docs.netgate.com/pfsense/en/latest/book/nat/nat-reflection.html for more info. I don’t know enough about networking to imagine all possible consequences of one setup vs. the other, but it’s been working flawlessly for me for a few years now and doesn’t require me to enter additional host overrides as I add web proxied hosts.
– pfSense also takes care of renewing the Let’s Encrypt wildcard certificates and copying them to FreeNAS via scp, provided you’ve set up passwordless key-based SSH access to FreeNAS.
Thank you for mentioning nginx access rules. I suspected they existed but never really took the time to look into them.
Ahhh, thanks for mentioning the NAT Reflection! I suspected that there was probably a better way to do it than just host overrides, but I didn’t come across anything. I’m going to look into this to see if it’s more appropriate for my use case 🙂
Nat reflection is what I ended up using as well. It works well.
Hey Samuel. On your advice I went and checked out bitwarden_rs which is a fork written in rust (which you probably know). This guide came in very useful, since I was able to spin up two linux VMs (on FreeNAS) — one for the reverse proxy and the other for the bw_rs implementation. This guide was really helpful in that I only expose the bw server to the internal LAN and the instructions from your reverse proxy were very very helpful in this step. I had used a docker image via docker-compose before, however that actually was relatively easy to setup. The hardest part was setting up postfix as a relay server — with my postfix installation located on the reverse proxy. I had to configure postfix as a n encrypted SMTP relay for the LAN machines so that bitwarden — when sending mail — would send to postfix (located on reverse proxy), which then would forward to gmail account. I wish I could bypass gmail, however I’m not really interested in wading into the world of setting up my own mail server and dealing all the overhead of management. One thing I really like about bw_rs is that it gives you all the premium features out of the box. It also works really well with the browser extension and mobile apps. Thanks for all your help.
Hello, I have the reverse proxy installed and it is working great! Thank You! However I would like to implement the configure ddns updates for my route53 and i have followed that part of your guide on installing nextcloud and have tried to use the ddns updates for route53 on the reverse proxy and I havent been able to get it to work. I did have to install bash. When i run “/usr/local/bin/bash /scripts/update-route53/update-route53.sh” I am getting an aws: error:
aws: error: the following arguments are required: –hosted-zone-id, –change-batch
/scripts/update-route53/update-route53.sh: line 92: –hosted-zone-id: command not found
/scripts/update-route53/update-route53.sh: line 93: –change-batch: command not found
I thought that maybe it was due to the fact i didnt have pip installed so i installed pip however i am now lost on what to look for next.
Thank You very much for your guides and help as I know that I have learned so much!
To use this script, you’ll also need to install the
awsclipackage and configure it with your credentials so that it can reach out to your aws profile (refer back to the nextcloud guide for detailed instructions). I’m glad it’s been working well for you 🙂 CheersThe nextcloud documentation says the following:
“The redirects for CalDAV or CardDAV does not work if Nextcloud is running behind a reverse proxy. The recommended solution is that your reverse proxy does the redirects.”
https://docs.nextcloud.com/server/18/admin_manual/configuration_server/reverse_proxy_configuration.html
Do you have any tips on configuring nginx to take care of these redirects?
Fixed it! See below:
nano /usr/local/etc/nginx/vdomains/subdomain1.example.com.confserver {listen 443 ssl http2;
server_name subdomain1.example.com;
access_log /var/log/nginx/cloud.access.log;
error_log /var/log/nginx/cloud.error.log;
include snippets/example.com.cert.conf;
include snippets/ssl-params.conf;
location / {
include snippets/proxy-params.conf;
proxy_pass http://192.168.0.0;
}
location = /.well-known/carddav {
return 301 https://subdomain1.example.com/remote.php/dav;
}
location = /.well-known/caldav {
return 301 https://subdomain1.example.com/remote.php/dav;
}
}
Interesting. I hadn’t seen that. I also can’t really speak to it; it hasn’t been an issue for me. I have CardDAV set up to sync my contacts on my phone, and I’m not having any issues with my current configuration. Aside from that though, I guess you might need to add a location block to your server directive that takes the target URL and redirects it appropriately? Might be worth seeing if the current configuration works or not though, I don’t see any reason why it wouldn’t. Hope this helps.
Also I got a “Request Entity Too Large” from my android Nextcloud client when trying to upload a ~3MB file, so I added this to the server {} line in nano /usr/local/etc/nginx/nginx.conf:
client_max_body_size 1999M;er sorry added to the http {} section
I had the same issue with Nextcloud, but I used this line to disable the checks entirely:
client_max_body_size 0;
https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size
You have to had this line in the ngnix.config
“client_max_body_size 10240M;”
bellow
“keepalive_timeout 65;”
I am trying to add a redirect for a generic TCP service using a stream { } argument, but I get an error while starting nginx:
nginx: [emerg] unknown directive "stream" in /usr/local/etc/nginx/vdomains/...
nginx -V shows “–with-stream=dynamic”, and my google-fu searching makes me think that has to be set to static. Any ideas?
Add to first line nginx.conf fixes the error:
load_module /usr/local/libexec/nginx/ngx_stream_module.so;But dealing with logic issues. I’m basically trying to do this for a different sub-domain:
https://stadicus.github.io/RaspiBolt/raspibolt_50_electrs.html#ssl-encryption
Great guide. Everything was going smoothly until I got to the part where I start up nginx. I get the following error:
[code]root@reverse-proxy:/usr/local/etc/nginx # service nginx start
Performing sanity check on nginx configuration:
nginx: [emerg] BIO_new_file(“/usr/local/etc/ssl/dhparam.pem”) failed (SSL: error :02001002:system library:fopen:No such file or directory:fopen(‘/usr/local/etc/s sl/dhparam.pem’,’r’) error:2006D080:BIO routines:BIO_new_file:no such file)
nginx: configuration file /usr/local/etc/nginx/nginx.conf test failed
Starting nginx.
nginx: [emerg] BIO_new_file(“/usr/local/etc/ssl/dhparam.pem”) failed (SSL: error :02001002:system library:fopen:No such file or directory:fopen(‘/usr/local/etc/s sl/dhparam.pem’,’r’) error:2006D080:BIO routines:BIO_new_file:no such file)
/usr/local/etc/rc.d/nginx: WARNING: failed to start nginx
[/code]
Never mind. I had to download the dhparam file. Doh!
Thank you for all your guides, I already used your nextcloud guide to understand what it is I’m going as opposed to just running a script. Great amount of detail and explanation, much appreciated.
I am having trouble setting up the reverse proxy, however. Apart from nextcloud I have a simple html repair manual on a different jail and want to run Onlyoffice. When I access everything locally, it all works (but isn’t going through the reverse proxy), but when I go through the proxy only nextcloud is available. Neither the repair manual is accessible nor does Onlyoffice work. So there is a problem with how I set up my reverse proxy, but I fail to understand where. When I look at the error logs of the repair manual I keep seeing some references to /remote/webdav-folders that nextcloud utilizes, but don’t get where the comes from, I’m trying again from scratch now.
The reason for setting up the reverse proxy is that I don’t want to expose all the different hosts directly and having to manage all the different certificates this entails. So in theory, is it not enough to have one certificate running on the reverse proxy and everything behind that is just running as http?
Hi Jens, this is exactly why I set mine up this way. With regards to your problem, it’s possible that it has something to do with the way DAV works. Another user reported similar issues, and resolved it by redirecting the DAV endpoints specifically. From their comment:
The difference here is that it redirects /.well-known/caldav and /.well-known/carddav to /remote.php/dav. You could try this and see how it goes, otherwise without posting any configuration or error messages there’s not much I can do to help, as I don’t use Onlyoffice. I’m also not sure what you mean when you say the repair manual isn’t available. How are you hosting it? You’ve said it’s in a jail but it’s not clear to me why/how it should be available. Have you created a vdomain entry for it? Hope this helps.
I will have a look at the “dav’s”, that’s an interesting point.
The repair manual is hosted via nginx in a seperate jail, it’s just a bunch of htmls and images that were created way back in the days of dial-up… it is available locally as “http://e24” or via its IP directly, and in the reverse proxy I’m pointing to it by “location /e24”, but that doesn’t work. It’s not finding a csrf token file that obviously in this stone-age website doesn’t exist, says the error log. This mainly served as a testbed for me to see if the “location /” setup works, before taking a deep dive at Onlyoffice and why that only works when served locally. I know there might be a few obstacles in Onlyoffice config to make it work behind a reverse proxy and think I have that figured out, but the fact that the “location /” is not working is throwing me off right now. I will have another look, but it’s been costing me much more time than I planned already, so I might just end up not using a reverse proxy and exposing all the services that are running locally that need exposure seperately and managing their certs… turns out the reverse proxy isn’t quite as easy as I though it would be… Your help, however, is much appreciated, either way!
Jens, I think you might have misunderstood how to configure a vdomain. If you want it to be available locally at https://e24, you’ll need to set the
server_namedirective to e24 and the location to/, i.e. something like the following in/usr/local/etc/nginx/vdomains/e24.conf:You’d then have a DNS entry to resolve https://e24 to your reverse proxy IP. Hope this helps.
Edit: Important to note that you won’t be able to get a LetsEncrypt certificate for the domain e24; the reason I subdomained all of my jails was to utilise the wildcard certificate that I could obtain for *.example.com. It might be better to host your jail at e24.yourdomain.com, and then get a wildcard for *.yourdomain.com, which would encrypt all of your sites.
Well, that makes sense now. The problem is I can only use CNAME in my (sub)domains to forward to the dyndns service built-in with my router (already a subdomain, as all Dyndns solutions I know of are), which in turn is going to generate a certificate error which I don’t want, so I guess I will revert to a different solution.
Anyways, thanks a lot Samuel. People like you make the Internet worth keeping 😉
Samuel — would you be open for a “guest” post submission? I don’t have my own wordpress website (at least right now but plans in the works). My topic would be setting up authelia which is a frontend to protect various domains or websites either via two factor authentication, duo push notification of YubiKey – https://github.com/authelia/authelia. This topic integrates nicely with your reverse proxy writeup and incorporates topics you’ve previously touched on (nginx, Let’s Encrypt Certs, smtp forwarding (gmail)) which also incorporating new topics such as docker, docker-compose that deal with container setup and administration.
I would love a write-up on this, but you don’t need Samuel’s blog, you can create a how-to on github, a lot of guides are uploaded there!
For example:
https://github.com/seth586/guides/blob/master/FreeNAS/README.md
Feel free to rip off the template and make a few pages! If you don’t want to make a github page, send me the writeup, and I can upload. I’ve been meaning to update my guide into a complete home server how-to page.
Hey man!
Thanks for the guide! I gave up doing this a few years back, but this writeup really helped me understand it all better! – just one evening made it happen!
I got it working using the acmesh script, as I’m on duck dns.
However, the last step my (ISP’s) router doesn’t seem to support, so I just thought I would skip that step, and to my surprise, it still works! Both internally and externally!
Does this mean I did something wrong?
Hi Mathias,
When you say “last step”, are you referring to adding a CAA record to your DNS provider? If so, this isn’t done with your ISP, it will be done with whoever you have registered your domain with. For me, this is AWS so I added an entry in Route 53. It’s an entirely optional step, but it’s a setting that prevents other DNS Providers from issuing valid certificates for your domain. Given you’re using duck dns, I’m guessing you don’t own a domain, so it’s not something to worry about.
Cheers,
Sam
Hey thank Samuel for the information. I’m in the process of writing this up which tends to be a lot more difficult than just setting things up since I need to completely verify every single step.
Also I recently learned about GitHub pages. I have no idea why I hadn’t heard of this application before. It’s really neat and nice for hosting things like this. It uses Jekyll as its provisioning app. I believe you write with WordPress which is likely more fully featured, however it seems for this project at least the jekyll platform is mostly adequate. I’m learning about markdown and scss — seems like there is always something to learn.
Anyway I have the template engine installed locally and have travis CI setup in the background to do the provisioning. I suspect I may have this writeup done in a week or show and I’ll submit you a link.
Hey Kev,
I appreciate the offer, but I have some uncertainty about the future of this blog at the moment. Not about it going down, but I’m looking at ways to implement CI/CD so that I can author all of these posts with Markdown and deploy from git commits. This seems to be reasonably easy to do for static websites without comments, but for dynamic sites such as WordPress this appears more complicated. I’m exploring a range of other CMS providers so there’s a possibility (probability) that migration would affect the availability/preservation of the posts and their historical comments/metadata.
With this in mind, it looks like you’ve found a good solution and I’d been keen to read your article when you post it – I recently bought a couple of YubiKeys and I’m still trying to work out a way to use them that works best for me 🙂
Cheers
I followed this tutorial and my reverse proxy is acting up. It first started with communicating with the FreeNAS host, the internal subdomain I setup kept getting 503s I think I recall? Well I found out it wasn’t able to receive pings back from the FreeNAS host, as a last ditch effort I changed the IP of the jail and it was able to see the FreeNAS host again.
But now I’m having issues with the server refusing to connect. It happens about 10-30 minutes after Nginx is loaded. If I reload Nginx, the issue goes away. The logs don’t have anything on these events. In fact I deleted them yesterday, nothing is in the error log since.
I’m forwarding TCP ports 80 and 443from my Google Wifi router to the jail’s IP. Any help would be appreciated.
Hi Tyler, I wish I could provide more help but this isn’t something I’ve ever seen before and I’m far from an expert on nginx. I suggest maybe checking out #nginx on freenode – they may be more helpful!
Hi there,
first I want to thank you for putting up your guides, I appreciate them a lot!
I am a total beginner concerning networking and hope I am describing my problem in an accurate way.
First of all: i am using a freeNAS system. There I am trying to setup a reverse proxy on a jail with ip 192.168.0.10 and am trying to route traffic to my nextcloud jail which is at 192.168.0.10.
While following the instructions several questions arose.
1) what is the resolver ip in the setup of ssl-param.conf
I assume it is the ip of my network? (I am sorry for such a newbie question)
2)i am using aws as dns resolver. There i have an dns entry for: example.com
Next I set up an alias (at aws) for my nextcloud which looks like nextcloud.example.com. I hope this is correct?
3) I forwarded in my router all incoming traffic to the reverse proxy (192.168.0.9) (ports: 80 & 443)
when I try to https://nextcloud.example.com It refuses connection due to safety issue.
I double checked all configuration files and I assume they are correct. Can anyone help me on this?
Cheers
Unfortunately i cannot edit my post.
I was able to solve the problem, as you pointed out in the guide: using intermediate ssl-config (with TLSv 1.2) solved my issues.
Feel free to remove this post.
And once again : thank you for your guides! 😉
Cheers
Samuel – did you set your Nginx Reverse Proxy to Proxy to your Apache Reverse Proxy to Proxy to your Nextcloud? Did you terminate the SSL connection at the reverse proxy or re-encrypt to the backend? If there are “two reverse proxies in place” was there anything you did on the first reverse proxy to configure it for nextcloud? Do you have to change anything on the backend to make this work?
Hey Kev,
I haven’t changed anything from what I detail in my Nextcloud guide. From Nextcloud’s perspective, I proxy php requests to the fcgi handler with Apache. I also use the nginx reverse proxy to handle traffic to Nextcloud, with SSL termination. However, since I haven’t changed my Nextcloud configuration since I first set it up, Nextcloud currently still serves itself via HTTPS. I plan to change this so that it’s served over HTTP and no longer handles any certificate configuration itself, but time is a factor for me at the moment (too much studying!). Ultimately, this means that the vdomain configuration file for my nextcloud instance looks like the following:
Note that the primary difference here is that it’s
proxy_pass-ing to a HTTPS address and not a HTTP address. I suppose to answer your question, there’s no Apache reverse proxy, per-se.Hello Samuel! Any best practices for updating nginx?
Just using the package manager as far as I’m aware, then restart the service 🙂
Hey Samuel — Quick question. I took your nextcloud blog information and just changed the webserver to nginx. It’s not that I don’t like Apache, its just there is a lot more info on configuring nextcloud with nginx. Anyway I want to put an nginx reverse proxy in front of my VM running nginx/nextcloud. I believe you have something similar with a VM running an nginx reverse proxy and an upstream VM with apache/nextcloud. Was there any specific headers you needed to use on the reverse proxy side when passing to the apache/nextcloud backend? I’m aware that the nextcloud config.php file likely needs the name of the reverse proxy included as a “trusted proxy”. Was there any additional changes you needed to make on the nextcloud end with the introduction of the reverse proxy? I’ve set up the reverse proxy and am in the process of trying to create a proxy pass to the backend using “proper” server and client authentication. I’ve got the https server authentication to the backend working on a test server (non nextcloud), and I’m slowly struggling but have a basic framework for client authentication certs with self-signed certs. I’m just missing the last nextcloud piece in the equation.
Hi Kev,
My vdomain configuration file for my nextcloud server looks like this:
Note the https in the
proxy_passdirective; this is there because I haven’t reconfigured nextcloud to serve via http yet; it’s still serving the page via https as per a previous iteration of my nextcloud installation guide (note: it still uses the wildcard certificate defined in my domain.com.cert.conf file). I haven’t made any adjustments to my config.php file or the “trusted_domains” field, it still works with just its own IP and the domain name, presumably because the proxy is forwarding the request with the ‘Host’: ‘cloud.domain.com’ header intact. Does this answer your question?Cheers
@Samuel
Hey thanks for the advice regarding the following header:
proxy_hide_header Strict-Transport-Security;
I wasn’t aware of this header
In terms of proxy_pass https://….
Nginx by default does not verify the upstream server. You could have the upstream server offer any certificate and nginx would accept it (by default). You can options however to verify the cert if you would like. For experimental purposes I’m trying to get my upstream working with server and client SSL certificates. Hopefully I’ll have a working example soon.
Cheers Kev, that’s good to know. This is really just a stopgap until I can reconfigure everything. The intention is to have SSL terminate at the reverse proxy, so I’m planning on removing any of the responsibility of serving SSL/TLS from nextcloud entirely.
Sorry to keep bothering you. I was able to setup an nginx reverse proxy in-front of an nginx/nextcloud installation (I used your original nextcloud documentation however I switched over to using nginx as the server rather than apache).
I’m able to reverse proxy to nextcloud however I’m wondering if you have a collabora installation as well. My collabora docker container functions properly with nextcloud in the absence of a reverse proxy, however when I add in the reverse proxy, things don’t exactly work.
Hey mate, I don’t unfortunately. I’d imagine it’s just a matter of forwarding the right traffic; but I haven’t looked at collabora at all. If you find a solution I’d be keen to hear what you had to do though!
Cheers
Hi Kevdog,
I’m interested in also switching NextCloud over to Nginx, instead of Apache. Did you find a good set of steps and config to follow by?
Angelo
I would guess as long as nginx is configured to process php correctly it should work.
I have a FEMP stack configuration for wordpress here https://github.com/seth586/guides/blob/master/FreeNAS/webserver/2_nginx.md
This is my vdomains file for collabora. Make sure to create a hostname on your router so your local IPs redirect collabora.mydomain.com to your reverse proxy IP address.
server {listen 443 ssl;
server_name collabora.mydomain.com;
server_tokens off;
include snippets/mydomain.com.cert.conf;
# static files
location ^~ /loleaflet {
proxy_pass http://192.168.84.247:9980;
proxy_set_header Host $http_host;
}
# WOPI discovery URL
location ^~ /hosting/discovery {
proxy_pass http://192.168.84.247:9980;
proxy_set_header Host $http_host;
}
# Capabilities
location ^~ /hosting/capabilities {
proxy_pass http://192.168.84.247:9980;
proxy_set_header Host $http_host;
}
# main websocket
location ~ ^/lool/(.*)/ws$ {
proxy_pass http://192.168.84.247:9980;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header Host $http_host;
proxy_read_timeout 36000s;
}
# download, presentation and image upload
location ~ ^/lool {
proxy_pass http://192.168.84.247:9980;
proxy_set_header Host $http_host;
}
# Admin Console websocket
location ^~ /lool/adminws {
proxy_pass http://192.168.84.247:9980;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header Host $http_host;
proxy_read_timeout 36000s;
}
}
Hey Sam,
How do you use this reverse proxy to redirect to your main domain blog without a subdomain?
For example, I currently have successful reverse-proxying of cloud.fubar.com but not http://www.fubar.com or fubar.com
Found the solution! Run certbot with the syntax:
certbot certonly --dns-route53 -d 'example.com,*.example.com'
Then make sure your DNS A records for example.com and http://www.example.com point to your reverse proxy!
Interesting. If you look at the certificate for this site, it’s a wildcard. I’m not sure why a wildcard wouldn’t work for http://www.example.com
Hey, which I have found this earlier, would have saved me a lot oft time. I set up my freeNAS and ended up with a simular setup. But I’m stuck with two thinks.
1 I have a Telekom Speedport Router (manufaxturer is Huawai I think) and found no way how to do the NAT sruff.
2 I can access my nextcloud from outside via my dynDNS Domain and all seams to work fine, but the nextcloud APP in Android (didn’t testet others) is looping endless in the last confirm Dialog to give access for the APP to nextcloud. No error logs. The only thing I see is in my access.log that the APP uses POST Request for lofin while the Browser uses GET.
Maybe you have an idea about my two issues.
Thank you and greatings from Germany.
I think you are having the same issue I had, I found in a forum somewhere to add the line below to config.php, it took care of the problem for me
‘overwriteprotocol’ => “https”,
I figured out the reason why TLS 1.3 won’t work:
FreeNAS is basically just FreeBSD 11.3, and so all jails run FreeBSD 11.3. The problem is that FreeBSD 11.3 ships with OpenSSL v1.0.2. The first version of OpenSSL that comes shipped with TLS 1.3 support is v1.1.1, so the solution is to somehow upgrade the base package OpenSSL so that it has TLS 1.3 support. I’m not sure how to do that with FreeNAS, but if your host is running FreeBSD >=12, you should be able to just upgrade the jail and it’ll work out of the box, since FreeBSD >=12 ships with OpenSSL 1.1.1. Hope this helps for others! (im so sorry if you spent more than 8 hours messing with nginx configs like me in a vain attempt to get it working when it turned out to just be an our of date package)
I’m sure this is part of the story, but perhaps not the whole story. I have 1.1.1g installed in my jail and it’s not working for me either. To upgrade, you can simply execute the following from within the jail:
When you install openssl using
pkg, the binary is installed into/usr/local/binrather than/usr/bin. So whennginxcallsopenssl, it calls the one bundled with FreeBSD and not the newer version (This should be confirmed if you runwhich openssl, or if you runopenssl version). Simply moving the binary from/usr/local/bindidn’t work for me for some reason, so upgrading the OS itself did the trick.@Nick Nnien
Sorry you had to suffer 8 hours to figure this out.
I’m running FreeNAS 11.U2-1.
I have a nextcloud jail (as per Samuel Dowling’s Guide), and I also have nginx with openssl 1.1.1
uname -mrs
FreeBSD 11.3-RELEASE-p7 amd64
nginx -V
nginx version: nginx/1.17.9
built with OpenSSL 1.1.1g 21 Apr 2020
TLS SNI support enabled
OpenSSL 1.1.1 introduces an entirely new API so any application that depends on openssl needs to be recompiled agains the new version (if you are installing from ports). If you’ve installed things from ports, you can check what is compiled against openssl via:
pkg info -r security/openssl
The following method will not change the base openssl for the system — just for port installed packages. (This is kind of confusing to explain however you’ll see this on the command line):
openssl version
OpenSSL 1.0.2s-freebsd 28 May 2019
pkg info openssl
openssl-1.1.1g,1
…
…
You what you need to do is:
#1 – install openssl 1.1.1
pkg install openssl
Check that 1.1.1g is installed via:
pkg info openssl
openssl-1.1.1g,1
#2 Prepare to build nginx from ports
a. Edit /etc/make.conf and add the following:
www_nginx-devel_DEFAULT_VERSIONS+=ssl=openssl111
DEFAULT_VERSIONS+=ssl=openssl
b. Install nginx-devel (from ports). This will conflict with the nginx pkg if you have this installed and it will remove it by default. If you’ve never installed from ports, you can do the following (these instructions are a little bit variable depending on the source your read, but this is what I did)
cd /usr/ports/www/nginx-devel
make clean
make config
This will bring up an ncurses menu where you can install any additional packages or modules with nginx. The source is here: https://www.freshports.org/www/nginx-devel. I just went with the defaults. You can always reinstall later if you find a missing missing package
make
make install (or reinstall if you are reinstalling)
You might be prompted about the conflicting nginx package at this point since you are installing nginx-devel. Go ahead and install nginx-devel.
Now check nginx version:
nginx -V
nginx version: nginx/1.17.9
built with OpenSSL 1.1.1g 21 Apr 2020
TLS SNI support enabled
That should be about it. You’ll see now that nginx-devel is now dependent on openssl 1.1.1
pkg info -r security/openssl
openssl-1.1.1g,1:
nginx-devel-1.17.9
nginx-devel will now need to be manually updated from ports rather than through the pkg manager with this method (I believe).
Hopefully this helps.
Ahhh maybe building
nginxfrom the ports would’ve fixed it. Oh well, it works nowHi, Thanks so much for this detailed write-up! I was able to get this working pretty easily. I have a question though, what if I have a webpage or server that requires websocket support, how do I set that up? I was using NGINX Reverse Proxy written by JC21 for docker, it has a web ui front end where I can enable websocket support. Is this possible with this particular install?
Thanks
Hi Joshua, certainly should be possible; there’s nothing special about this configuration that would exclude it. You would just need to add the right directives to
nginx.conf. Thestreamdirective might be appropriate; see if you can use the discussion here as a framework to adapt to your desired configurationThanks for the well written guide, and kudos on the streamlined command entering. A lot packed into this, but it went quickly with a bit of prior nginx tinkering. I’ve been scouring the web for guides more specific to my use case as this one is, and this is the best one I’ve found.
Cheers,
Tim
Samuel,
Perfect guide on how to set this up! I followed it step by step and all works fine!
I have created according this manual a jail with this reverse proxy and a jail with nextcloud which works like a charm!
I also created a jail with an FAMP (Apache2) stack with WordPress. WordPress works fine if I go from my internal network to the IP address of the jail but do you know what steps to take to have wordpress accessible from my external domain name? How would the vdomains file look like to have it forwarded to the WordPress stack? Do I need to adapt the WordPress stack as well?
I just did this very setup, heres a cheat sheet:
make your vdomains/example.com.conf:
If you are forwarding to http://www.example.com you do not need to change your SSL configuration. However, if you are using https://example.com you will need to add the domain to your letsencrypt
certbot certonly --dns-route53 -d 'example.com,*.example.com'
Add the following lines to your
wp-config.php:if (!empty($_SERVER['HTTP_X_FORWARDED_PROTO']) && $_SERVER['HTTP_X_FORWARDED_PROTO'] === 'https') {
$_SERVER['HTTPS'] = 'on';
}
define( 'WP_HOME', 'https://example.com' );
define( 'WP_SITEURL', 'https://example.com' );
Sam, before you approve moderation, can you please change my snippets/ .com domain on the above post and change it to example? I guess I didn’t proof-read, thanks!
Hi Seth,
Thanks a lot!!! This makes it work!!!
And thanks a lot for your quick reply!!
Sander
Hello, I have some questions.
On a VM mounted on virtualbox, I have FreeNAS installed.
This VM has a bridge configuration to take internet from my home network.
I have created a jail, there I am configuring a reverse proxy to attend to all incoming requests to my freeNAS.
My idea is to install a SSL Lets encrypt wilcard certificate over the jail with nginx. As investigated in:
https://letsencrypt.org/es/docs/challenge-types/
To be able to do it, lets encrypt says that I have to do the validation by DNS (DNS-01 challenge) and I have to create some TXT records and the DNS zone has to be accessible via API. They display a list of supported DNS services:
https://community.letsencrypt.org/t/dns-providers-who-easily-integrate-with-lets-encrypt-dns-validation/86438
In my case I plan to use Cloudflare.
My # 1 question is:
The proxy must be assigned a public IP so that it can resolve the DNS, but the jail has a local IP configured. In the jail I have a VNET + NAT configured without DHCP (fixed local IP).
Should I use a Dynamic DNS service to be able to link my dynamic IP (from the ISP) with the local IP of the jail and then do a port forwarding on my router? The problem I am having is that the jail is under another subnet, the Jail IP is 172.6.0.2.
When I want to configure port forwarding on my router with IP 172.6.0.2 it gives me the error that it is not on the same network.
My # 2 question is:
In the jail where I have the reverse proxy, how can I link my domain? What steps should I take?
My # 3 question is:
My FreeNAS private IP is 192.168.0.105 (NAT)
My Jail’s private IP (r-proxy) is 172.6.0.2 (NAT)
If I ping from my PC to the jail, I cannot access it.
You can help?
Cheers
Hi Alejandro, a few points:
1. The reverse proxy jail does not need a public IP. It just needs the appropriate traffic forwarded to it from your router. If this is to host a web server, usually this means ports 80 and 443, though there are some more uncommon ports that may also be appropriate.
2. A dynamic DNS service updates a DNS name server with your public IP, so that whatever domain name you have points to the correct IP if it is non-static (usually residential IP’s change semi-regularly)
3. The jail should not be under another subnet. The problem you’re having is that it literally is not on the same network, and you haven’t set up the routes to enable that. Best to give the jail an IP on your primary network to mitigate the need to implement any additional routing.
4. In my guide, you would specify your domain name in the
server_namedirective in the appropriate vdomain file. Strictly speaking, you just need a server block. This is how you handle requests to a given domain name.5. To make sure your server gets the requests, you need to configure a DNS entry for your domain (in Cloudflare) such that it points to your public IP.
6. Make your reverse proxy jail IP 192.168.0.106.
Spend some time going over the guide, I cover a lot of this in a lot more detail. It might also be worth watching some videos on how DNS works, and how networking works to understand some of the principles if this guide hasn’t been sufficient.
Hope this helps.
I have successfully installed the letsencrypt certificate with certbot in my reverse-proxy with nginx in a jail in FreeNAS with the -manual method (I am not using the cloudflare plugin because now the API is not accessible for free accounts).
The problem I am having is that when I run the command:
Details of the FreeNAS self-signed certificate appear to me, not the certificate that I installed in the jail corresponding to redacted:
I have configured my nginx.conf from jail so that it listens to port 443:
But by executing the following command, I get this result. Only port 80 is open:
I suspect the problem has to do with the CNAME setting (redacted) pointing to a Dynamic DNS of NO-IP. Therefore, when executing this CNAME, the freeNAS general interface is executed, when the correct thing would be to try to access the created jail.
What could be the problem?
My jail’s IP is 127.16.xxx.xxx (NAT), it is different from FreeNAS’s local IP (192.168.xxx.xxx). I understand that it is not possible to access the jail from the outside.
In my router I can only define port forwarding to my FreeNAS with 192.168.xxx.xxx, I cannot do it with the IP of jail 127.xxx.xxx.xxx (NAT).
What should I configure? In FreeNAS should I internally configure a forwarding from 192.168.xxx.xxx/24> 127.16.xxx.xxx/30?
What advice can you give me?
Regards
Alejandro, I’ve edited your comment to redact your domain, and in the process I messed up some of the formatting. I’ve tried to reconstruct it, but it may not have been perfect so if I’ve added
#in places it shouldn’t be, let me know.First of all, it doesn’t look like you’re using my guide. I don’t set my
nginx.confup this way. I’m not sure how I can be expected to support configurations that don’t follow my guide; I’m not an nginx guru or support channel.Secondly, this configuration shows all of your SSL parameters commented out. They aren’t in effect. You need to uncomment them if you expect a certificate to be issued.
I have corrected the nginx.conf:
…
#user nobody;
worker_processes 1;
This default error log path is compiled-in to make sure configuration parsing
errors are logged somewhere, especially during unattended boot when stderr
isn’t normally logged anywhere. This path will be touched on every nginx
start regardless of error log location configured here. See
https://trac.nginx.org/nginx/ticket/147 for more info.
#
#error_log /var/log/nginx/error.log;
#
#pid logs/nginx.pid;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr - $remote_user [$time_local] "$request" '# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
#access_log logs/access.log main;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
#gzip on;
server {
listen 80;
server_name r-proxy.nas.ethopolis.tech;
return 301 https://$server_name$request_uri;
#charset koi8-r;
#access_log logs/host.access.log main;
location / {
root /usr/local/www/nginx;
index index.html index.htm;
}
#error_page 404 /404.html;
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/local/www/nginx-dist;
}
# proxy the PHP scripts to Apache listening on 127.0.0.1:80
#
#location ~ \.php$ {
# proxy_pass http://127.0.0.1;
#}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
#location ~ \.php$ {
# root html;
# fastcgi_pass 127.0.0.1:9000;
# fastcgi_index index.php;
# fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name;
# include fastcgi_params;
#}
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
#location ~ /\.ht {
# deny all;
#}
}
# listen 8000;
# listen somename:8080;
# server_name somename alias another.alias;
# location / {
# root html;
# index index.html index.htm;
# }
#}
# HTTPS server
#
server {
listen 443 ssl;
server_name r-proxy.nas.ethopolis.tech;
ssl_certificate /usr/local/etc/letsencrypt/live/r-proxy.nas.ethopol
is.tech/fullchain.pem;
ssl_certificate_key /usr/local/etc/letsencrypt/live/r-proxy.nas.ethopol
is.tech/privkey.pem;
ssl_session_cache shared:SSL:1m;ssl_session_timeout 10m;
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
location / {
root html;
index index.html index.htm;
}
}
}
…
But by executing the following command:
…
openssl s_client -connect r-proxy.nas.ethopolis.tech:443
…
I get the message “gethostbyname failure”:
…
root@reverse-proxy:~ # openssl s_client -connect r-proxy.nas.ethopolis.tech:443
gethostbyname failure
connect:errno=0
…
Yeah, I don’t know what to tell you dude. Follow the guide I wrote? It works; I didn’t write it for no reason.
It looks like you have multiple problems here, I’d start with the networking issue as nothing will work if you can’t get traffic to the jail. Like Samuel said, you have the jail on a completely different subnet and there are no routing rules anywhere to get the traffic to the jail, port forwarding it sending traffic to the FreeNAS IP will not do anything as the jail is, in effect, a different box on a different network. While it is probably possible to put in a janky forward rule in the FreeBSD firewall, it is probably better and easier to just reconfigure your jail to be on the same network. Once you fix this, I think you’d be surprised to find most of your other issues disappear.
@ANGELO CORBO
Do you still need help with an nginx setup? I’m sorry I didn’t see your questions until now.
This was a great! Thank you so much, I’ve been wanting to host some projects running on my freenas publicly. I had a few issues setting up route53, but other than that all your steps were very easy to follow!
Thanks very much for the guide. I do not have it working yet
I got my wildcard cert up fine
Nginx is running with no errors, used modern config for ssl
In pfsense I could not figure out how to make my NAT look like your example…
I should add that I can access Nextcloud on the local nw using http when I put in a LAN rule. This was useful in reinstalling nextcloud which I did today.
I can ping my router from the jail and vice versa,
SO, any suggestions would be super helpful. I’m inclined to think it is my routing.
Route 53 confirms it’s working with the WAN addresses for pfsense
Thanks in advance Nic Greene
Nic, the modern configuration probably won’t work yet. From memory, the only protocol it lists is TLSv1.3, which requires OpenSSL1.1.1. FreeBSD 1.3 doesn’t ship with this version, and so the Nginx port isn’t built with compatibility for it out of the box. You could install the newer OpenSSL version and build the port manually against it if you so desire, or you could use the intermediate configuration. This thread has more information: https://forums.freebsd.org/threads/freebsd-11-tls-1-3.70968/
See if using the intermediate configuration helps.
Cheers.
Dear Samuel,
Thanks a lot . I’ll do that with the SSL Config,
Other than that, I guess I am trying to debug the routing systematically, is there a way to figure out where things break down on a setup like this?
Sincerely, Nic Greene
Update: that fixed it! Thanks
This is so cool!
I have these messages in Nextcloud:
• The reverse proxy header configuration is incorrect, or you are accessing Nextcloud from a trusted proxy. If not, this is a security issue and can allow an attacker to spoof their IP address as visible to the Nextcloud. Further information can be found in the documentation.
• Your web server is not properly set up to resolve “/.well-known/caldav”. Further information can be found in the documentation.
I believe the CalDav issue is addressed above,
Has anyone had success with updating their Nextcloud config.php file with trusted proxy? Could you post how you set it up?
Looks like you’ve got this solved, but note that this is addressed in the Nextcloud guide
Hi Samuel,
Thanks a lot for this and nextcloud build tutorial, worked almost like a charm:
A question for the audience:
My only issue, for now, is being stuck in the Account Access when we got redirection from the v2 wizard authentication on the nextcloud desktop app.
My nextcloud version is: 19.0.0 and the nextcloud desktop app version is:2.6.4
Any help will be appreciated
Thanks
For trusted proxies — I’m running my reverse proxy on a different machine than my nextcloud (well they are both virtualized). I also run pfsense as a router. Via a dhcp override I associated the physical IP addresses of both machines (Nextcloud, Reverse proxy) with names —
So for example
Reverse Proxy – IP address – 10.0.1.86 – Name – reverseproxy.domain.com
Nextcloud – IP address – 10.0.1.158 – Name – nextcloud.domain.com
With this information, I manually edited the config.php file and added this to the file (/usr/local/www/nextcloud/config/config.php)
‘trusted_proxies’ =>
array (
0 => ‘reverseproxy.domain.com’,
1 => ‘10.0.1.86’,
),
There is a way to use the command line to do this to avoid syntax errors, but I just found it easier to do manually. Please be very careful of the syntax since I think a malformed config.php file can render your nextcloud installation inoperable. I also don’t know if both the name and IP address are required (possibly you could you just one or the other). This should however give you a starting point.
Hope that helps.
Kevdog – that’s helpful – if the reverse proxy, i.e. this NGINX setup in a nextcloud jail, doesn’t have a name, then just use the ip address of the Reverse Proxy?
I tried this, with a DHCP override too and had no luck, it seemed to bork by config.php file. I did check the syntax and fixed funky single ‘ , but still no joy.
Basically my reverse-proxy is on 192.168.1.xx and nextcloud is on 192.168.1.yy – how could you express that as a trusted proxy statement?
@Nic Greene
Be very careful of the spacing:
‘trusted_domains’ =>
array (
0 => ‘192.168.1.yy’,
1 => ‘nextcloud.gohilton.com’,
),
‘trusted_proxies’ =>
array (
0 => ‘192.168.1.xx’,
),
Make sure to backup your config.php prior to editing and if you have syntax error, we can try something else. Hard to know since you haven’t posted the error you are getting.
Sorry here is the corrected syntax, possibly previous post could be redacted or deleted.
‘trusted_domains’ =>
array (
0 => ‘192.168.1.yy’,
),
‘trusted_proxies’ =>
array (
0 => ‘192.168.1.xx’,
),
Sorry here is the corrected syntax, possibly previous post could be redacted or deleted.
‘trusted_domains’ =>
array (
0 => ‘192.168.1.yy’,
),
‘trusted_proxies’ =>
array (
0 => ‘192.168.1.xx’,
),
Thanks – yes it was the syntax. I ended up copying and pastin the trusted domain statement and altering carefully. Thank you – All checks passed now!
Samuel –
Ok i have everything working now and this is great – added a subdomain to my home assistant RPI easily using the same domain and a different A record. Awesome.
I have one question – given that the Update Route 53 Script/setup in your Nextcloud guide is basically only really pertinent for this reverse proxy now, and it is this proxy that is scripted to update my SSL certs etc., I dont see why I couldnt set that scripting up on this Jail? Is there a reason you wouldn’t do that?
This sounds like a reasonable thing to do Nic, I might raise an issue on github to move it from nextcloud to the reverse proxy jail in a future update
Ok, nice. Thanks
I am following your guide.
To be able to connect to the jail from outside, do I have to have pfsense? or can i set special settings in freeNAS?
In my case I don’t have pfsense. Is it necessary to connect to the VLAN of the jail from outside?
I am not being able to connect to the internet in jail nor can I access it from the outside.
Regards
Alejandro, the configuration you’ve posted so far doesn’t follow anything I’ve specified in my guide, so try to get the configuration I’ve specified working and you might have more luck. As Josh has mentioned, the networking is going to be the place to start. When creating the jail, you specified a value for the
defaultrouterparameter (probably 192.168.0.1). This should be the IP address of your router. Assuming this is the IP address, your jail has to be on the same subnet. This means that in the jail creation command, you should be specifying something similar to the following:If you create it with
ip4_addr="vnet0|172.6.0.2/24"like you have indicated you have, it will not work unless you put routing rules in place to make this network accessible. I strongly advise against attempting to do this, as it seems like you’re new to networking and it’s an unnecessary complication. Better to start with the basics. This will give you internet access within the jail.Now, to connect to a domain from the outside, you need two things:
1. A DNS A record entry to point at your public IP address (mine is with Route 53, other popular services include Cloudflare or Dynamic DNS services)
2. A router that is capable of forwarding traffic using port forwards. I use pfSense for this, but it is not necessary – many routers should have this functionality but you’ll need to work out how to do it yourself.
When you connect to another computer via TCP, you use a socket. A socket is an IP:Port pair, for example 36.12.234.48:443. Port 443 is a common port, because this is the default port used for HTTPS connections. Since each DNS A record entry will just point to an IP address, and you may have multiple subdomains, i.e. Bitwarden.example.com and cloud.example.com, you have multiple subdomains pointing to the same IP address, i.e. both of these will resolve the socket 36.12.234.48:443 when accessed via a web browser. The problem posed here is, if we have multiple services that all point to the same IP, how can we differentiate them? The solution is to use the reverse proxy. This uses the ‘Host’ header as a differentiator, which contains the subdomain name specified (i.e, cloud.example.com; this is the value specified in the
server_namedirective). This means that we need the reverse proxy to handle the traffic. To do this, we need to accept the traffic at the router, and redirect it to the reverse proxy jail. This is what a port forward does.So to answer your question, no, you don’t need pfSense. Though, you do need a router capable of port forwarding. It’s worth noting that you could also set up a pfSense VM/jail and use that as your router. I don’t love this solution because it means connecting the unfiltered internet directly in to your NAS, so you would want to make sure you have a separate NIC, and have that NIC only available to the pfSense jail/VM, but this poses it’s own issues and is probably requires a lot of familiarity with networking and *nix.
There is nothing you can do from within FreeNAS to replace the role of the router, short of setting up a router in a VM or a jail.
Very good tutorial, amazing, thank you, I have a question, I have a comodo certificate, and Godaddy with DNS, how do I setup using this certificate? Thank you
Hello Samuel and others!
My nginx reverse proxy that I built using this guide is working great, but I’m trying to work through an issue I’m having.
I’m not sure if there are any folks using Standard Notes, but I’m setting up a syncing server on my debian machine. The nginx is on my FreeNAS machine, and the standard notes server is on a separate debian machine. The server is running and working well. All notes are able to sync via windows, web, and iOS using my FQDN.
I’m having issues with the extensions piece. It has to point to a specific folder on the debian machine located at: /home/phil/standardnotes-extensions/public. My nginx vdomain file is pasted below. I can navigate to the sync server just fine using notes.mydomain.com, but when I try to navigate to notes.mydomain.com/extensions/index.json I get a 404 director or file does not exist. I know the path is correct and the file does exist and I can cat the index.json items just fine. Any help you can provide would be fantastic. Thank you in advance!
Error log from ningx:
2020/08/28 11:38:38 [error] 31289#102740: *48868 open() “/home/phil/standardnotes-extensions/public/index.json” failed (2: No such file or directory), client: 192.168.150.1, server: notes.mydomain.com, request: “GET /extensions/index.json HTTP/2.0”, host: “notes.mydomain.com”
nginx vdomain file for the sync server:
server {
listen 443 ssl http2;
server_name notes.mydomain.com;access_log /var/log/nginx/notes.access.log;
error_log /var/log/nginx/notes.error.log;
include snippets/mydomain.com.cert.conf;
include snippets/ssl-params.conf;
location / {
include snippets/proxy-params.conf;
proxy_pass http://192.168.150.20:3000;
}
location ^~ /extensions {
autoindex off;
alias /home/phil/standardnotes-extensions/public;
# CORS HEADERS
if ($request_method = 'OPTIONS') {
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
#
# Custom headers and headers various browsers *should* be OK with but aren't
#
add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range';
#
# Tell client that this pre-flight info is valid for 20 days
#
add_header 'Access-Control-Max-Age' 1728000;
add_header 'Content-Type' 'text/plain; charset=utf-8';
add_header 'Content-Length' 0;
return 204;
}
if ($request_method = 'POST') {
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range';
add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range';
}
if ($request_method = 'GET') {
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range';
add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range';
}
}
}
Hi I perused your setup. What are the ip addresses of your machines?
Hey Kevdog. You’ve been very helpful in the past.
My nginx machine is on 192.168.150.15.
My debian machine is on 192.168.150.20
Redirected you too many times error.
And cloud.error.log showed empty. Any clue guys?
Figured it out, turns out it is DNS thats is making trouble. Once I set cloudfare to full encryption everything is fixed;/
Hello Samuel,
thank you for your tutorial.
Quick question, how can I install mod_security with coreruleset owasp?
Hi zibellon, that’s far from a quick question and pretty far afield from the content I’ve presented, but here are some links that should direct your research:
https://forums.freebsd.org/threads/install-mod_security-on-nginx-webserver.53286/
https://www.nginx.com/blog/compiling-and-installing-modsecurity-for-open-source-nginx/
https://www.freshports.org/security/modsecurity3-nginx/
https://github.com/SpiderLabs/ModSecurity-nginx
You can use
pkg search $KEYWORDSto identify what the appropriate packages in the freebsd repositories might be.Hope this helps.
Cheers.
suggestion: add log rotation, after couple of month you will get too much history there.
for me its a personal server with low traffic, so i set it to once a month, you can edit $M1D0 for it.
https://gist.github.com/killercup/5698316
how to: type
nano /etc/newsyslog.conf.d/nginx-reverse.conf
paste:
log rotation for nginx reverse proxy
logfilename [owner:group] mode count size when flags [/pid_file] [sig_num]
/var/log/nginx/*.log 640 7 * $M1D0 GB /var/run/nginx.pid 30
Hi, I have this running and working but can’t work out how to get reverse proxy working for multiple servers on same subnet. How does nginx know what you are wanting when you just go to https://domain or do you need to go to https://domain/ombi. Do you need to create a proxy_setup.conf and get nginx.conf to use.
Any help would be must appreciated and guide was the best I have found
Thanks,
Jay
Hi Jay, Nginx uses the
Hostheader to determine where the request should go. If you typehttps://subdomain.domain.comin to the URL bar in a browser, ‘subdomain.domain.com’ will be populate the ‘Host’ header in the request the browser sends. This is matched in theserverblock to theserver_namedirective. In order to have multiple servers, you need to have an A record that corresponds to each server, and a server block in your nginx configuration. If you’ve followed my guide, this will be satisfied by simply creating a new.conffile in thevdomains/directory; i.e.,vdomains/subdomain1.domain.com.confandvdomains/subdomain2.domain.com.conf, with appropriate values for theserver_namedirectives. You’ll also need to make sure that theproxy_passdirective points to the actual IP of the server the service runs on. Whether these servers are on the same subset or not is immaterial to this process provided you have the correct routing in place, otherwise having the servers on the same subnet actually makes everything easier.Hope this helps.
Cheers,
Sam
Edit: I’ve just re-read your question. If you’re asking about how you can differentiate between servers pointing to the same URL, then you’re right, you can’t. This is not the point of a reverse proxy. You must have either a subdomain (https://subdomain.domain.com) OR a unique path (https://www.domain.com/servicename). The guide I’ve provided here is for the former.
Hello,
When running “service nginx start” I get the following warning:
root@r-proxy:/usr/local/etc/nginx # service nginx start
Performing sanity check on nginx configuration:
nginx: [warn] “ssl_stapling” ignored, host not found in OCSP responder “ocsp.int-x3.letsencrypt.org” in the certificate “/usr/local/etc/letsencrypt/live/kittycooper.tk/fullchain.pem”
nginx: [warn] “ssl_stapling” ignored, host not found in OCSP responder “ocsp.int-x3.letsencrypt.org” in the certificate “/usr/local/etc/letsencrypt/live/kittycooper.tk/fullchain.pem”
nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful
root@r-proxy:/usr/local/etc/nginx #
I have the following code configured in “snippets/ssl-params.conf”:
ssl_session_timeout 1d;
ssl_session_cache shared:MozSSL:10m; # about 40000 sessions
ssl_session_tickets off;
modern configuration
ssl_protocols TLSv1.3;
ssl_prefer_server_ciphers off;
HSTS (ngx_http_headers_module is required) (63072000 seconds)
add_header Strict-Transport-Security “max-age=63072000” always;
OCSP stapling
ssl_stapling on;
ssl_stapling_verify on;
replace with the IP address of your resolver
resolver 8.8.8.8 ;
What could be the problem?
Hi Alex, looks like it’s probably related to your DNS Resolver. See this thread/similar for more information: https://community.letsencrypt.org/t/ssl-stapling-sometimes-fails-on-nginx/105926. If you google the warning you’ll be able to find other threads 🙂
Cheers
Hello again.
I have managed to configure the reverse proxy successfully.
I have a doubt.
If I access “heimdall.example.com” from my local network I have to be able to see the site, but if I try to access from a remote network or VPN, shouldn’t it let me in?
Because I did the tests and I can access “heimdall.example.com” from different networks. That is wrong, how is it possible?
Regards
Alex
Hi Alex, not sure specifically what you’re after, but my best guess is that you’re able to access
heimdall.example.comfrom networks that you don’t want to be able to access it. This is because your reverse proxy is routable from those networks. The only requirements for the services being proxied are:server_nameThis is what happens if you have an A record entry for a domain/subdomain with a DNS service provider (or on your own internal network). If you want to restrict this to requests from specific networks, you’ll have to use the Nginx access control directives such as
allowanddeny. You can read more about it here: https://docs.nginx.com/nginx/admin-guide/security-controls/controlling-access-proxied-tcp/According to what the documentation you have given me indicates: There it explains that you have to use the stream {} parameter for the restrictions to take effect.
Regarding the tutorial you published, it is observed that the file containing “allow” and “deny” directives in “internal-access-rules.conf” is inside the “server {}” parameter but it is not inside the stream { } parameter as mentioned in the documentation.
The instruction allows access from the 192.168.0.0/24 network; and denies everything else, but in practice that does not work.
Regards
Hello Samuel. Thank you for the thorough write-up! I was able to follow your instructions but it would have been helpful for a complete noob like me for you to spell out exactly what you should change your “resolver” to and how you (Samuel) have your network setup as (hierarchy). Anyways, I think I’m about there but have run into a specific issue. In my case I’m assuming this would be 192.168.1.0 with 192.168.1.1 being my Untangle UTM?
I have a server 192.168.1.10 running multiple services which I would like to have available outside my LAN (Emby, Nextcloud, Home Assistant, Zoneminder) and they all have their own https connection through a port forward. So therefore I’m assuming I can’t just have 443 forwarded to 192.168.1.10? My first vdomain is for Emby and is called emby.example.stream.conf. I was unable to access https://emby.example.stream as I was getting a 502 error. I then realized that the emby.example.stream.conf file had “listen 443 ssl http2;” so I changed it to my port forward for Emby “listen xxxxxx”. This then resulted in getting a ‘server not found’.
My question is – how do I setup each conf file so that each service can be reached externally via https://service.example.stream? Like I said, I’m completely new to reverse proxy and I’ve managed to muddle my way through thus far. If you could help, I would greatly appreciate it! Thanks!
Hey Diedrich, have a read over the guide again, it’s easy to miss detail on a once through. The first image of the conceptual role of a reverse proxy matches my network topology pretty closely for the purposes of this, and as for the resolver question, this means the DNS Resolver, that associates a URL with an IP. In my case, this is my router, 192.168.0.1. I use pfSense, which has a DNS Resolver function that lets me specify host overrides, and otherwise queries another upstream DNS server (i.e., Google, Cloudflare, OpenDNS) to resolve the hostnames it has to process. If you want to know more about how DNS works, consult the wikipedia article here. If your router doesn’t have this feature, still set your
resolverto be your router; I would imagine it would still forward these on (though I could be wrong). The only downside to not being able to specify a host override is that when you’re on your local network, the request would go out via the internet rather than just staying within your network.The problem you’re asking me about is exactly why you would want a reverse proxy. As I discuss in the guide, you forward port 443 on your WAN to port 443 of your reverse proxy jail on your LAN. Your reverse proxy jail (where your nginx reverse proxy lives), is what is listening on port 443, so you don’t want to change that. The only things that you should need to change in the vdomain configuration file (with the exception of application specific requirements), are the
server_nameandproxy_passdirectives. Theserver_namedirective is the URL you want to be able to access the service from externally. Theproxy_passdirective is the local IP/hostname of the service on your LAN. If your services are serving their own HTTPS, make sure you usehttps://in theproxy_passdirective rather than justhttp://.Read over the guide again a few times. I have a section that specifically deals with what you need to do to make your service available externally, or just internally, and a description of how it works.
Hope this helps.
Cheers.
Hi @deidrich not sure I can help you without some more information.
Can you describe the domain names of your reverse proxy and your Emby machines and their associated IP addresses?
Are you running a split DNS within Untangle?
I assume your using nginx as a reverse proxy? I’ve never set up Emby so I don’t know the configuration at all. Could you post your nginx server conf file for your Emby server?
Mr Dowling-
I’ve spent several hours over the last day trying to work out how to get a wildcard certificate. The problem was that I needed from a DNS service:
1. dynamic DNS for my domain
2. a plug-in or API (scratch off my registrar, Google Domains (not google cloud)
3. Easy setup for a beginner (eliminated aws / google cloud / azure / etc. for now)
4. Inexpensive
I found and settled on Dynu.
I still had trouble setting up the reverse proxy jail to get a certificate, but I found out that pfsense had the option to do the job.
I now have the certificates on pfsense (and I downloaded a cert and a key from pfsense), but now I don’t know how to implement the cert with nginx. Do I need to do this, or is it good enough to have pfsense maintaining and renewing the cert as needed?
Thanks,
Jason
Hi,
I am trying to put my existing Nextcloud server behind a reverse proxy, and was very happy to find this guide, as I used your guide for setting up Nextcloud in the first place. I´ve run through this guide, but since I´d like to keep Nextcloud working with its current certificate (cloud.mydomain.com), I picked another DNS name for this experiment (nextcloud.mydomain.com). I´ve also skipped the wildcard bit, and gone for a certificate by that name too.
After this I pointed my port forward to the proxy jail instead. Now, as I enter nextcloud.mydomain.com into a browser, it chnges to cloud.mydomain.com, and states that the connection is insecure.
I wonder what I might have to change on the Nextcloud server for it to work with the new certificate on the proxy jail. Any tips for me?
Thank, and again, thank you for your great guides!
Hey Andreas. It sounds like you don’t want the SSL termination component to the reverse proxy. You could just have the reverse proxy forward the https connection directly by using
proxy_pass https://cloud.domain.comand removing the SSL directives (ssl-params.conf, domain.com.conf). You might have to experiment with which parameters exactly need to be removed though. This is probably a better solution to your problem rather than having two endpoints.To answer your question though, the reason it’s directing you to cloud.domain.com is probably because you have a HTTPS redirect in your Nextcloud apache config. If you remove this redirect you’ll be able to access nextcloud by it’s direct IP over HTTP. if you
proxy_passto this IP, your reverse proxy should terminate SSL for that connection.Hope this helps.
Cheers.
Hi,
At first, I liked the idea of leaving Nextcloud alone, and just proxy_pass to https://cloud.domain.com. But then I realised that it won´t work, since the proxy and nextcloud share the same public IP. Right?
So, I think I need to move the certificate bit to the proxy, and then proxy_pass to the internal IP of the NC server. Unless I´m missing something? I can only find one forward to cloud.mydomain.com in the Apache config, and it is in the Virtualhost for port 80. I´m sorry to dump this on you, but if you can see something I´ve missed I´d really appreciate the help…
This is my Vhost file:
ServerName cloud.berglind.eu
Redirect permanent / https://cloud.mydomain.com/
ServerAdmin me@mydomain.com
ServerName cloud.mydomain.com
SetHandler “proxy:fcgi://127.0.0.1:9000/”
DirectoryIndex /index.php index.php
DocumentRoot /usr/local/www/nextcloud
SSLCertificateFile /usr/local/etc/letsencrypt/live/cloud.mydomain.com/fullchain.pem
SSLCertificateKeyFile /usr/local/etc/letsencrypt/live/cloud.mydomain.com/privkey.pem
SSLEngine on
# modern configuration, tweak to your needsSSLProtocol all -SSLv3 -TLSv1 -TLSv1.1
SSLCipherSuite ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA2>
SSLHonorCipherOrder on
SSLCompression off
SSLSessionTickets off
SSLOptions +StrictRequire
<IfModule mod_headers.c>
Header always set Strict-Transport-Security "max-age=15552000; includeSubDomains"
</IfModule>
And this is httpd.conf (Since it´s quite long, I took the liberty of removing every line starting with # from this post):
ServerRoot “/usr/local”
Listen 80
Listen 443
LoadModule mpm_prefork_module libexec/apache24/mod_mpm_prefork.so
LoadModule authn_file_module libexec/apache24/mod_authn_file.so
LoadModule authn_core_module libexec/apache24/mod_authn_core.so
LoadModule authz_host_module libexec/apache24/mod_authz_host.so
LoadModule authz_groupfile_module libexec/apache24/mod_authz_groupfile.so
LoadModule authz_user_module libexec/apache24/mod_authz_user.so
LoadModule authz_core_module libexec/apache24/mod_authz_core.so
LoadModule access_compat_module libexec/apache24/mod_access_compat.so
LoadModule auth_basic_module libexec/apache24/mod_auth_basic.so
LoadModule socache_shmcb_module libexec/apache24/mod_socache_shmcb.so
LoadModule reqtimeout_module libexec/apache24/mod_reqtimeout.so
LoadModule filter_module libexec/apache24/mod_filter.so
LoadModule mime_module libexec/apache24/mod_mime.so
LoadModule log_config_module libexec/apache24/mod_log_config.so
LoadModule env_module libexec/apache24/mod_env.so
LoadModule headers_module libexec/apache24/mod_headers.so
LoadModule setenvif_module libexec/apache24/mod_setenvif.so
LoadModule version_module libexec/apache24/mod_version.so
LoadModule proxy_module libexec/apache24/mod_proxy.so
LoadModule proxy_fcgi_module libexec/apache24/mod_proxy_fcgi.so
LoadModule ssl_module libexec/apache24/mod_ssl.so
LoadModule unixd_module libexec/apache24/mod_unixd.so
LoadModule status_module libexec/apache24/mod_status.so
LoadModule autoindex_module libexec/apache24/mod_autoindex.so
#LoadModule cgid_module libexec/apache24/mod_cgid.so
#LoadModule cgi_module libexec/apache24/mod_cgi.so
LoadModule dir_module libexec/apache24/mod_dir.so
LoadModule alias_module libexec/apache24/mod_alias.so
LoadModule rewrite_module libexec/apache24/mod_rewrite.so
Third party modules
IncludeOptional etc/apache24/modules.d/[0-9][0-9][0-9]_*.conf
User www
Group www
ServerAdmin me@mydomain.com
ServerName 192.168.1.235:80
AllowOverride none
Require all denied
DocumentRoot “/usr/local/www/nextcloud”
Options Indexes FollowSymLinks
AllowOverride All
Require all granted
DirectoryIndex index.html
Require all denied
ErrorLog “/var/log/httpd-error.log”
LogLevel warn
LogFormat “%h %l %u %t \”%r\” %>s %b \”%{Referer}i\” \”%{User-Agent}i\”” combined
LogFormat “%h %l %u %t \”%r\” %>s %b” common
<IfModule logio_module>LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
</IfModule>
CustomLog "/var/log/httpd-access.log" common
ScriptAlias /cgi-bin/ “/usr/local/www/apache24/cgi-bin/”
AllowOverride None
Options None
Require all granted
RequestHeader unset Proxy early
TypesConfig etc/apache24/mime.types
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
Include etc/apache24/extra/proxy-html.conf
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
SSLSessionCache “shmcb:/var/log/apache/ssl_gcache_data(512000)”
SSLSessionCacheTimeout 600
SSLUseStapling On
SSLStaplingCache “shmcb:/var/log/apache/ssl_stapling(32768)”
Include etc/apache24/Includes/*.conf
I haven’t looked at an apache config in a long time, so do your own research to confirm this, but I believe removing the second line should stop your nextcloud instance from redirecting to https://cloud.domain.com:
It shouldn’t matter that both the reverse proxy and your nextcloud instance share the same public ip, provided you have an internal host override for your domain name (you should regardless of your proxy, otherwise all of your requests to your local server are hitting the internet, which is wildly inefficient). I.e., if you have some configuration in your router to say that cloud.domain.com resolves to the internal ip (192.168.x.x), then it will work. If you don’t have this then you’re probably right.
And yes, I think it is a better idea to move SSL handling to the reverse proxy.
Cheers.
i got this working a while back. I revisited your nextcloud guide, which is updated. I took the new vhost from there, which is only for http now, since https has been moved to the proxy. After that it works great! So great in fact that adding more servers really is very simple! Thank you again. But, of course, there is always a question;
I´ve done this on a couple of installations now, and I noticed that the favicon for Nextcloud is no longer shown in the browser. Not exactly a huge problem, but I would like it back. Have you seen this before? do you know what to do about it?
Kind of solved it, but not really. I uploaded the favicon manually in the Theming section. Works, now it shows in the browser again
@ANDREAS BERGLIND
Way back when I ran into the same issue as you — first setting up my nextcloud server, then adding a reverse proxy and then realizing that — oops — I have a conflict with names since nextcloud or in your case cloud.mydomain.com now has to resolve to the reverse proxy rather than nextcloud.
So just some things you want to consider or ponder
It’s really helpful to run your own local DNS resolver (with backup to 9.9.9.9 or 8.8.8.8) on your network. I’m using pfSense with the unbound DNS resolver. It basically allows for a split DNS setup where names such as cloud.mydomain.com or nextcloud.mydomain.com will resolved to local IP addresses and not public facing IP addresses.
When adding a reverse proxy in the mix — you need to kind of figure out how you want it to act — Do you want an SSL terminating proxy that will proxy-pass to the backend using http? Do you want a TCP passthrough proxy that does nothing other than examine the SAN headers and pass the connection to the backend without touching the encryption, or do you want a decryption/re-encryption proxy that would decrypt the incoming traffic and then re-encrypt and proxy-pass to the backend over https? I know it’s a lot to ponder however its really going to depend on the capabilities of your reverse proxy.
I don’t know a lot about Apache as a reverse proxy in that I’ve mainly used it in the past as a webserver. I mostly run nginx/traefik and sometimes HAProxy as my reverse proxies. I’m aware apache can terminate the SSL connection and proxy-pass to the reverse end over http. Not sure if it can tls passthrough or decrypt/re-encrypt
Once you experiment and figure out how you want to run your reverse proxy, it will kind of determine how you want to setup the remaining pieces of your configuration.
Other things to think about and now I’m speaking totally off the cuff
1. You might want to consider changing the webserver associated with nextcloud from apache to nginx. In general I found it to be a little quicker and robust — robust in the sense that you begin learning how to setup both a webserver and many of the skills learned translate to setting up an nginx reverse proxy
2. Where is your reverse proxy located? On the same machine as nextcloud or another machine/VM/Container etc? I currently have my nextcloud running within a FreeNAS jail as instructed by this more than wonderful guide Sam Dowling has created and maintains. It’s really really good. The only issue I have with FreeBSD in general is it’s inability to run docker (See point #3)
3. If choosing to run a reverse proxy – there are some wonderful projects such traefik and nginx proxy manager that provide a GUI that really assists in setting up a reverse proxy. These reverse proxies will auto obtain and renew Let’s encrypt certs, so that’s one less thing to worry about. The GUI is also really nice since it very much helps trying to debug why the reverse proxy isn’t working as intended. It also helps when you don’t mess with the reverse proxy for months, then revisit it as you scratch your head — hmm I kind of remember messing with this before but forget the fine details
4. Traefik and Nginx Proxy Manager (NPM) have the limitation that they are both docker projects — and by nature docker projects require linux. Kind of a limitation with bsd for sure. To work around the limitation I’ve installed Ubuntu/Arch VMs within FreeNAS and then ran these reverse proxies through docker.
5. A straight nginx reverse proxy could be run within a freebsd jail with an acme backend for LE certs, however I’ll admit the automation of certificate management in using reverse proxies like traefik, NPM, and Caddy sure are nice.
Probably far too much information and rambling than what you wanted. Just some thoughts.
Hi, i get this error when i want to start nginx, and i dont know what to change 🙁
nginx: [emerg] “ssl_trusted_certificate” directive is duplicate in /usr/local/etc/nginx/snippets/ssl-params.conf:6
nginx: configuration file /usr/local/etc/nginx/nginx.conf test failed
Starting nginx.
nginx: [emerg] “ssl_trusted_certificate” directive is duplicate in /usr/local/etc/nginx/snippets/ssl-params.conf:6
/usr/local/etc/rc.d/nginx: WARNING: failed to start nginx
Hi Tita, I’ve never seen this error before but it seems to suggest you have
ssl_trusted_certificatedeclared in multiple locations. If you’ve changed the path, make sure the only instance lives insnippets/example.com.cert.conf. It looks like you also have it declared insnippets/ssl-params.conf– remove it from here. Hope this helps. Cheers.Samuel this is an amazing guide! Thank you for taking the time to put this together. I’ve never setup a reverse proxy before, but following this guide made it so simple.
I’m having an issue that’s probably more related to Radarr than the reverse proxy, but hoping that someone can help. I’ve created the conf file to reverse proxy to my local Radarr address and that appears to be working fine. I’m able to access the login page by going to radarr.mydomain.com. However, I have Radarr configured for authentication – using Forms (Login Page). I input my credentials to login to Radarr, but when I submit them I immediately receive an HTTP 400 error:
“This page isn’t working right now
If the problem continues, contact the site owner.
HTTP ERROR 400”
The interesting bit is I have Sonarr configured the exact same way as Radarr, both from an application perspective but also from a reverse proxy perspective. When I access sonarr.mydomain.com I’m able to login on the Forms (Login Page) and can actually access the app (no HTTP 400 error). There are no entries in the radarr.error.log file either. So at this point I’m not sure what to do next. Any suggestions?
Hi there. Not that familiar with Radarr, but some research indicated that there are some specific entries you could try. Try replacing the
include snippets/proxy-params.confline with the following:The key difference here is overriding the
proxy_set_header Connectiondirective to have the value$http_connectionrather than"upgrade", which is a new requirement for Radarr v3 – perhaps this is why it’s not an issue with sonarr? This comment on reddit provides some more detail.Hope this helps.
Cheers.
Yep, that was it! Working perfectly now. Thanks again!
Ok, got another one. Trying to configure Plex to use the reverse proxy. Has anyone done this before on FreeNAS? I’ve been able to find a couple of guides but nothing specific to FreeNAS.
Here’s what I’ve done so far:
– Created a separate config file for Plex in the vdomains directory.
– On the DNS side I’ve created a cname record that points plex.mydomain.com to my Dynamic DNS (WAN IP).
– Within Plex I’ve set the Custom server access URLs to https://plex.mydomain.com:443.
Outcome:
When I try to navigate to https://plex.mydomain.com I initially see a 401 Unauthorized error but then I’m quickly redirected to https://plex.mydomain.com/web/index.html where I see the Plex logo on the page but it never progresses to my plex server. Appears to just hang here.
I’m unsure if this is a plex config file issue in nginx or if I’m missing some other setting(s) outside of nginx. Below is my plex config file (found from other guides online):
server {
listen 443 ssl http2;
server_name plex.mydomain.com;
send_timeout 100m;resolver 1.1.1.1 valid=300s;
resolver_timeout 10s;
include snippets/mydomain.com.cert.conf;
ssl_stapling on;
ssl_stapling_verify on;
ssl_dhparam /usr/local/etc/nginx/snippets/dhparam.pem;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
#Intentionally not hardened for security for player support and encryption video streams has a lot of overhead with something like AES-256-GCM-SHA384.
ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';
#Plex has A LOT of javascript, xml and html. This helps a lot, but if it causes playback issues with devices turn it off. (Haven't encountered any yet)
gzip on;
gzip_vary on;
gzip_min_length 1000;
gzip_proxied any;
gzip_types text/plain text/css text/xml application/xml text/javascript application/x-javascript image/svg+xml;
gzip_disable "MSIE [1-6]\.";
#Nginx default client_max_body_size is 1MB, which breaks Camera Upload feature from the phones.
#Increasing the limit fixes the issue. Anyhow, if 4K videos are expected to be uploaded, the size might need to be increased even more
client_max_body_size 100M;
#Forward real ip and host to Plex
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Plex headers
proxy_set_header X-Plex-Client-Identifier $http_x_plex_client_identifier;
proxy_set_header X-Plex-Device $http_x_plex_device;
proxy_set_header X-Plex-Device-Name $http_x_plex_device_name;
proxy_set_header X-Plex-Platform $http_x_plex_platform;
proxy_set_header X-Plex-Platform-Version $http_x_plex_platform_version;
proxy_set_header X-Plex-Product $http_x_plex_product;
proxy_set_header X-Plex-Token $http_x_plex_token;
proxy_set_header X-Plex-Version $http_x_plex_version;
proxy_set_header X-Plex-Nocache $http_x_plex_nocache;
proxy_set_header X-Plex-Provides $http_x_plex_provides;
proxy_set_header X-Plex-Device-Vendor $http_x_plex_device_vendor;
proxy_set_header X-Plex-Model $http_x_plex_model;
proxy_set_header Host $server_addr;
proxy_set_header Referer $server_addr;
proxy_set_header Origin $server_addr;
#Websockets
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
#Buffering off send to the client as soon as the data is received from Plex.
proxy_redirect off;
proxy_buffering off;
access_log /var/log/nginx/plex.access.log;
error_log /var/log/nginx/plex.error.log;
location / {
proxy_pass http://IP_OF_FREENAS_JAIL:32400;
}
}
I’m running into a weird issue with nextcloud with the reverse proxy. I setup the vdomains for plex, nextcloud, and netdata. However, all subdomains go to nextcloud page that says not trusted domain.
plex.domain –> plex
monitor.domain –> netdata
cloud.domain –> nextcloud
ansda.domain –> nextcloud trusted domain page
anything.domain –> same as above.
Any ideas? I would like subdomains that don’t exist in my vdomains folder to go to 404 page.
Hi, I have the same issue.
In my case
nas.example.org – localy goes to truenas gui – (vdomain configured) this is good
cloud.example.org – goes to nextloud (vdomain configured) also good
example.com and any subdomain created in dns provider but not configured as vdomain goes to nextloud trusted domain page
Why nginix redirect not recognized requests to nextcloud jail?
Anyone know solution?
Best regards!
@rafal
I believe I’ve run into this problem before. It’s how you’ve set up your nginx.conf file (or files as in the case of this tutorial where you setup up various snippet files which nginx in the end concatenates all the individual files into one). Nginx is going to traverse the chain of defined domains looking for a match. I believe it if doesn’t find a match, it defaults to using the first named domain within the file. I’d recommend setting up your file(s) and defining a “catchall” domain at the end which could redirect to a specific site or 404.
I hope this made sense.
@KEVDOG,
Thanks a lot for the hint.
It was not easy to do, but it finally worked.
Many attempts and failures. Since we are using a general redirect to https, we cannot use catchall server with listening on 80 because after redirection it still hits one of the available subdomains – in my case cloud.example.com
I gave up configuring vdomains in separate files and moved all settings to nginx.conf.
I removed the “default_server” entry from server directive that listens to 80 and redirects to https.
Then at the end of the file I have added:
#catchall
server {
listen 443 ssl http2 default_server;
server_name catchall;access_log /var/log/nginx/catchall.access.log;
error_log /var/log/nginx/catchall.error.log;
include snippets / example.org.cert.conf;
include snippets / ssl-params.conf;
return 444;
}
return 444 will drop any connection to not recognized domain.
return 400 will reply that host not exist
I chose 400 because I think it is safer and uses less resources.
My nginx.conf file is such:
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
client_max_body_size 1024M;
# Redirect all HTTP traffic to HTTPSserver {
listen 80;
return 301 https://$host$request_uri;
}
# Import server blocks for all subdomains
# disabled include "vdomains/*.conf";
#truenas
server {
listen 443 ssl http2;
server_name nas.example.org;
access_log /var/log/nginx/nas.access.log;
error_log /var/log/nginx/nas.error.log;
include snippets/example.org.cert.conf;
include snippets/ssl-params.conf;
location / {
include snippets/proxy-params.conf;
include snippets/internal-access-rules.conf;
proxy_pass http://192.168.0.111;
}
}
#nextcloudserver {
listen 443 ssl http2;
server_name cloud.example.org;
access_log /var/log/nginx/cloud.access.log;
error_log /var/log/nginx/cloud.error.log;
include snippets/example.org.cert.conf;
include snippets/ssl-params.conf;
proxy_hide_header Strict-Transport-Security;
location / {
include snippets/proxy-params.conf;
proxy_pass http://192.168.0.221;
}
}
#test
server {
listen 443 ssl http2;
server_name test.example.org;
access_log /var/log/nginx/test.access.log;
error_log /var/log/nginx/test.error.log;
include snippets/example.org.cert.conf;
include snippets/ssl-params.conf;
location / {
include snippets/proxy-params.conf;
proxy_pass http://192.168.0.113;
}
}
#catchall
server {
listen 443 ssl http2 default_server;
server_name catchall;
access_log /var/log/nginx/catchall.access.log;
error_log /var/log/nginx/catchall.error.log;
include snippets/example.org.cert.conf;
include snippets/ssl-params.conf;
return 444;
}
}
Hello Micah
Solution in post below
Another great guide Samuel thank you.
Can you advise if nginx is installed with the stub_status? I was looking to moitor the reverse proxy with Grafana however when try to test if the HttpStubStatusModule is installed through nginx -V 2>&1 | grep -o with-http_stub_status_module
I get the responce Ambiguous output redirect.
Hi Sam,
This has been running beautifully for me, for over a year. Last week, I stupidly upgraded my jails to TrueNas 12.2 RELEASE p6. My other jails are functioning without issue, but I’m no longer able to connect to any of my domains through the reverse proxy setup. The jail is up and running. The services seem to all be there and up as well. Where do I look for a log for troubleshooting? Are you aware of any issues with the current release of FreeNas?
Thanks in advance
Soygen
Disregard! I just had to upgrade a pkg.
Thank you very much for this guide.
I got everything set up with nginx serving as the reverse proxy for my Home Assistant jail.
I haven’t quite gotten the automatic renewal cron going though.
I’m very much a layman, so I’m not sure what this call does:
“/usr/local/bin/python -c ‘import random; import time; time.sleep(random.random() * 3600)'”
Thanks again for the help,
CFC
For people who are looking to complete the DNS configuration, DNS resolver or host overrides feature. If you don’t have Pfsense. Look into “dnsmasq”. I believe it is the same function on a regular router.
In the dnsmasq.conf file add lines like this:
address=/example.com/
cname=www.lawrencesystems.com,lawrencesystems.com
I just found this solution have not tested yet.
Does anyone know how should I test this part? How do I know if hitting “cloud.example.com” does not go to public DNS and come back to my router? Instead, we want this to directly hit reverse-proxy’s IP.
@Peter
From command line from a computer sitting behind your router do a dns lookup of the address. The actual name of the program is going to depend on the host operating system. I usually do dns lookups trim command line. Part of the output of the dns lookup should show you the name or IP addresses of the definitive name servers. Dns lookups usually occur using port 53 if not using DOH or encrypted lookups…there shouldn’t be a firewall issue but just pointing this out.
On mac os terminal. You can use dig(1) command to do dns lookup. thanks @kev.
DNSmasq does work and is what you need for host overrides.
Starting nginx warning :
root@reverse-proxy:/ # service nginx start
Performing sanity check on nginx configuration:
nginx: [warn] low address bits of 192.168.x.x/xx are meaningless in /usr/local/etc/nginx/snippets/internal-access-rules.conf:1
nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful
Starting nginx.
nginx: [warn] low address bits of 192.168.x.x/xx are meaningless in /usr/local/etc/nginx/snippets/internal-access-rules.conf:1
Did anyone run into this?
Solved this, I edited the internal-access-rules.conf by accident. It should be kept like this. No edit needed.
allow 192.168.0.0/24;
deny all
After some time of wrangling with the machine. I am glad to report this still works as of the time of writting. Follow the guide exactly and don’t deviate. I thought I know what I am doing and altered a few things but eventually change back to the same config as the post.
I know Samuel suggested use intermediate or mordern ssl-settings. Check your OS version and make sure it works with mordern. Or just start with intermediate.
Hi everybody,
Hard to get it working when you are not familiar with that technology.
I managed to get the proxy up an running but the certificate seems to not work as I get a warning in my browser.
The acme chalenge is ok, accessing my nextcloud from outside is ok, I am not sure in which direction to look to find out what the error is.
If one of the eperimented guys in here can give me a hint it would be greatly appreciated.
thanks
FYI if you have issues uploading files in your local network through the Android app with a “no connection” error (even though you can upload from an external network): switch from modern to an intermediate ssl configuration.
I’m having a weird error where my websites aren’t available outside the network. I am running pfSense behind everything, not sure if this has anything to do with it.
On my phone using LTE I get ERR_SSL_PROTOCOL_ERROR, I’ve looked up this issue and nothing seems to fix it. I made sure my jail’s date and timezone were correct.
Any help would be appreciated.
I figured it out. I didn’t have my Port Forward for the reverse-proxy as the first rules so my email server was catching all the traffic for port 443. This resulted in getting the wrong SSL certificate and the mismatch prevented my websites from loading outside my network.
If someone encounters a problem with the nextcloud for windows client after configuring this proxy:
“The polling URL does not start with HTTPS despite the login URL started with HTTPS”
You have to add trusted proxy entry to config.php file in your nextcloud jail.
nano usr/local/www/nextcloud/config/config.php
add commands:
‘trusted_proxies’ => [‘put here reverse proxy’s IP address’],
‘overwriteprotocol’ => ‘https’,
https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/reverse_proxy_configuration.html
Hello, I’m lost on which one is the Secret access key form AWS I have the ID but where do I get the SecretKey?
I got this error:
An error occurred (InvalidClientTokenId) when calling the ListHostedZones operation: The security token included in the request is invalid.
To use certbot-dns-route53, configure credentials as described at https://boto3.readthedocs.io/en/latest/guide/configuration.html#best-practices-for-confi guring-credentials and add the necessary permissions for Route53 access.
Hi GIORIV,
I think you misunderstood it as I was at the beginning or maybe the article is misleading at this point.
Shouldn’t create a key pair, but a user in your IAM Dashboard:
https://console.aws.amazon.com/iam/
You have to choose access type:
Programmatic access
Enables an access key ID and secret access key for the AWS API, CLI, SDK, and other development tools.
Then it works. Regards.
Oh, thanks so much I was able to do so, but now here is my problem: Seems like I didn’t set my domain name correctly on aws I got this error:
Great now by mistake i place my domain name here for the whole world can see it. How do I erase that comment.?
I’ve censored it for you.
I really need to be point in the right direction. I’ve tried everything and I can’t get the certificate:
I create multiple records A, TXT, CNAME use my router IP, 0.0.0.0, etc. end nothing.
I always get this error:
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
root@reverse-proxy:~ # certbot certonly –dns-route53 -d ‘*.xxxx.net’
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Requesting a certificate for *.xxxx.net
Certbot failed to authenticate some domains (authenticator: dns-route53). The Certificate Authority reported these problems:
Domain: xxxx.net
Type: dns
Detail: DNS problem: SERVFAIL looking up TXT for _acme-challenge.xxxx.net – the domain’s nameservers may be malfunctioning
Hint: The Certificate Authority failed to verify the DNS TXT records created by –dns-route53. Ensure the above domains are hosted by this DNS provider, or try increasing –dns-route53-propagation-seconds (currently 10 seconds).
If I delete the zone on my Route53 I get as different error:
Encountered exception during recovery: certbot.errors.PluginError: Unable to fin d a Route53 hosted zone for _acme-challenge.xxxx.net
Unable to find a Route53 hosted zone for _acme-challenge.xxxx.net
Ask for help or search for solutions at https://community.letsencrypt.org. See t he logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
Do I need to do something on my router, an specific record that I have to create on my Route53 account?
You’re doing DNS validation here – certbot is trying to determine whether you have the authority to obtain a certificate for the domain you’re requesting one for. It’s doing this by querying the DNS servers to see if your Route 53 hosted zone administers this domain. So no, you won’t need to modify anything in your router. It’s probably something to do with how you’ve configured your hosted zone. Do you own the domain? Does your Route 53 hosted zone have valid entries for the name servers? I’m pretty sure what certbot will do here is add a
TXTrecord using the credentials you’ve provided to the hosted zone, and then check that thatTXTrecord is available for the domain name you’ve provided.Hey Samuel thanks for help! Sorry I replay on a new comment.
Thanks so much for your help and fixing the comment!
Yes I do won the domain and set Hostedzone for it, Now Rote53 sets its own records that can not be deleted which are these:
ns-636.awsdns-15.net.
ns-1546.awsdns-01.co.uk.
ns-316.awsdns-39.com.
ns-1149.awsdns-15.org
Should a created another A record that points to my router, a txt record or CNAME. I’m really trying to figure out what I’m doing wrong. Or maybe the credentials permissions. Is it a keypair or a IAM?
Did you buy the domain through AWS? For my domains to be discoverable i had to use the name services provided by the registrar. When I wrote the guide it was a key pair as per the instructions, and the keypair had limited access to things. I’m not sure if amazon have changed anything recently
Yes I purchase the domain via AWS. Also the only way I’m able to give permissions is via IAM and assing a policy permission rule base on:
https://certbot-dns-route53.readthedocs.io/en/stable/
It is very confusing it seems likes there’s something on Route53 side that is not working correctly, I spent hours off reading resources to set the permissions and seems like that’s the only way.
Now I’m trying via cloudflare it created the certificate but I’m stuck on the ‘
mv nginx.conf nginx.conf.bak ‘ part
Thank you for such a fantastic guide Samuel. I have managed to get things setup correctly, but I have a question about some behavior that doesn’t seem quite as intended. I’m able to access my servers via subdomain.example.com, but if I go to to example.com, or just randomcharacters.example.com, I get redirected to my plex server. I’d like any non-configured to subdomains to throw an error instead. How should I try and troubleshoot this?
Thank you,
-patrick
I’m trying to configure a Nginx reverse proxy with virtual domain to Nextcloud Server
here is my setting
Internet
Router (192.168.0.1)
Nginx (192.168.0.2)
Nextcloud Server (192.168.0.3)
Since I don’t want use 443 port, I set up a port forward in router. Router (port: 12345) -> Nginx (port: 443)
nginx.conf
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
# Redirect all HTTP traffic to HTTPSserver {
listen 80 default_server;
listen [::]:80 default_server;
return 301 https://$host$request_uri;
}
# Import server blocks for all subdomains
include "vdomains/*.conf";
}
and vdomains/test.conf
server {
listen 443 ssl http2;
server_name 'nextcloud.test.com';access_log /var/log/nginx/cloud.access.log;
error_log /var/log/nginx/cloud.error.log;
include snippets/nextcloud.test.com.cert.conf;
include snippets/ssl-params.conf;
location / {
include snippets/proxy-params.conf;
proxy_pass http://192.168.0.3;
}
location = /.well-known/carddav {
return 301 $scheme://$host/remote.php/dav;
}
location = /.well-known/caldav {
return 301 $scheme://$host/remote.php/dav;
}
client_max_body_size 1999M;
}
It seems work fine. if I direct port forward router 443 port to Nginx server.
When I type “https://nextcloud.test.com” in browser, it will redirect to “https://nextcloud.test.com/index.php/apps/dashboard/” <- the Nextcloud main page
But if I set router port 12345 forward to Nginx server port 443. It will not work.
When I type “https://nextcloud.test.com:12345” in browser, it still redirect to “https://nextcloud.test.com/index.php/apps/dashboard/” since it missing the port “:12345” in url, it cannot access the Nextcloud main page. The main page will work if I direct type “https://nextcloud.test.com:12345/index.php/apps/dashboard/”
Any idea?
Thanks!
Thank you so much for this guide, it is incredibly helpful.
This is a very simple question but I am stuck. How exactly do I download the dgparam.pem file?
curl https://ssl-config.mozilla.org/ffdhe2048.txt > /usr/local/etc/ssl/dhparam.pem returns the error
“/usr/local/etc/ssl/dhparam.pem: No such file or directory.”
Im sure it is simple I just cant seem to sort it out. Thanks!
I was able to do it as follows:
mkdir /usr/local/etc/ssl
pkg install curl
curl https://ssl-config.mozilla.org/ffdhe2048.txt > /usr/local/etc/ssl/dhparam.pem
Working now. Thanks again for the awesome guide!
Thanks for the guide! Of course, I have questions as a noob to reverse proxy.
Truenas 12.2 with nginx set up in a jail (192.168.10.20) as per this guide.
Certs all downloaded OK
Untangle FW
Problem:
If I configure nginx as per the guide but I am running my web files off the same jail with files in /usr/local/www/example.com I get ‘ERR_TOO_MANY_REDIRECTS’ when I hit the http://www.example.com URI.
I wondered if it was even possible to run the reverse proxy AND the webserver for one or more domains in the same jail so I spun up a new jail with a different IP, installed nginx, configured it as the host for http://www.example.com and I then get ‘ERR_CONNECTION_REFUSED’.
At this stage, I’m just trying to get it to work internally first, then config external access. I would prefer to run nginx reverse proxy AND the web servers for 3 or 4 domains from the same jail with a single IP if possible. If not, what is the best approach?
Hey Mark,
I’m not sure what’s causing the
ERR_TOO_MANY_REDIRECTSissue – this isn’t an issue I came across. It sounds like you’ve entered a redirect loop though. I’d check the logs to see what’s happening. It sounds like you’re also taking some deviations from my recommendations though. Adopting a service oriented architecture means isolating each service into its own container (jail, in this case). You want each jail to be responsible for only one thing. This improves debugging and maintenance – if you take one service down, you don’t have to take all of them down. Likewise, if one service has a problem, it won’t necessarily affect the other services you’re running. Having both services running in the same jail is absolutely possible, but I would advise against it.With this in mind, you should have:
* 1 jail for the reverse proxy
* 1 jail for nextcloud
* 1 jail for service 2
* 1 jail for service 3
This will give you separate internal IPs, and you will be able to have
proxy_passstatements tohttp://<NEXTCLOUD_IP>:80, for example. As discussed in the guide, the reverse proxy is the single point of entry to your internal network from the internet, and will direct these requests to the appropriate internal service (identifiable by it’s socket, IP:Port combination). In order to use this internally, you’ll need to have an internal DNS resolver that can associate “cloud.example.com” with the internal IP of your reverse proxy – the example I use for this is with pfSense, though I’m sure there are other options. This will depend on what your specific set up is though.Hi!
I set this up on my server following your guide and while most things work, I am getting a blank white page for nextcloud as well as guacamole. I made the recommended edits in nextcloud according to the docs (it has no cert in front of it and listens on port 80)
I have been banging my head against the wall for two weeks trying to get this fixed; maybe someone can take a brief look at my nginx.conf file and see where I am going wrong! everything below works except nextcloud and guacamole. They just result in blank white pages.
nginx.conf:
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
client_max_body_size 1024M;
Redirect all HTTP traffic to HTTPS
server {
listen 80;
return 301 https://$host$request_uri;
}
Import server blocks for all subdomains
disabled include “vdomains/*.conf”;
#truenas
server {
listen 443 ssl http2;
server_name nas.domain.com;
access_log /var/log/nginx/nas.access.log;
error_log /var/log/nginx/nas.error.log;
include snippets/domain.com.cert.conf;
include snippets/ssl-params.conf;
location / {
include snippets/proxy-params.conf;
include snippets/internal-access-rules.conf;
proxy_pass http://192.168.0.200;
}
}
#nextcloud
server {
listen 443 ssl http2;
server_name cloud.domain.com;
access_log /var/log/nginx/cloud.access.log;
error_log /var/log/nginx/cloud.error.log;
include snippets/domain.com.cert.conf;
include snippets/ssl-params.conf;
proxy_hide_header Strict-Transport-Security;
location / {
include snippets/proxy-params.conf;
proxy_pass http://192.168.0.4;
#proxy_set_header X-Forwarded-Proto $scheme;
#proxy_set_header X-Real-IP $remote_addr;
#proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
#proxy_set_header Host $host;
}
location /.well-known/carddav{
return 301 $scheme://$host/remote.php/dav;
}
location /.well-known/caldav{
return 301 $scheme://$host/remote.php/dav;
}
}
#guacamole
server {
listen 443 ssl http2;
server_name remote.domain.com;
access_log /var/log/nginx/guacamole.access.log;
error_log /var/log/nginx/guacamole.error.log;
include snippets/domain.com.cert.conf;
include snippets/ssl-params.conf;
location / {
include snippets/proxy-params.conf;
proxy_pass http://192.168.0.16:8080/guacamole/#/;
}
}
#radarr
server {
listen 443 ssl http2;
server_name radarr.domain.com;
access_log /var/log/nginx/radarr.access.log;
error_log /var/log/nginx/radarr.error.log;
include snippets/domain.com.cert.conf;
include snippets/ssl-params.conf;
location / {
include snippets/proxy-params.conf;
include snippets/internal-access-rules.conf;
proxy_pass http://192.168.0.15:7878;
}
}
#test
server{
listen 443 ssl http2;
server_name test.domain.com;
access_log /var/log/nginx/test.access.log;
error_log /var/log/nginx/test.error.log;
include snippets/domain.com.cert.conf;
include snippets/ssl-params.conf;
location / {
include snippets/proxy-params.conf;
proxy_pass http://192.168.0.11:2020;
}
}
#catchall
server {
listen 443 ssl http2 default_server;
server_name catchall;
access_log /var/log/nginx/catchall.access.log;
error_log /var/log/nginx/catchall.error.log;
include snippets/domain.com.cert.conf;
include snippets/ssl-params.conf;
return 444;
}
}
Hi, I’m stalled at “root@reverse-proxy:/ # pkg install py37-certbot” with the error : “No packages available to install matching ‘py37-certbot’ have been found in the repositories.
I have googled this extensively to no avail. Your help would be greatly appreciated.
I have tried this instruction set, but it’s not having any effect: https://kevin.deldycke.com/2020/10/truenas-configuration/
UPDATE: py37 is apparently defunct, and you need to use py38 at this point in time
Hello all!
I noticed my let’s encrypt cert hasn’t automatically updated and it’s less than 30 days until it expires. Checking the log I see the following error:
pkg_resources.ContextualVersionConflict: (jmespath 1.0.0 (/usr/local/lib/python3.8/site-packages), Requirement.parse(‘jmespath<1.0.0,>=0.7.1’), {‘boto3’})
It looks like py38-jmespath-1.0.0 is newer than what is expected. Any idea how to fix?
Well I’m down to a few days until my wildcard cert expires. Any Ideas I how I can fix this?
Looks like this bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=264929
I had the same issue. Running “pkg upgrade -f” inside the jail fixed it for me
Thank you Andrej! That fixed it for me as well. I appreciate the help!
Hi, great tutorial!
Quick question; when we say; proxy_pass http://192.168.0.113, it means the jail with IP 192.168.0.113 must have an nginx running (and configured) right? But how about configuration of the nginx inside the jail, where proxy_pass gets redirected?
It seems the article doesn’t mention configuration of the nginx inside the jail. Am I mistaken?
the same server definition should be inside the nginx.conf file within the jail (192.168.0.113 as example)?
Any idea would be much appreciated.
Is there any special fact/reason adding; add_header Strict-Transport-Security “max-age=63072000” always; into server block of nginx? Is this a must?
Hi.
Having a weird problem that I explained here, regarding the jail and nginx itself:
https://forums.freebsd.org/threads/freebsd-13-1-weird-jail-nginx-problem.85857/
Could anybody kindly point me to the error or to a possible workaround?
Hi Samuel,
I was wondering if it is possible to set up 2 different domains using the above procedures. I used this guide last year to get things up and running on my main domain and things have been working perfectly. I have recently gotten a second domain via AWS and would like to be able to run that domain through the same Reverse Proxy. I was going to start with
certbot certonly –dns-route53 -d ‘*.newDomain.com’
but I was just wondering if you happened to know if this would cause issues. Thanks as always, your guides are what sparked my interest in building out a robust home lab.
Hello I have followed your guide up to pkg install awscli but I am getting “No packages to install matching “awscli” have been found in the repositories. I am a bit of a TrueNAS noob so any help you could provide would be appreciated.
You can use the following command to search what packages are available:
pkg search awscli
It looks like it is now called “py39-awscli-1.27.16” so running the following shoould get the AWS Command Line Interface installed for you:
pkg install py39-awscli-1.27.16
Thank you sir for the prompt reply and that got it going.
You’re welcome. This guide and the corresponding next cloud guide were the 2 most impactful things that got me going when I started my homelab 2 years ago. Have fun!
Hi and thanks for the guide. My problem is my router, i cannot get ports 443 or 80 assigned to anything , the router blocks it even though he says he does not. any idea ? the ssl cannot be issued through another port, right?
In case someone (like me) can’t make wildcard ssh cert, you can use the HTTP-01 challenge.
certbot -v certonly –standalone –preferred-challenges http -d sub.domain.net
With the –standalone option we tell Certbot to handle the challenge using its own built-in web server. The only problem is, that nginx is using port 80, so we’ll need to stop it before and start afterwards.
Putting everything together for teh automatic renewal in the crone job:
0 0,12 * * * /usr/local/bin/python -c ‘import random; import time; time.sleep(random.random() * 3600)’ && /usr/local/bin/certbot renew –quiet –pre-hook “/usr/sbin/service nginx stop” –deploy-hook “/usr/sbin/service nginx start”
Really nice Explanation and Method – One thing noting is the ssl-parms.conf has an extra chain
verify chain of trust of OCSP response using Root CA and Intermediate certs
# ssl_trusted_certificate /path/to/root_CA_cert_plus_intermediates;
Thanks for this guide! It’s the only one that fit with TrueNas Core and I was able to make Overseerr work with nginx, after I was able to solve my issue.
I was about to ask a question when I was able to solve it myself. To everyone that’s using cloudflare as their dns/domain registrar, turn off the proxy setting because I was getting ‘Too many redirects’ error when visiting my website.
Samuel, is there anyway to be able to use the proxi setting in cloudflare and make everything work? It’s a nice feature that protect the origin server’s IP.