Setting up a Raspberry Pi 4 home server

With the release of the raspberry pi 4, which features a 1.5 Ghz processor, 1-4 Gb RAM, built-in networking and USB 3.0 ports, I decided to finally take the plunge and set up a home server with the pi. In this post we will go through setting up and configuring the pi to do 3 main things:

1. function as a media center on the local network using the tv as the only display
2. run an internet connected CI service using jenkins
3. run an internet connected file-sharing service using nextcloud

A couple of notes before we begin: since I was a pi noob, I went ahead and bought the raspberry pi 4 desktop kit which came with a 16 Gb SD-card pre-formatted with NOOBs (the os installer for the pi), a keyboard, a mouse, hdmi cables, a book on getting started with the pi and of course, the pi itself with 4Gb of RAM in a case. As mentioned above, the display for the pi was a tv via HDMI.

Initial setup

I was not aware of this when I bought the pi, but it is actually an ARM processor running a 32-bit operating system. It comes with Raspbian, a Debian-based operating system with a LXDE-based desktop called PIXEL. It is also possible to install other distributions like Ubuntu Server, Ubuntu MATE, but not all distributions had been certified for the pi 4 at the time of writing so I decided to keep it simple and stick with Raspbian. If you're like me and are completely new to the pi, connect the peripherals first and power up the pi by simply plugging it in. Follow the white rabbit and setup Raspbian via the wizard.

One of the first issues that popped up was the fact that the debian distribution that Raspbian is based on changed from testing to stable and thus the sources needed to be updated accordingly: sudo apt-get update --allow-releaseinfo-change . Once the os has been configured and all software updated, we begin by installing some basic utilities: sudo apt install tmux vim .

The next thing to do is to enable SSH via the Raspberry configuration. Once ssh has been setup, we will installfail2ban since we will be exposing the pi to the internet.

Finally, we will also install and configureconky which displays a plethora of system information on the desktop:

$ sudo apt-get install conky -y
$ wget -O /home/pi/.conkyrc https://raw.githubusercontent.com/novaspirit/rpi_conky/master/rpi3_conkyrc
$ sudo vim /usr/bin/conky.sh

#!/bin/sh
(sleep 4s && conky) &
exit 0

$ sudo vim /etc/xdg/autostart/conky.desktop

[Desktop Entry]
Name=conky
Type=Application
Exec=sh /usr/bin/conky.sh
Terminal=false
Comment=system monitoring tool.
Categories=Utility;

Read on for the various services that we can setup on the pi.

Internet access

Take a note of your lan ip (e.g. 192.168.1.100). Install nginx via: sudo apt install nginx and head to your router settings and port forward 80, 443, 22 and any other ports that you might want. I just went with opening all ports from 4001-9999. Test whether everything is working by trying to hit the nginx default home page via http://your-public-ip . You can find out your public ip address via whatismyipaddress.com .

Since your public ip is assigned via your ISP and might change once in a while, it is recommended to setup a dynamic dns which basically means that you get a domain which always points to your machine despite a changing ip address. This is accomplished by running some sort of a script from your pi which will update the ip address on the DNS server. There are a few free services out there which provided dynamic DNS,no-ip being one of them. Follow the instructionshere for setting up the no-ip dynamic update client. Note that if you would like to have the client also start on boot add the binary ( /usr/local/bin/noip2 ) to /etc/rc.local right before the exit 0 statement.

In my case, I already had a domain from namecheap and I was very happy to discover that they also provide dynamic dns entries. I settled for having a subdomain like home.mydomain.com as a pointer to my pi. I also added a wildcard *.home.mydomain.com so that I can run separate services like ci.home.mydomain.com , etc. Note that namecheap provides a really simple way of keeping the public ip address updated via a http request: https://dynamicdns.park-your-domain.com/update?domain=$DOMAIN&password=$PASS&ip=$IP . Someone created a bash script that can be run every once in a while to update the namecheap dynamic dns records: namecheap-ddns-update . I set up a cron job to run this script once a day:

1 1 */1 * * NC_DDNS_PASS=xxx /home/pi/namecheap-ddns-update/namecheap-ddns-update -e -d yourdomain.com -s 'home,*.home' > /home/pi/cron-logs/nc-ddns.log 2>&1

Don't forget to create the ~/cron-logs folder. Once the dns has been setup, the pi should be accessible from the internet ( http://home.yourdomain.com ) and we can continue with setting up further services.

Media center / file server

Since the SD card that came with the desktop kit was only 16 Gb, I decided to buy an external HDD to store media. I went with a simple 1 Terabyte Western Digital HDD rather than a SSD. This saved me a bit of money on the disk at the expense of a bit extra voltage. I setup an ext4 filesystem on the external drive and copied media such as videos and music under a /share folder. Plugging in the HDD to the pi (be sure to use the USB 3.0 port) will mount the drive under /media/pi/<disk-name> .

Setting up a file server on the pi is prettystraight-forward. Note that the samba service can be restarted with: sudo service smbd restart .

Once the file server is setup, we can proceed with installing media center software. There are a few choices but I decided to go with Kodi which comes with remote control apps for both Android and iOS. Installing kodi is as simple as sudo apt install kodi and it comes with some pretty sane defaults. What is amazing is that if you use an external drive then you can simply play videos via the "Videos" interface option. To run kodi simply type kodi in a terminal. In order to enable a smartphone based remote control follow the instructions here . If you've been following the instructions in this article to the letter, you probably don't want to allow access to kodi via the internet. To this effect, change the kodi http port to something above 10000 and ensure that you have a decently strong password configured!

Note that instead of a fully blown media center, you can also play videos from the command line usingomxplayer, e.g.: omxplayer -o hdmi filename.mp4

Continuous integration with Jenkins

Before we start installing jenkins, we will first install docker:

$ sudo apt-get update
$ sudo apt-get install apt-transport-https ca-certificates software-properties-common
$ curl -fsSL https://yum.dockerproject.org/gpg | sudo apt-key add -
$ sudo echo "deb https://download.docker.com/linux/raspbian/ stretch stable" > /etc/apt/sources.list.d/docker.list
$ sudo apt install --no-install-recommends docker-ce
$ sudo apt install docker-compose
$ sudo usermod -aG docker pi

$ sudo systemctl enable docker.service
$ sudo systemctl start docker.service

As you can see, installing docker is not as straight-forward unfortunately. The first thing that I tried was: curl -fsSL get.docker.com -o get-docker.sh && sh get-docker.sh and this failed due to a bad version specified in the sources. After fixing the sources file to use debian stretch, I tried installing docker without recommended packages and it still failed due to a bad aufs-dkms installation . The solution in my case was to sudo apt purge aufs-dkms before proceeding with the docker installation.

With docker out of the way, lets install java:

$ sudo apt install openjdk-8-jdk
$ sudo update-java-alternatives -l
$ sudo update-java-alternatives -s java-1.8.0-openjdk-armhf

We are now ready to install Jenkins: sudo apt install jenkins

Before we proceed with configuring jenkins, we will setup a virtual host on nginx to forward all requests on ci.home.yourdomain.com to jenkins:

server {

  server_name ci.home.*;
  
  index index.html index.htm;

  location / {
    proxy_pass          http://localhost:8080;
    
    proxy_set_header    Host             $host;
    proxy_set_header    X-Real-IP        $remote_addr;
    proxy_set_header    X-Forwarded-For  $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-NginX-Proxy true;

    proxy_redirect off;


    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
  }	

}

Don't forget to add jenkins to the docker group: sudo usermod -aG docker jenkins and reboot your machine in order to allow jenkins to run pipeline builds. Thus, our ci service is now reachable via the internet and to secure it further we can get SSL certificates vialetsencrypt:

$ sudo apt install certbot
$ sudo apt-get install -y python-certbot-nginx
$ sudo certbot --nginx -d home.yourdomain.com
$ sudo certbot --nginx -d ci.home.yourdomain.com

Database (Mariadb / MySQL)

Note that MySQL is not available for the pi, we need to use Mariadb instead:

$ sudo apt install mariadb-server
$ sudo mysql_secure_installation
$ sudo mysql -u root

CREATE DATABASE exampledb;
CREATE USER 'exampleuser'@'localhost' IDENTIFIED BY 'xxx';
GRANT ALL PRIVILEGES ON exampledb.* TO 'exampleuser'@'localhost';
FLUSH PRIVILEGES;

NAT loopback or why can I only access my PI from outside the LAN?

If it so happens that the pi is accessible with the public ip address / domain from outside your home but not from within the LAN then most likely your router does not support NAT loopback . This means that the router cannot find the pi via the public ip, it requires the local ip of the pi (e.g. 192.168.1.100 ).

If your client machine is always connected to the pi's local network, you can simply fix the DNS resolution of the domain to resolve to the local LAN address of the pi. In Ubuntu, this can be done by adding an entry the /etc/hosts file. However, a more elegant solution is to setup a DNS server on the local network and let all client devices use this local DNS server. This DNS server can be configured to direct any queries for the pi's domain to it's local LAN ip address.

We will thus installdnsmasq, a very simple and lightweight DNS server on the pi itself:

$ sudo apt-get update
$ sudo apt install dnsmasq dnsutils
$ sudo service dnsmasq stop
$ sudo vim /etc/dnsmasq.conf

# uncomment the following options
domain-needed         # Never forward plain names (without a dot or domain part)
bogus-priv            # Never forward addresses in the non-routed address spaces.

# save and quit
$ sudo service dnsmasq start
$ sudo journalctl -u dnsmasq -f

# check logs for errors
# test dns resolution on the pi returns the public ip address

$ dig home.mydomain.com

We will now configure the pi to resolve its domains to it's local LAN ip address and check that the DNS lookup now resolves correctly.

$ sudo vim /etc/hosts

192.168.1.100 home.mydomain.com
192.168.1.100 ci.home.mydomain.com

$ sudo service dnsmasq restart
$ dig home.mydomain.com

Once dnsmasq is setup on the pi, configure the machines on the LAN to use the pi as a DNS server. On Ubuntu with Gnome this can be configured via the Network Manager on a per connection level. Note that a restart of the network-manager might be required for the DNS settings to take effect. Once more test that the DNS resolution for home.mydomain.com resolves to the local LAN ip address.

Once this is setup, we can continue with the Nextcloud installation.

File sharing and sync with Nextcloud

Since we are running Nginx, we will do a manual Nextcloud installation. Download and untar the nextcloud archive under /var/www/nextcloud/ . Thereafter, prepare the system:

$ sudo apt install php-fpm php-ctype php-xml php-gd php-json php-mysql php-file php-bz2 php-bcmath php-apcu
$ sudo chown -R www-data:www-data /var/www/nextcloud/
$ sudo mysql -u root

CREATE USER 'username'@'localhost' IDENTIFIED BY 'password';
CREATE DATABASE IF NOT EXISTS nextcloud;
GRANT ALL PRIVILEGES ON nextcloud.* TO 'username'@'localhost' IDENTIFIED BY 'password';
FLUSH privileges;

Create the Nginx site configuration as instructed here (as is evident from the nextcloud installation location, pick "Nextcloud installed in the webroot of nginx"). Restart Nginx, navigate to your nextcloud: cloud.home.mydomain.com and setup your nextcloud instance! Note that if you choose a data directory other than /var/www/nextcloud/data , then you need to ensure that it has the correct owner via sudo chown -R www-data:www-data <data-dir> .

In my case, my router did not support NAT loopback but I managed to browse web-applications on the pi and even setup nextcloud without needing dnsmasq. Only when I setup my nextcloud client and started syncing data on my laptop, did the issue manifest itself. I noticed extremely slow upload/download speeds with the router randomly choking leading to both the pi and my laptop losing their wifi connections. Once I had the pi's DNS resolving to its local LAN address, everything worked smooth as butter :).

Issues

Please note the following issues at the time of writing:

• While setting up the pi, I noticed that there was a constant flashing thermometer on the display. This indicates that the pi is heating up. The temperature can be checked via: vcgencmd measure_temp . Note that you can also right click the top panel and add a "CPU temperature" panel item to keep an eye on the temperature. Anyways, in my case the temperature was constantly around 85 degrees C which I found quite high given that the CPU was mostly idle. It turns out that the pi4 has a heating issue which can be somewhat mitigated via a firmware patch . The firmware update reduced the pi's temperature to 77 degrees and I also removed the top of the case which further reduced the temperature to 60 degrees. I also ended up buying a heatsink for the pi which provided another 5 degrees worth of cooling to bring the total down to 55 degrees.
• The display on my tv is too big and at the time of writing the pi4 does not support overscan for HDMI output :(.

# uncomment if you get no picture on HDMI for a default "safe" mode
#hdmi_safe=1

# uncomment this if your display has a black border of unused pixels visible
# and your display can output without overscan
#disable_overscan=1

# uncomment the following to adjust overscan. Use positive numbers if console
# goes off screen, and negative if there is too much border
overscan_left=-8
overscan_right=-8
overscan_top=-16
overscan_bottom=-16

Ok well that's about it - despite the minor issues, I'm really happy to have a home server setup and I'll keep this article updated with any other services that I decide to install. Feel free to get in touch or discuss this on HN via the link below.

HackerNews submission / discussion