Seedbox and NAS on Raspberry Pi 4
With rtorrent, OpenMediaVault and Mullvad VPN
Table of Contents
- 1. Seedbox and NAS
- 1.1. Philosophy
- 1.2. Preparation
- 1.3. Pre-installation tasks on Raspberry Pi OS
- 1.3.1. DONE Transfer /root from SD-card to External SSD
- 1.3.2. DONE check that automated trim is in place
- 1.3.3. NEXT Transfer /boot from SD-card to External SSD
- 1.3.4. Measure the SSD performance
- 1.3.5. NEXT Enable TRIM on the SSD
- 1.3.6. Setup new hostname and a static IP
- 1.3.7. Install OS-software
- 1.3.8. Update firmware
- 2. Seedbox installation on Raspberry Pi OS with rtinst
-
- 2.0.1. Disable autostartup of rtorrent and irssidl
- 2.0.2. Remove rutorent password dialog
- 2.0.3. NEXT Fix errors with aptitude
- 2.0.4. FOLLOWUP Whether that source line is now needed to update with rtinst rtorrent etc.
- 2.0.5. FOLLOWUP OMV5 uninstalls libopts25, ntp and vsftpd that were installed with rtinst. Do I have now broken functionality in rtinst, ftp maybe? NTP breaks what, rtorrent?
- 2.0.6. Setup a new user
- 2.1. NEXT Research this topic once rtorrent is setup:
-
- 3. OMV5 Installation on Raspberry Pi OS
- 3.1. Post-installation tasks
- 3.1.1. Change the default logout time
- 3.1.2. Change the default GUI password for admin
- 3.1.3. Change network settings
- 3.1.4. Change email notification settings
- 3.1.5. NEXT Enable drive failure notifications
- 3.1.6. Enable system performance statistics collection
- 3.1.7. Install updates
- 3.1.8. Enable Testing, Extras and Backports repositories
- 3.1.9. NEXT Enable docker and Portainer
- 3.1.10. Format the HDD-disk
- 3.1.11. Create file system and mount the drive
- 3.1.12. DONE Add rest of the 4 drives to HDD enclosure, format and mount them
- 3.1.13. Create network share for music
- 3.1.14. DONE Setup SMB share
- 3.1.15. DONE Test SMB share in Windows
- 3.1.16. DONE Setup NFS share on host
- 3.1.17. DONE Test access to NFS share on a client
- 3.1.18. DONE Automate NFS mounting on client
- 3.1.19. DONE Install resetperms plugin
- 3.1.20. NEXT Install fatrace
- 3.1.21. DONE rsync from kapsi Nextcloud instance to /dev/sdd1 everyone's photos as backups
- 3.1.22. NEXT rsync from kapsi Nextcloud instance to /dev/sdd1 Tuomas's photos as backups
- 3.1.23. NEXT Apply same permissions resursively on srv/dev-disk-by-label-photos as on srv/dev-disk-by-label-data
- 3.2. Maintenance
- 3.3. Troubleshooting
- 3.4. Future direction
- 3.1. Post-installation tasks
- 4. NEXT Upgrade to OMV6
- 5. mullvad wireguard
- 5.1. NEXT Setup a single, isolated qbittorrent or rtorrent docker
- 5.2. Preparation
- 5.3. Install wireguard
- 5.4. NEXT add 10.0.1.240 back to pi-hole clients and observe during VPN testing
- 5.5. Test VPN speed with well seeded linux torrents
- 5.6. NEXT Investigate why this is, a pointer here:
- 5.7. NEXT Try setting /etc/iproute2/rt_tables as in here:
- 5.8. NEXT Test if with pihole I have now a dns leakage
- 5.9. NEXT After installing wireguard and connecting to mullvad, check connection:
- 5.10. NEXT and speed:
- 5.11. NEXT Check if I need additional firewall rules
- 5.12. NEXT Should I go with manual installation method:
- 5.13. NEXT Check speed with this:
- 6. NEXT Document why to keep host os as clean as possible
- 7. NEXT Install plugin resetperms and figure out what it does
- 8. NEXT Check contents from this article and compare contents to my article
1. Seedbox and NAS
1.1. Philosophy
OpenMediaVault is the next generation network attached storage (NAS) solution based on Debian Linux. […] primarily designed to be used in small offices or home offices, but is not limited to those scenarios. It is a simple and easy to use out-of-the-box solution that will allow everyone to install and administrate a Network Attached Storage without deeper knowledge.1
LibTorrent is a BitTorrent library written in C++ for *nix, with a focus on high performance and good code. The library differentiates itself from other implementations by transfering directly from file pages to the network stack.
- OpenMediaVault is a network-attached storage (NAS) based on Debian Linux distribution.
- OMV stores, shares my local volatile media copies (videos, family photos, personal music collection etc.) and backups anything important to an off-site Nextcloud instance, hosted at kapsi.fi.
- With a true gigabit ethernet controller of Raspberry Pi 4, RPi based NAS setup compared to commercial NAS options is now a lucrative choice in terms of performance vs. cost.
- Since network controller speed is no longer an issue, on same instance I have rtinst based seedbox installation.
- Rtinst installs a resource-friendly ruTorrent frontend based on rtorrent backend, which is ideal to run on a Raspberry Pi due to its low resource consumption.
- I also considered to combine HTPC with NAS, but decided not to due to their conflicting goals: HTPC needs to be small, quiet and slick next to a TV, but a NAS will have a bunch of noisy HDDs in it. HTPC needs to have decent performance if you might game on it, but you want a NAS to have low power draw and the graphics performance is irrelevant. HTPC wants flexibility to install new things and try out new use cases, but a NAS should have uninterrupted uptime.
1.2. Preparation
1.2.1. Acquire the hardware
- Raspberry Pi 4
- Flirc case (Argon Neo as an alternative).
- Yottamaster PS500U3-SV5 HDD enclosure with UASP. We want an enclosure without built-in hardware raid. This enclosure supports 5x 8TB 3.5" HDD SATA 6 Gbps drives through single USB3.0 cable (= 5Gbps transmission) to our Rpi4. Enclosure comes with a 12V 6.5A power supply, which is needed to spin our 3.5" HDDs.
Warning: Would not recommend this enclosure due to its UASP imcompatibilities with Linux kernel.2 A next step would be to investigate the proven SATA-technology with e.g. SATA-hat for rpi4 as USB connector in enclosure is a single point of failure: USB is pretty famous for being flaky and causing data loss.
Warning: Be sure you tell the difference between CMR and SMR type disks. SMR drives are not intended for random write IO use cases because the write performance is much slower than with a non-SMR drive. Therefore they are not recommended for NAS use cases featuring significant random write workloads. See arstechnica's article for more information. Then select disks e.g. based on reviews.3
- Kingston SA400 SATAIII SSD 2.5 Inch 120 GB (SA400S37)
- SATA 3 to USB-3 adapter with UASP (All You Need to Know About UASP). Check that UASP is enabled. Also make sure you choose a rpi4 compatible USB adapter.3
1.2.2. Install the Flirc case for Rpi4
- Why Flirc case?
- With Flirc we want to ensure the temperature stays below 80 °C; otherwise CPU clock speed is throttled from the standard 1,5 GHz down to 600 MHz.
- We want the most effective, silent, passive cooling solution as this box will be on prominent place in living room. Bonus points for Flirc: it is also stylished.
- Installation instructions for Flirc case.
1.2.3. Install the Yottamaster powered HDD enclosure
- Attach the enclosure's USB-cable into one of the two of RPI4's blue usb-3 ports.
- Attach the HDDs (I have 5) to trays and slide them in to enclosure and power it up.
1.2.4. Prepare Raspberry Pi OS image
- Install Etcher: https://etcher.io/
- With it, write minimal Raspberry Pi OS Lite on the SD card.
- To find a suitable SD-card, run a speed test for the card.
- Plug in the external SSD into usb3-port (blue colored) and Micro SD into SD-slot and boot up the Pi by attaching usb-c power cord in it. See: Setup raspberry.
- Raspberry starts when power cord is attached. Let it boot, then check its IP from router.
- Change Raspberry's IP to static by logging in to your Asus RT-N66U router: http://10.0.1.1/Main_Login.asp and go to: LAN, DHCP Server: Enable Manual Assignment: Yes.
- Select raspberrypi from the client's list and bind it to: 10.0.1.240 4.
- Once done, ssh into rpi4:
$ ssh pi@10.0.1.240
pass: raspberry.
The authenticity of host '10.0.1.240 (10.0.1.240)' can't be established. ECDSA key fingerprint is SHA256:l/LA0mZ8187cXSazV5b1nNvzRws6+5KfVAm5EJhrCgY. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added '10.0.1.240' (ECDSA) to the list of known hosts. pi@10.0.1.240's password:
1.3. Pre-installation tasks on Raspberry Pi OS
1.3.1. DONE Transfer /root from SD-card to External SSD
Update this fix eee-prom is now capable of booting fully though USB. See e.g. this video on how to do it.
: below is no longer needed as withRaspberry Pi 4 booting off of SDD has Positive performance gains.
- First, list all partitions:
$ sudo fdisk -l
- Seeing our external SSD has partition table at /dev/sda we launch fdisk like so:
$ sudo fdisk /dev/sda
Delete any existing partitions and write a new primary partition to SSD:
Welcome to fdisk (util-linux 2.33.1). Changes will remain in memory only, until you decide to write them. Be careful before using the write command. Command (m for help): p Disk /dev/sda: 111.8 GiB, 120034123776 bytes, 234441648 sectors Disk model: ASM105x Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 33553920 bytes Disklabel type: dos Disk identifier: 0x6c586e13 Device Boot Start End Sectors Size Id Type /dev/sda1 8192 532479 524288 256M c W95 FAT32 (LBA) /dev/sda2 532480 4390911 3858432 1.9G 83 Linux Command (m for help): d Partition number (1,2, default 2): 2 Partition 2 has been deleted. Command (m for help): d Selected partition 1 Partition 1 has been deleted. Command (m for help): p Disk /dev/sda: 111.8 GiB, 120034123776 bytes, 234441648 sectors Disk model: ASM105x Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 33553920 bytes Disklabel type: dos Disk identifier: 0x6c586e13 Command (m for help): n Partition type p primary (0 primary, 0 extended, 4 free) e extended (container for logical partitions) Select (default p): p Partition number (1-4, default 1): 1 First sector (65535-234441647, default 65535): Last sector, +/-sectors or +/-size{K,M,G,T,P} (65535-234441647, default 234441647): Created a new partition 1 of type 'Linux' and of size 111.8 GiB. Command (m for help): w The partition table has been altered. Failed to remove partition 1 from system: Device or resource busy Failed to add partition 1 to system: Device or resource busy The kernel still uses the old partitions. The new table will be used at the next reboot. Syncing disks. pi@raspberrypi:~ $ sudo reboot
As I'm getting above an error:
Failed to remove partition 1 from system: Device or resource busy Failed to add partition 1 to system: Device or resource busy
- I can workaround it by rebooting the rpi4. Otherwise, I can continue by formatting the partition for ext4 file format:
$ sudo mkfs.ext4 /dev/sda1
mke2fs 1.44.5 (15-Dec-2018) Creating filesystem with 29297014 4k blocks and 7331840 inodes Filesystem UUID: 34b431e6-5776-4387-be44-1a42b30f244c Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000, 23887872 Allocating group tables: done Writing inode tables: done Creating journal (131072 blocks): done Writing superblocks and filesystem accounting information: done
- Create a new directory called /media/newdrive, which you will use to mount your external drive:
$ sudo mkdir /media/newdrive
- Mount the new partition as /media/newdrive. Note that your partition name will be /dev/sda1, rather than just /dev/sda:
$ sudo mount /dev/sda1 /media/newdrive
- Copy all the files from your root directory:
$ sudo rsync -avx / /media/newdrive
- Open the /boot/cmdline.txt file for editing:
$ sudo nano /boot/cmdline.txt
Contents of cmdline.txt should be something like:
console=serial0,115200 console=tty1 root=PARTUUID=6c586e13-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait
- Replace root=PARTUUID=6c586e13-02 with root=/dev/sda1 so it becomes:
console=serial0,115200 console=tty1 root=/dev/sda1 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait
- Save the changes by hitting ctrl-w and then y. If I check now where the root partition is loaded from, it should be from the SD-card:
$ findmnt -n -o SOURCE /
/dev/mmcblk0p2
Reboot your Raspberry Pi 4 with $ sudo reboot
then issue $ findmnt -n -o SOURCE /
again, and
this time the result should be /dev/sda1. Rpi4 is now loading Raspberry Pi OS from
/root which is residing in the faster SSD-drive. Note that SD-card is
currently still needed for boot as Rpi4 firmware doesn't
yet
support full boot.
1.3.2. DONE check that automated trim is in place
- By doing
$ systemctl status fstrim.timer
as per recommendation here.
1.3.3. NEXT Transfer /boot from SD-card to External SSD
- As per https://lemariva.com/blog/2020/08/raspberry-pi-4-ssd-no-microsd-required
- Execute and document the step above, investigate how they can be combined under previous subtitle.
1.3.4. Measure the SSD performance
As per script by James Chambers:
$ sudo curl https://raw.githubusercontent.com/TheRemote/PiBenchmarks/master/Storage.sh | sudo bash
[...]
iozone test complete.
RandRead: 17621 - RandWrite: 20914 - Read: 33385 - Write: 21415
Enter a description of your storage and setup (Example: Kingston A400 SSD on Pi 4 using StarTech SATA to USB adapter)
Description: Kingston A400 SSD on Pi 4 using StarTech SATA to USB adapter
(Optional) Enter alias to use on benchmark results. Leave blank for completely anonymous.
Alias (leave blank for Anonymous):
Category Test Result
HDParm Disk Read 273.10 MB/s
HDParm Cached Disk Read 192.69 MB/s
DD Disk Write 98.2 MB/s
FIO 4k random read 14970 IOPS (59883 KB/s)
FIO 4k random write 9126 IOPS (36506 KB/s)
IOZone 4k read 33385 KB/s
IOZone 4k write 21415 KB/s
IOZone 4k random read 17621 KB/s
IOZone 4k random write 20914 KB/s
Score: 5861
Compare with previous benchmark results at:
https://storage.jamesachambers.com/
pi@raspberrypi:~ $
1.3.5. NEXT Enable TRIM on the SSD
As per tutorial by Jeff Geerling.
1.3.6. Setup new hostname and a static IP
Changing hostname is good practice in case we have multiple machines so we know which is which and human errors are minimized:
$ sudo nano /etc/hostname
Delete the old raspberrypi and setup a new name omv.
$ sudo nano /etc/hosts
Replace any occurrence of the existing computer name raspberrypi with your new one omv.
Set up a static IP and DNS that is of Mullvad's:
$ sudo nano /etc/dhcpcd.conf
Add to the end of file:
static ip_address=10.0.1.240/24 static routers=10.0.1.1 static domain_name_servers=193.138.218.74
Then restart dhcp-service:
$ sudo service dhcpcd restart
$ sudo systemctl daemon-reload
This is important tto prevent DNS leaks when using Mullvad's VPN. Check that DNS-server changed has taken place with:
$ cat /etc/resolv.conf
Reboot the system for rest of changes (hostname change) to take effect:
$ sudo reboot
1.3.7. Install OS-software
- Get aptitude cfg file (including no-recommended installations):
$ sudo wget -c http://iki.fi/~pyyhttu/debian/aptitude/apt.conf /etc/apt/
- Install aptitude (not mandatory as this is a personal preference for me to manage packages):
$ sudo apt update
$ sudo apt install aptitude
- Install and run localepurge to get rid of locales that we do not use, as well as localized packages and man pages.
- Install and run deborphan that searches for orphaned packages, i.e., which are not required by any other package:
$ sudo aptitude update
$ sudo aptitude install localepurge deborphan
- Leave locales "en" and "en_US.UTF-8". Choose "yes" for "Use dpkg --path-exclude?".
- Configure correct time zone:
$ sudo dpkg-reconfigure tzdata
Current default time zone: 'Europe/Helsinki'
Local time is now: Sun Feb 9 17:11:41 EET 2020.
Universal Time is now: Sun Feb 9 15:11:41 UTC 2020.
1.3.8. Update firmware
- Check the current firmware version:
$ /opt/vc/bin/vcgencmd version
Aug 15 2019 12:06:42
Copyright (c) 2012 Broadcom
version 0e6daa5106dd4164474616408e0dc24f997ffcf3 (clean) (release) (start)
- Then issue
$ sudo aptitude full-upgrade
. This will also update the latest stable firmware. Non-stable beta firmware, if needed, is installed with$ sudo rpi-update
.
Starting of Rpi4, device actually has onboard upgradable firmware stored on an EEPROM chip where the firmware upgrade is written, instead of the SD-card.
To check the status of the bootloader and VL805 code after full-upgrade, do:
$ sudo rpi-eeprom-update
BCM2711 detected BOOTLOADER: up-to-date CURRENT: Tue 10 Sep 10:41:50 UTC 2019 (1568112110) LATEST: Tue 10 Sep 10:41:50 UTC 2019 (1568112110) VL805: update required CURRENT: 00013701 LATEST: 000137ab
The bootloader code is up to date but a reboot is required to update the VL805 code to latest. So do:
$ sudo reboot
After reboot firmware should show: CURRENT: 000137ab. This version will fix network boot and overheating. See complete release notes for detailed changes.
I want to control when the updates are applied to avoid any unforeseen changes. So disabling autoupdating service with:
$ sudo systemctl mask rpi-eeprom-update
Created symlink /etc/systemd/system/rpi-eeprom-update.service → /dev/null.
The startup service has now been disabled and automatic bootloader firmware updates will be only applied if updating manually.
If I want to reenable the service, do unmasking with:
$ sudo systemctl unmask rpi-eeprom-update
To be notified about the updates, subscribe to rpi-eeprom releases rss-feed.
Once new version is released, check the availability with:
$ sudo rpi-eeprom-update
tuomas@omv:~ $ sudo rpi-eeprom-update [sudo] password for tuomas: BCM2711 detected Dedicated VL805 EEPROM detected *** UPDATE AVAILABLE *** BOOTLOADER: update available CURRENT: Thu Apr 16 17:11:26 UTC 2020 (1587057086) LATEST: Thu Sep 3 12:11:43 UTC 2020 (1599135103) FW DIR: /lib/firmware/raspberrypi/bootloader/default VL805: update available CURRENT: 000137ad LATEST: 000138a1 tuomas@omv:~ $
Then to update do:
$ sudo rpi-eeprom-update -a
tuomas@omv:~ $ sudo rpi-eeprom-update -a BCM2711 detected Dedicated VL805 EEPROM detected *** INSTALLING EEPROM UPDATES *** BOOTLOADER: update available CURRENT: Thu Apr 16 17:11:26 UTC 2020 (1587057086) LATEST: Thu Sep 3 12:11:43 UTC 2020 (1599135103) FW DIR: /lib/firmware/raspberrypi/bootloader/default VL805: update available CURRENT: 000137ad LATEST: 000138a1 BOOTFS /boot EEPROM updates pending. Please reboot to apply the update. tuomas@omv:~ $
Then issue a reboot for the update to take effect.
Alternatively, run $ raspi-config
.
2. Seedbox installation on Raspberry Pi OS with rtinst
rtinst script is intended for quick setup of seedboxes, but it is also handy in setting up a "one-click-install" of libtorrent/rtorrent, and its web facing frontend rutorrent, which is for easier management of torrents. rtinst also installs few "forced dependencies", such as git, vsftpd, Nginx, and autodl-irssi. However, since OMV5 install later automatically handles vsftpd (uninstalled) and Nginx (also used by OMV5), we don't have any other extra footprint than git and autodl-irssi, which does have its uses (more about that later).
To install the aforementioned software, do:
$ sudo bash -c "$(wget --no-check-certificate -qO - https://raw.githubusercontent.com/arakasi72/rtinst/master/rtsetup)"
Installing git Installing rtinst v1.8.9 Installation complete You can now run rtinst and the additional supporting scripts
Install with the following script options:
$ sudo rtinst --ssh-default -u tuomas -p password1 --webpass password2 --log
Where,
password1: ssh password for user tuomas
password2: rutorrent password for
Select the IP address to use: 1.) 10.0.1.240 2.) xx.xxx.xx.xxx 1 IP set to 10.0.1.240 Raspbian GNU/Linux 10 (buster) Checking the web sites we will need are accessible Checking Raspbian mirrors http://raspbian.raspberrypi.org/raspbian/: OK Checking major 3rd party components Rtorrent: OK xmlrpc-c: OK RuTorrent: OK Autodl-irssi: OK Your Server IP is 10.0.1.240 Is this correct y/n? y Your server's IP is set to 10.0.1.240 Your domain is set to omv.house User name is tuomas Adding user `tuomas' ... Adding new group `tuomas' (1001) ... Adding new user `tuomas' (1001) with group `tuomas' ... Creating home directory `/home/tuomas' ... Copying files from `/etc/skel' ... tuomas successfully created No additional users to add No more user input required, you can complete unattended It will take approx 10 minutes for the script to complete Updating package lists Upgrading packages Installing required packages rar/unrar install failed Completed installation of required packages Adding user `tuomas' to group `sudo' ... Adding user tuomas to group sudo Done. Configuring SSH SSH port set to 22 IP: 10.0.1.240 DN: omv.house Installing certbot Install Failed IP: 10.0.1.240 DN: omv.house Generating https/ssl certificates: done Self signed certificats installed Installing vsftpd FTP port set to 43861 Installing nginx Checking File Repositories xmlrpc-c: OK rtorrent: OK Debian 9, and Ubuntu 17.10 and later, are only supported by libtorrent-0.13.7/rtorrent-0.9.7 or later Installing libtorrent-0.13.8/rtorrent-0.9.8 Fetching source files ###################################################################################################################################################### 100.0% ###################################################################################################################################################### 100.0% Installing xmlrpc Installing libtorrent Installing rtorrent 0.13.8/0.9.8 installed Configuring rtorrent RuTorrent: OK Installing Rutorrent (v3.10-beta) Configuring Rutorrent Installing autodl-irssi Setting permissions, Starting services rtorrent was not running No session lock file Starting rtorrent. rtorrent has been started irssi was not running Starting irssi. irssi has been started no crontab for tuomas no crontab for tuomas Summary of Installation (Important Information, please read SSH Configured SSH port set to 22 root login directly from SSH disabled login with tuomas and switch to root using: sudo su FTP Server vsftpd 3.0.3-12 installed ftp port set to 43861 ftp client should be set to explicit ftp over tls using port 43861 rtorrent torrent client rtorrent 0.9.8 installed crontab entries made. rtorrent and irssi will start on boot for tuomas RuTorrent Web GUI RuTorrent 3.9 installed rutorrent can be accessed at https://10.0.1.240/rutorrent rutorrent password as set by user to change rutorrent password enter: rtpass If enabled, access https downloads at https://10.0.1.240/download/tuomas IMPORTANT: SSH Port set to 22 IMPORTANT: SSH Port set to 22 IMPORTANT: SSH Port set to 22 Please ensure you can login BEFORE closing this session The above information is stored in rtinst.info in your home directory. To see contents enter: cat /home/tuomas/rtinst.info To install webmin enter: sudo rtwebmin SCROLL UP IF NEEDED TO READ ALL THE SUMMARY INFO PLEASE REBOOT YOUR SYSTEM ONCE YOU HAVE NOTED THE ABOVE INFORMATION Thank You for choosing rtinst
2.0.1. Disable autostartup of rtorrent and irssidl
- This is done to make sure rtorrent does not connect before we have VPN-tunnel (to be setup later) established.
- rtinst script does the automatic startup with cron job. Open crontab:
$ crontab -e
Edit the file by commenting out lines with # so that it becomes:
#@reboot sleep 10; /usr/local/bin/rtcheck irssi rtorrent #*/10 * * * * /usr/local/bin/rtcheck irssi rtorrent
Notice, that the rtinst script also sets up rtorrent, rutorrent and autodl-irssi to run on startup. For now, we can inspect that rutorrent is spawned and running at https://10.0.1.240/rutorrent/.
2.0.2. Remove rutorent password dialog
- Not needed in local network
- Remove with: https://github.com/arakasi72/rtinst/issues/314
2.0.3. NEXT Fix errors with aptitude
For some reason, rtinst updates my sources.list with debian mirrors, so need to uncomment those:
$ sudo nano /etc/apt/sources.list
# http://deb.debian.org/debian buster-backports main
EDIT: Above is not really needed, as OMV5 install script cleans that up. EDIT2: Follow this up, as it may be wireguard that sets the above line.
2.0.4. FOLLOWUP Whether that source line is now needed to update with rtinst rtorrent etc.
2.0.5. FOLLOWUP OMV5 uninstalls libopts25, ntp and vsftpd that were installed with rtinst. Do I have now broken functionality in rtinst, ftp maybe? NTP breaks what, rtorrent?
2.0.6. Setup a new user
During the execution of the script, we also opted to add a new user, tuomas, which we will be using from now on for admin tasks instead of the default pi. Leaving the default user pi, and its default password raspberry is a security risk.
Check groups pi users is associated to:
$ groups
pi adm dialout cdrom sudo audio video plugdev games users input netdev ssh gpio i2c spi
Add user tuomas to all the same groups:
$ sudo usermod -a -G adm,dialout,cdrom,audio,video,plugdev,games,users,input,netdev,ssh,gpio,i2c,spi tuomas
Logout user pi with $ logout
. After this, ssh back in with user tuomas:
$ ssh tuomas@10.0.1.240
2.1. NEXT Research this topic once rtorrent is setup:
3. OMV5 Installation on Raspberry Pi OS
- As per tutorial from OpenMediaVault plugin developers.
Run a script that automates the install:
$ wget -O - https://github.com/OpenMediaVault-Plugin-Developers/installScript/raw/master/install | sudo bash
Let the script run, it'll take several minutes to finish. Script finishes with last line messages as:
[...]
Processing triggers for systemd (241-7~deb10u2+rpi1) ...
Enable and start systemd-resolved ...
Configure eth0 to use networkd ...
Enable networkd ...
It is recommended to reboot and then setup the network adapter in the openmediavault web interface.
pi@raspberrypi:~ $
Let's reboot with $ sudo reboot
.
3.1. Post-installation tasks
After 3 to 5 minutes, OMV can be logged into using the same IP address that was used for the SSH client, entered in a web browser address bar. The web GUI user is admin and the default password is openmediavault.
So login at 10.0.1.240.
3.1.1. Change the default logout time
- Go to System, General Settings and change Auto logout to 30 minutes.
3.1.2. Change the default GUI password for admin
- Go to System, General Settings, Web Administrator Password.
Add user tuomas to ssh group
- Go to Access Rights Management, User, highlight user tuomas and press
Edit. Add user to group ssh. This way
$ ssh tuomas@10.0.1.240
still works.
Delete old user pi
- Go to Access Rights Management, User, highlight user pi and press Delete. Note: This results in an error:
Failed to execute XPath query '//system/usermanagement/users/user[name='pi']'. Error #0: OMV\Config\DatabaseException: Failed to execute XPath query '//system/usermanagement/users/user[name='pi']'. in /usr/share/php/openmediavault/config/database.inc:344 Stack trace: #0 /usr/share/openmediavault/engined/rpc/usermgmt.inc(658): OMV\Config\Database->deleteByFilter('conf.system.use...', Array) #1 [internal function]: Engined\Rpc\UserMgmt->deleteUser(Array, Array) #2 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array) #3 /usr/share/php/openmediavault/rpc/rpc.inc(86): OMV\Rpc\ServiceAbstract->callMethod('deleteUser', Array, Array) #4 /usr/sbin/omv-engined(537): OMV\Rpc\Rpc::call('UserMgmt', 'deleteUser', Array, Array, 1) #5 {main}
But the user is still deleted.
3.1.3. Change network settings
- Go to System, Network, Interfaces, click on the + Add-button
and select Ethernet. For Name drop down select eth0 and under IPV4 for Method select DHCP. Note: At least since OMV version 5.5.2-1 (Usul), above is already filled out.
3.1.4. Change email notification settings
- Go to System, Notification, Settings and Enable notifications. Then fill in the SMTP settings:
SMTP server: mail.kapsi.fi SMTP Port: 587 Encryption mode: STARTTLS Sender mail: your-kapsi-mail Username: your-kapsi-username Password: your-kapsi-password
Recipient: your-notification-mail
Note: After this I will receive notification to my mail when: I login to openmediavault ("Your user account was used to log in…")
The notification is sent only once, provided that: a session cookie is present.
3.1.5. NEXT Enable drive failure notifications
As per: https://forum.openmediavault.org/index.php?thread/33893-omv-5-system-monitoring/&postID=248049 and https://www.realhardwarereviews.com/homebrew-nas/21/ also check instructions at https://forum.openmediavault.org/index.php?thread/34453-parity-on-a-cheap-nas-victim-of-silent-corruption/&postID=251372#post251372
- Go to System, Notification, Notifications and Enable Filesystems and S.M.A.R.T.
- Go to Storage, S.M.A.R.T., Settings and
3.1.6. Enable system performance statistics collection
- Go to Monitoring, toggle Enable on.
3.1.7. Install updates
- Go to System, Update Management, press Check, select everything with Package information and press Install.
3.1.8. Enable Testing, Extras and Backports repositories
- Go to OMV-Extras, check the Testing repo, Extra repo and Backports. After this the missing System, Plugins is populated with more packages after pressing Check.
3.1.9. NEXT Enable docker and Portainer
- Docker is the defacto plugin ecosystem offering various software for OMV5 nowadays. Trying to install software as plugins outside of docker containers is not recommended, otherwise breakage may occur.
- Portainer is a graphical frontend for docker.
- Go to OMV-Extras, Docker then select from Docker dropdown button Install. Once finished, select Portainer dropdown button, Install.
- Open a new tabl to browser and go to 10.0.1.240:9000. Create your Username and Password for portainer, then press Create user.
- Select Local to manage the local Docker environment and press Connect.
3.1.10. Format the HDD-disk
Note: When using USB-enclosure for HDDs, always partition and format a drive under the controller you intend to use it on. Otherwise, USB-adapters may silently change sector sizes.
To format a drive, go to Storage, Disks. Select the drive, in my case /dev/sdb and press Wipe (quick wipe is fine).
In cases with disks that have existing partitioning table, I've gotten an error:
Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; blockdev --rereadpt '/dev/sdc' 2>&1' with exit code '1': blockdev: ioctl error on BLKRRPART: Device or resource busy
And:
*************************************************************** Found invalid GPT and valid MBR; converting MBR to GPT format in memory. *************************************************************** Warning: The kernel is still using the old partition table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8) GPT data structures destroyed! You may now partition the disk using fdisk or other utilities. 1+0 records in 1+0 records out 4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.0213194 s, 197 MB/s
This may be due to device being mounted, but in my case this wasn't so. So to overcome this, I did "Secure wipe", which takes longer but allows me to wipe the disks.
3.1.11. Create file system and mount the drive
- Go to Storage, File Systems and press Create. Select from drop down /dev/sdb and Label it as data.
Mounting may fail with error message:
Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; partprobe '/dev/sdc' 2>&1' with exit code '1': Error: Partition(s) 1, 5 on /dev/sdc have been written, but we have been unable to inform the kernel of the change, probably because it/they are in use. As a result, the old partition(s) will remain in use. You should reboot now before making further changes. Error #0: OMV\ExecException: Failed to execute command 'export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin; export LANG=C.UTF-8; partprobe '/dev/sdc' 2>&1' with exit code '1': Error: Partition(s) 1, 5 on /dev/sdc have been written, but we have been unable to inform the kernel of the change, probably because it/they are in use. As a result, the old partition(s) will remain in use. You should reboot now before making further changes. in /usr/share/openmediavault/engined/rpc/filesystemmgmt.inc:648 Stack trace: #0 /usr/share/php/openmediavault/rpc/serviceabstract.inc(588): Engined\Rpc\OMVRpcServiceFileSystemMgmt->Engined\Rpc\{closure}('/tmp/bgstatusol...', '/tmp/bgoutputsZ...') #1 /usr/share/openmediavault/engined/rpc/filesystemmgmt.inc(688): OMV\Rpc\ServiceAbstract->execBgProc(Object(Closure), NULL, Object(Closure)) #2 [internal function]: Engined\Rpc\OMVRpcServiceFileSystemMgmt->create(Array, Array) #3 /usr/share/php/openmediavault/rpc/serviceabstract.inc(123): call_user_func_array(Array, Array) #4 /usr/share/php/openmediavault/rpc/rpc.inc(86): OMV\Rpc\ServiceAbstract->callMethod('create', Array, Array) #5 /usr/sbin/omv-engined(537): OMV\Rpc\Rpc::call('FileSystemMgmt', 'create', Array, Array, 1) #6 {main}
As the error says, this can be fixed by rebooting the rpi4.
- Once file system creation has finished with message The file system creation has completed successfully, press Close, select the drive, press Mount and Apply.
3.1.12. DONE Add rest of the 4 drives to HDD enclosure, format and mount them
- Create file systems like so:
Devices(s) | Label | Filesystem Type |
---|---|---|
/dev/sda1 | ext4 | |
/dev/mmcblk0p1 | boot | vfat |
/dev/sdb1 | data | ext4 |
/dev/sdc1 | music | ext4 |
/dev/sdd1 | photos | ext4 |
/dev/sde1 | backup | ext4 |
/dev/mmcblk0p2 | rootfs | ext4 |
- Test the HDD read/write speeds with hdparm:
tuomas@omv:$ sudo hdparm -tT /dev/sdb /dev/sdb: Timing cached reads: 1458 MB in 2.01 seconds = 726.53 MB/sec Timing buffered disk reads: 468 MB in 3.01 seconds = 155.67 MB/sec
3.1.13. Create network share for music
- Go to Access Rights Management, Shared Folders. Press Add, name it smb-music, device is data and path Music/.
3.1.14. DONE Setup SMB share
- Go to Services, SMB/CIFS. Under General Settings toggle Enable on, and for Extra options define:
min receivefile size = 16384 write cache size = 524288 getwd cache = yes socket options = TCP_NODELAY IPTOS_LOWDELAY mangled names = no catia:mappings = 0x22:0xa8,0x2a:0xa4,0x2f:0xf8,0x3a:0xf7,0x3c:0xab,0x3e:0xbb,0x3f:0xbf,0x5c:0xff,0x7c:0xa6
Note that there isn't other reasons to tweak smb settings.
- Go to Shares tab and press Add. Select music share and define for
Public
Only guests
. For Extra options definevfs objects = catia
. This makes sure the characters in filenames, such as '?', are transformed into '¿', which Windows can display. - The Samba share and mount point can be found in rpi4 host from /srv/dev-disk-by-label-data/Music.
Run $ testparm
to see if there are problems with samba config:
tuomas@omv:~ $ testparm Load smb config files from /etc/samba/smb.conf Processing section "[smb-music]" Loaded services file OK. Server role: ROLE_STANDALONE Press enter to see a dump of your service definitions # Global parameters [global] disable spoolss = Yes dns proxy = No load printers = No log file = /var/log/samba/log.%m logging = syslog map to guest = Bad User max log size = 1000 min receivefile size = 16384 multicast dns register = No pam password change = Yes panic action = /usr/share/samba/panic-action %d passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* . passwd program = /usr/bin/passwd %u printcap name = /dev/null server string = %h server socket options = TCP_NODELAY IPTOS_LOWDELAY catia:mappings = fruit:aapl = yes idmap config * : backend = tdb aio read size = 16384 aio write size = 16384 create mask = 0777 directory mask = 0777 mangled names = no printing = bsd use sendfile = Yes write cache size = 524288 [smb-music] create mask = 0664 directory mask = 0775 ea support = No force create mode = 0664 force directory mode = 0775 guest ok = Yes guest only = Yes hide special files = Yes inherit acls = Yes path = /srv/dev-disk-by-label-data/Music/ read only = No store dos attributes = No tuomas@omv:~ $
3.1.15. DONE Test SMB share in Windows
- Open Windows file explorer with win-e, focus address bar with ctrl-l and type \\10.0.1.240.
- A Windows 10 authentication window opens up. Select Other and type in admin and your-omv-password. Check Save your password.
Copy a file to
music
. Transfer speed should be… ?Continue troubleshooting the SMB speed with:
- https://www.reddit.com/r/OpenMediaVault/comments/emu2n4/expected_speeds_for_rpi_4/
- https://www.reddit.com/r/OpenMediaVault/comments/er9eqx/help_increasing_slow_11_mbs_pi4_nas_transfer/
- https://www.reddit.com/r/OpenMediaVault/search?q=SMB%20speed&restrict_sr=1
- https://www.google.fi/search?sclient=psy-ab&site=&source=hp&q=site%3Ahttps%3A%2F%2Fforum.openmediavault.org+rpi4+SMB+speed&=&=&oq=&pbx=1
- Probably slow transfer is due to wireless network, according to iper3 test over wlan: https://forum.openmediavault.org/index.php/Thread/28408-Poor-speed-while-copying-Files/?postID=211858#post211858
Similar speed issues: https://www.reddit.com/r/OpenMediaVault/comments/fwuicz/raspberry_pi4_slow_speeds/ –> maybe I have just reached max wifi speed: "30MB/s (300Mbps theoretical) is max for Wi-Fi" –> I get 25 MB/s with wifi).
Edit: This is correct, as is witnessed by in this reddit thread.
- Update: Potential solution to smb speed issues: https://www.reddit.com/r/OpenMediaVault/comments/11gwi1g/significant_samba_speedperformance_improvement_by/
Continue with guide Getting started with openmediavault 5 and from "OMV - Initial Configuration".
- Populate Samba-share with music
$ rsync -avuz user@host.com:"/path/to/music/backup/Music/*" /srv/dev-disk-by-label-data/Music
If directories or files copied to /srv/dev-disk-by-label-data/Music contain special characters (such as ()?:*), Windows explorer shows those something like M43J1E~0. To fix that:
- Go to Services, SMB/CIFS and under Extra Options add:
mangled names = no
.
- Go to Services, SMB/CIFS and under Extra Options add:
- Test access to Samba music share with iOS Files app
- Tap open in iOS the app Files.
- Tap the top right three dots (…) and select Connect to Server.
- Type in smb://10.0.1.240/music
- Connect as Guest.
3.1.16. DONE Setup NFS share on host
- Go to Services, NFS, Shares tab. Press Add and then +.
- For Name, give e.g. music-staging (share name must be unique), for Device, select music [/dev/sdc1]. Leave Path as default /Music and default for Permissions (Others: read only). Press Save. This will produce /etc/exports with content:
# This file is auto-generated by openmediavault (https://www.openmediavault.org) # WARNING: Do not edit this file, your changes will get lost. # /etc/exports: the access control list for filesystems which may be exported # to NFS clients. See exports(5) /export/music-staging (fsid=cbe0ded1-f420-4fa8-86eb-f7ab7487dcb7,rw,subtree_check,insecure,no_root_squash,async) # NFSv4 - pseudo filesystem root /export (ro,fsid=0,root_squash,no_subtree_check,hide)
- In Add share dialogue for Shared folder select music-staging. Client can be empty as we don't have requirements in LAN who is allowed to mount this (otherwise this could be (10.0.1.1/24). Privilege can be Read/Write. Extra Options setup as: subtree_check,insecure,no_root_squash,async, see here for more info. Press Save, then Apply.
- Lastly in Services, NFS toggle Enable to activate NFS, remember to Save and Apply changes.
- NFS is now mounted and visible in files system under /export:
$ ls -al /export/music-staging/
total 60 drwxr-sr-x 4 root users 4096 Jun 11 21:56 . drwxr-xr-x 3 root root 4096 Jun 12 09:27 .. drwxrwsrwx+ 1008 root users 36864 Apr 27 18:28 Music drwx------ 2 root root 16384 Jun 11 17:47 lost+found
3.1.17. DONE Test access to NFS share on a client
- With another linux box, install first nfs-common. This will make /sbin/mount.nfs4 and /sbin/showmount commands available:
$ sudo aptitude install nfs-common
- Create a mount point in the client box:
$ sudo mkdir -p /mnt/nfs/share
- Mount as NFSv4 file system:
$ sudo mount -t nfs4 10.0.1.240:/export/music-staging /mnt/nfs/share
Edit: Use instead:
$ sudo mount.nfs4 10.0.1.240:/ /mnt/nfs/share
Reason: You need to specify only / as fsid is set to 0. Source.
3.1.18. DONE Automate NFS mounting on client
- As per openmediavault forum post.
- Install autofs on the client:
$ sudo aptitude install autofs
- Create /etc/auto.master. This specifies the mount point for autofs:
# # Sample auto.master file # This is a 'master' automounter map and it has the following format: # mount-point [map-type[,format]:]map [options] # For details of the format look at auto.master(5). # #/misc /etc/auto.misc # # NOTE: mounts done from a hosts map will be mounted with the # "nosuid" and "nodev" options unless the "suid" and "dev" # options are explicitly given. # #/net -hosts # # Include /etc/auto.master.d/*.autofs # The included files must conform to the format of this file. # #+dir:/etc/auto.master.d # # Include central master map if it can be found using # nsswitch sources. # # Note that if there are entries for /net or /misc (as # above) in the included master map any keys that are the # same will not be seen as the first read key seen takes # precedence. # # +auto.master /srv/nfs /etc/auto.nfs --ghost
- Create /etc/auto.nfs. This specifies the nfs servers and mount settings:
music-staging -fstype=nfs4,hard,intr,nodev,nosuid,async,rsize=1048576,wsize=1048576 10.0.1.240:/export/music-staging
- Finally, add the user you're logged in, as a member to the group users. This needs to be on both on host (already done by OMV), and the client.
- On client, do
$ groups
tuomas@UX32LN:~/$ groups tuomas adm cdrom sudo dip plugdev lpadmin sambashare
- Add yourself to same users group as your user at omv-serve with
$ sudo usermod -a -G users tuomas
- Logout & login or reboot for group changes to take effect.
After this on client, /srv/nfs/music-staging (mapped to /export/music-staging) is automatically mounted on reboot and is writable by my regular user.
- If you need to add any other mount points & shares to auto.nfs, add a new line and restart autofs by doing:
$ sudo service autofs reload
3.1.19. DONE Install resetperms plugin
To easily see what shared folders are shared by which services (NFS, SMB, etc.) and to manage read/write/execute permissions of said folders, and reset the permissions to their defaults.
- Go to Services, Plugins and search for a plugin
openmediavault-resetperms 5.0. Select it and press Install.
After this, a new tab "Shared Folder In Use" appears on the shared folders page.
3.1.20. NEXT Install fatrace
Install fatrace as per the instructions.
3.1.21. DONE rsync from kapsi Nextcloud instance to /dev/sdd1 everyone's photos as backups
$ sudo mkdir /srv/dev-disk-by-label-photos/nextcloud/data
Backups for Tuomas:
$ sudo rsync -avhz --progress --delete --exclude={"cache","files_trashbin","files_versions","uploads","files/Music","files/Games","files/Shared"} pyyhttu@kapsi.fi:"~/siilo/sites/pyyhttu-siilo.kapsi.fi/www/nextcloud/data/Tuomas" /srv/dev-disk-by-label-photos/nextcloud/data/
Backups for Heidi:
$ sudo rsync -avhz --progress --delete --exclude={"cache","files_trashbin","files_versions","uploads"} pyyhttu@kapsi.fi:"~/siilo/sites/pyyhttu-siilo.kapsi.fi/www/nextcloud/data/Heidi" /srv/dev-disk-by-label-photos/nextcloud/data/
Backups for Aura
$ sudo rsync -avhz --progress --delete --exclude={"cache","files_trashbin","files_versions","uploads"} pyyhttu@kapsi.fi:"~/siilo/sites/pyyhttu-siilo.kapsi.fi/www/nextcloud/data/Aura" /srv/dev-disk-by-label-photos/nextcloud/data/
If data in group folders, then:
$ sudo rsync -avhz --progress --delete --exclude={"trash","versions","1/Music"} pyyhttu@kapsi.fi:"~/siilo/sites/pyyhttu-siilo.kapsi.fi/www/nextcloud/data/__groupfolders/" /srv/dev-disk-by-label-photos/nextcloud/data/__groupfolders/
3.1.22. NEXT rsync from kapsi Nextcloud instance to /dev/sdd1 Tuomas's photos as backups
$ sudo rsync -avhuz --progress --exclude={"appdata_ocg80o430zwr","audit.log*","files_external","__groupfolders",".htaccess","index.html","news","nexcloud.log*",".ocdata","ownbackup","pyyhttu","Tuomas/cache","Tuomas/files/Music","Tuomas/files/Shared","Tuomas/files_trashbin","Tuomas/files_versions","updater*"} user@host.fi:"~/siilo/sites/user-siilo.kapsi.fi/www/nextcloud/data/" /srv/dev-disk-by-label-photos/nextcloud/data/
3.1.23. NEXT Apply same permissions resursively on srv/dev-disk-by-label-photos as on srv/dev-disk-by-label-data
- So that sudo is not needed.
3.2. Maintenance
3.2.1. Install updates in openmediavault admin panel
- Go to System, Update Management, press Check, select everything with Package information and press Install.
These are the very same updates as one would login via ssh and do $ sudo aptitude update && sudo aptitude safe-upgrade
.
Alternatively, setup unattended upgrades.
3.2.2. Check reboot need periodically with needrestart
- As per this recommendation needrestart is installed, and it is operated then periodically as per this blog post. It will check which daemons need to be restarted after library upgrades, or if reboot need is due to new kernel.
To check the services, do before & after reboot:
$ sudo needrestart -b
$ sudo needrestart -u NeedRestart::UI::stdio -r l
3.2.3. FOLLOWUP Setup rsync as a scheduled job to backup files to remote server
In order for scheduled rsync job to run through establish passwordless login to remote server. This is needed so that the job can run from start to finish without stopping in between prompting the password.
Generate public/private rsa key pair on rpi4:
$ ssh-keygen -t rsa
- NEXT Document the key conversion to pem
https://github.com/openssl/openssl/issues/13947#issuecomment-822441502
As it will be needed in pem format in omv at System, Certificates and this is the only format that imports the key. The key is defined because of https://openmediavault.readthedocs.io/en/stable/administration/services/rsync.html to enable push rsync.
Generating public/private rsa key pair. Enter file in which to save the key (/home/tuomas/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/tuomas/.ssh/id_rsa. Your public key has been saved in /home/tuomas/.ssh/id_rsa.pub. The key fingerprint is: SHA256:5gfdY9ijaZENHpk+ATyeJ5mvWfckjpOjhGDLAW+wSbs tuomas@omv The key's randomart image is: +---[RSA 2048]----+ | . .=o | |* .. o. | |oB . | |+Bo / o | |o+B . S | |+/ o o b = | |@.O o . . | |E* B . | |+.. | +----[SHA256]-----+
Copy the public key to the server:
$ scp /home/tuomas/.ssh/id_rsa.pub user1@remote_server:~/
$ ssh user1@remote_server
On the remote server append the key to end of authorized key list:
$ cat ~/id_rsa.pub >> ~/.ssh/authorized_keys
If my omv box ever becomes compromised,
Under System, Scheduled Jobs, setup a rsync job to periodically backup config, data, etc. Note that there is also dedicated rsync backup method listed under Services, Rsync, but scheduled jobs should do just fine.
Press Add then for Command define the rsync backup command.
- Peridically backup beets database and config files
$ rsync -avhz --progress /home/tuomas/data/musiclibrary.db /home/tuomas/.config/beets/* user@remote.fi:~/public_html/debian/beets
- NEXT Periodically backup Music
$ rsync -avhz --progress --delete --dry-run /srv/dev-disk-by-label-data/Music/ user@remote.fi:~/siilo/sites/user-siilo.remote.fi/Music/
Warning: Be extra careful on setting this automated cron as
--delete
flag is involved. Run this only scheduled when you have versioned 3-2-1 backups in place for this data.
3.3. Troubleshooting
3.3.1. Changing default user and passwd
Experiment with new installation if changing hostname (omv) and new user (tuomas) are OK. Check that new user is created properly under Access Rights Management, User. I may still have to read tuomas to ssh group there, if adding that user prior running the script is removed from group (by the script).
3.3.2. DONE Disks won't spin down.
After reading not having disks to spin down is actually wanted as it will prolong the disks life span.
3.4. Future direction
3.4.1. NEXT go through the technodad installation tutorials:
3.4.2. NEXT with rpi4 8gb version, setup NFS backup
- Requires 64-bit Raspberry Pi OS image, but the benefits of NFS are lucrative.
3.4.3. NEXT Enable Borgbackup plugin
- and setup it correctly: https://forum.openmediavault.org/index.php?thread/32837-borgbackup-cronjob/
- Investigate into scheduling omv-backup and snapraid-diff.
3.4.4. NEXT Enable fail2ban plugin
3.4.5. NEXT ZFS Samba settings
3.4.6. NEXT Plugins
- Install and experiment usage of the plugins: "Nevertheless I still use a LOT of plugins which I think will never go away because they're all system related: snapraid, unionfs, backup, wakealarm, flashmemory and so on." – Source.
4. NEXT Upgrade to OMV6
- Changelog and short instructions here. Before upgrading, install beets web plugin and see if it clashes with omv5 webserver setup. Also, before upgrading, check this list.
- Once installed, install and test OMV6 regen script for disaster recovery.
4.1. NEXT Change or disable login page background
As per this.
5. mullvad wireguard
- Why mullvad here.
- We use Wireguard official client instead of mullvad's implementation, as it does not support arm architecture.
5.1. NEXT Setup a single, isolated qbittorrent or rtorrent docker
- Then setup another docker for wireguard-vpn and route torrent traffic there, experiment with it as per tutorial linked in here: https://old.reddit.com/r/selfhosted/comments/iwz9hj/simple_wireguard_docker_network_setup/
- Alternatively, try to setup the kill switch so that I can still ssh in from my local lan: https://www.reddit.com/r/mullvadvpn/comments/jm7anx/allow_my_local_lan_to_connect_with_kill_switch/
5.2. Preparation
- Create an account number and login with it to Mullvad.net. Note that if the account is bought with bitcoins, one is entitled for a 10% discount.
5.3. Install wireguard
$ sudo aptitude install raspberrypi-kernel-headers
- Disable 10.0.1.240 in pi-hole under Group management and Clients so that there are no DNS mixups etc. Then have pi-hole FTL to flush its internal domain-blocking cache with:
$ pihole restartdns reload-lists
- Disabling pihole for our omv seedbox (10.0.1.240) is an extra steps, but I want to be sure that when VPN is on, mullvad's DNS resolution is being used.
5.3.1. NEXT Investigate that mullvad completely bypassess pi-hole DNS-configuration
- As is written here.
- When VPN tunnel is on, restore pi-hole for 10.0.1.240.
- Install Wireguard to Raspberry Pi OS:
$ echo "deb http://deb.debian.org/debian/ unstable main" | sudo tee --append /etc/apt/sources.list.d/unstable.list
$ wget -O - https://ftp-master.debian.org/keys/archive-key-$(lsb_release -sr).asc | sudo apt-key add -
$ sudo aptitude update
$ sudo aptitude install wireguard -y
- Run Mullvad configuration script:
$ curl -LO https://mullvad.net/media/files/mullvad-wg.sh && chmod +x ./mullvad-wg.sh && ./mullvad-wg.sh
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 3291 100 3291 0 0 17412 0 --:--:-- --:--:-- --:--:-- 17505 [?] Please enter your Mullvad account number: xxxxxxxxxxxxxxxx [+] Contacting Mullvad API for server locations. [+] Generating new private key. [+] Contacting Mullvad API. [+] Writing WriteGuard configuration files. [+] Success. The following commands may be run for connecting to Mullvad: - Melbourne, Australia: $ wg-quick up mullvad-au3 - Melbourne, Australia: $ wg-quick up mullvad-au4 - Sydney, Australia: $ wg-quick up mullvad-au1 - Sydney, Australia: $ wg-quick up mullvad-au2 - Sydney, Australia: $ wg-quick up mullvad-au5 - Sydney, Australia: $ wg-quick up mullvad-au6 - Sydney, Australia: $ wg-quick up mullvad-au7 - Vienna, Austria: $ wg-quick up mullvad-at1 - Vienna, Austria: $ wg-quick up mullvad-at2 - Brussels, Belgium: $ wg-quick up mullvad-be1 - Brussels, Belgium: $ wg-quick up mullvad-be2 - Brussels, Belgium: $ wg-quick up mullvad-be3 - Brussels, Belgium: $ wg-quick up mullvad-be4 - Sao Paulo, Brazil: $ wg-quick up mullvad-br1 - Sofia, Bulgaria: $ wg-quick up mullvad-bg1 - Sofia, Bulgaria: $ wg-quick up mullvad-bg2 [...] - Salt Lake City, UT, USA: $ wg-quick up mullvad-us9 - Seattle, WA, USA: $ wg-quick up mullvad-us5 Please wait up to 60 seconds for your public key to be added to the servers. tuomas@omv:~ $
- Enable port forwarding: https://mullvad.net/en/help/wireguard-and-mullvad-vpn/
- Before enabling the Wireguard tunnel with Mullvad, check your current ip
with:
$ curl ipconfig.me
. - Now test the connection with some server, in this case with mullvad-se4:
$ wg-quick up mullvad-se4
[#] ip link add mullvad-se4 type wireguard [#] wg setconf mullvad-se4 /dev/fd/63 [#] ip -4 address add 10.67.107.184/32 dev mullvad-se4 [#] ip -6 address add fc00:bbbb:bbbb:bb01::4:6bb7/128 dev mullvad-se4 [#] ip link set mtu 1420 up dev mullvad-se4 [#] resolvconf -a mullvad-se4 -m 0 -x Too few arguments. Too few arguments. [#] wg set mullvad-se4 fwmark 51820 [#] ip -6 route add ::/0 dev mullvad-se4 table 51820 [#] ip -6 rule add not fwmark 51820 table 51820 [#] ip -6 rule add table main suppress_prefixlength 0 [#] ip6tables-restore -n [#] ip -4 route add 0.0.0.0/0 dev mullvad-se4 table 51820 [#] ip -4 rule add not fwmark 51820 table 51820 [#] ip -4 rule add table main suppress_prefixlength 0 [#] sysctl -q net.ipv4.conf.all.src_valid_mark=1 [#] iptables-restore -n
- Do ip check again and observe that the IP is now different:
$ curl ipconfig.me
5.4. NEXT add 10.0.1.240 back to pi-hole clients and observe during VPN testing
- If it has effect. If not, do nothing and remove this and do not remove 240 from pihole service. If effect (e.g. DNS is
changed or am.i.mullvad fails) then document here the effect and remove 10.0.1.240 from pi-hole.
- "When mullvad is running it completely bypasses the DNS configuration on your devices and routers, forcing a connection to their own DNS service. Your pihole won't receive any requests and won't perform any blocking. Mullvad is likely performing some blocking of their own. People have been asking them for years for the ability to add custom DNS servers so that it can be used with PiHole. […] "When mullvad is disabled your PiHole will start receiving requests again and take over blocking functionality." https://www.reddit.com/r/pihole/comments/hnqz6k/comment/fxd6g8d
5.5. Test VPN speed with well seeded linux torrents
If I experience horrible speeds with torrents, change the MTU speed in wireguard config as it usually fixes the speed issues.
- Finally, add the kill switch:
$ sudo su
# nano /etc/wireguard/mullvad-se4
- Add the following lines under the [Interface] section of the WireGuard configuration files found in /etc/wireguard/:
PostUp = iptables -I OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT && ip6tables -I OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT PreDown = iptables -D OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT && ip6tables -D OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT
- NOTE: After enabling the Wireguard tunnel now, note that SSH-connection is severed. I may want to retry the kill switch implementation with archwiki instructions.
5.6. NEXT Investigate why this is, a pointer here:
5.7. NEXT Try setting /etc/iproute2/rt_tables as in here:
5.8. NEXT Test if with pihole I have now a dns leakage
- am.i.mullvad –> google if can be used with cmd line? Also check the torrect check.
5.9. NEXT After installing wireguard and connecting to mullvad, check connection:
$ curl https://am.i.mullvad.net/connected
Additional info on checking connection.
You are connected to Mullvad (server se4-wireguard). Your IP address is
193.138.218.254
5.10. NEXT and speed:
5.11. NEXT Check if I need additional firewall rules
5.12. NEXT Should I go with manual installation method:
- https://mullvad.net/en/help/easy-wireguard-mullvad-setup-linux/
- and not setup ipv6, as: "Also uncheck IPv6 (because it sucks) or at the very
least if you are stupid enough to use that, make sure you are actually using IPv6 and have it enabled on your device and router lol… otherwise errors." What errors? (Note to self: Enable ipv6 if you need more speed) –> test max speed with linux iso torrents. Source: https://dietpi.com/phpbb/viewtopic.php?p=19461
5.13. NEXT Check speed with this:
- Nice for knowing your speeds whilst VPN is on/off without needing a browser!
- Speedtest command:
~ $ curl -s https://raw.githubusercontent.com/sivel/speedtest-cli/master/speedtest.py | python -~
6. NEXT Document why to keep host os as clean as possible
- Reason: https://www.reddit.com/r/OpenMediaVault/comments/hpbooe/add_pihole_capability_to_omv/
- Fix, install portainer with omv-templateurl from here: https://www.reddit.com/r/selfhosted/comments/gyltsd/the_selfhostedpro_and_nashosted_portainer_app/
7. NEXT Install plugin resetperms and figure out what it does
8. NEXT Check contents from this article and compare contents to my article
Footnotes:
OpenMediaVaulti's homepage.
Yottamaster PS5003-SV5 enclosure uses JMS567+JMB575 chipsets for USB
controller. Now, by doing lsusb -t
we see the command to return
Driver=usb-storage, 5000M, not Driver=uas. Reason is, that despite
Yottamaster claiming UASP compatibility, the JMicron JMS567 chipset they
use, its device id 152d is on kernel's internal
quirk
list. Meaning: Benefits of UASP are not achievable with this enclosure due
to reason that JMS567 chip
doesn't
handle FUA properly, which is required for uasp to
work. Unfortunately, this is a
widespread
issue in linux. Also
tested
in Win10 and there too legacy protocol Bulk Only Transport (BOT) speed was
registered, not UASP.
However, speed wise UASP is not critical
as HDDs
throughput
is limited anyway to 100-200 MB/s. BOT can offer 250 MB/s. One option for storage enclosure is to go with ASM1153, as
it is
listed
compatible (though for some reason not with Seagate drives).
2.5" HDDs are simpler to deal with in power and heat. 2.5" HDD drives typically don't need external power to operate (they can operate with 5v power), while 3.5" drives will pull enough amps to need more power than the USB ports on the Rpi4 can provide. The larger 3.5" drives will also demand active cooling (usually a fan) where the 2.5" HDDs don't need one. A very nice setup with Rpi4 and 2,5 inch drives.
Raspberry shows under "Network Map Clients List" with "Manual" IP, not "Static", but that should be the same. Manual vs. static IP explained.