| |
Containers have become the de facto way of running applications on the server, and while Docker is the most well-known way of doing this, there are other alternatives e.g. podman, LXC, containerd.
However, they are all built on the same foundational technologies provided by Linux, and this series of tutorials will start off by taking a look at these:
We'll then take a look at how these things are used, in practice:
Finally, we put all these pieces together and build an OCI-compliant image, manually and from scratch, and then run it:
The difference between images, containers and virtual machines
Before we start, it's worth clarifying what these things are, and the differences between them.
A single physical computer can be partitioned into several smaller "virtual machines". These operate as smaller, independent computers, with their own operating system, and can be rebooted or powered on and off individually, as if they were actually separate computers themselves. These don't have anything to do with the subject of this tutorial series - images and containers - so we won't be talking about them at all.
A container is one or more processes that run on the host computer, but are kept separate from everything else through the use of namespaces and control groups. Other than this, they are just normal processes running on the host, same as any other process.
An image is a set of files, and a bunch of settings (e.g. environment variables) that will be used to run a container. It can contain an entire operating system, but it doesn't have to. In other words, an image is a static thing, a template for creating containers, while a container is a dynamic runtime thing, consisting of one or more running processes.
For quite a few years, I've had a Raspberry Pi sitting on my local network, running a bunch of Docker'ized applications. It's managed by a bunch of shell scripts, and while it works, it's a bit hacky, and I've had a long-standing item on my to-do list to be a Big Boy and Do Things Properly i.e. run it all on Kubernetes 
My workload is nowhere near big enough to warrant a full Kubernetes cluster, so I'll be using K3s, which is intended for smaller workloads, and for extra style and difficulty points, I'll be running it on Fedora CoreOS, a minimal OS designed for running container workloads. In particular, it's immutable, so you don't e.g. apply updates and install new packages, you blow the whole thing away and replace it with a new image.
There are any number of tutorials out there on the intarwebs for setting up both K3s and CoreOS, but it's a pretty trivial process. However, the devil is, as always, in the details, and since there are quite a few niggling issues when setting up a usable server, I've documented the whole process here. It's relatively straight-forward once you've figured out how things work; the tricky thing is, of course, figuring out how things work
Tutorial index
- In which we set up the server
- In which we install CoreOS
- In which we install K3s
- In which we deploy a demo app
- In which we set up the Traefik dashboard
- In which we enable TLS for our demo app
- In which we enable SSH access into our demo container
- In which we install Open VM Tools into CoreOS
- In which we (don't) install other extra packages and tools into CoreOS
- In which we don't fix the problem of slow shutdowns
- In which we also don't configure the firewall
- Odds and ends.
If you liked this tutorial, there's another one here about setting up a bare-metal K8s cluster.
 One of the things that has really bugged me over the years is the trend of writing programs in such a way that they download a large number of third-party libraries, written mostly by unknown randos on the internet, and then running them as part of your program. It's an obvious security risk, and even if you go to the trouble of verifying that something is trustworthy, there's no guarantee that it will remain so over the life of your program.
With the rise of containerized applications, these dependencies are downloaded over and over and over again, each time the image is rebuilt, which is a problem if your internet access isn't great, and regardless, the engineer in me just detests this kind of waste and inefficiency, just on principle 
One way to solve these problems is to run a local proxy for the online repo's i.e. something that downloads the dependencies for you, and caches them. This doesn't address the issue of trusting a third-party library in the first place, but one thing that will help establish that trust is time i.e. after you've been using it for a while, and nothing has been reported on Hacker News or Reddit about problems with it, this will help increase your confidence that it's OK.
There are a few programs out there that provide this kind of service, but they are specific to their particular repository. However, Sonatype's Nexus Repository Manager offers support for a large number of online repositories, thus providing a one-stop solution, and this tutorial will go through the process of setting it up, and configuring it for some of the more popular online repo's.
Before we start
- I run my own DNS, and have set it up so that the name nexus3 resolves to the server running Nexus. This can cause problems when trying to access Nexus from inside a Docker container, which is discussed here. However, in all cases, you can also reference the server by its IP address.
- Nexus stores artifacts in the file system, and while it's not essential to do so, it's possible to keep things in separate sub-directories by creating a new storage blob for each one.
- Nexus doesn't seem to have an option to force downloaded artifacts to be immutable, but it's possible to configure repositories to never check back with the online repo to see if an artifact has changed (by setting the Maximum Component Age to -1), which is close enough.
Tutorial index
I've always been fond of the phrase "cattle, not pets", which refers to the idea that computer servers should be treated as cattle (i.e. you should have no problem killing them), as opposed to pets. One of the most important changes over the 35+ years I've been a professional developer is the rise of automating processes. Back in the day, if you wanted to set up a new server, you did it manually, carefully installing all the software and other dependencies, then even more carefully configuring them, and since you didn't want to have to do that work again, these servers were treated as precious pets. But today, with the rise of technologies such as Ansible and containers, servers are disposable - if one fails, just throw it away and run a script to create a new one.
 This approach introduces some new considerations (e.g. managing a fleet of servers, re-creating them when they fail, etc.), giving rise to a new class of software known as container orchestration. The king of these is Kubernetes, and since I recently did a bit of work with this, I wanted to set up a local instance for testing. While there are things like minikube, that let you set up a local cluster on your PC, there's nothing like a proper test environment that mirrors a real production environment as closely as possible.
The trend these days is, of course, to do everything in the cloud, so there's no shortage of information on how to set things up using e.g. AWS or GCP, but rather less on how to set up a bare-metal local cluster, so we'll remedy that here with a set of instructions on how to set up a local Kubernetes instance that has:
- a single VM that provides the control plane
It's possible to have a single server manage the control plane and act as a node (i.e. run containers), but in the interests of making this cluster as "real" as possible, we'll separate them out.
- two more VM's that will act as nodes
We want two of these, so that we can test things like distributing workloads over multiple servers, automatic failover if a server goes down, etc.
- dynamically-provisioned disk space
This gives us persistent disk storage, even as servers come and go.
- a local Docker registry to store images
So that we don't have to put them on Docker Hub.
Tutorial index
If you liked this tutorial, there's another one here about setting up K3s on Fedora CoreOS.
 Yeah, it's been a while I've been quietly chugging away in the background on Awasu client work, as well as other non-Awasu projects, but it's been some time since my last mega-tutorial, so let's remedy that with a deep dive into the internals of everyone's favorite source control system, git.
This tutorial assumes that you are familiar with using git (e.g. commits, branches, tags), and we'll take a look at the internals of git and how it works, and in particular, its file formats.
 A few years ago, I wrote a long series of tutorials showing how to embed Python into a C/C++ program, and periodically threatened to write another series showing how to go the other way i.e. extend Python by calling your own C/C++ code.
Well, I've finally made good on that promise and written some tutorials on how to write a Python extension module:
Never let it be said that I don't keep my promises!
 Many moons ago, I wrote a tutorial on how to set up an internet gateway on a Banana Pi, complete with DHCP, DNS, VPN, firewall and ad-blocking. It works well, I still use one today, and have even taken it with me on a few long backpacking trips. However, I worry about it being a bit fragile, and fear the day when an over-zealous customs officer decides it looks like something that could trigger a bomb , so I was overjoyed when I finally found my holy grail: something that does all of the above, in the form factor of a USB thumb drive.
GL-iNet's GL-USB-150 costs around USD 30, and comes with almost everything I need to get online when I'm on the road. This tutorial will be much shorter than the previous one, because nearly everything is already set up and ready to go 
Getting started
Plug it in, give it 30 or 40 seconds to start up, then open a browser and go to http://192.168.8.1. To login, the default password is goodlife; once you're in, change this under More Settings/Admin Password.
It runs a DHCP server, and your computer will have already been assigned an IP address in the 192.168.8.xxx range.
 Go to the Internet page, click on the Scan link, then connect to a WiFi network.
Open another browser window, and confirm that you're online.
Configuring the VPN
Go to the Management tab in the VPN/OpenVPN Client page, and upload your VPN configurations. This will typically be a ZIP of a bunch of .ovpn files, but if you have them, you will also need to include the .crt and/or .pem files.
Unfortunately, the stock firmware has a bug that prevents the ZIP file from being processed correctly, so you will need to upgrade the firmware first. Get the latest version from here, then install it via the Upgrade page.
 Once the VPN configurations have been installed, you will be able to select which one to use from the VPN/OpenVPN Client page. Check your IP address to confirm that you are going through the VPN.
Installing software
 To install additional software, go to the More Settings/Advanced page, and in the new browser window that opens, go to System/Software and update the package lists.
I installed the following packages:
- bash
- tmux
- openssh-sftp-server (so that I can scp files in and out)
- openssh-client (for a version of ssh that allows forwarding)
- coreutils (for GNU tools)
- bind-dig (for dig)
- mtr (a handy network monitoring tool)
To change your default shell to bash, update /etc/passwd.
The only down-side to this device: while you can just about install a minimal version of Python, the disk is so small, there won't be any room for anything else 
Ad-blocking
The only thing missing from this device is an ad-blocker. Since it uses dnsmasq for DNS, rather than bind as the Banana Pi does, the process is slightly different, but not much. Here's the script that I use:
# This script downloads blacklisted ad servers and updates dnsmasq to block them.
#
# The following line needs to be added to /etc/dnsmasq.conf:
# conf-file=/root/dns-blacklist
BLACKLIST_URL="http://pgl.yoyo.org/as/serverlist.php?hostformat=dnsmasq&mimetype=plaintext"
BLACKLIST_FNAME=/root/dns-blacklist
echo "Downloading the DNS blacklist..."
TMP_FNAME=/tmp/dns-blacklist
wget -O "$TMP_FNAME" "$BLACKLIST_URL"
if [ $? -ne 0 ] ; then exit 1 ; fi
echo
# fixup the entries so that they return "NX Domain"
echo "Updating the DNS blacklist..."
sed -i 's/address/server/g;s/127.0.0.1//g' "$TMP_FNAME"
echo
# install the new DNS blacklist
echo "Installing the DNS blacklist..."
echo " $TMP_FNAME => $BLACKLIST_FNAME"
mv "$TMP_FNAME" "$BLACKLIST_FNAME"
echo "Restarting dnsmasq..."
/etc/init.d/dnsmasq restart
echo
echo "All done."
The DNS blacklist is downloaded to a temp file, fixed up and then transferred to /root/dns-blacklist. You will need to tell dnsmasq to load this file by adding the following line to /etc/dnsmasq.conf:
conf-file=/root/dns-blacklist
This script can be configured to run periodically, or just run it manually every now and then.
Shutting down
There doesn't seem to be any way to shut down the device cleanly. I'm guessing it's been designed so that people can just pull the thing out of the USB port, but this really irks the sysadmin in me , so to shutdown cleanly, type the following in the console:
halt
The green LED light stays on, but the device will shutdown.
An Awasu user recently asked for some help in getting their Python code to work with Awasu, and while the problem turned out to be related to text encoding (which is not, strictly speaking, anything to do with Awasu), since this is such a common issue, I thought I'd write up some notes on how all it works.
The canonical introductory article on the subject is, of course, Joel Spolsky's The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)", but these notes will be a more practical guide, along the lines of "How do I get my code to *?@#'ing work?!?!".
Note that while this tutorial has separate sections on Python 2 and Python 3, even if you're only using one version of Python, you should read both sections if you want to really understand how things work.
 This stuff is tricky to get your head around at first, but once you figure it out, it's actually not too bad. The problem is that even when you've got your code right, you start receiving content from elsewhere that is wrong, which breaks your code, so you change it to handle that content, but then your code breaks when you receive content from somewhere else that is doing things correctly, and you get stuck in a cycle where your code never works properly  Hopefully, these notes will help you know when your code is right, and you can stick to your guns and start yelling at the other guy to fix their code... 
 A while back, I posted a massive tutorial on how to set up a NAS file server on a Banana Pi, and it was so much fun, I've written another even massiver one on how to set up an internet gateway on a Banana Pi.
 A gateway lets you isolate computers in your home network from the internet. To reach the internet, a computer has to through the gateway, which means that if you put a firewall or virus checker or ad-blocker here, all your computers will benefit from them.
As before, this series of tutorials will walk you through the whole process of setting up a gateway, including a lot of not-essential-but-nice-to-have stuff. We start off by setting up a bare-bones gateway:
Once this is up and running, we then take a look at installing some useful extra services:
These days, a firewall is pretty much a necessity, and it's quite eye-opening to watch the logs and see the constant stream of attacks, as people try to break into your computer. And even if you run an ad-blocker like uBlock or AdBlock, a DNS-based ad-blocker can be run along-side it, without affecting browser performance at all.
Just a quick follow up on my recent epic tome on setting up a Banana Pi as a file server. I mentioned that I configured my disks to use the ext3 file system, and while it's generally fine, it does have one weakness: it is very slow deleting large files. Even worse, it locks up the file system, impacting other activity and causing stalls, which kinda sucks if you're watching a movie at the time
depesz took a very detailed look at the problem and some possible solutions, the TL;DR being that it's better to progressively shrink the file until it's all gone rather than asking the operating system to delete it as a file.
For the benefit of anyone having problems with this, here's a script that I wrote that implements this idea:
#!/bin/bash
# parse the command-line arguments
if [ $# -lt 1 ] ; then
echo "$(basename $0) file1 file2 file3 ..."
echo " Delete file(s) slowly."
exit 1
fi
chunk_size=100000000
for fname in "$@"; do
# check if we were given a directory
if [ -d "$fname" ]; then
# yup - process each file, then remove the directory
#echo "Deleting directory: $fname"
find "$fname" -type f -exec $(readlink -e "$0") \{\} \;
rm -rf "$fname"
continue
fi
# check if we were given a file
if [ ! -f "$fname" ]; then
# nope - ignore it
continue ;
fi
# yup - delete the file slowly
#echo "Deleting file: $fname"
while true; do
# get the current size of the file
fsize=$(ls -l "$fname" | cut -d' ' -f 5)
if [ $fsize -lt $chunk_size ] ; then
# the file is small enough to just delete
#echo "- Deleting file."
rm -f "$fname"
break
fi
# truncate the file, then loop back
#echo "- Truncating file: $fsize"
truncate -c -s -$chunk_size "$fname" || exit
sleep 0.25
done
done
Pass in a list of files and/or directories and it will slow-delete them. It will take longer to run, but will have far less impact on the rest of the system.
Nothing to do with Awasu, but hopefully someone out there in Internet-land will find it useful...
|
|
|
I am a 
