This is a maintenance release, with mostly bug fixes and performance improvements, and since it's been a while since the last release, the support period for the paid versions has been extended. So, if you purchased (or renewed to) version 3.2, you'll be eligible for this version.
It's been a while since the last release, but the first release candidate for the next version of Awasu is available here.
This is a maintenance release, with mostly bug fixes and performance improvements, and since it's been a while since the last release, the support period for the paid versions has been extended. So, if you purchased (or renewed to) version 3.2, you'll be eligible for this version.
One of the things that has really bugged me over the years is the trend of writing programs in such a way that they download a large number[1]Sometimes hundreds, or even thousands! of third-party libraries, written mostly by unknown randos on the internet, and then running them as part of your program. It's an obvious security risk[2]There have been quite a few of these so-called "supply chain attacks" in recent times., and even if you go to the trouble of verifying that something is trustworthy, there's no guarantee that it will remain so over the life of your program[3]For example, hackers could take control of a repository, or perhaps someone takes over an expired domain name, and thus its email addresses, which might give them control of a library's repository. .
With the rise of containerized applications, these dependencies are downloaded over and over and over again, each time the image is rebuilt, which is a problem if your internet access isn't great, and regardless, the engineer in me just detests this kind of waste and inefficiency, just on principle
One way to solve these problems is to run a local proxy for the online repo's i.e. something that downloads the dependencies for you, and caches them. This doesn't address the issue of trusting a third-party library in the first place, but one thing that will help establish that trust is time i.e. after you've been using it for a while, and nothing has been reported on Hacker News or Reddit about problems with it, this will help increase your confidence that it's OK[4]This requires that the artifacts that have been cached in your local proxy are immutable, to prevent attacks by a malicious party on the version stored in the online repo..
There are a few programs out there that provide this kind of service[5]For example, devpi, local-npm, or a Docker pull-through cache., but they are specific to their particular repository[6]Pulp offers support for different types of repository, although the list doesn't seem quite as extensive as Nexus.. However, Sonatype's Nexus Repository Manager offers support for a large number of online repositories, thus providing a one-stop solution, and this tutorial will go through the process of setting it up, and configuring it for some of the more popular online repo's.
Before we start
I run my own DNS, and have set it up so that the name nexus3[7]I originally used nexus, but this causes problems when trying to open the admin interface in a browser, because that name is on the HSTS preload list, which means that the browser will force the use of HTTPS resolves to the server running Nexus[8]Actually, a Docker container fronted by nginx.. This can cause problems when trying to access Nexus from inside a Docker container[9]Since DNS often operates differently there., which is discussed here. However, in all cases, you can also reference the server by its IP address[10]Unless, of course, you're running Nexus in a Docker container fronted by nginx .
Nexus stores artifacts in the file system, and while it's not essential to do so, it's possible to keep things in separate sub-directories by creating a new storage blob for each one.
Nexus doesn't seem to have an option to force downloaded artifacts to be immutable, but it's possible to configure repositories to never check back with the online repo to see if an artifact has changed (by setting the Maximum Component Age to -1), which is close enough.
For example, hackers could take control of a repository, or perhaps someone takes over an expired domain name, and thus its email addresses, which might give them control of a library's repository.
This requires that the artifacts that have been cached in your local proxy are immutable, to prevent attacks by a malicious party on the version stored in the online repo.
I originally used nexus, but this causes problems when trying to open the admin interface in a browser, because that name is on the HSTS preload list, which means that the browser will force the use of HTTPS
I've always been fond of the phrase "cattle, not pets", which refers to the idea that computer servers should be treated as cattle (i.e. you should have no problem killing them), as opposed to pets. One of the most important changes over the 35+ years I've been a professional developer is the rise of automating processes. Back in the day, if you wanted to set up a new server, you did it manually, carefully installing all the software and other dependencies, then even more carefully configuring them, and since you didn't want to have to do that work again[1]And it was often the case that you couldn't re-create them, even if you wanted to, because of all the minor tweaks and changes that were invariably made over time, that didn't get documented., these servers were treated as precious pets. But today, with the rise of technologies such as Ansible and containers, servers are disposable - if one fails, just throw it away and run a script to create a new one.
This approach introduces some new considerations (e.g. managing a fleet of servers, re-creating them when they fail, etc.), giving rise to a new class of software known as container orchestration. The king of these is Kubernetes, and since I recently did a bit of work with this, I wanted to set up a local instance for testing. While there are things like minikube, that let you set up a local cluster on your PC, there's nothing like a proper test environment that mirrors a real production environment as closely as possible.
The trend these days is, of course, to do everything in the cloud, so there's no shortage of information on how to set things up using e.g. AWS or GCP, but rather less on how to set up a bare-metal local cluster, so we'll remedy that here with a set of instructions on how to set up a local Kubernetes instance that has:
It's possible to have a single server manage the control plane and act as a node (i.e. run containers), but in the interests of making this cluster as "real" as possible, we'll separate them out.
two more VM's that will act as nodes[2]These will actually run the containers.
We want two of these, so that we can test things like distributing workloads over multiple servers, automatic failover if a server goes down, etc.
And it was often the case that you couldn't re-create them, even if you wanted to, because of all the minor tweaks and changes that were invariably made over time, that didn't get documented.
Yeah, it's been a while I've been quietly chugging away in the background on Awasu client work, as well as other non-Awasu projects[1]And to be honest, I haven't felt much like writing , but it's been some time since my last mega-tutorial, so let's remedy that with a deep dive into the internals of everyone's favorite source control system, git.
This tutorial assumes that you are familiar with using git (e.g. commits, branches, tags), and we'll take a look at the internals of git and how it works, and in particular, its file formats.
A few years ago, I wrote a long series of tutorials showing how to embed Python into a C/C++ program, and periodically threatened to write another series showing how to go the other way i.e. extend Python by calling your own C/C++ code[1]Typically because you want better performance, or because you want to run it multi-threaded, which Python is known to not handle very well..
Well, I've finally made good on that promise and written some tutorials on how to write a Python extension module:
Many moons ago, I wrote a tutorial on how to set up an internet gateway on a Banana Pi, complete with DHCP, DNS, VPN, firewall and ad-blocking. It works well, I still use one today, and have even taken it with me on a few long backpacking trips. However, I worry about it being a bit fragile, and fear the day when an over-zealous customs officer decides it looks like something that could trigger a bomb , so I was overjoyed when I finally found my holy grail: something that does all of the above, in the form factor of a USB thumb drive.
GL-iNet's GL-USB-150 costs around USD 30, and comes with almost everything I need to get online when I'm on the road. This tutorial will be much shorter than the previous one, because nearly everything is already set up and ready to go
Getting started
Plug it in, give it 30 or 40 seconds to start up, then open a browser and go to http://192.168.8.1. To login, the default password is goodlife; once you're in, change this under More Settings/Admin Password.
It runs a DHCP server, and your computer will have already been assigned an IP address in the 192.168.8.xxx range.
Go to the Internet page, click on the Scan link, then connect to a WiFi network.
Open another browser window, and confirm that you're online.
Configuring the VPN
Go to the Management tab in the VPN/OpenVPN Client page, and upload your VPN configurations. This will typically be a ZIP of a bunch of .ovpn files, but if you have them, you will also need to include the .crt and/or .pem files.
Unfortunately, the stock firmware has a bug that prevents the ZIP file from being processed correctly, so you will need to upgrade the firmware first. Get the latest version from here[1]Version 3.026 worked for me., then install it via the Upgrade page.
Once the VPN configurations have been installed, you will be able to select which one to use from the VPN/OpenVPN Client page. Check your IP address to confirm that you are going through the VPN.
Installing software
To install additional software, go to the More Settings/Advanced page, and in the new browser window that opens, go to System/Software and update the package lists[2]This doesn't seem to persist after a reboot, so you have to remember to do this every time .
I installed the following packages:
bash
tmux
openssh-sftp-server (so that I can scp files in and out)
openssh-client (for a version of ssh that allows forwarding)
coreutils (for GNU tools)
bind-dig (for dig)
mtr (a handy network monitoring tool)
To change your default shell to bash, update /etc/passwd.
The only down-side to this device: while you can just about install a minimal version of Python, the disk is so small, there won't be any room for anything else
Ad-blocking
The only thing missing from this device is an ad-blocker. Since it uses dnsmasq for DNS, rather than bind as the Banana Pi does, the process is slightly different, but not much. Here's the script that I use:
# This script downloads blacklisted ad servers and updates dnsmasq to block them.
#
# The following line needs to be added to /etc/dnsmasq.conf:
# conf-file=/root/dns-blacklist
BLACKLIST_URL="http://pgl.yoyo.org/as/serverlist.php?hostformat=dnsmasq&mimetype=plaintext"
BLACKLIST_FNAME=/root/dns-blacklist
echo "Downloading the DNS blacklist..."
TMP_FNAME=/tmp/dns-blacklist
wget -O "$TMP_FNAME" "$BLACKLIST_URL"
if [ $? -ne 0 ] ; then exit 1 ; fi
echo
# fixup the entries so that they return "NX Domain"
echo "Updating the DNS blacklist..."
sed -i 's/address/server/g;s/127.0.0.1//g' "$TMP_FNAME"
echo
# install the new DNS blacklist
echo "Installing the DNS blacklist..."
echo " $TMP_FNAME => $BLACKLIST_FNAME"
mv "$TMP_FNAME" "$BLACKLIST_FNAME"
echo "Restarting dnsmasq..."
/etc/init.d/dnsmasq restart
echo
echo "All done."
The DNS blacklist is downloaded to a temp file, fixed up and then transferred to /root/dns-blacklist. You will need to tell dnsmasq to load this file by adding the following line to /etc/dnsmasq.conf:
conf-file=/root/dns-blacklist
This script can be configured to run periodically, or just run it manually every now and then.
Shutting down
There doesn't seem to be any way to shut down the device cleanly. I'm guessing it's been designed so that people can just pull the thing out of the USB port, but this really irks the sysadmin in me , so to shutdown cleanly, type the following in the console:
halt
The green LED light stays on, but the device will shutdown.
Another year, another Awasu Day, and it's with great pleasure I can release the latest and greatest in the long line of Awasu releases: Awasu v3.2.
This release is faster, slicker and just damn prettier than ever before, with numerous improvements, in particular, to the search engine[1]Many thanks to Jacek for prodding me into action on this..
Check it out, not least of all because there will be, yet again, no price increase , and if you purchased in the last 2 years, this release will be included as part of that i.e. completely free.
And once you're up and running, don't forget about one of the key features of Awasu, it's extensibility, with many free plugins here (e.g. skins, Office integration, geo-location and translation tools), as well as a bunch of paid ones e.g. monitoring:
One thing that was quietly introduced in the recent 3.2.rc1 release was support for running on Linux, under Wine. This is something that I've been looking at for quite a while already, and Awasu has run reasonably well like this, but I've finally bit the bullet and made some changes to the code to help with the process.
Most of the issues relate to the embedded browser. They seem to have included a version of Gecko, tricked out to look like Internet Explorer, which mostly works, but Awasu has some sophisticated interactions with its embedded browser, which may or may not work. All the issues I came across have been documented in the wiki, along with work-arounds and other notes.
I'm one of those people who have become increasingly disillusioned with the direction Windows has taken recently, and while I've used Linux on the server for many years[1]And UNIX before that , it's never been much fun on the desktop. However, Gnome on Fedora is not bad these days, and given that my recent foray into the iUniverse has left me scarred for life, it looks like Linux might it, and so this version of Awasu will be getting a lot more dog-fooding in the near future...
Awasu 3.2.rc1 has been released here. This is a release candidate for 3.2, so it comes as an installer, and all you need to do is run it and it will auto-magically upgrade your installation[1]You must be running Awasu 3.0 or later..
This release contains numerous optimizations and performance improvements to speed up all operations, and... nah, just kidding , all that work's been done already. This release contains a few bug fixes and UI tweaks, but there is one change of note: the Personal Edition now allows SSL feeds i.e. feeds that use the more secure https:// URL's (instead of http://).
There's been a big push over the past year or two to get people to use SSL, in particular with the major browsers slowly starting to flag web sites that don't use it as potentially insecure. SSL feeds had previously been available only in paid versions of Awasu, but can now be used even in the free Personal Edition, as well. It's a small thing, in the grand scheme of things, but it's our contribution to helping make the web a safer place
Have fun, and if all goes well, this will become the final 3.2 release.
Awasu and the stylized Japanese character in the orange box are trademarks of Awasu Pty. Ltd. Other brands and product names are trademarks of their respective owners. Awasu Pty. Ltd. believes the information in this publication is accurate as of its publication date. Such information is subject to change without notice. Awasu Pty. Ltd. is not responsible for inadvertent errors.
One of the things that has really bugged me over the years is the trend of writing programs in such a way that they download a large number
This approach introduces some new considerations (e.g. managing a fleet of servers, re-creating them when they fail, etc.), giving rise to a new class of software known as container orchestration. The king of these is 
A few years ago, I wrote a long series of 
Many moons ago, I wrote a 
I am a 
