An Awasu user recently asked for some help in getting their Python code to work with Awasu, and while the problem turned out to be related to text encoding (which is not, strictly speaking, anything to do with Awasu), since this is such a common issue, I thought I'd write up some notes on how all it works.
Note that while this tutorial has separate sections on Python 2 and Python 3, even if you're only using one version of Python, you should read both sections if you want to really understand how things work.
This stuff is tricky to get your head around at first, but once you figure it out, it's actually not too bad. The problem is that even when you've got your code right, you start receiving content from elsewhere that is wrong, which breaks your code, so you change it to handle that content, but then your code breaks when you receive content from somewhere else that is doing things correctly[2]Or also doing things incorrectly, but in a different way , and you get stuck in a cycle where your code never works properly Hopefully, these notes will help you know when your code is right, and you can stick to your guns and start yelling at the other guy to fix their code...
A gateway lets you isolate computers in your home network from the internet. To reach the internet, a computer has to through the gateway, which means that if you put a firewall or virus checker or ad-blocker here, all your computers will benefit from them.
As before, this series of tutorials will walk you through the whole process of setting up a gateway, including a lot of not-essential-but-nice-to-have stuff. We start off by setting up a bare-bones gateway:
These days, a firewall is pretty much a necessity, and it's quite eye-opening to watch the logs and see the constant stream of attacks, as people try to break into your computer. And even if you run an ad-blocker like uBlock or AdBlock, a DNS-based ad-blocker can be run along-side it, without affecting browser performance at all[1]Browser plugins tend to slow the browser down noticeably, and can use huge amounts of memory..
Just a quick follow up on my recent epic tome on setting up a Banana Pi as a file server. I mentioned that I configured my disks to use the ext3 file system, and while it's generally fine, it does have one weakness: it is very slow deleting large files[1]Since I use my NAS for back ups, some of my files are well over 100GB.. Even worse, it locks up the file system, impacting other activity and causing stalls, which kinda sucks if you're watching a movie at the time
depesz took a very detailed look at the problem and some possible solutions, the TL;DR being that it's better to progressively shrink the file until it's all gone rather than asking the operating system to delete it as a file.
For the benefit of anyone having problems with this, here's a script that I wrote that implements this idea:
#!/bin/bash
# parse the command-line arguments
if [ $# -lt 1 ] ; then
echo "$(basename $0) file1 file2 file3 ..."
echo " Delete file(s) slowly."
exit 1
fi
chunk_size=100000000
for fname in "$@"; do
# check if we were given a directory
if [ -d "$fname" ]; then
# yup - process each file, then remove the directory
#echo "Deleting directory: $fname"
find "$fname" -type f -exec $(readlink -e "$0") \{\} \;
rm -rf "$fname"
continue
fi
# check if we were given a file
if [ ! -f "$fname" ]; then
# nope - ignore it
continue ;
fi
# yup - delete the file slowly
#echo "Deleting file: $fname"
while true; do
# get the current size of the file
fsize=$(ls -l "$fname" | cut -d' ' -f 5)
if [ $fsize -lt $chunk_size ] ; then
# the file is small enough to just delete
#echo "- Deleting file."
rm -f "$fname"
break
fi
# truncate the file, then loop back
#echo "- Truncating file: $fsize"
truncate -c -s -$chunk_size "$fname" || exit
sleep 0.25
done
done
Pass in a list of files and/or directories and it will slow-delete them. It will take longer to run, but will have far less impact on the rest of the system.
Nothing to do with Awasu, but hopefully someone out there in Internet-land will find it useful...
I've been a big fan of NAS's for many years, that is, a small file server that sits on my network and serves up music, movies, provides space for backups, etc. In the past, I've had Synology and QNAP units, and while they were both nice, they were both were relatively expensive, loaded with features I never used. They also both only lasted a few years, and rebuilding a NAS with 5-6 TB of data is a painfully long process
So for the next one, the plan was to grab an old laptop, load it up with FreeNAS, and then just hang a few disks off it. If and when the laptop dies, I can just set up a new one and the external disks, with all the data on them, should just plug straight in.
However, this is a bit of clunky solution, so when the Raspberry Pi came out, I got very interested in the idea of using that. Unfortunately, the rPi has one big drawback that makes it unsuitable for use as a file server: it only has 10/100 Mbps ethernet. All the computers on my network have gigabit ethernet, and since I'm moving 100's of GB's of data every night for backups, my file server also needs to have gigabit ethernet.
Enter the Banana Pi. Released in late 2014 by LeMaker in China, it's slightly more expensive but significantly more powerful, notably with gigabit ethernet and a SATA port. Add in a case, and I'll be able to build my own future-proof NAS for well under a hundred bucks, plus the cost of the disks.
There are quite a few tutorials floating around that explain how to set up a Banana Pi as a NAS, but they invariably only talk about how to set up the factory image of Open Media Vault[1]This is the successor to FreeNAS, written by one of the FreeNAS guys, that runs on Linux instead of FreeBSD. (which is relatively easy to do), but this series of tutorials will also talk about the many things you need to do after that to get a usable system.
A while back, I posted a tutorial that showed how easy it is to extend Awasu through the use of plugins and channel hooks, and continuing on from that, here's another series that shows how you can control your Awasu via its API.
Whether you just want to find out what state your channels or reports are in, or if you want to programmatically create, update and delete them, the Python and PHP libraries available make it a breeze.
Have a play with them, hope you find them useful and, as always, feel free to ask questions in the forums.
As promised, the tutorial on writing Awasu plugin channels and channel hooks is now up here.
Part 1 talks about how to get set up with the new awasu_tools library, and how to generate a basic feed, then Part 2 shows how to convert a basic script into something that can be called by Awasu.
Part 3 explores some more of the features offered by the awasu_tools library, while Part 4 rounds things off by explaining how to compile your extension, ready for distribution.
Writing Awasu extensions has always been relatively easy, but there was a bit of a learning curve, mostly spent looking at the samples supplied with Awasu. Hopefully, this tutorial will make things a bit easier, and the new awasu_tools library really makes things a breeze
As an aside, I've been programming computers for around 30 years (professionally for 25), yet publishing my first bit of code on PyPI and GitHub somehow makes me suddenly feel like a Real Programmer
So, one of our long-time users has been asking about how RSS tracks authors, which reminded me that I've been meaning to talk about this insanely cool feature of Awasu that I'm sure not many people know about.
RSS feed items typically come with metadata associated with them, like who the author was, what time they were published, etc. There are also non-standard things like licensing details, geocodes, information about who's commented or linked to the blog post. Not only is Awasu able to slurp up all this yummy data, it doesn't even have to be a standard type of metadata. You can even embed your own custom metadata in a feed and Awasu will be able to extract it - awesome if you're working in an enterprise environment.
One of the things you can do with this metadata is include it in the item pane, the list of feed items that appears in the top (by default) of a channel's window. Here's a screenshot of a channel I have using the <foreshadowing> new and improved plugin that monitors email accounts </foreshadowing>, with the email's sender and timestamp in the item pane.
Setting this up is pretty easy:
Open the channel's Properties dialog and on the first tab (Channel), click on the Item pane button.
Click on the green plus icon, to add a new column.
Enter the metadata value you want to show in the column. The most commonly-used ones are provided in the droplist, but you can enter anything you want.
Then enter the name of the column.
And ta da! Awasu will now show the relevant metadata in the item pane.
This is a pretty nice feature, and Awasu is not limited to just showing these values in the item pane, they can be included in the browser pane as well, which means they will show up in reports, or emails. Post in the forums if you need a hand sending any of this stuff up.
Those of you who know me know that I'm a big fan of Python. It's a really powerful and flexible scripting language that sits nicely in that space between shell scripts that have gotten too big (i.e. more than 10 lines ), and large-scale applications written in C++ or Java. I remember trying to print out the manual for 1.5.2[1]IIRC, this was the iconic version of Python, much like 3.1 was for Windows for a long time. way back in the late-90's[2]Yes kids, we often used printed manuals, back in the day :-O, and I'd already been using it for a few years then, so I guess that means I've been using it for nearly 20 years now
Awasu has had an embedded Python interpreter for some time, and one of the things that has been lurking in the murky depths of my to-do list for many years now is a write-up on how to actually do this, since the documentation is not particularly great. Inexplicably, I found myself with a bit of time recently and have finally managed to cross this one off my list
In which we look at what you need to distribute with your program for Python to work.
Hopefully, I'll be able to follow this series up with another one, that explains how to go the other way - extending Python with your own custom features. Just give me a couple of years...
Awasu and the stylized Japanese character in the orange box are trademarks of Awasu Pty. Ltd. Other brands and product names are trademarks of their respective owners. Awasu Pty. Ltd. believes the information in this publication is accurate as of its publication date. Such information is subject to change without notice. Awasu Pty. Ltd. is not responsible for inadvertent errors.
.
This stuff is tricky to get your head around at first, but once you figure it out, it's actually not too bad. The problem is that even when you've got your code right, you start receiving content from elsewhere that is wrong, which breaks your code, so you change it to handle that content, but then your code breaks when you receive content from somewhere else that is doing things correctly[2]Or also doing things incorrectly, but in a different way 
, and you get stuck in a cycle where your code never works properly 
A while back, I posted a massive tutorial on 
Those of you who know me know that I'm a big fan of 
), and large-scale applications written in C++ or Java. I remember trying to print out the manual for 1.5.2[1]
I am a 
