Autarchy of the Private Cave

I’m moving from (a kind of…) a dedicated server to a VPS, to decrease my frightful anticipation of hardware failures.
Honestly though, that server had been freezing up and restarting spontaneously for several months now, causing sometimes really long down-times…
That server is now about 6-7 years old, built with off-the-shelf components, some of which (the HDD ) had weird noises from the very start.
Definitely time to move!

I’ve purchased a fairly cheap VPS with an easy, one-click upgrade option for after I’m done configuring it.
It comes with a wide selection of OSes to pre-install; I’ve chosen Debian Jessie, version 8.3 as of this writing.

I wanted to use btrfs from the beginning, so could have installed Debian myself, but… VPS provider does some initial configuration (like their Debian mirror and some other things), so I’ve felt that converting to btrfs after the fact would be easier. Now that I’ve done this – I guess it was fairly easy, although preparation did take some time.

Below, I’m providing step-by-step instructions on how to convert your root filesystem from (most likely) ext4 to btrfs.
Read the rest of this entry »

So… You had finally built a nice LXC container for your web-facing application, and even configured Apache (Debian package version 2.14.18-1 in my case) to serve some static/web-only components.
From your client-side JavaScript UI you talk (in JSON) to the API, which is implemented as a separate node.js/Python/etc server – say, on port 8000 in the same LXC container.

The simplest solution to forward requests from the web-frontend to your API is by using mod_proxy.
If you want to forward any requests to /api/* to your custom back-end server on port 8000, you just add the following lines to your VirtualHost configuration:

ProxyPass “/api” “http://localhost:8000″
ProxyPassReverse “/api” “http://localhost:8000″

I’d suggest not wrapping this fragment with the classical IfModule: as your application will not really work without its API back-end, you actually want Apache to fail as soon as possible if mod_proxy is missing.

That was easy, right? What, it doesn’t work? Can’t be! It’s dead simple! No way you could make a mistake in 2 lines of configuration!!! :mad_rage:

Oh wait… I remember I had this problem before… Read the rest of this entry »

• sysv-rc-conf: text-UI, allows editing multiple runlevels, but seems to be failing recently
• rcconf: another good-looking text-UI alternative; can only modify current runlevel
• bum: GUI tool (BootUp Manager)

More details about these tools and runlevels (in German).

The usual, or even classical way is to create the list of installed packages with sudo dpkg --get-selections > package_list, and then restore when/if necessary with cat package_list | xargs sudo apt-get -y install.

As VihangD points out in his serverfault answer, the same can be achieved with aptitude, while also excluding dependent, automatically installed packages (which are included by the classical method). To create the list of packages, run aptitude search -F '%p' '~i!~M' > package_list. Here, -F '%p' asks aptitude to only print package names (instead of the default output, which also contains package state and description); search term ‘~i!~M’ asks for all non-automatically installed packages.

To install packages using the created list, run xargs aptitude --schedule-only install < package_list; aptitude install. The first of these two commands instructs aptitude to mark all the packages from the list as scheduled for installation. The second command actually performs the installation.

Hamish Downer suggests an alternative way of getting the initial package_list: using the deborphan utility, deborphan -a --no-show-section > package_list. This command asks deborphan to show a list of packages, which have no dependencies on them. Sounds very similar to what we did with aptitude above, but using deborphan will most likely result in a much shorter list of packages (on my system, deborphan printed 291 package names, aptitude printed 847, and dpkg printed 3650 package names). One more potentially important difference between aptitude- and deborphan-produced package lists is that aptitude only specifies package architecture when it is different from native (e.g. 'googleearth:i386' on a 64-bit system), while deborphan specifies architectures for all the packages (resulting in e.g. 'google-talkplugin:amd64' and 'googleearth-package:all' on a 64-bit system).

SPF is nice for protecting your mail server from spam, but sometimes there is a need to bypass SPF checking. For example, if you rely on 3rd party servers to do spam protection for you

Current setup:

• MX records point to the spam protection mail servers, which then
• connect to my server and deliver (hopefully spam-free) mail.

Problem: some senders (like last.fm) do have proper, strict SPF records. Tumgreyspf on my server then rejects emails relayed through the spam-protection service.

If these spam protection relay servers are the only which send mail to your server, then it makes sense to fully disable/uninstall tumgreyspf. Putting tumgreyspf into the permanent “learning mode” (set defaultSeedOnly = 1 in /etc/tumgreyspf/tumgreyspf.conf) may not fix the SPF problem described above, as SeedOnly seems to only affect greylisting, and not rejecting unauthorized senders.

Solution: whitelist relay server IPs.
Read the rest of this entry »

Assumptions:

• current HDD is /dev/sda, it has a GPT (with bios_grub being /dev/sda1), separate /boot partition (/dev/sda2), and a physical LVM volume (/dev/sda3), where LVM holds all the remaining partitions (root, /home, /srv, …); LVM is properly configured, and system reboots with no problems
• your new drive is /dev/sdb, it is identical to /dev/sda, and it comes empty from the manufacturer (this is important! wipe the drive if it is not empty, especially if it used to be a part of another RAID)
• your system is Debian or Debian-based; in this exact example I’ve been using Ubuntu Server 10.04
• your LVM volume group is named vg0
• make sure you understand what each command does before executing it
• you do have an external backup of all your important data, and you do understand that the following operations are potentially dangerous to your data integrity

Inspired by: Debian Etch RAID guide, serverfault question.
Read the rest of this entry »

rtorrent is an excellent ncurses rtorrent client.

There are a few highly-popular patches, which haven’t yet made it into the Squeeze’s rtorrent version 0.8.6.

Note: ArchLinux already has an rtorrent-extended package, so you may want to look at that one (instead of my Debian package below). I might use Arch’s patchset next time to generate a truly extended rtorrent package.

I’ve incorporated 3 of such patches (#239 ipfilter, #2064 GeoIP support, and #2171 colours/eyecandy), and built a squeeze amd64 rtorrent-enhanced package.

A few excerpts to explain how these extra features are to be configured and used (all come from the above-mentioned trac tickets, some were edited/extended).

ipfilter

ipfilter allows to selectively blacklist/whitelist peers – based on IP address range files (so-called ipfilter files). The ipfilter files may, for example, come from bluetack.co.uk. Each line of the file contains a record in this format: range_description:start_IP-end_IP, where start_IP should be less than or equal to end_IP. Multiple files can be used. Overlapping ranges are merged automatically. Both incoming and outgoing connections are checked against the filter. Exclusions are not supported, so connection to/from IPs in all the loaded ranges will be disallowed and dropped.

Include “ip_filter=” directive in .rtorrent.rc. For example (paths are specified relative to user’s home directory):
ip_filter=ipfilter/level1,ipfilter/level2

It is probably a good idea to reload ipfilter files once in a while, so you can also include “reload_ip_filter” directive on schedule to refresh the filter. The same files named in “ip_filter” will be reloaded.

schedule = filter,18:30:00,24:00:00,reload_ip_filter=

To support the feature you may want to setup cron job to reload and unzip files from bluetack. Examples here and here.

GeoIP

In the Peers view of each torrent a new column – CC, country code – is added, which shows peer’s country code. When examining each peer, you will see more data from the GeoIP database (if you have the relevant files installed): AS num and city. At the very least, you need to have geoip-database installed. You can get free “lite” versions of city/AS-databases, rename them (removing “lite”) and put into an appropriate location to make rtorrent+geoip use them as well. Geop-isp data support isn’t integrated into this rtorrent package.

Colours

This patch somewhat changes the way rtorrent shows torrents. Seeding torrents have bold titles, there are no half-displayed torrents at the top/bottom when scrolling, and you can configure colours for active/done torrents (ticket libtorrent.rakshasa.no/ticket/1382, which seems to be gone now). Source code and testing hint at these configurable colours: done_fg_color, done_bg_color, active_fg_color, active_bg_color, and at these possible values: 1 (red), 2 (green), 3 (yellow).

Geek’s cellar
A related (though not used in any way for the preparation of the package) resource is rtorrent mods page.

Relatively schematically, applying patches and building the package was performed in these steps (starting within some newly-created directory):

1. sudo aptitude install cdbs devscripts [and whatever else you find you're missing]
2. apt-get source rtorrent
3. dpkg-source -x rtorrent_0.8.6-1.dsc
4. cd rtorrent-0.8.6
5. cdbs-edit-patch 01-ipfilter.patch
6. patch -p1 < /path/to/patch/239
7. exit 0
8. cdbs-edit-patch 02-geoip.patch
9. patch -p1 < /path/to/patch/2064-after-ipfilter
10. exit 0
11. cdbs-edit-patch 03-eyecandy.patch
12. patch -p1 < /path/to/patch/2171-mod-with-canvas
13. exit 0 [you could do all 3 patches together, but I prefer cleaner and manageable approaches]
14. [some weird operation to be explained below]
15. dpkg-buildpackage

Weird operation: I didn’t know how to make dpkg-buildpackage run autorun.sh (which is required for one of the patches I’ve used) before configure (could someone enlighten me how to do that?), so I started a new patch with cdbs-edit-patch, then ran one by one `autoreconf -if`, `./autorun.sh`, `./configure`, exited with zero status and was done with that problem. The resulting “patch” was 2MB large