I’m moving from (a kind of…) a dedicated server to a VPS, to decrease my frightful anticipation of hardware failures.
Honestly though, that server had been freezing up and restarting spontaneously for several months now, causing sometimes really long down-times…
That server is now about 6-7 years old, built with off-the-shelf components, some of which (the HDD ) had weird noises from the very start.
Definitely time to move!

I’ve purchased a fairly cheap VPS with an easy, one-click upgrade option for after I’m done configuring it.
It comes with a wide selection of OSes to pre-install; I’ve chosen Debian Jessie, version 8.3 as of this writing.

I wanted to use btrfs from the beginning, so could have installed Debian myself, but… VPS provider does some initial configuration (like their Debian mirror and some other things), so I’ve felt that converting to btrfs after the fact would be easier. Now that I’ve done this – I guess it was fairly easy, although preparation did take some time.

Below, I’m providing step-by-step instructions on how to convert your root filesystem from (most likely) ext4 to btrfs.
Read the rest of this entry »

So… You had finally built a nice LXC container for your web-facing application, and even configured Apache (Debian package version 2.14.18-1 in my case) to serve some static/web-only components.
From your client-side JavaScript UI you talk (in JSON) to the API, which is implemented as a separate node.js/Python/etc server – say, on port 8000 in the same LXC container.

The simplest solution to forward requests from the web-frontend to your API is by using mod_proxy.
If you want to forward any requests to /api/* to your custom back-end server on port 8000, you just add the following lines to your VirtualHost configuration:

ProxyPass “/api” “http://localhost:8000″
ProxyPassReverse “/api” “http://localhost:8000″

I’d suggest not wrapping this fragment with the classical IfModule: as your application will not really work without its API back-end, you actually want Apache to fail as soon as possible if mod_proxy is missing.

That was easy, right? What, it doesn’t work? Can’t be! It’s dead simple! No way you could make a mistake in 2 lines of configuration!!! :mad_rage:

Oh wait… I remember I had this problem before… Read the rest of this entry »

Midnight Commander is a convenient two-panel file manager with tons of features.

You can create hard links and symbolic links using C-x l and C-x s keyboard shortcuts. However, these two shortcuts invoke two completely different dialogs.

While for C-x s you get 2 pre-populated fields (path to the existing file, and path to the link – which is pre-populated with your opposite file panel path plus the name of the file under cursor; simply try it to see what I mean), for C-x l you only get 1 empty field: path of the hard link to create for a file under cursor. Symlink’s behaviour would be much more convenient…

Fortunately, a good man called Wiseman1024 created a feature request in the MC’s bug tracker 6 years ago. Not only had he done so, but he had also uploaded a sample mc user menu script (local copy), which works wonderfully! You can select multiple files, then F2 l (lower-case L), and hard-links to your selected files (or a file under cursor) will be created in the opposite file panel. Great, thank you Wiseman1024!

Word of warning: you must know what hard links are and what their limitations are before using this menu script. You also must check and understand the user menu code before adding it to your mc (by F9 C m u, and then pasting the script from the file).

Word of hope: 4 years ago Wiseman’s feature request was assigned to Future Releases version, so a more convenient C-x l will (sooner or later) become the part of mc. Hopefully.

• sysv-rc-conf: text-UI, allows editing multiple runlevels, but seems to be failing recently
• rcconf: another good-looking text-UI alternative; can only modify current runlevel
• bum: GUI tool (BootUp Manager)

More details about these tools and runlevels (in German).

About 2 years ago I had already reviewed some parallel (and not) compressing utilities, settling at that time on pbzip2 – it scales quasi-linearly with the number of CPUs/cores, stores compressed data in relatively small 900k blocks, is fast, and has good compression ratio. pbzip2 was (and still is) a very good choice.

Yesterday I got somewhat distracted, and thus found lbzip2 -

an independent, multi-threaded implementation of bzip2. It is commonly the fastest SMP (and uniprocessor) bzip2 compressor and decompressor

- as it says in the Debian package description. Is it really “commonly the fastest” one? How does it compare to pbzip2? Should I use lbzip2 instead of pbzip2?

This minor distraction had grown into a full-scale web-search and comparison, adding to the mix plzip (a parallel version of lzip), xz, and lrzip. After reading thousands of characters, all of these were put to a simple test: compressing an about 2 gigabyte FASTQ file with default options.

All the external links and benchmarks, as well as my own mini-benchmark results, are provided below.

The conclusion is that out of all the tested compressors lbzip2 is indeed the best one (for my practical use). It is only slightly better than the trusty pbzip2, which takes the second place. All the other compressors performed so poorly, that they do not get any place in my practical rating…

So, let us first ask internet wisdom/foolishness, if lbzip2 or pbzip2 is faster/better?
Read the rest of this entry »

After fixing offline uncorrectable sector warning email, I have taken a closer look at my /etc/smartd.conf, and now it looks like this:

DEFAULT -d sat -H -f -p -t -W 0,40,45 -n standby -S on -m example@example.com
# Attributes 1, 230, and 231 are very important (-r 1! -r 230! -R 230! -r 231! -R 231!), but likely covered by -t.
/dev/sda -s (S/../../6/01|L/../(01|02|03|04|05|06|07)/7/00) -C 0 -I 189 -I 194
# -a implies -f and -p (through -t)
DEFAULT -d sat -a -I 194 -W 0,40,45 -n standby -o on -S on -m example@example.com
/dev/sdb -s (S/../../6/02|L/../(01|02|03|04|05|06|07)/7/02)
# This drive does not decrement Offline_Uncorrectable (198) after re-allocation,
# so only monitoring for increase, not for non-zero value.
/dev/sdc -s (S/../../6/03|L/../(01|02|03|04|05|06|07)/7/04) -U 198+
# This drive has 40 “normally”.
/dev/sdd -s (S/../../6/04|L/../(01|02|03|04|05|06|07)/7/06) -W 0,42,45

Note: explanations below are intentionally simplified; please consult man smartd.conf for more precise, complete, and up-to-date information.

Ok, so what do these settings mean, and how is this different from default settings?
Read the rest of this entry »

A few days ago my smartd daemon (from the smartmontools package) notified me about a +1 increase in Current_Pending_Sector (197) and Offline_Uncorrectable (198) SMART attributes. The 2.5″ Fujitsu laptop hard-drive these appeared on is very old, and it also has been working 24/365 since a little over a year.

Running a short SMART self-test (sudo smartctl -t short /dev/sdc) produced a read error at sector 1289:

Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
1 Short offline Completed: read failure 80% 22339 1289

Looking at the partition table of /dev/sdc, we see that this sector is outside of the only RAID partition on the disk, which starts at sector 2048:

Device Boot Start End Blocks Id System
/dev/sdc1 2048 117209087 58605088 fd Lnx RAID auto

Read the rest of this entry »