Autarchy of the Private Cave*nix »

tudu: a beautiful ncurses to-do list application

Bogdan — Tue, 11 Oct 2011 12:15:21 +0000

tudu is just what it says in the title, and is written in C.
All the functions are mapped to keys. Extremely flexible: you can use it either as a simple to-do list, or utilize a bunch of optional features (priorities, schedules, deadlines, categories, tags, maybe something else).

If you (like me) love ncurses – you will love tudu Just give it a try.

Screenshot copyright: Ruben Pollan (tudu author).

Good advice: /bin/false is not security

Bogdan — Sat, 01 Oct 2011 21:38:30 +0000

SSH Security and You – /bin/false is *not* security.

How to remotely convert live 1xHDD/LVM Linux server to 2xHDD RAID1/LVM (GRUB2, GPT)

Bogdan — Tue, 17 May 2011 13:23:05 +0000

Assumptions:

current HDD is /dev/sda, it has a GPT (with bios_grub being /dev/sda1), separate /boot partition (/dev/sda2), and a physical LVM volume (/dev/sda3), where LVM holds all the remaining partitions (root, /home, /srv, …); LVM is properly configured, and system reboots with no problems
your new drive is /dev/sdb, it is identical to /dev/sda, and it comes empty from the manufacturer (this is important! wipe the drive if it is not empty, especially if it used to be a part of another RAID)
your system is Debian or Debian-based; in this exact example I’ve been using Ubuntu Server 10.04
your LVM volume group is named vg0
make sure you understand what each command does before executing it
you do have an external backup of all your important data, and you do understand that the following operations are potentially dangerous to your data integrity

Inspired by: Debian Etch RAID guide, serverfault question.

Create the GPT on the new drive:
parted /dev/sdb mklabel gpt
Get the list of partitions on /dev/sda:
parted -m /dev/sda print
Create /dev/sdb partitions similarly to what you have on /dev/sda (my example numbers follow, use your numbers here):
parted /dev/sdb mkpart bios_grub 1049kB 2097kB
parted /dev/sdb mkpart boot 2097kB 258MB
parted /dev/sdb mkpart lvm 258MB 2000GB
Set proper flags on partitions:
parted /dev/sdb set 1 bios_grub on (GPT doesn’t have MBR, so you create a 1-MB partition instead to hold grub2′s boot code)
(possibly optional) parted /dev/sdb set 2 raid on
(possibly optional) parted /dev/sdb set 3 raid on
(possibly optional) To make sure /dev/sdb1 (the bios_grub) indeed contains grub’s boot code, I did dd if=/dev/sda1 of=/dev/sdb1
apt-get install mdadm
Note: at this point, older tutorials suggest adding a bunch of raid* kernel modules to /etc/modules and to grub’s list of modules to load. I’m not sure this is really necessary, but do see the tutorials mentioned at the top for more information. If you do modify the lists of modules – don’t forget to run update-initramfs -u.
Create two initially-degraded RAID1 devices (one for /boot, another for LVM):
mdadm ––create /dev/md0 ––level=1 ––raid-devices=2 /dev/sdb2 missing
mdadm ––create /dev/md1 ––level=1 ––raid-devices=2 /dev/sdb3 missing
Store the configuration of your RAID1 to the mdadm.conf file (important! this is not done automatically!)
mdadm -Es >> /etc/mdadm/mdadm.conf
Verify the contents of your mdadm.conf:
cat /etc/mdadm/mdadm.conf
dpkg-reconfigure mdadm, and enable booting in degraded mode
Copy your current /boot (/dev/sda2) to the new /dev/md0 /boot partition:
(one can use dd here as well, but for some reason my attempt at dd failed writing 1 last byte of data)
mkdir /mnt/md0
mount /dev/md0 /mnt/md0
cp -a /boot/* /mnt/md0/
umount /dev/md0
rmdir /mnt/md0
Now extend your existing volume group to include the newly-created /dev/md1:
pvcreate /dev/md1
vgextend vg0 /dev/md1
Verify the list of logical volumes you curently have: enter lvm shell, and type lvs. Here’s what I had:
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
home vg0 -wi-ao 1.70t
logs vg0 -wi-ao 4.66g
root vg0 -wi-ao 10.24g
srv vg0 -wc-ao 100.00g
swap vg0 -wi-ao 1.86g
tmp vg0 -wi-ao 4.66g
Now, you can move all the logical volumes to new physical volume in one command: pvmove /dev/sda3 /dev/md1. Personally, remembering the problem I had with dd from /dev/sda2 to /dev/md0, I decided to move all logical volumes one-by-one; as this takes time, you may consider joining these operations with ; or &&, and putting the /tmp last (as the easiest one to re-create if it fails to move):
pvmove ––name home /dev/sda3 /dev/md1
pvmove ––name srv /dev/sda3 /dev/md1
pvmove ––name logs /dev/sda3 /dev/md1
pvmove ––name swap /dev/sda3 /dev/md1
pvmove ––name root /dev/sda3 /dev/md1
pvmove ––name tmp /dev/sda3 /dev/md1
To be safer, I ran FS check on a few volumes I could umount:
umount /dev/mapper/vg0-srv
fsck -f /dev/mapper/vg0-srv
mount /dev/mapper/vg0-srv
umount /dev/mapper/vg0-tmp
fsck -f /dev/mapper/vg0-tmp
mount /dev/mapper/vg0-tmp
Remove /dev/sda3 from the physical space available to your volume group:
vgreduce vg0 /dev/sda3
Install grub2 to both drives, so as to make them both bootable in case of failure:
grub-install ‘(hd0)’
grub-install ‘(hd1)’
Edit /etc/fstab, pointing /boot to /dev/md0. You may use UUIDs here, but please do not use UUIDs from mdadm.conf – those are different from FS-UUIDs, instead do ls -l /dev/disk/by-uuid to find the UUID of /dev/md0. Personally, I had no problems just using /dev/md0.
Now is the time to add your original /dev/sda to the RAID1; be absolutely sure you have moved all the data off that drive, because these commands will destroy it:
mdadm ––manage ––add /dev/md0 /dev/sda2
mdadm ––manage ––add /dev/md1 /dev/sda3
Re-syncing array will take some time.
To be on the safe side, you may want to run again update-initramfs -u and update-grub; I have also edited /etc/grub.d/40_custom, adding there 2 more boot options: from /dev/sda2 and /dev/sdb2 (/boot on both drives) – have no idea if that will work, but having more boot options didn’t hurt
Reboot into your new system. Actually, at this point reboot is only necessary to verify that your system is bootable – you may delay this reboot as long as you want to.
Many tutorials also suggest testing your RAID1 by manually “degrading” it, trying to boot, and then rebuilding it back. I haven’t done that, but you may want to.

Improvement suggestions, criticism and thank-you are welcome in the comments.

Generate .mood moodbar files for your whole music collection

Bogdan — Sun, 10 Apr 2011 20:21:46 +0000

Amarok moodbar wiki page has 2 nice scripts to generate .mood files for your whole music collection (to be displayed by amarok when playing).

Both scripts should be started from the directory where you keep your music. The .mood files will be generated next to the source music files (in the same directories). You can modify the scripts to have the moods stored to $HOME/.kde/share/apps/amarok/moods/, so as not to clutter your music directories.

Script for multicore CPUs (will not overwrite existing mood files):

PLAIN TEXT

CODE:

#!/bin/bash
NUMCPU="$(grep ^processor /proc/cpuinfo | wc -l)"
find . -type f -regextype posix-awk -iregex '.*\.(mp3|ogg|flac|wma)' | while read i ; do
while [ `jobs -p | wc -l` -ge $NUMCPU ] ; do
sleep 0.1
done
TEMP="${i%.*}.mood"
OUTF=`echo "$TEMP" | sed 's#$.*$/$[^,]*$#\1/.\2#'`
if [ ! -e "$OUTF" ] ; then
moodbar -o "$OUTF" "$i" &
fi
done

Script for single-threaded moods generation (will only overwrite the last-generated mood file from a previous run):

PLAIN TEXT

CODE:

#!/bin/bash
DIR=${1:-.}
LAST=.moodbar-lastreadsong
C_RET=0
control_c() # run if user hits control-c
{
echo "$1"> "$LAST"
echo "Exiting..."
exit
}
if [ -e "$LAST" ]; then
read filetodelete <"$LAST"
rm "$filetodelete" "$LAST"
fi
exec 9<<(find "$DIR" -type f -regextype posix-awk -iregex '.*\.(mp3|ogg|flac|wma)') # you may need to add m4a and mp4
while read i
do
TEMP="${i%.*}.mood"
OUTF=`echo "$TEMP" | sed 's#$.*$/$[^,]*$#\1/.\2#'`
trap 'control_c "$OUTF"' INT
if [ ! -e "$OUTF" ] || [ "$i" -nt "$OUTF" ]; then
moodbar -o "$OUTF" "$i" || { C_RET=1; echo "An error occurred!">&2; }
fi
done <&9
exec 9<&-
exit $C_RET

That same page has a link to an older, somewhat more polished ruby script for the same task. It requires a running instance of Amarok, and uses it's media collection to find music files and create mood files for them. This script can potentially (after a little editing) store .mood files either with the music or to the $HOME/.kde/... path mentioned above. It is for an older version of Amarok, and thus is not likely to work with e.g. Amarok 2.4

PLAIN TEXT

RUBY:

#!/usr/bin/ruby
# Simple moodbar file management utility by Joe Rabinoff
# This is also my first ever ruby script so bear with me
$moodbar_folder = "#{ENV['HOME']}/.kde/share/apps/amarok/moods/"
def usage()
print "This is the moodbar file management utility\n"
print "\n"
print "Usage is:\n"
print " moodbar_util.rb -rename Rename mood files" \
+ " from any old naming scheme to the current one\n" \
+ " (not applicable when mood" \
+ " files are stored with music)\n"
print " moodbar_util.rb -calcall Calculate all un-calculated" \
+ " mood files\n"
exit(1)
end
def check_amarok_running()
result = `dcop 'amarok*'`.split("\n")
return (result.length> 0)
end
def get_devices()
result = `dcop amarok collection query 'SELECT id, lastmountpoint FROM devices;'`.split("\n")
ret = Hash.new()
result.each_index do |i|
ret[result[i].to_i] = result[i+1] if i % 2 == 0
end
return ret
end
def mood_name(url, deviceid)
ret = url.gsub(/\//, ',')
return deviceid.to_s + "," + ret
end
def rename()
print "Renaming mood files from outdated naming schemes...\n"
# As far as I can tell, the best way to do this is just select all
# tracks, check the old possible moodbar names, and rename them
tracks = `dcop amarok collection query 'SELECT url, deviceid FROM tags;'`.split("\n")
devices = get_devices
0.step(tracks.length-1, 2) do |i|
url = tracks[i].sub(/\.[^\.]*$/, '.mood')
deviceid = tracks[i+1].to_i
path = url.sub(/\./, '')
if devices.has_key?(deviceid)
path = devices[deviceid] + path
end
moodfile = $moodbar_folder + mood_name(url, deviceid)
moodfile1 = $moodbar_folder + path.gsub(/\//, ',')
next if FileTest.exists?(moodfile)
if FileTest.exists?(moodfile1)
system("mv", "-f", moodfile1, moodfile)
print "Moved ", moodfile1, " to ", moodfile, "\n"
end
end
end
def calcall()
print "Calculating all nonexisting moodbars...\n\n"
tracks = `dcop amarok collection query 'SELECT url, deviceid FROM tags;'`.split("\n")
devices = get_devices
withMusic = (`dcop amarok script readConfig MoodsWithMusic`.chomp == "true")
0.step(tracks.length-1, 2) do |i|
url = tracks[i]
deviceid = tracks[i+1].to_i
moodfile = ""
songpath = url.sub(/\./, '')
if devices.has_key?(deviceid)
songpath = devices[deviceid] + songpath
end
if withMusic
moodfile = songpath.sub(/\.[^\.]*$/, '.mood')
moodfile = File.dirname(moodfile) + "/." + File.basename(moodfile)
else
moodfile = $moodbar_folder \
+ mood_name(url.sub(/\.[^\.]*$/, '.mood'), deviceid)
end
next if FileTest.exists?(moodfile) && FileTest.size(moodfile)> 0
next unless FileTest.exists?(songpath)
print "Saving moodbar for ", songpath, "\n"
print " to file ", moodfile, "\n"
system("moodbar", "-o", moodfile, songpath)
print "\n"
end
end
$mode = ARGV[0]
if !check_amarok_running
print "Amarok is not running! Please start Amarok and re-run " \
"this script.\n"
end
if $mode == "-rename"
rename
elsif $mode == "-calcall"
calcall
else
usage
end

rtorrent-enhanced with ipfilter and GeoIP: Debian Squeeze amd64 package

Bogdan — Thu, 31 Mar 2011 23:15:49 +0000

rtorrent is an excellent ncurses rtorrent client.

There are a few highly-popular patches, which haven't yet made it into the Squeeze's rtorrent version 0.8.6.

Note: ArchLinux already has an rtorrent-extended package, so you may want to look at that one (instead of my Debian package below). I might use Arch's patchset next time to generate a truly extended rtorrent package.

I've incorporated 3 of such patches (#239 ipfilter, #2064 GeoIP support, and #2171 colours/eyecandy), and built a squeeze amd64 rtorrent-enhanced package.

A few excerpts to explain how these extra features are to be configured and used (all come from the above-mentioned trac tickets, some were edited/extended).

ipfilter

ipfilter allows to selectively blacklist/whitelist peers - based on IP address range files (so-called ipfilter files). The ipfilter files may, for example, come from bluetack.co.uk. Each line of the file contains a record in this format: range_description:start_IP-end_IP, where start_IP should be less than or equal to end_IP. Multiple files can be used. Overlapping ranges are merged automatically. Both incoming and outgoing connections are checked against the filter. Exclusions are not supported, so connection to/from IPs in all the loaded ranges will be disallowed and dropped.

Include "ip_filter=" directive in .rtorrent.rc. For example (paths are specified relative to user's home directory):
ip_filter=ipfilter/level1,ipfilter/level2

It is probably a good idea to reload ipfilter files once in a while, so you can also include "reload_ip_filter" directive on schedule to refresh the filter. The same files named in "ip_filter" will be reloaded.
schedule = filter,18:30:00,24:00:00,reload_ip_filter=

To support the feature you may want to setup cron job to reload and unzip files from bluetack. Examples here and here.

GeoIP

In the Peers view of each torrent a new column - CC, country code - is added, which shows peer's country code. When examining each peer, you will see more data from the GeoIP database (if you have the relevant files installed): AS num and city. At the very least, you need to have geoip-database installed. You can get free "lite" versions of city/AS-databases, rename them (removing "lite") and put into an appropriate location to make rtorrent+geoip use them as well. Geop-isp data support isn't integrated into this rtorrent package.

Colours

This patch somewhat changes the way rtorrent shows torrents. Seeding torrents have bold titles, there are no half-displayed torrents at the top/bottom when scrolling, and you can configure colours for active/done torrents (ticket libtorrent.rakshasa.no/ticket/1382, which seems to be gone now). Source code and testing hint at these configurable colours: done_fg_color, done_bg_color, active_fg_color, active_bg_color, and at these possible values: 1 (red), 2 (green), 3 (yellow).

Geek's cellar
A related (though not used in any way for the preparation of the package) resource is rtorrent mods page.

Relatively schematically, applying patches and building the package was performed in these steps (starting within some newly-created directory):

sudo aptitude install cdbs devscripts [and whatever else you find you're missing]
apt-get source rtorrent
dpkg-source -x rtorrent_0.8.6-1.dsc
cd rtorrent-0.8.6
cdbs-edit-patch 01-ipfilter.patch
patch -p1 < /path/to/patch/239
exit 0
cdbs-edit-patch 02-geoip.patch
patch -p1 < /path/to/patch/2064-after-ipfilter
exit 0
cdbs-edit-patch 03-eyecandy.patch
patch -p1 < /path/to/patch/2171-mod-with-canvas
exit 0 [you could do all 3 patches together, but I prefer cleaner and manageable approaches]
[some weird operation to be explained below]
dpkg-buildpackage

Weird operation: I didn't know how to make dpkg-buildpackage run autorun.sh (which is required for one of the patches I've used) before configure (could someone enlighten me how to do that?), so I started a new patch with cdbs-edit-patch, then ran one by one `autoreconf -if`, `./autorun.sh`, `./configure`, exited with zero status and was done with that problem. The resulting "patch" was 2MB large

How to easily install any PyPi/easy_install python module on Debian

Bogdan — Wed, 16 Feb 2011 17:22:20 +0000

Imagine you need to install pycassa (which uses easy_install). Here are the 2 (at maximum) very simple steps to have it properly debianized and installed on your Debian/Ubuntu:

if you don't have the python-stdeb package: sudo aptitude install python-stdeb
pypi-install pycassa

That's it.

Refer to stdeb readme for more information. You will need that if there are dependencies - which might not be resolved automatically by stdeb.

Before stdeb, it wasn't exactly trivial to make a .deb from python module.

My favourite command line for mirroring with wget

Bogdan — Sun, 09 Jan 2011 12:30:27 +0000

wget ––continue ––mirror ––page-requisites ––adjust-extension ––convert-links ––backup-converted ––limit-rate=500k ––wait=2 URL

(short form: wget -cmpEkK ––limit-rate=500k -w 2 URL)

––continue does no harm on initial run, but when run again this instructs wget to pick up where it was interrupted the previous time
––mirror turns on several options useful for mirroring (read wget manual for details)
––page-requisites instructs wget to also fetch linked resources, which would otherwise be skipped
––adjust-extension (or ––html-extension in older wget versions) will append the .html extension to files without it, but with document type text/html or equivalent
––convert-links will convert the links for convenient local viewing
––backup-converted helps to avoid re-downloading some files, especially when ––convert-links is specified
––limit-rate=500k and ––wait=2 prevent overloading the target website (being nice to someone offering information you need is a must)

How to relay outgoing postfix emails via another mail server (e.g. your ISP)

Bogdan — Sat, 04 Dec 2010 18:01:37 +0000

Here's a simple and clear guide for gmail, which also definitely works with other relay hosts. I've used it to configure my ISP's mail relay (they block outgoing port 25) on a Debian Squeeze laptop.

How to replace newlines with commas, tabs etc (merge lines)

Bogdan — Tue, 16 Nov 2010 08:20:45 +0000

Imagine you need to get a few lines from a group of files with missing identifier mappings. I have a bunch of files with content similar to this one:

ENSRNOG00000018677 1368832_at 25233
ENSRNOG00000002079 1369102_at 25272
ENSRNOG00000043451 25353
ENSRNOG00000001527 1388013_at 25408
ENSRNOG00000007390 1389538_at 25493

In the example above I need '25353', which does not have corresponding affy_probeset_id in the 2nd column.

It is clear how to do that:

PLAIN TEXT

CODE:

sort -u *_affy_ensembl.txt | grep -v '_at' | awk '{print $2}'

This outputs a column of required IDs (EntrezGene in this example):

116720
679845
309295
364867
298220
298221
25353

However, I need these IDs as a comma-separated list, not as newline-separated list.

There are several ways to achieve the desired result (only the last pipe commands differ):

PLAIN TEXT

CODE:

sort -u *_affy_ensembl.txt | grep -v '_at' | awk '{print $2}' | gawk '$1=$1' ORS=', '

PLAIN TEXT

CODE:

sort -u *_affy_ensembl.txt | grep -v '_at' | awk '{print $2}' | tr '\n' ','

PLAIN TEXT

CODE:

sort -u *_affy_ensembl.txt | grep -v '_at' | awk '{print $2}' | sed ':a;N;$!ba;s/\n/, /g'

PLAIN TEXT

CODE:

sort -u *_affy_ensembl.txt | grep -v '_at' | awk '{print $2}' | sed ':q;N;s/\n/, /g;t q'

PLAIN TEXT

CODE:

sort -u *_affy_ensembl.txt | grep -v '_at' | awk '{print $2}' | paste -s -d ","

These solutions differ in efficiency and (slightly) in output. sed will read all the input into its buffer to replace newlines with other separators, so it might not be best for large files. tr might be the most efficient, but I haven't tested that. paste will re-use delimiters, so you cannot really get comma-space ", " separation with it.

Sources: linuxquestions 1 (explains used sed commands), linuxquestions 2, nixcraft.

How to record Skype calls on Linux: use free Skype Call Recorder

Bogdan — Thu, 11 Nov 2010 13:06:53 +0000

Just came across Skype Call Recorder - an awesome in its functionality+simplicity tool to record skype calls. Highly recommended!

It worked immediately for me, and default settings are good enough not to bother tweaking. Well, I know that because I did tweak a few to get more nerdiness, but normal people don't need that.

SCR download page has packages for Ubuntu, Debian/i386, Xandros, RPM-based distributions, Gentoo - and as its free, you can of course just use the fsource~~, Luke~~!

At the time of writing, a package for Debian/amd64 was not available, but it is really easy to build one.
Here's mine: skype-call-recorder-debian_0.8_amd64.deb

Linux server remote backup options

Bogdan — Sun, 03 Oct 2010 19:40:37 +0000

Recently I finally managed to configure remote (aka off-site) backup for my Debian server. As always, I started with a comparison of existing solutions.

Debian has a number of packages enabling remote (over the network) backup: backupPC, backupninja, backup-manager, dirvish, duplicity, luckybackup, rdiff-backup, and some others.

In my setup I wanted backups to be initiated by the external backup server (as opposed to internal, production server initiated backup). Also, I was going to backup linux server to another linux box. Thus, it would be preferable to have support for rsync on both boxes and hardlinks on the backup box. I needed backup to be run regularly, I didn't need any kind of GUI, and I would prefer to keep a bunch of older backups around (in case the most recent backup is of a compromised system, for example). Also, having an incremental backup of SQL databases was yet another requirement.

A few tools more or less matched my requirements: backupPC, duplicity, rdiff-backup, and dirvish.

backupPC, judging from its description and a few tutorial articles, is a cool piece of software. I think I might use it next time. But this time I really had no need for over-SMB backups (intended for Windows boxes), and no need for a nice administrative web-interface to backupPC. If not these extra features (which I don't currently need) I'd go for backupPC. Here's a guide for Ubuntu.

Duplicity claims to still be in beta, thus I haven't given it a try. rdiff-backup seemed a bit too simple (and with no ready templates to get working regular backups faster), so I skipped that one as well.

Dirvish really caught my attention, partially thanks to a good and relatively short quick-start tutorial. That tutorial also mentions automysqlbackup - a nice, almost no-configuration tool to take regular dumps of MySQL databases. In combination, these two now regularly backup up my whole server to another linux box.

I would definitely recommend using dirvish+automysqlbackup for remote/off-site backup needs. However, other solutions might be as good or better for your use case.

Linux: how to label swap partition w/o losing swap UUID

Bogdan — Fri, 16 Jul 2010 12:41:13 +0000

In short: sudo mkswap -L new_swap_label -U old_swap_UUID /dev/sd_swap_device.
If you don't care about the UUID: just sudo mkswap -L new_swap_label /dev/sd_swap_device.

Step-by-step:

Identify current swap UUID: grep swap /etc/fstab ; you should see a line starting with UUID=b4e6e... - note that UUID; let us assume our UUID is b4e6e5d8-8854-4a3e-8edb-f423ded31e2a
Identify swap partition device. If you are using Debian or Ubuntu, device might have been shown to you when running command #1 as a comment of the form
# /dev/sdX4 none swap sw 0 0
Just to be sure, run cat /proc/swaps (or sudo swapon -s), to be shown something like this:
Filename Type Size Used Priority /dev/sdX4 partition 2104504 0 -1
You can also use partition managers to identify swap device, e.g. sudo fdisk -l. Let us assume our partition device is sdX4.
If that swap partition is in use, you need to disable it first: sudo swapoff /dev/sdX4
Now that you have both the UUID and the device, run
sudo mkswap -L new_swap_label -U b4e6e5d8-8854-4a3e-8edb-f423ded31e2a /dev/sdX4
Do not forget replacing sample values above (label, UUID and device) with your actual values.
Enable swap: sudo swapon /dev/sdX4

Now, if you wish so, you may proceed to edit /etc/fstab and replace UUID=.... with LABEL=your_new_swap_label, leaving all other spaces and values in that line untouched.

Best Linux software to import still images from a digital camera

Bogdan — Sat, 01 May 2010 22:08:01 +0000

Back in the times when Windows was my OS more than 75% of the time in front of the computer, I really liked a simple photos import utility, supplied for Canon digital still image cameras. It allowed the import of all the new media items (photos and movies) from a camera into a pre-configured location in one click, and it also automatically created date-based directories for new photos. Given the removable media autoplay feature, after connecting the camera it was really a matter of one click (or one button press) to have everything copied.

Today, I decided to have a look at what the FOSS community has to match that simple and useful Canon utility. Here's my "rating":

digikam: nice and easy import; supports date-based auto-creation of albums (directories); feature-reach, suitable as a primary images-viewing application; my personal "number one" for now.
f-spot: easy import; has nice date-based photos browser; on import, converts photo filenames to lower case; my second choice, will have a closer look at its features.
gThumb: good import, but haven't found a way to configure/create target directory during import.
gtkam: unusual interface; while testing, it died once unexpectedly; otherwise easy to use.
picasa: haven't tried to import with a recent version, but I generally dislike the idea of using software which requires an extra layer to run (wine in this case), unless it is unique. Picasa is not unique, and thus looses (in my opinion) to digikam.

The list of Linux music players with CUE sheet support

Bogdan — Fri, 30 Apr 2010 18:45:02 +0000

Last updated: 2011-12-13

Clementine music player
DeaDBeeF (as in 0xDEADBEEF) is an audio player for GNU/Linux systems with X11 (though now it also runs in plain console without X, in FreeBSD, and in OpenSolaris) with CUE sheets support
AudioPlayer51 is a free, open-source, cross-platform audio player written to handle FLAC files and CUE sheets (exclusively). If you don't use FLAC files and CUE sheets, you won't find AudioPlayer51 very useful.
Amarok: only for WAV+CUE, doesn't currently support FLAC+CUE; version 2.3.1 is promised to fully support CUE sheets
Audacious (through the .cue plugin)
QMMP (haven't tested this one)
XMMS through mp3cue (also untested)
foobnix (untested, suggested in this comment)
cueplay

Some extra links:

screen mastery with only 3 keyboard shortcuts

Bogdan — Mon, 01 Mar 2010 17:11:57 +0000

screen is a must-use for anyone familiar with SSH sessions; it saves your time by keeping your console apps running even after (an unexpected) disconnect from the remote session.

The easiest way to start with screen is screenie - an interface to screen. To use it, you only need to memorize a single keyboard shortcut: C-a d (Ctrl+a, d). This shortcut detaches (hides) your current screen session, allowing to resume later - either with the help of screenie, or by issuing screen -r command.

After a while, with increased screen (and screenie) usage, you will find it inconvenient detaching every time you need a new screen session. Here comes the second keyboard shortcut: C-a c (Ctrl+a, c), which creates a new window within a current screen session, and switches to it.

Finally, you will find it tiresome detaching to then re-attach to a different screen. C-a " (Ctrl+a, Shift+') will display a list of screen windows for you to select and switch to.

Screenie, an excellent beginner's interface to screen, is no longer needed after you memorize these 3 keyboard shortcuts.

Many will say that this is a bare minimum, and also there is much more to screen than what you present here, and that will be true. However, these 3 shortcuts are sufficient for simple but convenient everyday screen sessions management.

ntfstruncate binary for Debian (resetting NTFS bad clusters counter)

Bogdan — Mon, 01 Mar 2010 16:55:56 +0000

There is an excellent step-by-step instruction on resetting the bad clusters counter of an NTFS partition with linux-ntfs tools. I've checked - it works as expected:

Back up important data from partition just in case
Find out size of '$Bad' attribute in $Badclus using ntfsinfo -i 8 partition (partition is for example /dev/sda1). It will be the “Allocated size” value in the “Dumping attribute $DATA (0x80)” (there will be two 0x80 attributes. Only one has an “Allocated size” line). Let us write down (remember) this size as ntfs_size.
Use ntfstruncate partition 8 0x80 '$Bad' 0 to set $Bad's attribute length to zero.
Use ntfstruncate partition 8 0x80 '$Bad' ntfs_size to set $Bad's attribute length back to proper value ntfs_size which was recorded in step 2.
Boot into Windows and run chkdsk -f diskname. It will find errors and should fix them.

However, Debian's ntfsprogs package does not have the ntfstruncate binary.

Here's how you can easily build one yourself (you may need a few extra packages with build tools for that):

apt-get source ntfsprogs
cd linux-ntfs-2.0.0
./configure && make extra
cd ntfsprogs && ./ntfstruncate

You can also install ntfstruncate system-wide, but I have no idea if that will interfere with the already-installed ntfsprogs package, so I just used this binary directly from linux-ntfs-2.0.0/ntfsprogs.

Bad clusters happen not only when cloning an NTFS partition between 2 physical hard disk drives. Once, after a brief blackout (with a turned on PC, and not on a UPS), the SMART status of my new HDD started showing 1 'Pending sector'. Correspondingly, all the SMART tests (performed with a smartctl utility) reported read errors. To help my HDD examine and fix the problem itself, I performed a write operation to that single unreadable sector, which restored it (and reset the 'Pending sector' counter back to zero). It was possible, because not the physical sector itself, but the hidden CRC field had errors after power failure; modern hard drives can detect and fix this kind of errors upon write operations (read operations only mark the sector as 'pending', because data read is not consistent with the stored CRC).

However, NTFS partition still had 4096 bytes in 'bad clusters'. A little research revealed the presence of internal NTFS bad clusters counter, which can be reset following the procedures above.

Chandler 1.0.3 package for Debian testing amd64

Bogdan — Mon, 25 Jan 2010 08:52:26 +0000

Recently, I have come across the Chandler project. Chandler is a sophisticated organizer, including tasks, calendar, appointments, invitations, and many more.

The project does provide a 64-bit package for Ubuntu 8.10 "Intrepid Ibex", but it has a dependency on python-zope-interface - which in Debian testing is a virtual package, currently provided by python-zope.interface. That single dependency made my aptitude complain all the time, so I edited the .deb-file, renaming depends/python-zope-interface to depends/python-zope.interface (following nice and simple instructions). After that - no more aptitude complaints, and Chandler works fine.

Download the modified Chandler 1.0.3 Debian testing amd64 package (md5: 239071715977bea2eb75f3bb54491a02).

fail2ban and Google translate: how to easily cut your WP blog traffic

Bogdan — Sat, 14 Nov 2009 00:11:08 +0000

fail2ban has a php-url-fopen rule.

WordPress has a Global Translator plugin, which - among others - uses Google Translate service.

If someone uses Google Translate (e.g. using Global Translate's mini-language-flags), and goes back to your blog - that someone might get banned by fail2ban (especially if you have set maxretry to 1), as the referrer will contain the php-URL-fopen attack signature. The bad thing is that you will not realize that until after you check one or several translations yourself, as a random site visitor experiencing the problem is highly unlikely to bother reporting this problem - especially when your blog's Contact page is also inaccessible.

Clearly, Google Translate is not the only legitimate service which will trigger that rule.

Solution: The only solution I have found is to specify the whitelist regex for the php-URL-fopen rule.

R under Debian testing/i386: permanently set pdfviewer option

Bogdan — Wed, 21 Oct 2009 14:06:37 +0000

If you get this message when opening vignettes:

Error in openPDF(vif) :
getOption('pdfviewer') is ''; please use 'options(pdfviewer=...)'

and you are tired of running this command every time:

> options(pdfviewer="okular")

then you should check if your system-wide Renviron file has proper PDF viewer set:

grep -i pdf /etc/R/Renviron

R_PDFLATEXCMD=${R_PDFLATEXCMD-${PDFLATEX-'/usr/bin/pdflatex'}}
R_RD4PDF=${R_RD4PDF-'times,hyper'}
## Default PDF viewer
R_PDFVIEWER=${R_PDFVIEWER-''}

It wasn't in my case.

To set one, either edit the system-wide Renviron (e.g. by editing the R_PDFVIEWER=${R_PDFVIEWER-''} line to look like R_PDFVIEWER=${R_PDFVIEWER-'/usr/bin/xdg-open'} - this will use MIME types to open your preferred PDF viewer), or one of the per-user/per-directory Renviron files to fix this minor annoyance (format is the same, e.g. R_PDFVIEWER=okular ).

Convenient design and debugging of regular expressions under Linux

Bogdan — Sat, 10 Oct 2009 15:51:18 +0000

Regular expressions (regexps) are powerful indeed. But debugging non-trivial regexps is a burden even if you understand how regexps work, and remember most (if not all) regexp syntax.

Miscellaneous tools exist to ease this task. This post was inspired by redet's comparison of regexp helper tools - it could be sufficient to read only that, if you're going to try the mentioned tools yourself. Otherwise, read on.

Kodos is an easy-to-use (but powerful enough) Python tool to verify regular expressions against sample user-provided strings. Available for both Linux and Windows. Generates ready-to-use Python regexp code.

redet assumes some knowledge of REs. Unlike all the other helper tools, it has the widest code generation coverage - just go to "Tools->Save command information" to generate code (Python, Perl, C, grep - just to name a few of the 57 (!) languages/tools/editors supported). At the same time, redet does not depend on specific versions of all supported RE engines - instead, redet automatically tests for features in available versions.

visual-regexp is similar to kodos in features.

txt2regex is a regex wizard written in bash, capable of generating code for 24 languages/editors/programs (by default, these are enabled: perl, php, postgres, python, sed and vim). It is really simple and convenient, though some understanding of "groups" and "character ranges" is highly recommended.

KRegExpEditor was a part of kde3-utils, and seems to be missing from current KDE 4.3. It is similar to kodos and visual-regexp in its philosophy and features.

I would recommend using one of kodos/visual-regexp/kregexpeditor to quickly test or design your regular expression, then use redet to generate appropriate source code. As an alternative (and when working in a console), just use txt2regexp to generate the code you need.

Tools I haven't tried include regex coach and regexpviewer.

Please also see this extensive list of regular expression and pattern-matching related software.