I do not really recollect anything that large. Well, not counting dropbox’s 4-hour window of “any password fits all accounts”, but that was different.

Here are some of the news items about this seriously large hacking incident:

• NYTimes (Dec. 25, 2011)
• WSJ (Dec. 25, 2011)
• CNN (Dec. 25, 2011)
• relatively above-average write-up from Wired (Dec. 26, 2011)
• ABCNews (Dec. 26, 2011)
• The Register (Jan. 3, 2012)

Here come more technical reports:

• short pastebin document, supposedly by the hackers
• cryptome keeps track of the data being removed from the internet
• a 1MB report by the hackers
• TheTechGerald has some analysis of the leaked stratfor passwords (Jan. 2, 2012)

TheTechGerald’s analysis linked to above got my attention. Unfortunately, a while ago I’ve subscribed to stratfor’s “free intelligence mailing list”, and was wondering if my account information is now publicly available. I was the most worried about the password I’ve used to subscribe, because of the risk of using the same password somewhere else.

Unlike TheTechGerald, I haven’t used any dictionaries – just the default configuration of a well-known tool for finding weak passwords. Within a single hour, ~100k passwords were decrypted (~12% of all). Till the end of the day, ~50k more passwords were decrypted (totalling 17.4% of 860k). At this point my password was still safe, and I’ve found a way to verify that it is not used anywhere else, so I’ve aborted further decryption.

There are a few simple conclusions:

• anybody who had a stratfor account must verify that he/she isn’t using that password anywhere else, because if 1 PC can get 17% of all the passwords in less than a day, it is only a matter of short time until all the leaked passwords will be decrypted and made publicly available in various “md5 decryption databases”
• system owners should run periodic screenings for weak passwords (and implement policies to prevent creating obviously weak passwords from the very beginning)
• md5 is very fast to decrypt/bruteforce – a much slower hashing function wouldn’t hurt; also, using a more complex hashing approach, maybe even with a closed-source shared library, could help
• single-factor authentication (password-based) is likely to get replaced with 2-factor authentication in the nearest future
• one may enjoy increased personal data safety by using throw-away passwords in conjunction with antispam mailboxes like spam.la and mailinator.com (at least 1600 users – 0.186% – did use these services).

Fortunately, the top 10 passwords (by their counts) were exclusively “throw-away”, and added up to ~10% of the decrypted passwords. (I’m not showing any, as that would unnecessarily simplify further decryption – maybe thetechgerald should have also been more vague about actual passwords.)

Sooner or later this significant-size corpus of real-life passwords will find its way (after decryption by those who would actually use leaked passwords to gain unauthorized access) into various wordlists and wordlist mutation rules, making it even easier to decrypt any future leaks. This is where 2-factor authentication will, hopefully, come in handy to protect against similar leaks.

I wonder if I should put up a page “Check if my password was among those 860k”, to help people easily identify if they should change theirs – not even necessarily being a Stratfor subscriber. Unless similar pages/services had already been put up by others.

It is also unclear what will the future of Stratfor be, taking into account that their website is still dysfunctional.

It is sad to see Drupal (stratfor.com’s CMS) involved here. However, I have no idea if their installation was up to date, and if their website was the point of entry. The hacklog suggests that attackers somehow obtained the password of one of the system administrators, and then used it for SSH access, which would save Drupal’s face (Drupal’s security record to date was pretty reassuring).

If you (like me) love ncurses – you will love tudu Just give it a try.

Screenshot copyright: Ruben Pollan (tudu author).

Also as a PDF.

• current HDD is /dev/sda, it has a GPT (with bios_grub being /dev/sda1), separate /boot partition (/dev/sda2), and a physical LVM volume (/dev/sda3), where LVM holds all the remaining partitions (root, /home, /srv, …); LVM is properly configured, and system reboots with no problems
• your new drive is /dev/sdb, it is identical to /dev/sda, and it comes empty from the manufacturer (this is important! wipe the drive if it is not empty, especially if it used to be a part of another RAID)
• your system is Debian or Debian-based; in this exact example I’ve been using Ubuntu Server 10.04
• your LVM volume group is named vg0
• make sure you understand what each command does before executing it
• you do have an external backup of all your important data, and you do understand that the following operations are potentially dangerous to your data integrity

Inspired by: Debian Etch RAID guide, serverfault question.

1. Create the GPT on the new drive:
   parted /dev/sdb mklabel gpt
2. Get the list of partitions on /dev/sda:
   parted -m /dev/sda print
3. Create /dev/sdb partitions similarly to what you have on /dev/sda (my example numbers follow, use your numbers here):
   parted /dev/sdb mkpart bios_grub 1049kB 2097kB
   parted /dev/sdb mkpart boot 2097kB 258MB
   parted /dev/sdb mkpart lvm 258MB 2000GB
4. Set proper flags on partitions:
   parted /dev/sdb set 1 bios_grub on (GPT doesn’t have MBR, so you create a 1-MB partition instead to hold grub2′s boot code)
   (possibly optional) parted /dev/sdb set 2 raid on
   (possibly optional) parted /dev/sdb set 3 raid on
5. (possibly optional) To make sure /dev/sdb1 (the bios_grub) indeed contains grub’s boot code, I did dd if=/dev/sda1 of=/dev/sdb1
6. apt-get install mdadm
7. Note: at this point, older tutorials suggest adding a bunch of raid* kernel modules to /etc/modules and to grub’s list of modules to load. I’m not sure this is really necessary, but do see the tutorials mentioned at the top for more information. If you do modify the lists of modules – don’t forget to run update-initramfs -u.
8. Create two initially-degraded RAID1 devices (one for /boot, another for LVM):
   mdadm ––create /dev/md0 ––level=1 ––raid-devices=2 /dev/sdb2 missing
   mdadm ––create /dev/md1 ––level=1 ––raid-devices=2 /dev/sdb3 missing
9. Store the configuration of your RAID1 to the mdadm.conf file (important! this is not done automatically!)
   mdadm -Es >> /etc/mdadm/mdadm.conf
10. Verify the contents of your mdadm.conf:
    cat /etc/mdadm/mdadm.conf
    dpkg-reconfigure mdadm, and enable booting in degraded mode
11. Copy your current /boot (/dev/sda2) to the new /dev/md0 /boot partition:
    (one can use dd here as well, but for some reason my attempt at dd failed writing 1 last byte of data)
    mkdir /mnt/md0
    mount /dev/md0 /mnt/md0
    cp -a /boot/* /mnt/md0/
    umount /dev/md0
    rmdir /mnt/md0
12. Now extend your existing volume group to include the newly-created /dev/md1:
    pvcreate /dev/md1
    vgextend vg0 /dev/md1
13. Verify the list of logical volumes you curently have: enter lvm shell, and type lvs. Here’s what I had:
    LV VG Attr LSize Origin Snap% Move Log Copy% Convert
    home vg0 -wi-ao 1.70t
    logs vg0 -wi-ao 4.66g
    root vg0 -wi-ao 10.24g
    srv vg0 -wc-ao 100.00g
    swap vg0 -wi-ao 1.86g
    tmp vg0 -wi-ao 4.66g
14. Now, you can move all the logical volumes to new physical volume in one command: pvmove /dev/sda3 /dev/md1. Personally, remembering the problem I had with dd from /dev/sda2 to /dev/md0, I decided to move all logical volumes one-by-one; as this takes time, you may consider joining these operations with ; or &&, and putting the /tmp last (as the easiest one to re-create if it fails to move):
    pvmove ––name home /dev/sda3 /dev/md1
    pvmove ––name srv /dev/sda3 /dev/md1
    pvmove ––name logs /dev/sda3 /dev/md1
    pvmove ––name swap /dev/sda3 /dev/md1
    pvmove ––name root /dev/sda3 /dev/md1
    pvmove ––name tmp /dev/sda3 /dev/md1
15. To be safer, I ran FS check on a few volumes I could umount:
    umount /dev/mapper/vg0-srv
    fsck -f /dev/mapper/vg0-srv
    mount /dev/mapper/vg0-srv
    umount /dev/mapper/vg0-tmp
    fsck -f /dev/mapper/vg0-tmp
    mount /dev/mapper/vg0-tmp
16. Remove /dev/sda3 from the physical space available to your volume group:
    vgreduce vg0 /dev/sda3
17. Install grub2 to both drives, so as to make them both bootable in case of failure:
    grub-install ‘(hd0)’
    grub-install ‘(hd1)’
18. Edit /etc/fstab, pointing /boot to /dev/md0. You may use UUIDs here, but please do not use UUIDs from mdadm.conf – those are different from FS-UUIDs, instead do ls -l /dev/disk/by-uuid to find the UUID of /dev/md0. Personally, I had no problems just using /dev/md0.
19. Now is the time to add your original /dev/sda to the RAID1; be absolutely sure you have moved all the data off that drive, because these commands will destroy it:
    mdadm ––manage ––add /dev/md0 /dev/sda2
    mdadm ––manage ––add /dev/md1 /dev/sda3
    Re-syncing array will take some time.
20. To be on the safe side, you may want to run again update-initramfs -u and update-grub; I have also edited /etc/grub.d/40_custom, adding there 2 more boot options: from /dev/sda2 and /dev/sdb2 (/boot on both drives) – have no idea if that will work, but having more boot options didn’t hurt
21. Reboot into your new system. Actually, at this point reboot is only necessary to verify that your system is bootable – you may delay this reboot as long as you want to.
22. Many tutorials also suggest testing your RAID1 by manually “degrading” it, trying to boot, and then rebuilding it back. I haven’t done that, but you may want to.

Improvement suggestions, criticism and thank-you are welcome in the comments.

Both scripts should be started from the directory where you keep your music. The .mood files will be generated next to the source music files (in the same directories). You can modify the scripts to have the moods stored to $HOME/.kde/share/apps/amarok/moods/, so as not to clutter your music directories.

Script for multicore CPUs (will not overwrite existing mood files):

Script for single-threaded moods generation (will only overwrite the last-generated mood file from a previous run):

That same page has a link to an older, somewhat more polished ruby script for the same task. It requires a running instance of Amarok, and uses it's media collection to find music files and create mood files for them. This script can potentially (after a little editing) store .mood files either with the music or to the $HOME/.kde/... path mentioned above. It is for an older version of Amarok, and thus is not likely to work with e.g. Amarok 2.4

There is an excellent, in-depth, well-argumented article by Geoff Chappell on the issue. Highly recommended in its entirety to those who want a complete understanding (additional side-reading and facts verification might be necessary).

A single citation to get you started:

There is already on the Internet and elsewhere an awful lot of rubbish to read about this question. Hardly any of it would be worth citing even if I didn’t want to spare the authors the embarrassment. A surprising number of people who claim some sort of attention as expert commentators would have you believe that using more than 4GB of memory is mathematically impossible for any 32-bit operating system because 2 to the power of 32 is 4G and a 32-bit register can’t form an address above 4GB. If nothing else, these experts don’t know enough history: 2 to the 16 is only 64K and yet the wealth of Microsoft is founded on a 16-bit operating system that from its very first version was designed to use 640KB of RAM plus other memory in a physical address space of 1MB. Some remember this history and add seemingly plausible qualifications that exceeding 4GB is possible only at the price of nasty hacks that require everyone—well, all programmers—to jump through hoops. Fortunately, Intel’s processors are a lot more advanced than the 8086 from all those years ago.

P.S. Unfortunately, patching the kernel won't help make Windows XP see more than 4GB RAM: even though the kernel itself does support more RAM (with PAE), starting with SP2 the HAL was modified in a way prohibiting access to any RAM beyond 4GB. Patching may only be suggested to devoted geeks with Vista's and 7's.

There are a few highly-popular patches, which haven't yet made it into the Squeeze's rtorrent version 0.8.6.

Note: ArchLinux already has an rtorrent-extended package, so you may want to look at that one (instead of my Debian package below). I might use Arch's patchset next time to generate a truly extended rtorrent package.

I've incorporated 3 of such patches (#239 ipfilter, #2064 GeoIP support, and #2171 colours/eyecandy), and built a squeeze amd64 rtorrent-enhanced package.

A few excerpts to explain how these extra features are to be configured and used (all come from the above-mentioned trac tickets, some were edited/extended).

ipfilter

ipfilter allows to selectively blacklist/whitelist peers - based on IP address range files (so-called ipfilter files). The ipfilter files may, for example, come from bluetack.co.uk. Each line of the file contains a record in this format: range_description:start_IP-end_IP, where start_IP should be less than or equal to end_IP. Multiple files can be used. Overlapping ranges are merged automatically. Both incoming and outgoing connections are checked against the filter. Exclusions are not supported, so connection to/from IPs in all the loaded ranges will be disallowed and dropped.

Include "ip_filter=" directive in .rtorrent.rc. For example (paths are specified relative to user's home directory):
ip_filter=ipfilter/level1,ipfilter/level2

It is probably a good idea to reload ipfilter files once in a while, so you can also include "reload_ip_filter" directive on schedule to refresh the filter. The same files named in "ip_filter" will be reloaded.

schedule = filter,18:30:00,24:00:00,reload_ip_filter=

To support the feature you may want to setup cron job to reload and unzip files from bluetack. Examples here and here.

GeoIP

In the Peers view of each torrent a new column - CC, country code - is added, which shows peer's country code. When examining each peer, you will see more data from the GeoIP database (if you have the relevant files installed): AS num and city. At the very least, you need to have geoip-database installed. You can get free "lite" versions of city/AS-databases, rename them (removing "lite") and put into an appropriate location to make rtorrent+geoip use them as well. Geop-isp data support isn't integrated into this rtorrent package.

Colours

This patch somewhat changes the way rtorrent shows torrents. Seeding torrents have bold titles, there are no half-displayed torrents at the top/bottom when scrolling, and you can configure colours for active/done torrents (ticket libtorrent.rakshasa.no/ticket/1382, which seems to be gone now). Source code and testing hint at these configurable colours: done_fg_color, done_bg_color, active_fg_color, active_bg_color, and at these possible values: 1 (red), 2 (green), 3 (yellow).

Geek's cellar
A related (though not used in any way for the preparation of the package) resource is rtorrent mods page.

Relatively schematically, applying patches and building the package was performed in these steps (starting within some newly-created directory):

1. sudo aptitude install cdbs devscripts [and whatever else you find you're missing]
2. apt-get source rtorrent
3. dpkg-source -x rtorrent_0.8.6-1.dsc
4. cd rtorrent-0.8.6
5. cdbs-edit-patch 01-ipfilter.patch
6. patch -p1 < /path/to/patch/239
7. exit 0
8. cdbs-edit-patch 02-geoip.patch
9. patch -p1 < /path/to/patch/2064-after-ipfilter
10. exit 0
11. cdbs-edit-patch 03-eyecandy.patch
12. patch -p1 < /path/to/patch/2171-mod-with-canvas
13. exit 0 [you could do all 3 patches together, but I prefer cleaner and manageable approaches]
14. [some weird operation to be explained below]
15. dpkg-buildpackage

Weird operation: I didn't know how to make dpkg-buildpackage run autorun.sh (which is required for one of the patches I've used) before configure (could someone enlighten me how to do that?), so I started a new patch with cdbs-edit-patch, then ran one by one `autoreconf -if`, `./autorun.sh`, `./configure`, exited with zero status and was done with that problem. The resulting "patch" was 2MB large

• if you don't have the python-stdeb package: sudo aptitude install python-stdeb
• pypi-install pycassa

That's it.

Refer to stdeb readme for more information. You will need that if there are dependencies - which might not be resolved automatically by stdeb.

Before stdeb, it wasn't exactly trivial to make a .deb from python module.

Back in 2007 I wrote a brief review of web-based project management tools. After that, I started using dotProject for personal projects management. I'm still using it, but for collaborative project management, communication, and tasks/milestones tracking dotProject isn't perfect.

I need a tool, which is

• collaborative
• web-based (to allow effective collaboration)
• preferably free
• has concise per-project activity log
• minimal required functionality: tasks, milestones, files, and status updates.

After trying a few things, our small team settled for now on using github + pivotaltracker jira + confluence + flowdock.

Here's a full list of tools briefly reviewed. I've been already using ProjectPier, so I'll start with this software.

ProjectPier (used myself)

• dashboard: all events log
• interface similar to Basecamp; themable/skinnable
• all the basic features are there (milestones, tasks, task lists, messages, files)
• modular (functionality is in plugins)
• easy to install (requires PHP and MySQL)
• is being maintained/developed (maybe slowly, but that doesn't mean much)

Not much to add. Simple, functional, worked good for a 1-person "team" (that is, for personal projects management). Have no idea how it scales to more people.

Collabtive

• desktop: just an overview, no log of events; project view has 'activities' log
• [too much?] eye-candy, JS-reach default interface (themable/skinnable)
• projects, tasks, milestones, messages, files
• calendar, time tracking
• is being maintained/developed

Open Atrium

• Drupal-based, thus probably the most flexible (but requires time investments to change functionality)
• 6 features: blog, wiki, calendar, to-do list, shoutbox, and a dashboard to manage it all
• has "recent activity" log
• issues tracking
• I guess it is heavier than others in use patterns: requires more clicking and typing (as it has more features), and there seem to be no concepts of milestones and projects - just tasks

Projectfork

• possibly Joomla-based
• free, with commercial add-ons, themes, and maybe support
• projects, milestones, tasks with priorities, files
• calendar, discussion board, time tracking
• activity stream (premium add-on)

EGroupware

• hosted, commercial
• free community version is available for download
• projects, tasks, sub-tasks, files
• address book, calendar, chat, issue tracking system, time tracking
• knowledge base, wiki
• news, polls
• interface seems very responsive (JS-reach)
• large, feature-reach: might be an overkill where basecamp would do just fine
• actively developed

]project-open[

• not reviewed: seems even more feature-reach (complicated) than EGroupware

Redmine

• doesn't seem to use "milestone" and "task" concepts
• issue tracking, gantt charts, calendar, time tracking
• wiki, files, forums, roadmap (similar to trac)
• repository browser (among others, git and svn are supported)
• is maintained/developed

Codebase

• non-free
• issue tracker for git/mercurial/others with project management features
• wiki, time tracking, milestones, files

Ofuz

• paid hosted version (free up to 5 projects), free version available for download
• contacts, time tracking, invoices
• projects, tasks, documents, files
• tight email integration (e.g. continue discussions by email, with replies logged to Ofuz)

RailsCollab

• activecollab-inspired, ProjectPier-based Ruby software
• interface (and features) very similar to ProjectPier
• tasks and task lists, milestones, files, messages
• time-tracking
• development/maintenance stalled in Feb 2010

Teambox (used myself)

• hosted service (free up to 3 projects), community edition available for download; RoR-based
• free plan has search disabled
• projects, tasks, task lists, files
• dashboard
• pages/wiki/writeboard, discussions
• gantt charts, calendar, twitter-like status updates, time-tracking
• light interface
• clients for mobile devices
• email notifications and email-to-web functionality

Seems best for conversations-oriented projects. A few times posted updates took lots of time to become visible to other team members (far not immediate, so comparison to twitter does not give the right idea), and page refreshes (even forced) didn't help. Tasks system is basically an extension of conversations: once you created a task, you can only "extend" it with comments, but not edit. Personally, I found the tasks implementation too awkward to use - it might be different for writing-related projects. I liked the Pages functionality: it provides a good (easy and quick) way of organizing information accumulated by the project. Basically, we ended up using Teambox as a repository for external and internal documentation - but not for status updates, chats or planning.

As free time permits, I would love to compile a feature table, comparing all these tools, together with subjective "easy-of-use" scores (maybe collected with a poll of some kind). Any contributions towards this simple goal are welcome. If comments fail to work for you - use the contact page.

A few more related web-tools follow.

Pivotal Tracker (currently using)

• agile projects management
• concepts: icebox, backlog, current, done
• has: features, bugs, chores, releases; each of these can have description, comments and short tasks (all very easy to add and organize)
• features can have their complexity estimated in points, which are then used to calculate weekly team velocity, and also to move tasks from the backlog panel into current panel

I'm new to agile development tools, and after getting used to it - Pivotal Tracker is good. It is also useful as a place to keep the things you would like to eventually implement - just append these to the end of the icebox, and then start-move to backlog/current when determined to implement.

Flowdock (currently using)

• web-chat with history saved as an infinite scrollable page
• has a concept of "flows" (similar to chat rooms in campfire)
• tags (tab-autocomplete possible when writing messages); can be added/removed to/from existing entries
• files can be inserted directly into chat stream
• separate views for posted URLs and files
• full-text search (a recent feature), and search by tags
• mobile device support (haven't tried)
• various desktop notification tools for all platforms (Linux, Mac, Windows); has minimally-configurable sound notifications
• tracks online/idle/offline statuses (e.g. idle for X hours or offline for Y hours)
• mails can be sent to a flow, and they can have tags
• Influx: an aggregator of external events (github, twitter, RSS, mails, PivotalTracker, Confluence and others)

Flowdock is just... convenient. After trying teambox, present.ly and campfire, we seem to have settled on this one for in-project communication (our team currently has only 3 people, though). The most convenient feature is probably the built-in aggregator.

WeDoist

• collaborative to-do lists
• tasks (maybe also sub-tasks), status updates, group chat
• hosted solution

ToDoist

• 1-person projects, tasks, sub-tasks
• hosted solution
• opera widget at http://widgets.opera.com/widget/15372/

SlimTimer (using this one)

• simple (perfect? ) tasks-based timetracker with nice reports feature
• hosted solution, has free plan

Present.ly (used myself, would use again)

• "corporate twitter"
• hash-tags autocompletion
• files can be attached
• "attach text" - when 140 symbols is not enough
• direct messaging and replies; replies can be viewed in threaded mode
• mobile devices support (haven't tried)
• configurable email alerts
• concepts helping organize data: topics, feeds, tags
• separate views for files and links to find them faster

Overall, present.ly is very cool for within-team status updates - that is, to keep track of what anybody's doing.

Campfire (used myself)

• web-chat with "rooms" (e.g. by topic, by department etc)
• each day is saved as a transcript of chats
• files can be attached directly within the chat flow
• full-text searchable
• free use tier implies chat-stream embedded ads (can be removed with adblock+ and element hiding helper)
• can be configured to track external resources (e.g. github commits), though those do not look as good as in flowdock

Overall, campfire is a nice chat. The best thing they have is the event sound - probably the best I've heard.

Finally, nice mantra (except for the very last phrase) from ToDoist - "The Zen of Todoist":

Now is better than later.
Later is better than never.
Organized is better than messy.
Big things are composed by smaller things.
Smaller things are done by action.
Think like a person of action.
Act like a person of thought.
The beginning is half of every action.
The longest journey starts with the first step.
Everything should be made as simple as possible.
But not simpler.
Celebrate any progress.
Don't wait to get perfect.
Deadlines and stress are a part of life.

Just a single quote from Glyn Moody, Computerworld UK:

The criticisms made in the video are not really the point - they are mostly about OpenOffice.org not being a 100% clone of Microsoft Office, and compatibility problems with Microsoft's proprietary formats. The key issue is the exactly the same as it was for the Mindcraft benchmarks. You don't compare a rival's product with your own if it is not comparable. And you don't make this kind of attack video unless you are really, really worried about the growing success of a competitor.

See also what Savio Rodriguez (Infoworld) has to say about that video.

ENSRNOG00000018677 1368832_at 25233
ENSRNOG00000002079 1369102_at 25272
ENSRNOG00000043451 25353
ENSRNOG00000001527 1388013_at 25408
ENSRNOG00000007390 1389538_at 25493

In the example above I need '25353', which does not have corresponding affy_probeset_id in the 2nd column.

It is clear how to do that:

This outputs a column of required IDs (EntrezGene in this example):

116720
679845
309295
364867
298220
298221
25353

However, I need these IDs as a comma-separated list, not as newline-separated list.

There are several ways to achieve the desired result (only the last pipe commands differ):

These solutions differ in efficiency and (slightly) in output. sed will read all the input into its buffer to replace newlines with other separators, so it might not be best for large files. tr might be the most efficient, but I haven't tested that. paste will re-use delimiters, so you cannot really get comma-space ", " separation with it.

Sources: linuxquestions 1 (explains used sed commands), linuxquestions 2, nixcraft.

It worked immediately for me, and default settings are good enough not to bother tweaking. Well, I know that because I did tweak a few to get more nerdiness, but normal people don't need that.

SCR download page has packages for Ubuntu, Debian/i386, Xandros, RPM-based distributions, Gentoo - and as its free, you can of course just use the fsource, Luke!

At the time of writing, a package for Debian/amd64 was not available, but it is really easy to build one.
Here's mine: skype-call-recorder-debian_0.8_amd64.deb

I have found many tools which claim to facilitate this procedure. Some of them are reviewed below (in no specific order).

Pathway Explorer by Bernhard Mlecnik was last updated in 2007, but is fully functional (I believe it is being maintained without changes to the last-updated date). Both online and downloadable Java applications are available. Note that for downloadable application you will need to obtain a license key - the procedure is well documented and was very fast for me.

Pathway Explorer supports import from 3 sources: KEGG xml files, biocarta URLs, and GenMAPP URLs. Import from KEGG does work as described in the short manual, and seems functional (I had some problems exporting/saving the resulting picture, but didn't investigate further). Biocarta import seems to work, but for some reason does not display expression levels of pathway components. I could not test the import of GenMAPP pathways, because they are not available online.

I found Pathway Explorer good, but then switched to PathVisio (reviewed next), because for some reason Зathway Explorer was recognising only a small fraction of genes from my expression data. It could be that identifiers mappings are outdated, but this is just a guess.

PathVisio appears to be a spin-off of GenMAPP and WikiPathways. It excells at importing/visualizing WikiPathways data, which even comes bundled with PathVisio Java application. It is easier to use than Pathway Explorer, and it seems to recognize more genes (although still not all the genes which are present in the data). There is KEGG pathways support, but it is not always usable - many edges (links between genes/proteins) are absent, so instead of a pathway you get a bunch of nodes relevant to a pathway, but cannot really see how they are connected. PathVisio supports an insanely long list of database identifiers, so it is highly unlikely that you will have to map your data to use a different identifier. This pathway mapper exports to several formats, including PNG and PDF.

I could not fully test AffyWEB, because it doesn't list rat arrays we used. Trying their barley genome example did work, so the tool is probably functional. It overlays your expression data onto KEGG pathways.

G-language Microarray System is a comparatively simple pathway visualizer. It accepts CSV files containing EntrezGene IDs column with a single column of expression values normalized to 1-100 range, fetches requested KEGG pathway, and generates a Flash (SWF) object depicting that pathway with coloured components. It does work with sample data. I was too lazy to normalize my expression data to [1;100] range, and SWF is not exactly a usable format, so I haven't tested this tool any further (you can right-click to zoom in the flash pathway below).

If time permits (or work requires) this post may be extended with the reviews of GenMAPP, GEPAT, KEGG2heatmap script, EGAN, MapMan, Pathway Miner, ArrayXPath, VisANT, SpotXplore, and maybe others.

Please comment to share your experience using pathway expression overlaying tools or to suggest other tools.

That's my first encounter of Shapado, so it was interesting to read Shapado authors' justification and a related question on meta.SO.

Fortunately, Alexey Smirnov (the author of DeaDBeeF) maintains a github repository deadbeef-debian which has simple instructions on installing DeaDBeeF in Debian (citing with minor edits):

1) download deadbeef from sourceforge (0.4.2 as of the moment of writing)
2) tar xvf deadbeef-X.Y.Z.tar.bz2
3) $ cd deadbeef-X.Y.Z/
4) $ git clone git://github.com/asv/deadbeef-debian.git debian
5) $ dpkg-buildpackage -rfakeroot -us -uc -i -I".git"
6) (install any dependencies reported by dpkg-checkbuilddeps, wait for the build to finish)
7) $ sudo dpkg -i ../deadbeef_X.Y.Z-R_yourarch.deb
Enjoy DeaDBeeF!

Hint: dpkg-buildpackage accepts -jNumJobs option for multi-core/multi-CPU systems.

Debian has a number of packages enabling remote (over the network) backup: backupPC, backupninja, backup-manager, dirvish, duplicity, luckybackup, rdiff-backup, and some others.

In my setup I wanted backups to be initiated by the external backup server (as opposed to internal, production server initiated backup). Also, I was going to backup linux server to another linux box. Thus, it would be preferable to have support for rsync on both boxes and hardlinks on the backup box. I needed backup to be run regularly, I didn't need any kind of GUI, and I would prefer to keep a bunch of older backups around (in case the most recent backup is of a compromised system, for example). Also, having an incremental backup of SQL databases was yet another requirement.

A few tools more or less matched my requirements: backupPC, duplicity, rdiff-backup, and dirvish.

backupPC, judging from its description and a few tutorial articles, is a cool piece of software. I think I might use it next time. But this time I really had no need for over-SMB backups (intended for Windows boxes), and no need for a nice administrative web-interface to backupPC. If not these extra features (which I don't currently need) I'd go for backupPC. Here's a guide for Ubuntu.

Duplicity claims to still be in beta, thus I haven't given it a try. rdiff-backup seemed a bit too simple (and with no ready templates to get working regular backups faster), so I skipped that one as well.

Dirvish really caught my attention, partially thanks to a good and relatively short quick-start tutorial. That tutorial also mentions automysqlbackup - a nice, almost no-configuration tool to take regular dumps of MySQL databases. In combination, these two now regularly backup up my whole server to another linux box.

I would definitely recommend using dirvish+automysqlbackup for remote/off-site backup needs. However, other solutions might be as good or better for your use case.