Autarchy of the Private Cave

Your software needs tons of RAM, and runs a bit too slow on your super-duper HPC cluster? Read this: Slow memory allocation due to Transparent Huge Pages (THP)

LinuxAteMyRAM (also as a PDF: Linux disk caching effects)

To examine the behavior of your Linux box disk caching under specific loads, see Linux write cache mystery (PDF).

To understand what is going on, see also The Linux Page Cache and pdflush (PDF) by the same author, Gregory Smith.

Another useful resource is OpenSUSE’s Tuning the Memory Management Subsystem, which nicely explains some of the kernel cache/memory-related configuration options.

Here’s a simple and clear guide for gmail, which also definitely works with other relay hosts. I’ve used it to configure my ISP’s mail relay (they block outgoing port 25) on a Debian Squeeze laptop.

Imagine you need to get a few lines from a group of files with missing identifier mappings. I have a bunch of files with content similar to this one:

ENSRNOG00000018677 1368832_at 25233
ENSRNOG00000002079 1369102_at 25272
ENSRNOG00000043451 25353
ENSRNOG00000001527 1388013_at 25408
ENSRNOG00000007390 1389538_at 25493

In the example above I need ’25353′, which does not have corresponding affy_probeset_id in the 2nd column.

It is clear how to do that:

This outputs a column of required IDs (EntrezGene in this example):

116720
679845
309295
364867
298220
298221
25353

However, I need these IDs as a comma-separated list, not as newline-separated list.

There are several ways to achieve the desired result (only the last pipe commands differ):

These solutions differ in efficiency and (slightly) in output. sed will read all the input into its buffer to replace newlines with other separators, so it might not be best for large files. tr might be the most efficient, but I haven’t tested that. paste will re-use delimiters, so you cannot really get comma-space “, ” separation with it.

Sources: linuxquestions 1 (explains used sed commands), linuxquestions 2, nixcraft.

Just came across Skype Call Recorder – an awesome in its functionality+simplicity tool to record skype calls. Highly recommended!

It worked immediately for me, and default settings are good enough not to bother tweaking. Well, I know that because I did tweak a few to get more nerdiness, but normal people don’t need that.

SCR download page has packages for Ubuntu, Debian/i386, Xandros, RPM-based distributions, Gentoo – and as its free, you can of course just use the fsource, Luke!

At the time of writing, a package for Debian/amd64 was not available, but it is really easy to build one.
Here’s mine: skype-call-recorder-debian_0.8_amd64.deb

In my previous post on CUE sheet support in Linux music players I mentioned DeaDBeeF. Unfortunately, DeaDBeeF is not yet available as a Debian package.

Fortunately, Alexey Smirnov (the author of DeaDBeeF) maintains a github repository deadbeef-debian which has simple instructions on installing DeaDBeeF in Debian (citing with minor edits): Read the rest of this entry »

Recently I finally managed to configure remote (aka off-site) backup for my Debian server. As always, I started with a comparison of existing solutions.

Debian has a number of packages enabling remote (over the network) backup: backupPC, backupninja, backup-manager, dirvish, duplicity, luckybackup, rdiff-backup, and some others.

Read the rest of this entry »