Autarchy of the Private Cave

Tiny bits of bioinformatics, [web-]programming etc

    • Archives

    • Recent comments

    Slow memory allocation due to Transparent Huge Pages (THP)

    6th August 2018

    Your software needs tons of RAM, and runs a bit too slow on your super-duper HPC cluster? Read this: Slow memory allocation due to Transparent Huge Pages (THP)

    Share

    Posted in *nix, Links | No Comments »

    Hands-on examination of Linux disk caching effects

    8th July 2013

    LinuxAteMyRAM :) (also as a PDF: Linux disk caching effects)

    To examine the behavior of your Linux box disk caching under specific loads, see Linux write cache mystery (PDF).

    To understand what is going on, see also The Linux Page Cache and pdflush (PDF) by the same author, Gregory Smith.

    Another useful resource is OpenSUSE’s Tuning the Memory Management Subsystem, which nicely explains some of the kernel cache/memory-related configuration options.

    Share

    Posted in *nix, Software | No Comments »

    How to relay outgoing postfix emails via another mail server (e.g. your ISP)

    4th December 2010

    Here’s a simple and clear guide for gmail, which also definitely works with other relay hosts. I’ve used it to configure my ISP’s mail relay (they block outgoing port 25) on a Debian Squeeze laptop.

    Share

    Posted in *nix, how-to, Links, Notepad, Software | No Comments »

    How to replace newlines with commas, tabs etc (merge lines)

    16th November 2010

    Imagine you need to get a few lines from a group of files with missing identifier mappings. I have a bunch of files with content similar to this one:

    ENSRNOG00000018677 1368832_at 25233
    ENSRNOG00000002079 1369102_at 25272
    ENSRNOG00000043451 25353
    ENSRNOG00000001527 1388013_at 25408
    ENSRNOG00000007390 1389538_at 25493

    In the example above I need ’25353′, which does not have corresponding affy_probeset_id in the 2nd column.

    It is clear how to do that:

    1. sort -u *_affy_ensembl.txt | grep -v '_at' | awk '{print $2}'

    This outputs a column of required IDs (EntrezGene in this example):

    116720
    679845
    309295
    364867
    298220
    298221
    25353

    However, I need these IDs as a comma-separated list, not as newline-separated list.

    There are several ways to achieve the desired result (only the last pipe commands differ):

    1. sort -u *_affy_ensembl.txt | grep -v '_at' | awk '{print $2}' | gawk '$1=$1' ORS=', '
    1. sort -u *_affy_ensembl.txt | grep -v '_at' | awk '{print $2}' | tr '\n' ','
    1. sort -u *_affy_ensembl.txt | grep -v '_at' | awk '{print $2}' | sed ':a;N;$!ba;s/\n/, /g'
    1. sort -u *_affy_ensembl.txt | grep -v '_at' | awk '{print $2}' | sed ':q;N;s/\n/, /g;t q'
    1. sort -u *_affy_ensembl.txt | grep -v '_at' | awk '{print $2}' | paste -s -d ","

    These solutions differ in efficiency and (slightly) in output. sed will read all the input into its buffer to replace newlines with other separators, so it might not be best for large files. tr might be the most efficient, but I haven’t tested that. paste will re-use delimiters, so you cannot really get comma-space “, ” separation with it.

    Sources: linuxquestions 1 (explains used sed commands), linuxquestions 2, nixcraft.

    Share

    Posted in *nix, Bioinformatics, how-to, Notepad, Software | 2 Comments »

    How to record Skype calls on Linux: use free Skype Call Recorder

    11th November 2010

    Just came across Skype Call Recorder – an awesome in its functionality+simplicity tool to record skype calls. Highly recommended!

    It worked immediately for me, and default settings are good enough not to bother tweaking. Well, I know that because I did tweak a few to get more nerdiness, but normal people don’t need that.

    SCR download page has packages for Ubuntu, Debian/i386, Xandros, RPM-based distributions, Gentoo – and as its free, you can of course just use the fsource, Luke!

    At the time of writing, a package for Debian/amd64 was not available, but it is really easy to build one.
    Here’s mine: skype-call-recorder-debian_0.8_amd64.deb

    Share

    Posted in *nix, Links, Software | 1 Comment »

    Installing DeadBeef music player on Debian

    16th October 2010

    In my previous post on CUE sheet support in Linux music players I mentioned DeaDBeeF. Unfortunately, DeaDBeeF is not yet available as a Debian package.

    Fortunately, Alexey Smirnov (the author of DeaDBeeF) maintains a github repository deadbeef-debian which has simple instructions on installing DeaDBeeF in Debian (citing with minor edits): Read the rest of this entry »

    Share

    Posted in Links, Software | 3 Comments »

    Linux server remote backup options

    3rd October 2010

    Recently I finally managed to configure remote (aka off-site) backup for my Debian server. As always, I started with a comparison of existing solutions.

    Debian has a number of packages enabling remote (over the network) backup: backupPC, backupninja, backup-manager, dirvish, duplicity, luckybackup, rdiff-backup, and some others.

    Read the rest of this entry »

    Share

    Posted in *nix, Links, Software | 1 Comment »