Autarchy of the Private Cave

Tiny bits of bioinformatics, [web-]programming etc

    • Archives

    • Recent comments

    Archive for the '*nix' Category

    rtorrent-enhanced with ipfilter and GeoIP: Debian Squeeze amd64 package

    1st April 2011

    rtorrent-eyecandy
    rtorrent is an excellent ncurses rtorrent client.

    There are a few highly-popular patches, which haven’t yet made it into the Squeeze’s rtorrent version 0.8.6.

    Note: ArchLinux already has an rtorrent-extended package, so you may want to look at that one (instead of my Debian package below). I might use Arch’s patchset next time to generate a truly extended rtorrent package.

    I’ve incorporated 3 of such patches (#239 ipfilter, #2064 GeoIP support, and #2171 colours/eyecandy), and built a squeeze amd64 rtorrent-enhanced package.

    A few excerpts to explain how these extra features are to be configured and used (all come from the above-mentioned trac tickets, some were edited/extended).

    ipfilter

    ipfilter allows to selectively blacklist/whitelist peers – based on IP address range files (so-called ipfilter files). The ipfilter files may, for example, come from bluetack.co.uk. Each line of the file contains a record in this format: range_description:start_IP-end_IP, where start_IP should be less than or equal to end_IP. Multiple files can be used. Overlapping ranges are merged automatically. Both incoming and outgoing connections are checked against the filter. Exclusions are not supported, so connection to/from IPs in all the loaded ranges will be disallowed and dropped.

    Include “ip_filter=” directive in .rtorrent.rc. For example (paths are specified relative to user’s home directory):
    ip_filter=ipfilter/level1,ipfilter/level2

    It is probably a good idea to reload ipfilter files once in a while, so you can also include “reload_ip_filter” directive on schedule to refresh the filter. The same files named in “ip_filter” will be reloaded.

    schedule = filter,18:30:00,24:00:00,reload_ip_filter=

    To support the feature you may want to setup cron job to reload and unzip files from bluetack. Examples here and here.

    GeoIP

    In the Peers view of each torrent a new column – CC, country code – is added, which shows peer’s country code. When examining each peer, you will see more data from the GeoIP database (if you have the relevant files installed): AS num and city. At the very least, you need to have geoip-database installed. You can get free “lite” versions of city/AS-databases, rename them (removing “lite”) and put into an appropriate location to make rtorrent+geoip use them as well. Geop-isp data support isn’t integrated into this rtorrent package.

    Colours

    This patch somewhat changes the way rtorrent shows torrents. Seeding torrents have bold titles, there are no half-displayed torrents at the top/bottom when scrolling, and you can configure colours for active/done torrents (ticket libtorrent.rakshasa.no/ticket/1382, which seems to be gone now). Source code and testing hint at these configurable colours: done_fg_color, done_bg_color, active_fg_color, active_bg_color, and at these possible values: 1 (red), 2 (green), 3 (yellow).

    Geek’s cellar
    A related (though not used in any way for the preparation of the package) resource is rtorrent mods page.

    Relatively schematically, applying patches and building the package was performed in these steps (starting within some newly-created directory):

    1. sudo aptitude install cdbs devscripts [and whatever else you find you're missing]
    2. apt-get source rtorrent
    3. dpkg-source -x rtorrent_0.8.6-1.dsc
    4. cd rtorrent-0.8.6
    5. cdbs-edit-patch 01-ipfilter.patch
    6. patch -p1 < /path/to/patch/239
    7. exit 0
    8. cdbs-edit-patch 02-geoip.patch
    9. patch -p1 < /path/to/patch/2064-after-ipfilter
    10. exit 0
    11. cdbs-edit-patch 03-eyecandy.patch
    12. patch -p1 < /path/to/patch/2171-mod-with-canvas
    13. exit 0 [you could do all 3 patches together, but I prefer cleaner and manageable approaches]
    14. [some weird operation to be explained below]
    15. dpkg-buildpackage

    Weird operation: I didn’t know how to make dpkg-buildpackage run autorun.sh (which is required for one of the patches I’ve used) before configure (could someone enlighten me how to do that?), so I started a new patch with cdbs-edit-patch, then ran one by one `autoreconf -if`, `./autorun.sh`, `./configure`, exited with zero status and was done with that problem. The resulting “patch” was 2MB large :)

    Share

    Posted in *nix, Software | No Comments »

    How to easily install any PyPi/easy_install python module on Debian

    16th February 2011

    Imagine you need to install pycassa (which uses easy_install). Here are the 2 (at maximum) very simple steps to have it properly debianized and installed on your Debian/Ubuntu:

    • if you don’t have the python-stdeb package: sudo aptitude install python-stdeb
    • pypi-install pycassa

    That’s it.

    Refer to stdeb readme for more information. You will need that if there are dependencies – which might not be resolved automatically by stdeb.

    Before stdeb, it wasn’t exactly trivial to make a .deb from python module.

    Share

    Posted in *nix, how-to, Notepad, Python, Software | 1 Comment »

    My favourite command line for mirroring with wget

    9th January 2011

    wget ––continue ––mirror ––page-requisites ––adjust-extension ––convert-links ––backup-converted ––limit-rate=500k ––wait=2 URL

    (short form: wget -cmpEkK ––limit-rate=500k -w 2 URL)
    Read the rest of this entry »

    Share

    Posted in *nix, Notepad | No Comments »

    How to relay outgoing postfix emails via another mail server (e.g. your ISP)

    4th December 2010

    Here’s a simple and clear guide for gmail, which also definitely works with other relay hosts. I’ve used it to configure my ISP’s mail relay (they block outgoing port 25) on a Debian Squeeze laptop.

    Share

    Posted in *nix, how-to, Links, Notepad, Software | No Comments »

    How to replace newlines with commas, tabs etc (merge lines)

    16th November 2010

    Imagine you need to get a few lines from a group of files with missing identifier mappings. I have a bunch of files with content similar to this one:

    ENSRNOG00000018677 1368832_at 25233
    ENSRNOG00000002079 1369102_at 25272
    ENSRNOG00000043451 25353
    ENSRNOG00000001527 1388013_at 25408
    ENSRNOG00000007390 1389538_at 25493

    In the example above I need ’25353′, which does not have corresponding affy_probeset_id in the 2nd column.

    It is clear how to do that:

    1. sort -u *_affy_ensembl.txt | grep -v '_at' | awk '{print $2}'

    This outputs a column of required IDs (EntrezGene in this example):

    116720
    679845
    309295
    364867
    298220
    298221
    25353

    However, I need these IDs as a comma-separated list, not as newline-separated list.

    There are several ways to achieve the desired result (only the last pipe commands differ):

    1. sort -u *_affy_ensembl.txt | grep -v '_at' | awk '{print $2}' | gawk '$1=$1' ORS=', '
    1. sort -u *_affy_ensembl.txt | grep -v '_at' | awk '{print $2}' | tr '\n' ','
    1. sort -u *_affy_ensembl.txt | grep -v '_at' | awk '{print $2}' | sed ':a;N;$!ba;s/\n/, /g'
    1. sort -u *_affy_ensembl.txt | grep -v '_at' | awk '{print $2}' | sed ':q;N;s/\n/, /g;t q'
    1. sort -u *_affy_ensembl.txt | grep -v '_at' | awk '{print $2}' | paste -s -d ","

    These solutions differ in efficiency and (slightly) in output. sed will read all the input into its buffer to replace newlines with other separators, so it might not be best for large files. tr might be the most efficient, but I haven’t tested that. paste will re-use delimiters, so you cannot really get comma-space “, ” separation with it.

    Sources: linuxquestions 1 (explains used sed commands), linuxquestions 2, nixcraft.

    Share

    Posted in *nix, Bioinformatics, how-to, Notepad, Software | 2 Comments »

    How to record Skype calls on Linux: use free Skype Call Recorder

    11th November 2010

    Just came across Skype Call Recorder – an awesome in its functionality+simplicity tool to record skype calls. Highly recommended!

    It worked immediately for me, and default settings are good enough not to bother tweaking. Well, I know that because I did tweak a few to get more nerdiness, but normal people don’t need that.

    SCR download page has packages for Ubuntu, Debian/i386, Xandros, RPM-based distributions, Gentoo – and as its free, you can of course just use the fsource, Luke!

    At the time of writing, a package for Debian/amd64 was not available, but it is really easy to build one.
    Here’s mine: skype-call-recorder-debian_0.8_amd64.deb

    Share

    Posted in *nix, Links, Software | 1 Comment »

    Linux server remote backup options

    3rd October 2010

    Recently I finally managed to configure remote (aka off-site) backup for my Debian server. As always, I started with a comparison of existing solutions.

    Debian has a number of packages enabling remote (over the network) backup: backupPC, backupninja, backup-manager, dirvish, duplicity, luckybackup, rdiff-backup, and some others.

    Read the rest of this entry »

    Share

    Posted in *nix, Links, Software | 1 Comment »