Autarchy of the Private Cave

Tiny bits of bioinformatics, [web-]programming etc

    • Archives

    • Recent comments

    Compressors galore: pbzip2, lbzip2, plzip, xz, and lrzip tested on a FASTQ file

    28th March 2015

    About 2 years ago I had already reviewed some parallel (and not) compressing utilities, settling at that time on pbzip2 – it scales quasi-linearly with the number of CPUs/cores, stores compressed data in relatively small 900k blocks, is fast, and has good compression ratio. pbzip2 was (and still is) a very good choice.

    Yesterday I got somewhat distracted, and thus found lbzip2 -

    an independent, multi-threaded implementation of bzip2. It is commonly the fastest SMP (and uniprocessor) bzip2 compressor and decompressor

    - as it says in the Debian package description. Is it really “commonly the fastest” one? How does it compare to pbzip2? Should I use lbzip2 instead of pbzip2?

    This minor distraction had grown into a full-scale web-search and comparison, adding to the mix plzip (a parallel version of lzip), xz, and lrzip. After reading thousands of characters, all of these were put to a simple test: compressing an about 2 gigabyte FASTQ file with default options.

    All the external links and benchmarks, as well as my own mini-benchmark results, are provided below.

    The conclusion is that out of all the tested compressors lbzip2 is indeed the best one (for my practical use). It is only slightly better than the trusty pbzip2, which takes the second place. All the other compressors performed so poorly, that they do not get any place in my practical rating…

    So, let us first ask internet wisdom/foolishness, if lbzip2 or pbzip2 is faster/better?
    Read the rest of this entry »

    Posted in *nix, Comparison, Links, Misc, Software | 6 Comments »

    Looking for a perfect Android app for TSW/GTD use

    13th March 2015

    I have recently realized that my planning habits are quite similar to what The Secret Weapon promotes. However, my planning is not as elaborate and detailed/structured as TSW, and I am using several tools:

    • Google Keep, an awesome note-taking and to-do lists application with a really good web-interface, and free;
    • Trello, convenient lists/projects/tasks management platform (especially for group work), and free;
    • Google Calendar, the de facto calendar standard for Android phones, and free;
    • my A5 format weekly paper planner, and… the only not free component.

    It is easy to see that I am using too many tools.

    In an effort to use less tools, and also to try some of the features of TSW, I’ve performed a brief search for GTD/TSW-compatible Android apps.

    TSW website is built around the Evernote app. However, I am not sure if this would be a good solution for me, as I have been already using Evernote since several years for longer-term note-keeping, and thus already have a bunch of notepads, notes, and tags there. Moreover, Evernote’s website mentions something about “offline notes” in the Premium (non-free) tier for mobile apps; this hints at the requirement to have internet connectivity to be able to work with TSW+Evernote efficiently through the day.

    Oh, before I forget: all the 4 tools that I am using have their purpose, with overlap between Keep and Trello.
    My A5 format paper planner (weekview compact 2015) is not a simple weekly planner; it has a structure that stimulates goal-oriented planning.
    More specifically, it provides means to plan:
    Read the rest of this entry »

    Posted in Comparison, Links, Misc, Notepad, Software | No Comments »

    Ukraine: from the Past to the Future (presentation)

    1st March 2015

    Presentation about Ukraine

    • Goal: general introduction of Ukraine.
    • Target audience: German and international students, including PhD students.
    • Duration: approximately 25-30 minutes.
    • Presented: January, 16, 2015, in Canossa, Universität des Saarlandes, Campus, 66123 Saarbrücken, Deutschland, during the Ukrainian Evening (Landerabend Ukraine).
    • Presentation planning group: Dima Panfilenko (Landerabend organizer), Bogdan Tokovenko, Varvara Obolonchykova, Zarema Ibragimova, Ivan Pryvalov.
    • Slides: Dima Panfilenko (initial version of tourism slides and text), Bogdan Tokovenko (all the other slides/text and final tourism slides/text).
    • Useful feedback and criticism: Varvara Obolonchykova.

    Презентація про Україну

    1st slide
    Ukraine: from the Past to the Future (v5 slides, PDF)
    1st slide with notes
    Ukraine: from the Past to the Future (v5 slides with notes and extra information, PDF)

    You can also download slide notes as a text file.
    Original PowerPoint presentation file is available on request.
    Read the rest of this entry »

    Posted in Kyiv, Life, Society, Ukraine | No Comments »

    My smartd.conf, explained

    28th February 2015

    After fixing offline uncorrectable sector warning email, I have taken a closer look at my /etc/smartd.conf, and now it looks like this:

    DEFAULT -d sat -H -f -p -t -W 0,40,45 -n standby -S on -m example@example.com
    # Attributes 1, 230, and 231 are very important (-r 1! -r 230! -R 230! -r 231! -R 231!), but likely covered by -t.
    /dev/sda -s (S/../../6/01|L/../(01|02|03|04|05|06|07)/7/00) -C 0 -I 189 -I 194
    # -a implies -f and -p (through -t)
    DEFAULT -d sat -a -I 194 -W 0,40,45 -n standby -o on -S on -m example@example.com
    /dev/sdb -s (S/../../6/02|L/../(01|02|03|04|05|06|07)/7/02)
    # This drive does not decrement Offline_Uncorrectable (198) after re-allocation,
    # so only monitoring for increase, not for non-zero value.
    /dev/sdc -s (S/../../6/03|L/../(01|02|03|04|05|06|07)/7/04) -U 198+
    # This drive has 40 “normally”.
    /dev/sdd -s (S/../../6/04|L/../(01|02|03|04|05|06|07)/7/06) -W 0,42,45

    Note: explanations below are intentionally simplified; please consult man smartd.conf for more precise, complete, and up-to-date information.

    Ok, so what do these settings mean, and how is this different from default settings?
    Read the rest of this entry »

    Posted in *nix, how-to, Notepad, Software | No Comments »

    How to fix Offline Uncorrectable sector outside of a partition

    26th February 2015

    A few days ago my smartd daemon (from the smartmontools package) notified me about a +1 increase in Current_Pending_Sector (197) and Offline_Uncorrectable (198) SMART attributes. The 2.5″ Fujitsu laptop hard-drive these appeared on is very old, and it also has been working 24/365 since a little over a year.

    Running a short SMART self-test (sudo smartctl -t short /dev/sdc) produced a read error at sector 1289:

    Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
    1 Short offline Completed: read failure 80% 22339 1289

    Looking at the partition table of /dev/sdc, we see that this sector is outside of the only RAID partition on the disk, which starts at sector 2048:

    Device Boot Start End Blocks Id System
    /dev/sdc1 2048 117209087 58605088 fd Lnx RAID auto

    Read the rest of this entry »

    Posted in *nix, how-to, Software | 1 Comment »

    Russia shelled Ukrainians from within its own territory

    20th February 2015

    A recent publication in The Guardian references a high-quality conclusive report of Bellingcat proving that in the summer of 2014 Russia used artillery and multiple rocket launch systems (MRLS) from its own territory to strike Ukrainian Army positions, effectively stopping the successful anti-terrorist campaign, and then overtaking long stretches of Ukrainian border (to enable easy supplies of more weapons, tanks, MRLS, and anti-aircraft systems). Shelling was followed by the full-scale Russian ground forces invasion around the 24th of August, which is Independence Day in Ukraine.

    The Bellingcat shelling report is available in several languages and file formats:

    Posted in Life, Ukraine | No Comments »

    I want a physical QWERTY keyboard Android

    27th October 2014

    My motivation is simple, and Kevin Michaluk had nicely written down the benefits of physical keyboards: Why there’s still demand in 2014 for phones with physical keyboards. For many years I’ve been using Nokia E71, and now, after switching to a screen-only Android 4.4.2 device, I dearly miss the physical keyboard (and the compact, durable metal phone body, too, as well as week-long battery life…).

    I really hope that phone manufacturers offer something that:

    • is about the size of E71, possibly slightly higher/wider/thicker
    • has a physical QWERTY keyboard, either E71-style or a slider
    • runs Android Lollipop :P
    • has decent (but not necessarily the best) smartphone hardware (5-8+MP camera, 1.4+GHzCPU, 1.5+GB RAM, 8+GB internal storage, etc)
    • has a good screen, e.g. HD (~4.3-4.7″) for a slider-keyboard version, or half-HD for Nokia-style keyboard
    • is minimally waterproof (can withstand heavy rain for 30 minutes)

    Reasonable price would also be welcome…

    Posted in Links | No Comments »