Autarchy of the Private Cave

Tiny bits of bioinformatics, [web-]programming etc

    • Archives

    • Recent comments

    How to: easily add swap partition to a live system on btrfs

    14th April 2016

    Recently I had a need to add a swap file to my Debian installation.
    However, I am now using btrfs, and – as with any other COW filesystem – it is not possible to simply create a swap file and use it.
    There are workarounds (creating a file with a COW attribute removed, and then loop-mounting it), but I just did not like them.

    So I have decided to add a swap partition.
    It worked amazingly (and very easily), there was even no need to reboot – at all.
    I still did restart, just to make sure the system is bootable – and all was perfectly fine.

    My initial setup is very simple: a single /dev/sda1 partition on the /dev/sda disk, fully used by btrfs.
    Different important paths/mountpoints are btrfs subvolumes, using flat hierarchy.
    For this example, let us assume that /dev/sda (and /dev/sda1) is 25GB large, and that I want to add a 2GB swap /dev/sda2 after /dev/sda1.

    Brief explanation before we start:

    1. shrink btrfs filesystem by more than 2GB;
    2. shrink btrfs partition by 2GB;
    3. create new 2GB partition for the swap;
    4. resize btrfs filesystem to full size of its new-size partition;
    5. initialize swap and turn it on.

    Here are the very easy steps! Just make sure you do not make mistakes anywhere ;)
    Read the rest of this entry »

    Posted in *nix, how-to | No Comments »

    Yandex probing for vulnerabilities in .UA domains?

    11th April 2016

    Here is a recent entry from my web-server’s access log: – - [09/Apr/2016:15:53:22 +0300] “GET /categories/programming?_SERVER[DOCUMENT_ROOT]= HTTP/1.1″ 200 13158 “-” “Mozilla/5.0 (compatible; YandexBot/3.0; +”

    Client’s IP does belong to Yandex network range.


    • Had Yandex started looking for vulnerabilities in the web-sites it scans?
    • Does it only look for vulnerabilities in the .UA web-sites/domains?
    • Does Yandex really use a Korean web-site to host malicious code?

    In fact, there are more entries like that one, also from one of Yandex IPs: – - [04/Apr/2016:00:14:22 +0300] “GET /categories/programming/page/5? HTTP/1.1″ 200 12607 “-” “Mozilla/5.0 (compatible; YandexBot/3.0; +” – - [04/Apr/2016:00:19:31 +0300] “GET /categories/programming/page/4? HTTP/1.1″ 200 12174 “-” “Mozilla/5.0 (compatible; YandexBot/3.0; +”

    I can see 3 explanations, and all of them are bad for Yandex:

    • Yandex now belongs to KGB, and it does scan [.UA] web-sites for vulnerabilities;
    • some/many of Yandex crawler servers are compromised, and are used by malicious 3rd parties;
    • there was a public malicious link somewhere (???) to my blog, and Yandex blindly followed it.

    Posted in Misc, Web | No Comments »

    Choosing a budget heart-rate monitor for an amateur/casual runner

    26th March 2016

    Beurer PM 25Sigma PC 25.10Polar FT60FPolar M400

    Since 2014 I am participating in a local 5km mass-run (with over 15k participants). My time is between nothing special and fairly good. I will run this year as well.

    Previously, I did not train systematically. I would start only when it was comfortably warm (mid-late April), running 4-6km 1-3 times a week.
    I would also stop training very soon after the run, and definitely stop if it was getting colder (early October).
    After every winter, it actually felt like I am starting my training from scratch.

    Eventually, I have adopted an 18-session training regimen, which I found to be very easy for me, and also very efficient.
    I had also tried to keep training for longer after the run (which is in summer).

    This year, thanks to highly-enthusiastic co-workers, we started light jogging already in January.
    One of my co-workers has a nearly-full set of gadgets, including a heart-rate monitor, and was keeping his pulse under a certain threshold.
    I have only used a very basic Xiaomi Mi Band fitness/sleep tracker before, and got interested in a more quantified self.
    After some reading, I have decided to buy a heart-rate monitor.

    Here are my requirements for a heart-rate monitor:

    • should measure heart-rate well; based on some reading, I’d prefer to have a chest strap: optical pulse measuring is a tad less precise, it may work worse with hairy (man’s) hands, and (for now) I only plan measuring my heart rate when I’m actually running/bicycling, not all day long;
    • should be convenient to use – a wrist display is fine, but carrying a smartphone during a run isn’t;
    • should not be expensive: I do not know yet how useful it will be to me, so I do not want to spend more than 150 EUR on it, so the likes of Polar V800 are, unfortunately, outside the scope of my comparison…

    Read the rest of this entry »

    Posted in Notepad, Welfare | No Comments »

    How to: export only notes to PDF from LibreOffice Impress 5

    28th February 2016

    If you want to export Notes to a PDF from LibreOffice Impress 5,
    and dutifully set the appropriate checkbox in PDF export dialog,
    then you will get all slides twice: first just all the slides as with usual PDF export, and then all the Notes pages.

    There is an easy solution to get Notes-only without editing the PDF.

    If you have a PDf printer installed (most Linux distributions, and Windows 10), just do File -> Print from Impress,
    then under the Print sub-header choose Notes from the Document drop-down (see picture).
    Make sure to set the proper paper format for the PDF printer (A4 in my case).
    Then print, and save the resulting PDF.
    Read the rest of this entry »

    Posted in how-to, Notepad, Software | No Comments »

    Preprint servers and open journals

    28th February 2016

    Let’s start with some definitions.

    With Open Journals I’m referring to open/public peer-review journals.
    With preprint servers, I’m referring to services which allow you to publish your manuscript with a DOI, for pre-submission interest and feedback collection.

    I am aware of the following public peer-review journals:

    • F1000 Research: your submission is made public without any editorial pre-screening within an average of 7 days, but only gets indexed in PubMed/Scopus/Scholar after a successful public peer review. Public means that a reviewer-signed evaluation appears together with the submitted manuscript. Authors may respond to criticism, and upload revisions of their submission. I believe a submission passes peer review after two positive reviews. Note that even your initial submission receives a DOI, and is thus citable (as well as all subsequent revisions). Brief examination of articles in some of the topics tells me that F1000 Research is a good place to publish, esp. because it is a kind of pre-print + journal in one package. You pay per-submission, there are 3 tiers by word count.
    • The Winnower: submit-review-revise, but here you pay for the DOI after your submission is reviewed. Before review your submission is thus not citable (except for by URL, which isn’t tracked as easily as DOI references). I haven’t formed an opinion on how attractive the winnower is for submitting, but I did find this quite interesting story for you to enjoy :)
    • Science Open: this project encompasses 5 mostly medical journals. It lists over 11 million articles on the front page, but those are sourced from other publications; Science Open itself seems to have several hundred publications across all 5 journals. Submissions get a DOI, then can undergo public review. It is not clear to me in which direction Science Open will be moving – towards becoming an excellent research papers aggregator, or towards becoming a publishing platform, or – like now – towards both.

    I’m also aware of the following preprint servers:
    Read the rest of this entry »

    Posted in Science | No Comments »

    How to use mkfifo named pipes with

    24th February 2016 is a utility written in Perl for preprocessing NGS reads, also in FASTQ format.
    It can read sequences both from files and from stdin (if you only have 1 sequence).

    I wanted to use it with compressed (gzipped/bzipped2) FASTQ input files.
    As I do not need to store decompressed input files, the most efficient solution is to use pipes.
    This works well for a single file, but not for 2 files (paired-end reads).

    For 2 files, named pipes (also known as FIFOs) can be used.
    You can create a named pipe in Linux with the help of mkfifo command, for example mkfifo R1_decompressed.fastq.
    To use it, start decompressing something into it (either in a different terminal, or in background), for example zcat R1.fastq.gz > R1_decompressed.fastq &;
    we can call this a writing/generating process, because it writes into a pipe.
    (If you are writing software to use named pipes, any processes writing into them should be started in a new thread, as they will block until all the data is consumed.)
    Now if you give the R1_decompressed.fastq as a file argument to some other program, it will see decompressed content (e.g. wc -l R1_decompressed.fastq will tell you the number of lines in the decompressed file); we can call program reading from the named pipe a reading/consuming process.
    As soon as a consuming process had consumed (read) all of the data, the writing/generating process will finally exit.

    This, however, does not work with (version 0.20.4 or earlier), with a broken pipe error. Read the rest of this entry »

    Posted in *nix, Bioinformatics, Software | No Comments »

    ZFS is the FS for Containers in Ubuntu 16.04 and how it compares to btrfs

    20th February 2016

    Recently in hacker news the following was posted: ZFS is the FS for Containers in Ubuntu 16.04.

    I must admit the 16.04 demo does look very pleasant to work with.

    However, bringing in ZFS into Linux reminded me of a fairly recent comparison of ZFS and btrfs that I had to do when building my home NAS.
    At that time, few months ago, I’ve arrived (among others) at the following conclusions:

    • ZFS on FreeBSD is reliable, though a memory hog;
    • on Debian, OpenVault seems to be a good NAS web-management interface;
    • on FreeBSD, FreeNAS is good (there is also Nas4Free fork of an older version, but I haven’t looked into it deep enough);
    • running ZFS on linux (even as a kernel module) is the least efficient solution, at least partially because kernel’s file caching and ZFS’s ARC cache are two separate entities;
    • although btrfs offers features very similar to ZFS, as of few months ago OpenVault did not offer btrfs volumes support from the web-interface.

    In the end, I’ve decided to go with FreeNAS, and it seems to work well so far.

    But had anything changed in the btrfs vs ZFS on Linux field?
    Read the rest of this entry »

    Posted in *nix | No Comments »