Linux: how to remove trailing ^M (carriage return)

Bogdan — Mon, 30 Mar 2009 09:43:11 +0000

Imagine you have some styles.css transferred from Win machine to Linux via FTP in binary mode instead of ASCII mode; then

cat styles.css | tr -d "\r" > styles-nocarriage.css

will create styles-nocarriage.css with ^M’s removed.

Alternative syntax:

tr -d "\r" < styles.css > styles-nocarriage.css

Most editors have global replace features which allow to get rid of control characters using regular expressions (exact instructions are editor-specific).

For multiple files, try this:

for f
do
mv $f ${f}~ && tr -d "\r" <${f}~ >$f
rm ${f}~
done

Save this shell script as a file (e.g. dos2unix.sh), then do ./dos2unix.sh . This script accepts wildcards (e.g. ./dos2unix.sh *.php), so be careful!

Python: iterate (and read) all files in a directory (folder)

Bogdan — Sun, 12 Aug 2007 11:33:57 +0000

To iterate through all the files within the specified directory (folder), with ability to use wildcards (*, ?, and [ ]-style ranges), use the following code snippet:

import os
import glob
path = 'sequences/'
for infile in glob.glob( os.path.join(path, '*.fasta') ):
print "current file is: " + infile

If you do not need wildcards, then there is a simpler way to list all items in a directory:

import os
path = 'sequences/'
listing = os.listdir(path)
for infile in listing:
print "current file is: " + infile

print was promoted from a statement to a function in Python 3 (use print(infile) instead of print infile).

One should use ‘os.path.join()’ part to make the script cross-platform-portable (different OS use different path separators, and hard-coding path separator would stop the script from executing under a different OS).

Python docs mention that there is also iglob(), which is an iterator and thus working on directories with way too many files it will save memory by returning only single result per iteration, and not the whole list of files – as glob() does.

Autarchy of the Private Cave » files

Linux: how to remove trailing ^M (carriage return)

Python: iterate (and read) all files in a directory (folder)