Autarchy of the Private Cave » list https://bogdan.org.ua Tiny bits of bioinformatics, [web-]programming etc Wed, 28 Dec 2022 16:09:04 +0000 en-US hourly 1 https://wordpress.org/?v=3.8.27 Python performance: set vs listhttps://bogdan.org.ua/2011/08/15/python-performance-set-vs-list.html https://bogdan.org.ua/2011/08/15/python-performance-set-vs-list.html#comments Mon, 15 Aug 2011 09:29:04 +0000 http://bogdan.org.ua/?p=1673 Sometimes there is a need to be sure that no identifier is processed twice – for example, when parsing a file into a database, with file potentially containing duplicate records. An obvious solution is to properly wrap the DB insertion code into try…except block, and process duplicate primary ID exceptions. Another, sometimes more desired solution is to maintain a set/list of processed IDs internally, and check against that list prior to attempting the insertion of anything. So is it a set or a list?

There are already quite a few internet resources discussing “python set vs list”, but probably the simplest while elegant way to test that is below.

First, test the speed of adding/appending to a set or a list (here, I’m mimicking the real-life application, thus the test case has an optional loop):

  1. $python -mtimeit -s 'myset = set()' 'for x in xrange(1000): myset.add(x)'
  2. 10000 loops, best of 3: 133 usec per loop
  3. $python -mtimeit -s 'tmp = list()' 'for x in xrange(1000): tmp.append(x)'
  4. 10000 loops, best of 3: 116 usec per loop

As we can see, set and list are comparable in the speed of adding new items, with list being slightly (~12%) faster than set.

Now, the speed of membership testing: ‘x in tmp’. For this test, I’ve deliberately chosen an imbalance of True (1%) and False (99%) results for the test – again, mimicking the real problem I have at hand:

  1. $python -mtimeit -s 'tmp = set()' -s 'for x in xrange(1000): tmp.add(x)' 'for x in xrange(100000): x in tmp'
  2. 100 loops, best of 3: 7.27 msec per loop
  3. $python -mtimeit -s 'tmp = list()' -s 'for x in xrange(1000): tmp.append(x)' 'for x in xrange(100000): x in tmp'
  4. 10 loops, best of 3: 2.12 sec per loop

List is much slower for membership testing, while sets were designed to be fast for doing just that.

Share

]]> https://bogdan.org.ua/2011/08/15/python-performance-set-vs-list.html/feed 1 List of free DNS serviceshttps://bogdan.org.ua/2006/10/28/list-of-free-dns-services.html https://bogdan.org.ua/2006/10/28/list-of-free-dns-services.html#comments Sat, 28 Oct 2006 13:25:01 +0000 http://www.bogdan.org.ua/2006/10/28/list-of-free-dns-services.html Information taken from: freedom.org.ru

  1. www.zoneedit.com; 5 names; A, MX, NS; mail forwarding, web forwarding
  2. www.everydns.net; 20 names; A, MX, NS; web forwarding, dynamic domains
  3. www.dnspark.com; 3 names; A, MX, NS; web forwarding, mail forwarding, backup MX
  4. www.domain-dns.com; unlimited names; A, MX; web forwarding, paid mail forwarding
  5. www.xname.org; no limit; A, MX, NS (I keep several domains there; quite good as for a free service)
  6. primaryns.kiev.ua (UA-IX only); unlimited names; A, MX, NS; primary only
  7. secondary.net.ua; unlimited names; secondary only
  8. ns2.trifle.net; unlimited names; secondary only
  9. FreeDNS.ws (suggested in comments; looks promising – I now have one domain there)

Share

]]> https://bogdan.org.ua/2006/10/28/list-of-free-dns-services.html/feed 8