<?xml version="1.0" encoding="UTF-8"?> <rss
version="2.0"
xmlns:content="http://purl.org/rss/1.0/modules/content/"
xmlns:wfw="http://wellformedweb.org/CommentAPI/"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:atom="http://www.w3.org/2005/Atom"
xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
> <channel><title>Autarchy of the Private Cave &#187; performance</title> <atom:link href="https://bogdan.org.ua/tags/performance/feed" rel="self" type="application/rss+xml" /><link>https://bogdan.org.ua</link> <description>Tiny bits of bioinformatics, [web-]programming etc</description> <lastBuildDate>Wed, 28 Dec 2022 16:09:04 +0000</lastBuildDate> <language>en-US</language> <sy:updatePeriod>hourly</sy:updatePeriod> <sy:updateFrequency>1</sy:updateFrequency> <generator>https://wordpress.org/?v=3.8.27</generator> <item><title>Python performance: set vs list</title><link>https://bogdan.org.ua/2011/08/15/python-performance-set-vs-list.html</link> <comments>https://bogdan.org.ua/2011/08/15/python-performance-set-vs-list.html#comments</comments> <pubDate>Mon, 15 Aug 2011 09:29:04 +0000</pubDate> <dc:creator><![CDATA[Bogdan]]></dc:creator> <category><![CDATA[Notepad]]></category> <category><![CDATA[Programming]]></category> <category><![CDATA[Python]]></category> <category><![CDATA[list]]></category> <category><![CDATA[membership]]></category> <category><![CDATA[performance]]></category> <category><![CDATA[set]]></category> <guid
isPermaLink="false">http://bogdan.org.ua/?p=1673</guid> <description><![CDATA[Sometimes there is a need to be sure that no identifier is processed twice &#8211; for example, when parsing a file into a database, with file potentially containing duplicate records. An obvious solution is to properly wrap the DB insertion code into try&#8230;except block, and process duplicate primary ID exceptions. Another, sometimes more desired solution [&#8230;]]]></description> <content:encoded><![CDATA[<p>Sometimes there is a need to be sure that no identifier is processed twice &#8211; for example, when parsing a file into a database, with file potentially containing duplicate records. An obvious solution is to properly wrap the DB insertion code into try&#8230;except block, and process <em>duplicate primary ID</em> exceptions. Another, sometimes more desired solution is to maintain a set/list of processed IDs internally, and check against that list prior to attempting the insertion of anything. So is it a set or a list?</p><p>There are already quite a few internet resources discussing &#8220;python set vs list&#8221;, but probably the simplest while elegant way to test that is below.<br
/> <span
id="more-1673"></span><br
/> First, test the speed of adding/appending to a set or a list (here, I&#8217;m mimicking the real-life application, thus the test case has an optional loop):</p><div
id="ig-sh-1" class="syntax_hilite"><div
class="code"><ol
class="code" style="font-family:monospace;"><li
style="font-weight: normal; vertical-align:top;"><div
style="font: normal normal 1em/1.2em monospace; margin:0; padding:0; background:none; vertical-align:top;">$python -mtimeit -s 'myset = set()' 'for x in xrange(1000): myset.add(x)'</div></li><li
style="font-weight: normal; vertical-align:top;"><div
style="font: normal normal 1em/1.2em monospace; margin:0; padding:0; background:none; vertical-align:top;">10000 loops, best of 3: 133 usec per loop</div></li><li
style="font-weight: normal; vertical-align:top;"><div
style="font: normal normal 1em/1.2em monospace; margin:0; padding:0; background:none; vertical-align:top;">$python -mtimeit -s 'tmp = list()' 'for x in xrange(1000): tmp.append(x)'</div></li><li
style="font-weight: normal; vertical-align:top;"><div
style="font: normal normal 1em/1.2em monospace; margin:0; padding:0; background:none; vertical-align:top;">10000 loops, best of 3: 116 usec per loop</div></li></ol></div></div><p>As we can see, set and list are comparable in the speed of adding new items, with list being slightly (~12%) faster than set.</p><p>Now, the speed of membership testing: &#8216;x in tmp&#8217;. For this test, I&#8217;ve deliberately chosen an imbalance of True (1%) and False (99%) results for the test &#8211; again, mimicking the real problem I have at hand:</p><div
id="ig-sh-2" class="syntax_hilite"><div
class="code"><ol
class="code" style="font-family:monospace;"><li
style="font-weight: normal; vertical-align:top;"><div
style="font: normal normal 1em/1.2em monospace; margin:0; padding:0; background:none; vertical-align:top;">$python -mtimeit -s 'tmp = set()' -s 'for x in xrange(1000): tmp.add(x)' 'for x in xrange(100000): x in tmp'</div></li><li
style="font-weight: normal; vertical-align:top;"><div
style="font: normal normal 1em/1.2em monospace; margin:0; padding:0; background:none; vertical-align:top;">100 loops, best of 3: 7.27 msec per loop</div></li><li
style="font-weight: normal; vertical-align:top;"><div
style="font: normal normal 1em/1.2em monospace; margin:0; padding:0; background:none; vertical-align:top;">$python -mtimeit -s 'tmp = list()' -s 'for x in xrange(1000): tmp.append(x)' 'for x in xrange(100000): x in tmp'</div></li><li
style="font-weight: normal; vertical-align:top;"><div
style="font: normal normal 1em/1.2em monospace; margin:0; padding:0; background:none; vertical-align:top;">10 loops, best of 3: 2.12 sec per loop</div></li></ol></div></div><p>List is much slower for membership testing, while <a
href="http://en.wikipedia.org/wiki/Collection_(computing)#Sets">sets were designed to be fast for doing just that</a>.</p><p><a
class="a2a_button_citeulike" href="https://www.addtoany.com/add_to/citeulike?linkurl=https%3A%2F%2Fbogdan.org.ua%2F2011%2F08%2F15%2Fpython-performance-set-vs-list.html&amp;linkname=Python%20performance%3A%20set%20vs%20list" title="CiteULike" rel="nofollow noopener" target="_blank"></a><a
class="a2a_button_pocket" href="https://www.addtoany.com/add_to/pocket?linkurl=https%3A%2F%2Fbogdan.org.ua%2F2011%2F08%2F15%2Fpython-performance-set-vs-list.html&amp;linkname=Python%20performance%3A%20set%20vs%20list" title="Pocket" rel="nofollow noopener" target="_blank"></a><a
class="a2a_button_kindle_it" href="https://www.addtoany.com/add_to/kindle_it?linkurl=https%3A%2F%2Fbogdan.org.ua%2F2011%2F08%2F15%2Fpython-performance-set-vs-list.html&amp;linkname=Python%20performance%3A%20set%20vs%20list" title="Kindle It" rel="nofollow noopener" target="_blank"></a><a
class="a2a_button_evernote" href="https://www.addtoany.com/add_to/evernote?linkurl=https%3A%2F%2Fbogdan.org.ua%2F2011%2F08%2F15%2Fpython-performance-set-vs-list.html&amp;linkname=Python%20performance%3A%20set%20vs%20list" title="Evernote" rel="nofollow noopener" target="_blank"></a><a
class="a2a_button_pinterest" href="https://www.addtoany.com/add_to/pinterest?linkurl=https%3A%2F%2Fbogdan.org.ua%2F2011%2F08%2F15%2Fpython-performance-set-vs-list.html&amp;linkname=Python%20performance%3A%20set%20vs%20list" title="Pinterest" rel="nofollow noopener" target="_blank"></a><a
class="a2a_dd addtoany_share_save addtoany_share" href="https://www.addtoany.com/share#url=https%3A%2F%2Fbogdan.org.ua%2F2011%2F08%2F15%2Fpython-performance-set-vs-list.html&#038;title=Python%20performance%3A%20set%20vs%20list" data-a2a-url="https://bogdan.org.ua/2011/08/15/python-performance-set-vs-list.html" data-a2a-title="Python performance: set vs list"><img
src="https://static.addtoany.com/buttons/share_save_120_16.png" alt="Share"></a></p>]]></content:encoded> <wfw:commentRss>https://bogdan.org.ua/2011/08/15/python-performance-set-vs-list.html/feed</wfw:commentRss> <slash:comments>1</slash:comments> </item> <item><title>Intel i915 integrated graphics under Debian: how to get rid of sluggish 2D performance</title><link>https://bogdan.org.ua/2009/03/02/intel-i915-integrated-graphics-under-debian-how-to-get-rid-of-sluggish-2d-performance.html</link> <comments>https://bogdan.org.ua/2009/03/02/intel-i915-integrated-graphics-under-debian-how-to-get-rid-of-sluggish-2d-performance.html#comments</comments> <pubDate>Mon, 02 Mar 2009 20:41:03 +0000</pubDate> <dc:creator><![CDATA[Bogdan]]></dc:creator> <category><![CDATA[*nix]]></category> <category><![CDATA[Hardware]]></category> <category><![CDATA[how-to]]></category> <category><![CDATA[Debian]]></category> <category><![CDATA[driver]]></category> <category><![CDATA[i915]]></category> <category><![CDATA[performance]]></category> <category><![CDATA[slow]]></category> <category><![CDATA[video]]></category> <guid
isPermaLink="false">http://bogdan.org.ua/?p=623</guid> <description><![CDATA[I assume you already have configured and working desktop environment, but want to improve performance. First of all, sudo aptitude install mesa-utils. Then run in a Terminal/Konsole glxgears, and wait for ~15 seconds; if your FPS is ~400 or less, then you do have sluggish video performance (usually manifesting itself as slow scrolling in Firefox/Iceweasel, [&#8230;]]]></description> <content:encoded><![CDATA[<p>I assume you already have configured and working desktop environment, but want to improve performance.</p><p>First of all, <strong>sudo aptitude install mesa-utils</strong>. Then run in a Terminal/Konsole <strong>glxgears</strong>, and wait for ~15 seconds; if your FPS is ~400 or less, then you do have sluggish video performance (usually manifesting itself as slow scrolling in Firefox/Iceweasel, slow window switching/minimziing/maximizing etc).</p><p>After reading through several forums and bug reports and blog posts, I&#8217;ve ended with the following modifications to my <strong>/etc/X11/xorg.conf</strong>:<br
/> <span
id="more-623"></span></p><ol><li>backup your current xorg.conf: <strong>sudo cp /etc/X11/xorg.conf /etc/X11/xorg.conf.before-exa</strong></li><li>open your /etc/X11/xorg.conf with a favourite editor; I&#8217;ve added these lines to section &#8220;module&#8221;:<br
/><blockquote><p> Load &#8220;dbe&#8221;<br
/> Load &#8220;xtrap&#8221;<br
/> Load &#8220;record&#8221;<br
/> Load &#8220;GLcore&#8221;</p></blockquote><p>These are <strong>not</strong> really <strong>performance-related</strong> and could be skipped; I&#8217;m giving them here, because they were suggested as default entries by dexconf or some other xorg.conf-generator utility. Again &#8211; they are not supposed to improve performance, so you can skip these.</li><li>my &#8220;Device&#8221; section is now:<br
/><blockquote><p> Section &#8220;Device&#8221;<br
/> Identifier	&#8220;Intel Corporation 82915G/GV/910GL Integrated Graphics Controller&#8221;<br
/> BoardName	&#8220;82915G/GV/910GL Integrated Graphics Controller&#8221;<br
/> Vendorname	&#8220;Intel Corporation&#8221;<br
/> #Driver		&#8220;i810&#8243;<br
/> Driver		&#8220;intel&#8221;<br
/> BusID		&#8220;PCI:0:2:0&#8243;<br
/> Option	&#8220;DRI&#8221;	&#8220;true&#8221;<br
/> Option	&#8220;AccelMethod&#8221;	&#8220;exa&#8221;<br
/> Option	&#8220;MigrationHeuristic&#8221;	&#8220;greedy&#8221;<br
/> Option	&#8220;ExaNoComposite&#8221;	&#8220;false&#8221;<br
/> EndSection</p></blockquote><p><strong>Driver i810</strong> (xserver-xorg-video-i810 package) was about 10-20 FPS slower than <strong>Driver intel</strong> (xserver-xorg-video-intel package) for me. The most important lines here are <strong>Option AccelMethod exa</strong> (which enables EXA acceleration instead of the older XAA; if you run 2.6.28 or later kernel, you can try UXA instead of EXA) and <strong>Option MigrationHeuristic greedy</strong>. <strong>Option ExaNoComposite false</strong> doesn&#8217;t make a difference for me, and <strong>Option DRI true</strong> might be redundant (didn&#8217;t bother trying to remove it).</li><li>at the end of xorg.conf I have two more sections:<br
/><blockquote><p> Section &#8220;DRI&#8221;<br
/> Mode	0666<br
/> EndSection</p><p>Section &#8220;Extensions&#8221;<br
/> Option	&#8220;Composite&#8221;	&#8220;enable&#8221;<br
/> Option	&#8220;MIT-SHM&#8221;	&#8220;Yes&#8221;<br
/> EndSection</p></blockquote></li><li><strong>sudo nano /etc/environment</strong>, and add one line: <strong>INTEL_BATCH=1</strong> (if you have graphics stability issues or screen corruption &#8211; try removing this change first)</li><li>you can test if your xorg.conf is syntactically correct by running X -config /etc/X11/xorg.conf (provided that you edited this exact file)</ol><p>This is it. Before these modifications, my FPS in glxgears was around 430; after these modifications it is ~915.</p><p>As time permits, I might try the XAA configuration, suggested at <a
href="http://foolcontrol.org/?p=181">foolcontrol</a>.</p><p><ins
datetime="2009-04-13T09:53:47+00:00">Update</ins>: with compiz enabled, XAA configuration recommended at <a
href="http://foolcontrol.org/?p=181">foolcontrol</a> is definitely faster in 2D.</p><p>Improvements, suggestions, corrections are welcome.</p><p><a
class="a2a_button_citeulike" href="https://www.addtoany.com/add_to/citeulike?linkurl=https%3A%2F%2Fbogdan.org.ua%2F2009%2F03%2F02%2Fintel-i915-integrated-graphics-under-debian-how-to-get-rid-of-sluggish-2d-performance.html&amp;linkname=Intel%20i915%20integrated%20graphics%20under%20Debian%3A%20how%20to%20get%20rid%20of%20sluggish%202D%20performance" title="CiteULike" rel="nofollow noopener" target="_blank"></a><a
class="a2a_button_pocket" href="https://www.addtoany.com/add_to/pocket?linkurl=https%3A%2F%2Fbogdan.org.ua%2F2009%2F03%2F02%2Fintel-i915-integrated-graphics-under-debian-how-to-get-rid-of-sluggish-2d-performance.html&amp;linkname=Intel%20i915%20integrated%20graphics%20under%20Debian%3A%20how%20to%20get%20rid%20of%20sluggish%202D%20performance" title="Pocket" rel="nofollow noopener" target="_blank"></a><a
class="a2a_button_kindle_it" href="https://www.addtoany.com/add_to/kindle_it?linkurl=https%3A%2F%2Fbogdan.org.ua%2F2009%2F03%2F02%2Fintel-i915-integrated-graphics-under-debian-how-to-get-rid-of-sluggish-2d-performance.html&amp;linkname=Intel%20i915%20integrated%20graphics%20under%20Debian%3A%20how%20to%20get%20rid%20of%20sluggish%202D%20performance" title="Kindle It" rel="nofollow noopener" target="_blank"></a><a
class="a2a_button_evernote" href="https://www.addtoany.com/add_to/evernote?linkurl=https%3A%2F%2Fbogdan.org.ua%2F2009%2F03%2F02%2Fintel-i915-integrated-graphics-under-debian-how-to-get-rid-of-sluggish-2d-performance.html&amp;linkname=Intel%20i915%20integrated%20graphics%20under%20Debian%3A%20how%20to%20get%20rid%20of%20sluggish%202D%20performance" title="Evernote" rel="nofollow noopener" target="_blank"></a><a
class="a2a_button_pinterest" href="https://www.addtoany.com/add_to/pinterest?linkurl=https%3A%2F%2Fbogdan.org.ua%2F2009%2F03%2F02%2Fintel-i915-integrated-graphics-under-debian-how-to-get-rid-of-sluggish-2d-performance.html&amp;linkname=Intel%20i915%20integrated%20graphics%20under%20Debian%3A%20how%20to%20get%20rid%20of%20sluggish%202D%20performance" title="Pinterest" rel="nofollow noopener" target="_blank"></a><a
class="a2a_dd addtoany_share_save addtoany_share" href="https://www.addtoany.com/share#url=https%3A%2F%2Fbogdan.org.ua%2F2009%2F03%2F02%2Fintel-i915-integrated-graphics-under-debian-how-to-get-rid-of-sluggish-2d-performance.html&#038;title=Intel%20i915%20integrated%20graphics%20under%20Debian%3A%20how%20to%20get%20rid%20of%20sluggish%202D%20performance" data-a2a-url="https://bogdan.org.ua/2009/03/02/intel-i915-integrated-graphics-under-debian-how-to-get-rid-of-sluggish-2d-performance.html" data-a2a-title="Intel i915 integrated graphics under Debian: how to get rid of sluggish 2D performance"><img
src="https://static.addtoany.com/buttons/share_save_120_16.png" alt="Share"></a></p>]]></content:encoded> <wfw:commentRss>https://bogdan.org.ua/2009/03/02/intel-i915-integrated-graphics-under-debian-how-to-get-rid-of-sluggish-2d-performance.html/feed</wfw:commentRss> <slash:comments>5</slash:comments> </item> <item><title>How to improve MySQL application performance</title><link>https://bogdan.org.ua/2008/03/06/how-to-improve-mysql-application-perfromance-performance.html</link> <comments>https://bogdan.org.ua/2008/03/06/how-to-improve-mysql-application-perfromance-performance.html#comments</comments> <pubDate>Thu, 06 Mar 2008 14:48:43 +0000</pubDate> <dc:creator><![CDATA[Bogdan]]></dc:creator> <category><![CDATA[Links]]></category> <category><![CDATA[Programming]]></category> <category><![CDATA[Web]]></category> <category><![CDATA[application]]></category> <category><![CDATA[developer]]></category> <category><![CDATA[hints]]></category> <category><![CDATA[mysql]]></category> <category><![CDATA[performance]]></category> <category><![CDATA[presentation]]></category> <category><![CDATA[storage engine]]></category> <guid
isPermaLink="false">http://bogdan.org.ua/2008/03/06/how-to-improve-mysql-application-perfromance.html</guid> <description><![CDATA[Juicy presentation, even for seasoned MySQL developers. &#124; View You can also download the PDF version of the presentation. Many more MySQL performance presentations.]]></description> <content:encoded><![CDATA[<p>Juicy presentation, even for seasoned MySQL developers.</p><div
style="margin-bottom: 15px;width:425px;text-align:left" id="__ss_245799"><object
style="margin:0px" width="425" height="355"><param
name="movie" value="http://static.slideshare.net/swf/ssplayer2.swf?doc=how-to-kill-mysql-performance-1201635569837936-3"/><param
name="allowFullScreen" value="true"/><param
name="allowScriptAccess" value="always"/><embed
src="http://static.slideshare.net/swf/ssplayer2.swf?doc=how-to-kill-mysql-performance-1201635569837936-3" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="425" height="355"></embed></object><div
style="font-size:11px;font-family:tahoma,arial;height:26px;padding-top:2px;"><a
href="http://www.slideshare.net/?src=embed"><img
src="http://static.slideshare.net/swf/logo_embd.png" style="border:0px none;margin-bottom:-5px" alt="SlideShare"/></a> | <a
href="http://www.slideshare.net/techdude/how-to-kill-mysql-performance?src=embed" title="View 'How to Kill Mysql Performance' on SlideShare">View</a></div></div><p><span
id="more-269"></span><br
/> You can also download the <a
href="http://bogdan.org.ua/wp-content/uploads/2008/03/jay_pipes-kill_mysql_performance.pdf" title="15 ways to kill MySQL application performance">PDF version of the presentation</a>.</p><p>Many more <a
href="http://www.mysqlperformanceblog.com/mysql-performance-presentations/">MySQL performance presentations</a>.</p><p><a
class="a2a_button_citeulike" href="https://www.addtoany.com/add_to/citeulike?linkurl=https%3A%2F%2Fbogdan.org.ua%2F2008%2F03%2F06%2Fhow-to-improve-mysql-application-perfromance-performance.html&amp;linkname=How%20to%20improve%20MySQL%20application%20performance" title="CiteULike" rel="nofollow noopener" target="_blank"></a><a
class="a2a_button_pocket" href="https://www.addtoany.com/add_to/pocket?linkurl=https%3A%2F%2Fbogdan.org.ua%2F2008%2F03%2F06%2Fhow-to-improve-mysql-application-perfromance-performance.html&amp;linkname=How%20to%20improve%20MySQL%20application%20performance" title="Pocket" rel="nofollow noopener" target="_blank"></a><a
class="a2a_button_kindle_it" href="https://www.addtoany.com/add_to/kindle_it?linkurl=https%3A%2F%2Fbogdan.org.ua%2F2008%2F03%2F06%2Fhow-to-improve-mysql-application-perfromance-performance.html&amp;linkname=How%20to%20improve%20MySQL%20application%20performance" title="Kindle It" rel="nofollow noopener" target="_blank"></a><a
class="a2a_button_evernote" href="https://www.addtoany.com/add_to/evernote?linkurl=https%3A%2F%2Fbogdan.org.ua%2F2008%2F03%2F06%2Fhow-to-improve-mysql-application-perfromance-performance.html&amp;linkname=How%20to%20improve%20MySQL%20application%20performance" title="Evernote" rel="nofollow noopener" target="_blank"></a><a
class="a2a_button_pinterest" href="https://www.addtoany.com/add_to/pinterest?linkurl=https%3A%2F%2Fbogdan.org.ua%2F2008%2F03%2F06%2Fhow-to-improve-mysql-application-perfromance-performance.html&amp;linkname=How%20to%20improve%20MySQL%20application%20performance" title="Pinterest" rel="nofollow noopener" target="_blank"></a><a
class="a2a_dd addtoany_share_save addtoany_share" href="https://www.addtoany.com/share#url=https%3A%2F%2Fbogdan.org.ua%2F2008%2F03%2F06%2Fhow-to-improve-mysql-application-perfromance-performance.html&#038;title=How%20to%20improve%20MySQL%20application%20performance" data-a2a-url="https://bogdan.org.ua/2008/03/06/how-to-improve-mysql-application-perfromance-performance.html" data-a2a-title="How to improve MySQL application performance"><img
src="https://static.addtoany.com/buttons/share_save_120_16.png" alt="Share"></a></p>]]></content:encoded> <wfw:commentRss>https://bogdan.org.ua/2008/03/06/how-to-improve-mysql-application-perfromance-performance.html/feed</wfw:commentRss> <slash:comments>0</slash:comments> </item> </channel> </rss>