WordPress has a Global Translator plugin, which – among others – uses Google Translate service.

If someone uses Google Translate (e.g. using Global Translate’s mini-language-flags), and goes back to your blog – that someone might get banned by fail2ban (especially if you have set maxretry to 1), as the referrer will contain the php-URL-fopen attack signature. The bad thing is that you will not realize that until after you check one or several translations yourself, as a random site visitor experiencing the problem is highly unlikely to bother reporting this problem – especially when your blog’s Contact page is also inaccessible.

Clearly, Google Translate is not the only legitimate service which will trigger that rule.

Solution: The only solution I have found is to specify the whitelist regex for the php-URL-fopen rule.

As a result, I’m now using Eclipse PDT with Xdebug and Xdebug Helper (Firefox extension). Now I don’t understand how I used to debug my PHP programs before!

After proper configuration (I’m using local Apache, but it is also possible to debug remotely), my work flow is rather simple:

• use my web-app as usual, e.g. tweaking and testing here and there
• if something server-side goes wrong: click the XDebug helper icon in Firefox, and perform some server-request action (e.g. load a page)
• debugging is started in Eclipse PDT, where I can step through the code, set breakpoints, and examine all variables
• as soon as the problem is fixed – click the XDebug helper icon again to continue using the site normally (w/o invoking the debugger)

It takes some time to get used to, but then it’s a breeze.

Some advice:

• don’t use apt-get/aptitude to install Eclipse; it will be much easier both in the short and long run to use some all-in-one package from the Eclipse PDT site; all you need to do – download, extract, run!
• before actually starting to do anything, tweak the eclipse.ini file by increasing heap size from 40 MiB (default) to some larger value (I used 128MiB). If you don’t do this, then at some point your debugging will become painfully sloooow, and then you’ll start getting tons of “out of heap memory” errors, each one suggesting that you quit Eclipse immediately
• install XDebug with apt-get/aptitude – worked perfectly, and there’s /etc/php5/conf.d/xdebug.ini not to mess with php.ini
• do read XDebug guide for PDT 2.x (I’m assuming you got the 2.x version); it should be the only document you will really need to configure everything

I only wish Eclipse was faster – that is, written not in Java but e.g. C or C++.

Update: it seems as though since that short MySQL outage everything is faster at GoDaddy shared hosting. Did they upgrade database server(s)? I have no idea, but I like the change.

Update: the original script had an error, it would generate queries likes this one (note the bold part):

ALTER TABLE `links` CHANGE `link_rel` `link_rel` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL DEFAULT NULL;

This is clearly wrong syntax (and logic). I fixed this by making comparison to NULL strict (with three equal signs instead of two):

// Does the field default to null, a string, or nothing?
if ($row['Default'] === NULL)

Update 2: based on comment by banesto, I modified the script; now it does not require specifying the from_collation, it’s sufficient to specify to_collation (which will be used for all the fields and tables). The modified code is:

if ($row['Collation'] == ” || $row['Collation'] == $convert_to)
continue;

Update 3: the long-lasting, re-appearing NOT NULL DEFAULT NULL problem is finally fixed.

Update 4: incorporated Russ’s fix to skip numeric fields (in order to leave autoincrement values intact).

Here’s the script itself: (to copy-paste: first click the “Plain text” header)

1. Process browser’s request – find out, if the browser is asking for content for the first time, or is looking for an updated version. If the browser’s content is fresh, just reply that it’s fresh. Otherwise, proceed to step 2. Also proceed to step 2, if this is a first-time request.
2. If this is a first-time request, check if we have a fresh locally cached content, and send it in response, to bypass de novo contet generation. If local cache is stale, proceed to step 3.
3. Generate new (fresh) content cache item, and send it to browser, together with cache-enabling headers.

I decided to put all the header-related processing into one function, and all the cache-creation and sending – into another. If you prefer, you can make an object with two or more methods, but I see no need in doing so.

Here’s the headers part:

This function checks two caching-related request headers: If-None-Match and If-Modified-Since. Use of two headers is redundant, but this was done with an idea to provide different caching methods for different content types in the future. Timestamp-based approach should be sufficient in most cases, but having ETag as well does not hurt.

The parameter this function takes – $timestamp – is the date of the locally cached content item. We should check if the cached item exists before calling HTTP_caching; and if cached item does not exist, we should create it and then call HTTP_caching with the timestamp of the newly generated cached content item.

Function is utterly simple, but needs some explaining comments:

• There is a convenient function “apache_request_headers”, but as its name implies, it works only under apache. Thus I had to use global PHP variables for fetching specific request headers. However, this approach also doesn’t work on all hosting providers – you just do not get those headers, though they are sent by the browser. But you still benefit from local caching, and the only drawback of being unable to process request headers is that you waste traffic sending data which could have been unsent.
• I’m also sending two equal headers – standard HTTP/1.1, and outdated Status header. This is redundant (sending Status might be removed), but depending on how your PHP is installed (as apache-httpd module or CGI) you may want to leave both in place, or experiment and leave the one which actually works.

Now, let’s move on to actual caching. The simplest and quite reliable method of identifying any object within your cache is md5(url) – that is, the hash of the request URL. Note, that you might want to hash not the complete URL (starting with http://), but only the part after the TLD’s slash, e.g. for complete URL http://bogdan.org.ua/2006/10/27/xnameorg-down-largest-ddos-they-ever-had.html you would hash only the “xnameorg-down-largest-ddos-they-ever-had.html” part (or “2006/10/27/xnameorg-down-largest-ddos-they-ever-had.html”, if the filename part of the path might be non-unique). Evidently, this will save you from generating cache both for “http://www.bogdan.org.ua/2006/10/27/xnameorg-down-largest-ddos-they-ever-had.html” and for “http://bogdan.org.ua/2006/10/27/xnameorg-down-largest-ddos-they-ever-had.html” (differing only in “www.” part).

Here’s what we need to do (some evident micro-tunings dropped for clarity):

1. Hash the request URL.
2. Check if that cache item exists. If yes – go to step 4. If not – step 3.
3. Create cache item (cache file), set the file name to the hash of the request URL, and store the file in your cache directory (e.g. ‘/cache’).
4. Read cache file modification timestamp.
5. Call HTTP_caching with that timestamp. If it returns true – do nothing. Else – continue to step 6.
6. Read cache file and send it to browser.

To make this part of caching system wrapped in a single universally applicable function, you will need to define some “content-generating” handler function, which will be called when local cache file must be regenerated. “Content-generation” might be as simple as just reading some file from the disk; or you could wrap your index.php file in cache-checking block, so that page call and generation occur only if there is no readily available cached page.

function caching($handler_function)
{
// first, check if we have existing cache
// of currently requested resource in the ‘cache’ directory
$cachedfile = ‘cache/’ . md5($_SERVER['REQUEST_URI']);
if (file_exists($cachedfile) && is_file($cachedfile))
{ // cache exists. now check if it is up-to-date
$filetime = filemtime($cachedfile);
$modif = time() – $filetime;
// CACHE_PERIOD is a constant defined somewhere else,
// e.g. define(‘CACHE_PERIOD’,300)
if ($modif <= (int)CACHE_PERIOD) { if ( HTTP_caching($filetime) === false ) { // we need to send cached content to browser - no // if-modified-since or if-none-match request headers $expires = gmdate('D, d M Y H:i:s', ($filetime + CACHE_PERIOD) ) . ' GMT'; send_cached_file($cachedfile, CACHE_PERIOD, $expires) } // if HTTP_caching returned 'true' - we are done with '304' header. // finally, ensure nothing else happens after sending contents... exit(); } // if cache is old - just call $handler_function, // assuming that it returns all the content $new_cache = $handler_function(); // if instead of using $handler_function you wrap some code: // ob_start(); // include "page_generation.php"; // $new_cache = ob_get_contents(); // ob_end_clean(); $fp = fopen ($cachedfile, "w"); fwrite ($fp, $new_cache); fclose ($fp); $filetime = filemtime($cachedfile); $expires = gmdate('D, d M Y H:i:s', ($filetime + CACHE_PERIOD) ) . ' GMT'; send_cached_file($cachedfile, CACHE_PERIOD, $expires); } } [/php] Below is a function which sends cached files to the browser. Note, that it lacks content-type parameter (or content-type detection code). [php] function send_cached_file($cachedfile, $cache, $expires) { // I omitted Content-Length and Content-Type for simplicity, // though it's better to send them header("HTTP/1.1 200 OK"); header("Status: 200 OK"); header("Pragma: cache"); header("Cache-Control: max-age=".$cache.", min-fresh=".$cache.", no-transform"); // Content-Type needs adjustment on a per-content-item basis header("Content-Type: text/html"); header("Content-Length: " . filesize($cachedfile)); header("Expires: $expires"); readfile($cachedfile); } [/php] (Note: function 'caching' was not tested; if you find any errors, or if it just doesn't work for you - let me know via the contact form, I’ll try to fix my error.)

For the explanation of what headers mean, refer to my previous post.

This is it for now. Coming next might be some useful information on differences in HTTP headers handling between PHP installed as a module and PHP as CGI.

As always, comments/suggestions are welcome.

If your proxy does not need authorization – just drop the proxy_user and proxy_pass parameters.

• source-level cross-platform scripting: your script will run anywhere, where Python compiles; expanding this statement – your script will run anywhere, where there is a C compiler (needed to build Python itself)
• high-level language: you can iterate all the lines in a text file with as little as one ‘for’-statement, for example (see the actual example below)
• simple/minimalist syntax: no curly braces around blocks of statements, no semicolons after each and every line of code, etc. Python at a glance looks much more understandable, than, for example, Perl.
• the power of C in a language-interpreting system
• it is interpreted! This gives easyness of debugging: modify, execute, see the trouble – with no compile/link stages
• and, despite being interpreted, it is fast!

For the comparison (in speed, memory use, program size) with other computer programming languages, please see the “Computer Language Shootout Benchmarks”. Here I provide the link only to the comparison of Python with Perl and comparison of Python with PHP (which can also be used as shell-scripting language, albeit after some tinkering with settings and stuff)

Below is an example of the 2-minute script in Python, which counts the number of occurrences of some string in a file.