As you can see in the following image the Google App Engine have got lots of trouble in this June and we cannot afford to lose customers.

For that reason we are now serving again our static content from our servers until the situation in Google App Engine normalizes.

You can follow the google app engine downtimes here

Thank you

What we want?:

• Create a CDN easy to update and free of charge for static resources (images, css, js)
• Consume as less bandwidth as possible leveraging the If-Modified-Since/Last-Modified/304 Not Modified model

Hands-on:

The first approach, of course, was to look on Google for some help, the post of Andreas Krohn helped a lot to start.

But I want to go further and take care of modern browsers If-Modified-Since requests, then the google framework and a little of Python comes to the rescue.

Note: I’m assuming you’ve already installed the Python environment and the Google App Engine SDK

First of all let me give you two little .bat files that are useful:

Start the test webserver (test.bat):
dev_appserver.py c:\ipsojobscloud

Upload your application to the cloud (update.bat):
appcfg.py update c:\ipsojobscloud

Note: simply change c:\ipsojobscloud for the folder you are working in and contains your app.yaml

Then I’ve setup the app.yaml, it’s very simple (16 lines):

application: ipsojobscloud
version: 1
runtime: python
api_version: 1

handlers:
- url: /favicon.ico
  static_files: favicon.ico
  upload: favicon.ico

- url: /images/favicon.ico
  static_files: favicon.ico
  upload: favicon.ico

- url: /.*
  script: cacheheaders.py

This app.yaml simply tells the GAE the name of the application (ipsojobscloud) the version we’re working on (use only the major release number, GAE automatically takes care of the .x when you upload).

Then we specify two handlers for the favicon.ico static file and a catch-all handler that redirects our requests to the Python script cacheheaders.py

With that environment set, we simply code the cacheheaders.py file, let’s see it in detail:

The skeleton of the file is:

import wsgiref.handlers
from google.appengine.ext import webapp

class MainPage(webapp.RequestHandler):

  def get(self, dir, file, extension):
...

def main():
  application = webapp.WSGIApplication([(r'/(.*)/([^.]*).(.*)', MainPage)], debug=False)
  wsgiref.handlers.CGIHandler().run(application)

if __name__ == "__main__":
  main()

Here we are importing the webapp framework and setting the class MainPage, in the main section the only change in the sample GAE is
the regular expression that we used to match our requests, the expression r’/(.*)/([^.]*).(.*)’ is telling that we are using regular expressions (r)
, then take one slash, followed by an arbitray number of characters and another slash /(.*)/ the parentesis tells the regular expression to keep the string beetween the two slashes as a variable. The next part ([^.]*). takes all caracters except a dot and puts them in to the second variable and finally, we’ll take the rest of the input as a variable with (.*)

This regular expression is designed to only capture paths like /images/helloworld.gif where variables are images, helloworld and gif respectively

Note: Of course that’s not a complete solution, we can only have one folder depth, but it’s a good readers exercice to improve that

The part that you need to know is that when a request arrives it’s mapped to the get function with the parameters dir, file and extension (and don’t forget the first “self” parameter)

Let’s see the code of the get function in detail:

First, check the validity of the parameters received and set the correct content-type based on the extension:

  def get(self, dir, file, extension):
    if (dir!='js' and dir!='css' and dir!='images'):
      self.error(404)
      return

    if (extension!='js' and extension!='css' and extension!='jpg' and extension!='png' and extension!='gif'):
      self.error(404)
      return

    if extension=='js':
      self.response.headers['Content-Type'] = 'application/x-javascript'
    elif extension=='css':
      self.response.headers['Content-Type'] = 'text/css'
    elif extension=='jpg':
      self.response.headers['Content-Type'] = 'image/jpeg'
    elif extension=='gif':
      self.response.headers['Content-Type'] = 'image/gif'
    elif extension=='png':
      self.response.headers['Content-Type'] = 'image/png'

Note: the firts two ifs are completely optional, we check if the dir variable is in our valid list of dirs (js, css, images) and if the extension of the file is in our allowed list (js, css, jpg, png, gif), you have to change that check or completely remove it at your convenience.

And now the tricky part:

    try:
      import os
      import datetime
      path = dir+'/'+file+"."+extension
      info = os.stat(path)
      lastmod = datetime.datetime.fromtimestamp(info[8])
      if self.request.headers.has_key('If-Modified-Since'):
        dt = self.request.headers.get('If-Modified-Since').split(';')[0]
        modsince = datetime.datetime.strptime(dt, "%a, %d %b %Y %H:%M:%S %Z")
        if modsince >= lastmod:
        # The file is older than the cached copy (or exactly the same)
          self.error(304)
          return
        else:
        # The file is newer
          self.output_file(path, lastmod)
      else:
        self.output_file(path, lastmod)
    except:
      self.error(404)
      return

First we import some packages (os, datetime), then create a variable “path” with the full path of the file we want to retrieve

path = dir+'/'+file+"."+extension

Then, take the info of the file from the Operating System and keep the last modified date into lastmod variable, note that if an error occurs (non existing file for example, the except part will be executed, returning a 404 not found response to the browser).

In the following lines we scan the headers of the request, looking for an If-Modified-Since header, if we found it take the date part

      if self.request.headers.has_key('If-Modified-Since'):
        dt = self.request.headers.get('If-Modified-Since').split(';')[0]
        modsince = datetime.datetime.strptime(dt, "%a, %d %b %Y %H:%M:%S %Z")

Then compare the last modification date of the file against the ifmodifiedsince date and act accordingly, note that self.error(304) will return a response code 304 (Not-Modified) to the browser:

        if modsince >= lastmod:
        # The file is older than the cached copy or the same
          self.error(304)
          return
        else:
        # The file is newer
          self.output_file(path, lastmod)

The self.output_file(path, lastmod) is a function we have defined to avoid code duplication:

  def output_file(self, path, lastmod):
    import datetime
    try:
      self.response.headers['Cache-Control']='public, max-age=31536000'
      self.response.headers['Last-Modified'] = lastmod.strftime("%a, %d %b %Y %H:%M:%S GMT")
      expires=lastmod+datetime.timedelta(days=365)
      self.response.headers['Expires'] = expires.strftime("%a, %d %b %Y %H:%M:%S GMT")
      fh=open(path, 'r')
      self.response.out.write(fh.read())
      fh.close
      return
    except IOError:
      self.error(404)
      return

As you can see we imported datetime to manipulate dates and try to do the following:

• Set the header Cache-Control, to be as much cacheable as posible
• Set the header Last-Modified (IMPORTANT ! when we send for the first time the file to the browser it keeps the Last-Modified date of the file, this value is the value that will send in the next If-Modified-Since requests, when we usually will respond 304 not-modified!)
• Calculate an expires date in the future (we’ve put 365 days)
• Set the Expires header with this value (last-modified+365 days)
• Open the file and send it to the output and finally close the file
• return, because when we output the file we’re done

Note: If something happens we returned an standard response of Not Found (404)

Conclusions:

We’ve improved the latency in the requests of static files putting them into the cloud, and keep the bandwidth used in the cloud to a minimum answering correctly to the If-Modified-Since requests and only in about 70 lines of code

One of the advantatges of Google App Engine above Amazon S3 is that GAE is free up 5 million page views a month, that give us a good chance to try this kind of features without spending cash.

You can see the speed improvement on-line in all the ipsojobs.com pages rigth now !

Some screenshots taken from firebug:

First request:

Second request:

Detail of a request:

Full source of cacheheaders.py:

import wsgiref.handlers
from google.appengine.ext import webapp

class MainPage(webapp.RequestHandler):

  def output_file(self, path, lastmod):
    import datetime
    try:
      self.response.headers['Cache-Control']='public, max-age=31536000'
      self.response.headers['Last-Modified'] = lastmod.strftime("%a, %d %b %Y %H:%M:%S GMT")
      expires=lastmod+datetime.timedelta(days=365)
      self.response.headers['Expires'] = expires.strftime("%a, %d %b %Y %H:%M:%S GMT")
      fh=open(path, 'r')
      self.response.out.write(fh.read())
      fh.close
      return
    except IOError:
      self.error(404)
      return

  def get(self, dir, file, extension):
    if (dir!='js' and dir!='css' and dir!='images'):
      self.error(404)
      return

    if (extension!='js' and extension!='css' and extension!='jpg' and extension!='png' and extension!='gif'):
      self.error(404)
      return

    if extension=='js':
      self.response.headers['Content-Type'] = 'application/x-javascript'
    elif extension=='css':
      self.response.headers['Content-Type'] = 'text/css'
    elif extension=='jpg':
      self.response.headers['Content-Type'] = 'image/jpeg'
    elif extension=='gif':
      self.response.headers['Content-Type'] = 'image/gif'
    elif extension=='png':
      self.response.headers['Content-Type'] = 'image/png'

    try:
      import os
      import datetime
      path = dir+'/'+file+"."+extension
      info = os.stat(path)
      lastmod = datetime.datetime.fromtimestamp(info[8])
      if self.request.headers.has_key('If-Modified-Since'):
        dt = self.request.headers.get('If-Modified-Since').split(';')[0]
        modsince = datetime.datetime.strptime(dt, "%a, %d %b %Y %H:%M:%S %Z")
        if modsince >= lastmod:
        # The file is older than the cached copy (or exactly the same)
          self.error(304)
          return
        else:
        # The file is newer
          self.output_file(path, lastmod)
      else:
        self.output_file(path, lastmod)
    except:
      self.error(404)
      return

def main():
  application = webapp.WSGIApplication([(r'/(.*)/([^.]*).(.*)', MainPage)], debug=False)
  wsgiref.handlers.CGIHandler().run(application)

if __name__ == "__main__":
  main()

Google AJAX Feed API

The idea is simple, just put some keywords, google will look for relevant RSS feeds and generate a simple but elegant RSS slideshow with the latest “news” contained in those feeds.

How can help you to put the latest job offers of ipsojobs.com in your website?

EASY
Just go to http://www.google.com/uds/solutions/wizards/dynamicfeed.html

In the Style, check the best suited for your site (if you choose “Vertical Stacked” be sure to choose an appropiate title too)

In the “Feeds Expression” enter your favorite ipsojobs sites, for example:

ipsojobs barcelona, ipsojobs sabadell

The comma, separates the cities and it’s important to put ipsojobs every time to be sure that google takes the correct RSS

When you click the Preview button be sure that in the “Direct Feed URL” are appearing valid ipsojobs RSS feeds, like in the example:

ipsojobs barcelona: http://barcelona.ipsojobs.com/rss/controller.php
ipsojobs sabadell: http://sabadell.ipsojobs.com/rss/controller.php

Finally, go to Generate Code and copy and paste it in to your favorite CMS.

That’s what we have done

Sample integration in a blogger page

As you may know Ipsojobs.com is a global job bank, fast easy and free for everybody !

The new API allows you to locate job offers near you, you only need to give us your latitude and longitude and we will return you a nice XML containing the nearest Ipsojobs cities and the last job postings in that city (if any).

You can see a couple of examples:
Jobs near Barcelona
Jobs near Madrid

As you may see in the examples the XML structure is quite simple:

<cities>
<city>
<home>URL of the Ipsojobs city home</home>
<rss> URL of the RSS of that city</rss>
<title>Title of the city, for example “Badalona, Spain”</title>
<current_posts>Number of posts active in that city</current_posts>
<lat>Latitude of the city</lat>
<lng>Longitud of the city</lng>
<posts>
<post>
<title>Title of the job posting</title>
<urlnice>URL of the job posting</urlnice>
</post>
…
</posts>
</city>
…
</cities>

You can use the API, just pointing to the URL http://www.ipsojobs.com/api/nearby.php with two parameters, lat and lng, as in this example.

http://www.ipsojobs.com/api/nearby.php?lat=40.416706&lng=-3.703270

If you think this API is nonsense, we’ve already our FIRST IMPLEMENTATION, check this site:

http://www.esofid.com/

This is an organization of “Language Schools” all around Spain, if you search in the top box, look for “Barcelona” for example, you will see all the “Official Language Schools in that area”, then if you click on one of them, you will see three tabs, the first are the basic info of the School, but the second contains a Panoramio’s picture integration and the Jobs near that location with the help of the API we’ve just launched, see the picture.

Nice, uh ?

Thanks to Raúl for the idea and the first integration !

Ya puestos a crear un widget, lo hemos creado con UWA, aunque es posible que si salen incompatibilidades debamos crear n-widgets para las n-plataformas sobre las que queramos desarrollar.

El widget pretende ser algo un poco más complejo que un simple lector de RSS, en concreto los objetivos del widget son:

• Permitir seleccionar fácilmente la ciudad sobre la que queremos consultar las ofertas de trabajo de ipsojobs.
• La selección de la ciudad debe ser “dinámica” dado que cada día aparecen ciudades nuevas no puede ser una lista estática.
• Cuando estas viendo las ofertas de una ciudad siempre podrás cambiar a otra ciudad fácilmente.
• El número de ofertas por ciudad debe ser personalizable.
• Una vez seleccionada una ciudad el widget debe “recordar la preferencia” y mostrar las ofertas de la ciudad hasta que el usuario decida cambiarla.

Si quereis probarlo simplemente podéis visitar http://www.ipsojobs.com/widgets/uwa.html

El código del widget UWA es relativamente sencillo, se basa en un sólo fichero XHTML con todos los javascripts y los CSS incrustados.

Declaración:

<?xml version=”1.0″ encoding=”utf-8″?>
<!DOCTYPE html PUBLIC “-//W3C//DTD XHTML 1.0 Strict//EN” “http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd”>
<html xmlns=”http://www.w3.org/1999/xhtml” xmlns:widget=”http://www.netvibes.com/ns/” >
<head>

<meta name=”author” content=”Agusti Pons, omatech.com” />
<meta name=”description” content=”ipsojobs.com find jobs easy in your city” />

<meta name=”apiVersion” content=”1.0″ />
<meta name=”autoRefresh” content=”20″ />
<meta name=”debugMode” content=”true” />

<link rel=”stylesheet” type=”text/css” href=”http://www.netvibes.com/themes/uwa/style.css” />
<script type=”text/javascript” src=”http://www.netvibes.com/js/UWA/load.js.php?env=Standalone”>

De esta parte, sólo cabe destacar que se debe modificar el autor y la descripción así como el parámetro debugMode, que en producción deberá ser false. El debugMode=true es muy practico porque manda los mensajes de error a la consola del navegador y se puden consultar con el Firebug por ejemplo.

Javascripts útiles:

<script type=”text/javascript”>
<![CDATA[
function showhide(e)
{
//el = document.getElementById(e);
el = widget.body.getElementsByClassName(e)[0];
el.style.display = el.style.display == “block” ? “none” : “block”;
}

function getlang()
{
return widget.lang.substr(0, 2);
}

function link(url)
{
if (widget && widget.openURL)
{
widget.openURL(url);
}
}

function getSelected(ctrl)
{
for(i=0;i
{
if(ctrl[i].checked)
{
return ctrl[i].value;
}
}
return ”;
}

function setcity(url)
{
widget.setBody(’<p>…</p>’);
widget.setValue(’url’, url);
UWA.Data.getFeed(widget.getValue(’url’)+’?lang=’+getlang(), ipsowidget.display);
}

function selectcity()
{
widget.setTitle(’ipsojobs.com widget – setting…’);
UWA.Data.getText(’http://www.ipsojobs.com/api/controller.php?action=zones_sites_list&xsl=netvibes〈=’+getlang(), ipsowidget.zones_sites_processor);
}

function replace(texto,s1,s2)
{
return texto.split(s1).join(s2);
}

function get_change_city()
{
var lang = getlang();
if (lang==’es’)
{
return ‘Cambiar ciudad’;
}
if (lang==’ca’)
{
return ‘Canviar ciutat’;
}
if (lang==’fr’)
{
return ‘Changer de ville’;
}
else
{
return ‘Select city’;
}
}
]]>
</script>

Las dos funciones más interesantes son setcity() y selectcity() que detallo a continuación:

function setcity(url)
{
widget.setBody(’<p>…</p>’);
widget.setValue(’url’, url);
UWA.Data.getFeed(widget.getValue(’url’)+’?lang=’+getlang(), ipsowidget.display);
}

Recibe como parámetro la url del RSS que nos han clicado en la pagina de selección de la ciudad, primero ponemos unos puntos suspensivos para indicar que estamos cargando (widget.setBody(’<p>…</p>’);) posteriormente actualizamos el valor de la preferencia url (recordad que es oculta y el usuario no la puede ver) con la instrucción widget.setValue(’url’, url); y finalmente cargamos el feed RSS en el body, fijaos que utilizamos la url de la preferencia que acabamos de actualizar pero le añadimos el parámetro lang basándonos en el language del navegador del usuario UWA.Data.getFeed(widget.getValue(’url’)+’?lang=’+getlang(), ipsowidget.display);.

Para mostrar el RSS feed en el body fijaros que llamamos a la función display del widget que veremos más abajo declarada como ipsowidget.display = function(feed) esta función es muy parecida a la que se explica en el HOWTO de como mostrar un RSS en la web de Netvibes.

function selectcity()
{
widget.setTitle(’ipsojobs.com widget – setting…’);
UWA.Data.getText(’http://www.ipsojobs.com/api/controller.php?action=zones_sites_list&xsl=netvibes〈=’+getlang(), ipsowidget.zones_sites_processor);
}

La función selectcity, sólo tiene dos lineas, la primera modifica el título del widget widget.setTitle(’ipsojobs.com widget – setting…’); indicando que no estamos en ningúna ciudad sino en la pantalla de setting. La segunda linea carga una url especial de ipsojobs que genera el html necesario para mostrar el seleccionable de las distintas zonas y ciudades UWA.Data.getText(’http://www.ipsojobs.com/api/controller.php?action=zones_sites_list&xsl=netvibes〈=’+getlang(), ipsowidget.zones_sites_processor); con el html resultante se llama a la función ipsowidget.zones_sites_processor que veremos más abajo y simplemente machaca el body con el nuevo html. Cabe destacar que el html contiene la lista de zonas y sus ciudades ocultas, al clicar sobre la zona se llama a la función showhide para mostrar las ciudades de esta zona. Con iGoogle he comprovado que el desplegado y el plegado no funcionan muy bien porque el widget no se redimensiona correctamente, en cambio si arrastras el titulo del widget como si lo fueses a reposicionar, entonces se recalcula la altura del mismo correctamente.

Preferencias y CSSs:

<title>ipsojobs.com widget
<link rel=”icon” type=”image/png” href=”http://www.netvibes.com/favicon.ico” />

<widget:preferences>
<preference name=”url” type=”hidden” label=”URL” defaultValue=”" />
<preference name=”limit” type=”range” label=”Number of items to display” defaultValue=”10″ step=”1″ min=”1″ max=”25″ />
</widget:preferences>

<style type=”text/css”>
/* your CSS rules */

body {
height: 400px;
}

a:hover {
cursor: pointer;
}

h2 {
color: #8CC402;
}

h3 {
color: #8CC402;
}

</style>

Aquí podemos ver que creamos una preferencia llamada url de tipo hidden, es la que nos servira para almacenar la url del recurso RSS de la ciudad seleccionada. También creamos el parametro limit que permitirá al usuario seleccionar el numero de ofertas a mostrar en el rango de 1 a 25.

El meollo de la cuestión:

<script type=”text/javascript”>
<![CDATA[
var ipsowidget = {}

ipsowidget.feed = false;

ipsowidget.display = function(feed)
{
// get the number of items to display
widget.preferences[1].max = feed.items.length;

var toolbar=widget.createElement(’h2′);
var website_url=replace(widget.getValue(’url’), ‘/rss/controller.php’, ”);
toolbar.setHTML(’‘+feed.title+’‘);

var toolbar2=widget.createElement(’h3′);
toolbar2.setHTML(’‘+get_change_city()+’‘);

widget.setTitle(’ipsojobs.com – ‘+feed.title);

// create the ‘ul’ element, applying it the CSS class ‘nv-feedList’
var feedList = widget.createElement(’ul’);

// your ‘ul’ element MUST make use of the ‘nv-feedList’ class
// to ensure your widget uses the UWA UI library
feedList.className = ‘nv-feedList’;

// number of parsed items
var j = 0;

// loop through the downloaded items
for(var i=0; i < feed.items.length; i++)
{
// if the limit is reached, stop looping
if (j >= widget.getValue(’limit’)) break;

// for each item, create the ‘li’ element
var item = feed.items[i];
var li = widget.createElement(’li’);

// create and fill the ‘a’ element of the item with the item’s link
var a = widget.createElement(’a');
a.href = item.link;

// fill the ‘a’ element with the item’s title
var displayTitle = item.title;
a.innerHTML = displayTitle;

// build the title from the 255 first characters of the content
// remove the content’s HTML tags along the way
var title = item.content.stripTags().truncate(255);
a.title = title;

// set a tooltip on the ‘a’ element, with the item’s content
a.onmouseover = function()
{
UWA.Utils.setTooltip(this, this.content, 250);
}

// finally append the ‘a’ element we just filled, into the ‘li’ element
li.appendChild(a);

// …and append the ‘li’ element to the main ‘ul’ element
feedList.appendChild(li);
j++;
}

var general=widget.createElement(’div’);
general.appendChild(toolbar);
general.appendChild(toolbar2);
general.appendChild(feedList);
// once the needed items have been parsed,
// send the main ‘ul’ element to the HTML body tag
widget.setBody(general);
}

ipsowidget.zones_sites_processor = function(text)
{
widget.setBody(text);
}

widget.onLoad = function()
{
if (widget.getValue(’url’)==”)
{
selectcity();
}
else
{
//alert(getlang());
UWA.Data.getFeed(widget.getValue(’url’)+’?lang=’+getlang(), ipsowidget.display);
}
}
]]>
</script>

La mayoría de este código es muy parecido al HOWTO de como hacer un RSS widget con UWA (solucionando algún pequeño bug) con las siguientes diferencias:

var toolbar=widget.createElement(’h2′);
var website_url=replace(widget.getValue(’url’), ‘/rss/controller.php’, ”);
toolbar.setHTML(’‘+feed.title+’‘);

var toolbar2=widget.createElement(’h3′);
toolbar2.setHTML(’‘+get_change_city()+’‘);

widget.setTitle(’ipsojobs.com – ‘+feed.title);

En este fragmento creamos el elemento DOM que nos muestra el nombre de la ciudad seleccionada (con un link a la misma) y el elemento que nos permite cambiar de ciudad. En la última linea también actualizamos la barra de título del widget con el nombre de la ciudad seleccionada.

Al final, hemos añadido los fragmentos DOM creados.

var general=widget.createElement(’div’);
general.appendChild(toolbar);
general.appendChild(toolbar2);
general.appendChild(feedList);
// once the needed items have been parsed,
// send the main ‘ul’ element to the HTML body tag
widget.setBody(general);

El onLoad también esta un poco tuneado, en el ejemplo siempre carga la url, pero en nuestro caso puede ser que el widget se haya instalado por primera vez y no tengamos ninguna url en concreto, por eso hacemos lo siguiente:

widget.onLoad = function()
{
if (widget.getValue(’url’)==”)
{
selectcity();
}
else
{
//alert(getlang());
UWA.Data.getFeed(widget.getValue(’url’)+’?lang=’+getlang(), ipsowidget.display);
}
}

Simplemente detecta si ya tenemos una url fijada y sino llamamos a selectcity que ya se encarga de hacer la llamada AJAX correspondiente para mostrar el selector de ciudades. En caso contrario (este usuario ya tenia la preferencia url fijada para este widget) simplemente la cargamos llamando a UWA.Data.getFeed, fijaros que obtengo el lenguaje del navegador y intento llamar con el lenguaje más apropiado al RSS que me interesa.

Finalmente el body de la página

</head>
<body>
<p>Welcome to the ipsojobs.com widget </p>
<p>…</p>
</body>
</html>

Esta parte no tiene misterio, simplemente mostramos un body, que en seguida se machacará con la función widget.onLoad, que hemos visto más arriba.

Your Web application can use this files if:
- You can use .htaccess in your hosting
- You have CSS only and JS only folders to apply to the whole folder
- You can use PHP in your hosting

If all the above conditions are met, you can simply add three files to your CSS and the same three files to your JS folder and you are done.

Please, take with caution and do some testing, don’t use directly in a production environment.

Of course, I’ll not give any warranty of success and you have to check the results using the Firebug Extension of Firefox, the “Live HTTP Headers” extension and the YSlow plug in of the Firebug Extension.

Let’s see the three files involved:

First file .htaccess:
Modify your .htaccess file to include the following:

AddHandler application/x-httpd-php .css .js
php_value auto_prepend_file gzip-start.php
php_value auto_append_file gzip-end.php

This code, tells Apache to automatically run the gzip-start.php before the requested javascript or css file and to run the file gzip-end.php after the conclusion of the requested file.

It’s a powerful way to create common headers and footers to pages or files, but be aware that it comes at a cost, it’s more fast to serve the static file alone than to run an php script before and one after the file, of course, you have to do some testing and see if it’s worthy to do so.

Second file, the magical gzip-start.php:

<?php

function get_http_mdate()
{
return gmdate(’D, d M Y H:i:s’, filemtime($_SERVER['DOCUMENT_ROOT'].$_SERVER['PHP_SELF'])).’ GMT’;
}

function check_modified_header()
{// This function is based on code from http://ontosys.com/php/cache.html
$headers=apache_request_headers();
$if_modified_since=preg_replace(’/;.*$/’, ”, $headers['If-Modified-Since']);
if(!$if_modified_since)
{
return;
}

$gmtime=get_http_mdate();

if ($if_modified_since == $gmtime)
{
header(”HTTP/1.1 304 Not Modified”);
exit;
}
}

check_modified_header();

// Open a gzipped buffer
ob_start (”ob_gzhandler”);

if (strpos($_SERVER['PHP_SELF'], ‘.js’)>0)
{// Javascript file
header(”Content-type: text/javascript”);
}
elseif (strpos($_SERVER['PHP_SELF'], ‘.css’)>0)
{// CSS file
header(”Content-type: text/css”);
}

$offset = 60000000; // Far far away in time

// Expires
header(”Expires: “.gmdate(”D, d M Y H:i:s”, time() + $offset) . ” GMT”);

// Cache-Control
header(”Cache-Control: must-revalidate, max-age=”.(time()+$offset));

header(”Last-Modified: “.get_http_mdate());

// generate unique ID, using the modification date and the absolute path to the file
$hash = md5(filemtime($_SERVER['DOCUMENT_ROOT'].$_SERVER['PHP_SELF']).$_SERVER['DOCUMENT_ROOT'].$_SERVER['PHP_SELF']);
header(’Etag: “‘.$hash.’”‘);

?>

The concept is clear, this file is interpreted by the PHP runtime right before the output of the requested file.

Let’s see the different parts of the file:

check_modified_header();

This function looks for the modification date of the requested file (filemtime($_SERVER['DOCUMENT_ROOT'].$_SERVER['PHP_SELF'])) and returns a HTTP 304 (Not Modified) if the file is fresh, to do that, first looks for a request header If-Modified-Since and then compare the dates.

// Open a gzipped buffer
ob_start (”ob_gzhandler”);

if (strpos($_SERVER['PHP_SELF'], ‘.js’)>0)
{// Javascript file
header(”Content-type: text/javascript”);
}
elseif (strpos($_SERVER['PHP_SELF'], ‘.css’)>0)
{// CSS file
header(”Content-type: text/css”);
}

These lines opens a PHP buffer with the trick that it’s using GZip compression, this is a very good feature of PHP that enables you to output gzipped content without any modification. Afterwards we put the Content-Type of the resulting file, depending of the extension of the file we put text/css or text/javascript.

$offset = 60000000; // Far far away in time

// Expires
header(”Expires: “.gmdate(”D, d M Y H:i:s”, time() + $offset) . ” GMT”);

// Cache-Control
header(”Cache-Control: must-revalidate, max-age=”.(time()+$offset));

These lines put the Expires and Cache-Control headers, both far away in time, we create the variable offset with a constant 60 million seconds (about 2 years) and use that to generate a Expires header and a Cache-Control header with the value of two years in the future.

Be warned, your CSS and your JS files will be cached in a lot of proxies and client browsers, you have to use a “rename file policy” in your HTML to prevent the users from using old versions of the files. We’ve got little javascripts and css files to change, but I recommend to split these kind of files in different folders, for instance /js/usually-modified/omatech.js and apply only the caching technique to the folder usually-modified, then rename each time the .js file with a timestamp, say omatech_20071010.js, and change your html code to reflect the new version.

header(”Last-Modified: “.get_http_mdate());

$hash = md5(filemtime($_SERVER['DOCUMENT_ROOT'].$_SERVER['PHP_SELF']).$_SERVER['DOCUMENT_ROOT'].$_SERVER['PHP_SELF']);
header(’Etag: “‘.$hash.’”‘);

The last lines, creates a Last-Modified header, used to indicate the client browser the freshness of the file for future requests, this have interactions with the Cache-Control/max-age and the Expires headers (out of the reach of this post).

Finally we generate an unique Etag header, based on the update time of the requested file and it’s full path.

Third and last file, the not less magical gzip-end.php:

<?php
header(’Content-Length: ‘ . ob_get_length());
ob_end_flush();
?>

This file simply creates dynamically a Content-Length header with the size of the gzipped buffer we have been using for this request, and then flushes it’s contents.

Simple uh ?

I hope you enjoyed and finded useful, feel free to change and play with the code.

I would like to thank the sources where I took inspiration and code:

The tutorial of CSS compression,
Facilitate User Experience with CSS Compression, from Max Kiesler

Good introduction to the problem of speeding up the webpages through caching and zipping javascripts
Serving Javascript Fast

And always a good tutorial even for novices (in Spanish)
¿Por qué y cómo crear un espacio web “cache friendly”? from jcea