ff

Zlib compression can reach very high compression rates with XMPP (up to 90%), however starting a session with a mobile client can be still expensive. The best solution would be less greediness of carriers, but I'm not dealing with science fiction, so I prefer to reduce data ;) These are the rough figures about a login with my roster (~250 contacts, with ~60-70 online) and with compression enabled using Lampiro:

• ~ 1.5KB for the authentication and  stream setup
• ~ 12-13KB for the presence broadcast and other related stanzas 
• ~ 4KB for roster retrieval

Can we do better? There is a new proposal, Stanza Interception and Filtering Technology (SIFT ), which may help a little by filtering incoming stanzas and allowing session "hush". Which is the importance of hush, besides not receiving unecessary presence updates? As Dave Cridland pointed out in a private chat it can improve Zlib compression by stuffing more data during hush and delaying flushes. The question was: how relevant is this action? So I did few experiments with a log of a login of mines and these are the results:

• A complete login session:
  
• Initial data size: 136175
• Compression of the whole block 16036
• Compression flushing each packet 22739
• Only presence:
• Initial data size:55729
• Compression of the whole block 4881
• Compression flushing each packet 7323

 Almost 1/3 of bandwidth gained, SIFT is definitely good!

Today I had an "interesting" exchange of point of views with some people working at a large mobile operator, and the result has been enlightening. Their business model is still the same model that didn't work in the nineties for close portals like MSN or AOL, thinking to owe both the Internet and users, and they don't really get that:

• with the iPhone and Android, that they are happily selling, no user will consent a locked down network access
  
• the later they realize that their closed services don't work (e.g. all their closed and unused messengers), the harder they'll be hit by the market
• once users have tried some sort of freedon, they don't get back any more

We are pleased to announce the first release of Lampiro in 2009! If you are impatient just go to google code and grab it (or to the official site for directly downloading to your mobile).  

These are the news:

• Added: Roster push from known contacts (xep: 144 roster item exchange)
• Added: Automatic recognition of phone platform and key autoconfiguration (got rid from the confusing setup screen at the beginning)
• Improved: Better and more usable contact subscription handling
• Improved: Simplified the interaction with gateways

What we think is really nice is how we now handle multiple subscription requests, which is an annoying behaviour in most clients  when subscribing to IM gateways (we know that other IM networks suck, but incredibly most people still prefer MSN and in some way we must cope with that!).

Usually you get overwhelmed with tens of popups asking for authorizing all the new contacts, which is not the best approach for users migrating from other networks and importing their old contacts. In order to avoid this we have designed a new screen allowing handling multiple requests with few clicks (below there is a screenshot:)

• all new subscription requests are queued to a list in a single screen, where they can be reviewed
  
• each contact can be singularly added or rejected by clicking on it
• it's possible add a bunch of contacts with one click by pressing "accept all"
• it's possible to trust the gateway once for all (useful if you have already approved requests by other means, as it happens for our facebook gateway)

Few days ago I've found on Launchpad an interesting project by Thomas Hervé, the twisted interface to OTP, which allows to connect to erlang nodes using Twisted Matrix. The idea is incredibily powerful because it allows to easily build mixed-distributed applications using both erlang and python/twisted where they are best. So I gave it a try with ejabberd which has hooks for adding callbacks to its internal components (i.e. it's possible to intercept and modify packets). Unfortunately I've discovered that the present implementation of ejabberd hooks allows to connect only callbacks from the same node of ejabberd, i.e. as an extension module, but ejabberd folks have been incredibily quick in releasing a patch which allows distributed hooks, by adding the "CallNode" atom as parameter. The patch may be not stable yet, but it works perfectly with the last release of ejabberd (2.0.2), and here is an example of a twisted node which intercepts and logs all packets sent and received by any user (here is the full code, change the node names accordingly to your setup).

It works very simply. The hooks are written as methods of a regular python class with the "remote_" prefix:

class UserMonitor(object):
    
    def remote_receive_packet(self, jid, frm, to, packet):
        print ">>",  to_domish(packet).toXml()
    
    def remote_send_packet(self, frm, to, packet):
        print "<<",  to_domish(packet).toXml()

which is published by using a PersistentPortMapper:

nodeName = buildNodeName("uccaro@olindo.bluendo.priv")
epmd = PersistentPortMapperFactory(nodeName, cookie, "olindo.bluendo.priv")
epmd.publish(proxy = UserMonitor()) 

After this it's possible to call the methods proxy:receive_packtes and proxy:send_packet from any erlang node.

Then we register with ejabberd using a second node which calls the register methods, using the ejabberd api:

And bingo, after this ejabberd happily starts calling our hooks!

As bonus in the code you find also a function which translates from the ejabberd xml representation to twisted domish nodes, which is far more usable.

One of the great advantages of free and open source projects is the feedback you receive from users. So few days after the release we've been able to fix many quirks while casting the roadmap for the  next major release. In particular:

• fixed "user add" in the menu, which was not working
• improved the usability in roster navigation
• adjusted the "ok" and "cancel" buttons in the composition forms, which were creating some troubles in some phone models

Lampiro 8.11.2 is out, go and get it.