4 Jun, 2010

Methods of opening a bi-directional socket from a Browser

Posted by Bhavin Turakhia | (6) Comments

It is no surprise that 6 of the top 10 desktop applications by usage time are browsers (source: Wakoopa). We all have our gripes with a browser as an application container – sandboxing, cross browser compatibility issues,  no access to native APIs. The developments over the last few years however have been very promising – Ajax, Flex,HTML5, Web Sockets, Web Hooks, Google gears – with all thats afoot a browser application nowadays provides a near native experience.

One of my many personal peeves has been the lack of raw socket connection capabilities and bi-directional communication from a browser. This too has changed considerably over the years. This article lists various bi-directional communication methods that one can use from a browser -

  • Comet: Comet is more a collection of techniques that provide bi-directional communication between a browser and a server. It is a superset of Long-polling, BOSH, and other such techniques
  • Long-polling: This merely refers to an HTTP connection that is maintained for a long duration, without disconnection. A server, upon receiving a request, keeps the connection with the client open, and sends streams of data back to the client. The response is never deemed to have completed, hence the server can continue to keep pushing data to a client over this connection, thus emulating push
  • BOSH: A BOSH library uses upto 2 connections to a server  - one connection for the client to send data to the server, and another for the server to send data to the client. The client opens a first connection and sends a request to the server. The server does not respond, and then subsequently can use this connection to send a response whenever it is ready. If the client meanwhile needs to send data to the server it does so through a request from a second connection. The moment the server receives this request from a second connection, it sends a response out to the first connection thus reversing the roles of the the two connections
  • Flash: One can use Flash to establish a socket connection to a server. This is a far more efficient method for bi-directional communication. However it has certain limitations. Firstly flash supports two types of socket connections – XMLSocket and a Raw TCP Socket. So no UDP. Secondly from a Flash widget, one can only make a socket connection to the domain from where the page was loaded. No cross-domain calls are permitted unless an explicit cross domain policy file is provided for by the server you are making a connection to. Therefore one cannot load a flash widget from server1 and make a socket connection to server2. For instance, if one were to write a flash based MSN client, the client would not be able to directly connect to the MSN servers. One solution would be to proxy the connection through a TCP proxy installed on your server. However this would mean that you would need server infrastructure to relay the connection. A nice article describing how to achieve this is available here.
  • Web Sockets: Plagiarising from Wikipedia – “WebSockets is a technology providing for bi-directional, full-duplex communications channels, over a TCP socket and is being standardized by the W3C and IETF”. Websockets is still limited in the sense that it is not a protocol-independent binary socket connection. A reference implementation (client and server) is available at http://jwebsocket.org/
  • Java Applets: Java applets are way more powerful than flash when it comes to raw socket capability. You have a choice of protocols (UDP/TCP) and most of the java stack at your disposal. Java applets too have a sandboxing restriction, which though is easier to circumvent than Flash. As a Java applet, you can only make a socket connection to the server from where the page was loaded, unless the applet is a signed applet. So one can use a signed applet to setup a socket connection to any server without having to use a proxy server. To my mind this would be the ideal method if only I had the slightest confidence in the applet working as advertised. In the last few years I am yet to see a single Java applet run without error in my browser. Have never bothered to troubleshoot it, but it does not give me confidence :)
  • Browser plugin: One can write a browser plugin to perform the socket communication. The plugin would expose a function in Javascript enabling a web page to use it to make socket connections. There is a certain degree of friction for the user who would need to download install the browser plugin. Browser plugins are also difficult to maintain given that there are several browsers each running on various platforms. Soon the cross-browser cross-OS compatibility can become a nightmare
  • External application: I now come to the most elegant method of achieving powerful bi-directional access between a browser and any server with complete native capabilities. Infact this method is the raison d’etre for this article. While most of the above methods would work in most scenarios, they still lack the power of a native desktop application (barring the browser plugin). Most of the methods above are sand-boxed, inefficient, require server proxies, and cannot access underlying native OS functionality. This brings me to a far simpler yet superior method – writing a native application that runs on the users machine and exposes a web server (or some socket server) to which the app in the browser can communicate using … you guessed it … any of the above methods (Flash/BOSH/Comet/HTTP). Seemingly Google’s video chat plugin works in this manner. All the cool P2P, UDP, ICE, NAT traversal magic is written as an external application that the user downloads. The data is then streamed from this out-of-process app into the browser and can be played using the Flash player. This method infact reminds me very much of how Rhomobile works on the mobile phone. As a part of my research I also came across numerous other applications that use this technique. Another interesting project worth mentioning is Littleshoot by Adam Fisk. LittleShoot is an opensource implementation of P2P in the browser. It works by downloading an application that runs on your machine as a service, and then when you visit the LittleShoot website the webpage detects that you have the app installed and can use the app (which is a mini-web server with complete OS access) to pretty much do anything.
  • Other methods: I havent researched ActiveX and Silverlight, but I would assume that ActiveX behaves much like Java w.r.t socket connections and Silverlight possibly behaves much like Flash :) . I also came across this other cool tool – Orbited – which essentially provides a server proxy and a client javascript library to simulate a TCP socket implementation within javascript. Essentially a combination of the techniques I have described in the first few methods – but pre-packaged for you.
Category : 0-cosmos | TechTalk

13 Apr, 2010

My mini OAuth resource compendium

Posted by Bhavin Turakhia | (2) Comments

We are beginning implementation of OAuth in one of our projects. I just finished reading up a ton of resources. In the end I only needed to readup a few. Here they are in the recommended order -

OAuth is a fairly simple protocol, especially if you are familiar with the basics of HTTP, nonce, basic encryption/digital signatures etc.

Category : 0-cosmos | TechTalk

20 Jan, 2009

HTTP vs REST vs SOAP

Posted by Bhavin Turakhia | (17) Comments

I have been an active proponent of SOAP since its inception. SOAP revolutionzed RPC and loose coupling to a great extent. However off late I have been giving APIs and interfaces considerable thought and am leaning a lot more towards simple HTTP based APIs with an XML or JSON response format as opposed to SOAP. In this post I pen down some random thoughts on the merits and demerits of each.

Introduction
Let me first clarify the terminology -

  • SOAP refers to Simple Object Access Protocol
  • HTTP based APIs refer to APIs that are exposed as one or more HTTP URIs and typical responses are in XML / JSON. Response schemas are custom per object
  • REST on the other hand adds an element of using standrdized URIs, and also giving importance to the HTTP verb used (ie GET / POST / PUT etc)

Typing
SOAP provides relatively stronger typing since it has a fixed set of supported data types. It therefore guarantees that a return value will be available directly in the corresponding native type in a particular platform. Incase of HTTP based APIs the return value needs to be de-serialized from XML, and then type-casted. This may not represent much effort, especially for dynamic languages. Infact, even incase of copmlex objects, traversing an object is very similar to traversing an XML tree, so there is no definitive advantage in terms of ease of client-side coding.

Client-side effort
Making calls to an HTTP API is significantly easier than making calls to a SOAP API. The latter requires a client library, a stub and a learning curve. The former is native to all programming languages and simply involves constructing an HTTP request with appropriate parameters appended to it. Even psychologically the former seems like much less effort.

Testing and Troubleshooting
It is also easy to test and troubleshoot an HTTP API since one can construct a call with nothing more than a browser and check the response inside the browser window itself. No troubleshooting tools are required to generate a request / response cycle. In this lies the primary power of HTTP based APIs

Server-side effort
Most Programming languages make it extremely easy to expose a method using SOAP. The serialization and deserialization is handled by the SOAP Server library. To expose an object’s methods as an HTTP API can be relatively more challenging since it may require serialization of output to XML. Making the API Rest-y involves additional work to map URI paths to specific handlers and to import the meaning of the HTTP request in the scheme of things. Offcourse many frameworks exist to make this task easier. Nevertheless, as of today, it is still easier to expose a set of methods using SOAP than it is to expose them using regular HTTP.

Caching
Since HTTP based / Rest-ful APIs can be consumed using simple GET requests, intermediate proxy servers / reverse-proxies can cache their response very easily. On the other hand, SOAP requests use POST and require a complex XML request to be created which makes response-caching difficult

Conclusions
In the end I believe SOAP requires greater implementation effort and understanding on the client side while HTTP based or REST based APIs require greater implementation effort on the server side. API adoption can increase considerably if a HTTP based interface is provided. Infact an HTTP-based API with XML/JSON responses represents the best of both breeds and is easy to implement on the server as well as easy to consume from a client

Category : 0-cosmos | TechTalk