26 Nov, 2007

Lucky 13: Our rank in the Deloitte and Touche Fast 50 list

Posted by Bhavin Turakhia | (6) Comments

Directi was ranked in the Deloitte and Touche Technology Fast 50 list for the 3rd consecutive year (2006-2007). This time we improved upon last year, garnering 13th position (as compared to 17th last year) with a compounded annual growth rate of 547%. The title is somewhat of a misnomer given that luck had nothing to do with this. Credit goes to the relentless effort of our amazing team.

Directi was the only company in the Top 15 to have won this award 3 years in a row, a commendable feat, considering that each year becomes the baseline for the next year.

(Divyank collecting the award - Looking smart as ever )

As I always say … this is just the beginning, and there are many more to come

16 Nov, 2007

Building Platforms vs Building Applications - (Part 1 of 3)

Posted by Bhavin Turakhia | (0) Comments

Platforms are in these days and everyone who has a half-decent half-popular Internet application is providing APIs and plugin frameworks to make the application extensible. I have been wanting to pen down my thoughts on the advantages of building a platform as opposed to a monolithic application, and the thought process required for the same, since a while now. Some spare time on a weekend, and this article grew to over 6000 words. So I decided to divide it into a multi-part post to make it an easier read.

Index

• PART I - Advantages of building a Platform
• PART II - How to go about building a Platform - Techniques and Concepts
• PART III - Principles to keep in mind when designing a Platform

This first part will cover the advantages of building a platform as compared to an application. Check the index above for links to the rest of this article.

What is a Platform?

Firstly, lets cover, briefly - what exactly is a Platform vis-a-vis an Application. Marc Andressons blog post does a good job of introducing platforms, and categorizing the types of platforms that exist today. In a nutshell however, any application, which provides ways to -

1. Customize the experience of the application
2. Extend the functionality of the application
3. Pull data out of the application
4. Interact with the application through a programming interface… is a Platform

Advantages of creating a platform

A while ago I had written an article (within my internal company blog) on how in the Web 2.0 era, extensibility and collaborativeness are essential for the success of an application. People wonder about the run-away success of popular Web 2.0 brands such as digg, flickr, facebook, youtube etc and how they achieved such momentum within such a short span of time. One of the fundamental reasons was that all these applications were collaborative by nature ie they made it natural for a user to “want to” - scratch that - “need to” invite other users in order to reap greater benefits from the application. Extensibility in some sense provides overlapping as well as distinct advantages to collaborativeness.

So lets take a look why one should build platforms vis-a-vis stand-alone applications.

Well … to start with - everyone is doing it. And yea, the geek in you wants to :). However building a platform has many commercial advantages apart from just being able to brag about it at the next BarCamp. Here is a braindump of some of them, in no particular order -

Better Functionality
This is by far the most obvious benefit. By building an extensible application you enable others to extend the functionality of your application in ways you didnt think of, or didnt have resources for. This results in more features, developed at a faster pace. There are thousands of applications that currently extend facebooks functionality. I doubt Mark Zuckerberg and his team could have even thought of those many features, both extensive and creative, let alone build them.

Free resources
There is such a thing as free labor. Extensible applications can leverage the time and resources of hundreds and thousands of curious minds, evangelists, organizations, or just geeky users, creating a large free workforce pool. The commercial value of such development can sometimes even outweigh the capital investment by the original company. For instance, the countless plugins built by the community for SalesForce, Atlassian, Facebook etc represent millions of dollars of free development for the benefit of the community as well as the company.

Viral effect / Increased adoption / Greater Momentum
There is no denying the logic that the more people that are invested in an application, the greater the momentum behind it. Microsoft became the behemoth that it is today, primarily because it built a platform and engaged thousands of companies worldwide to create applications for it, thus automatically tying their success to its own success. The more people that are building / customizing your platform, the more people will depend on its adoption, the more people will talk about it, the more people will publicise it, and the more people will propel its growth.

Greater stickitivity from the developer community
You generally feel closer to something you have built yourself. Extensible applications create a sticky following, of individuals who have invested time and effort behind the platform. They will generally not migrate over to competition very easily for the fear of having to start from scratch.

Greater stickitivity from the user community
With a larger number of features ongoingly introduced on your platform, end-users who begin using these extensions / plug-ins also increase their investment in your application, and thus their stickivity. For instance, anyone who begins using various facebook extensions (funwall, slideshare et al), as a user, has invested time in learning it, using it, and has accumulated personal information within it. So a competitor application offering similar functionality would not be sufficient incentive for a user to move their loyalty, given their investment into the incumbant.

Feeling of Ownership and Contribution
A Platform gives users / developers the ability to make a difference, to your application - strengthening their bond / feeling of ownership towards the Platform. You always generate a closer bond towards something you have customized. Remember the good ol’ samurai / nintendo video-gaming days when you spent hours designing and playing custom tracks for excite-bike?

Additionally, everyone likes being acknowledged and recognised by their peer group. Platforms provide opportunities to the ever-growing population of geeks and techies, and even the regular users, to contribute, and hence opens a window for conversation and recognition.

This sums up a basic list of advantages of creating a Platform as opposed to building a closed application.

Click on any of the links below to read further on how one can go about building a platform -

PART II - How to go about building a Platform - Techniques and Concepts »
PART III - Principles to keep in mind when designing a Platform »

16 Nov, 2007

How to go about building a Platform - Techniques and Concepts (Part 2 of 3)

Posted by Bhavin Turakhia | (1) Comments

This is part 2 of a 3 part article on Building Platforms vs Building Applications. Refer the below index to review the other two parts. It is recommended that you read them in order - Part 1, Part 2 and Part 3.

Index

• PART I - Advantages of building a Platform
• PART II - How to go about building a Platform - Techniques and Concepts
• PART III - Principles to keep in mind when designing a Platform

This second part will cover techniques that one can use to build a Platform.

So … how does one go about creating a Platform
The first step is to begin thinking of your application as a platform, early on in your development lifecycle, preferably in the very beginning. Applications need to be designed as a platform from inception, since the cost of post-conversion is high and in some cases results in ugly patchwork.

Secondly, you really need to understand the meaning of creating plugin frameworks, providing APIs etc. It always helps to closely investigate various existing applications that can be classified as a platform. For instance applications like facebook, flickr etc. One of my favorite application development company is Atlassian. They have done an amazing job with JIRA and Confluence, their flagship products. Both applications have an extensive array of plugins contributed by a large number of users worldwide - the result of a very mature, easy-to-use, flexible, well thought out extensible platform. I would strongly recommend any application developer to study existing APIs, mashups, plugin frameworks of existing popular applications to broaden their horizons. It helps identify potential opportunities in terms of providing extensibility, and provides examples of tech implementations of creating a extensible application, cuz believe-you-me good platforms are thought of, and built, ground-up as “platforms”.

Lets investigate the opportunities that any given application has in terms of extensibility and customization -

Theming / Skinning

Your application can allow the user to change colour schemes, fonts, layouts and styles. This can be achieved by creating the application in such a manner that the entire user interface is encoded in separate files which form the skin. You can supply a basic set of skins. If your application becomes popular you may even have third party companies developing and distributing skins for your application. With regards to Theming/Skinning, I will broadly divide the need into two categories -

• The first - applications which are single-user - and their interfaces are experienced only by that user. Most desktop applications fit into this category. For eg Winamp. These applications do not really require to allow skin customization. The only advantage derived would be decorative. Users may like to decorate the application in their own way. However since the application is for their eyes only, it may not make much of a difference
• The second - applications that involve publishing - where a user can use the application to publish and share content with other users. For eg Blogs, Social Networks etc. Here individualism does become important. People want their blogs to look different from their friend’s blogs. Users want the ability to customize the look and feel of their published content. For eg, I may want my facebook wall to look different from other facebook walls. Publishing is kind of like inviting people to your home. If all homes looked the same one would not be excited about showing them off. Each house reflects the personality and tastes of each home owner. For any application that provides a publishing opportunity it would likely make sense to offer some sort of skinning ability

In order to provide skinning ability, in a clean manner, the user interface of your application must be extracted out into separate files which can be plugged back into the application. The files should essentially define the layout of the data / components, style, fonts, colors, style-associated images. For instance, a clean separation of style using CSS / images for any HTML based application would achieve this. Users can be provided the ability to upload and apply an entirely new theme, or modify components of the existing theme.

If the separation is clean, one can encourage third party companies to build templates and themes, and even theme modification tools for your application. For instance one can expose an API or an upload interface which enables 3rd-party vendors to download the current active themes of a user, and provide an easy interface to manipulate them and re-upload them, or create their own.

Customizing / Moving Widgets

Most applications consist of several independent widgets and controls. Applications may allow a user to customize the position, size, prominence, accessibility etc of these widgets. Typically this functionality is only provided for the main dashboard of the application, where a user maybe allowed to add or remove widgets, move them around etc. This type of functionality is useful since different users have different usage patterns in any application.

If a user is allowed to reorganize his/her application dashboard they can move the widgets that they commonly use to areas of the screen that are easily accessible to them. This is sort of like how Windows users manipulate their desktop, taskbar position, systray position etc. Or how facebook allows users to close or move widgets in the facebook profile page of a user.

In order to achieve this it is important to design the UI in a manner where each widget is cleanly separated from the others in its UI definition. Each widget should be a wholly contained object and cleanly defined with no dependencies on other widgets on the page

Content Embedding

Content Embedding refers to embedding content and widgets of a source application in other applications. Generally there are two ways in which an application can show data / information from another application. One is via API calls where a caller application requests data from the callee application using an API call, and the actual formatting and display of the same is handled by the caller application. Content Embedding however provides an easy way to integrate data from one application into another without any complex programming, API integration or UI creation. Generally both the applications must use the same display markup language (in most cases HTML). Content embedding can be done using protocols such as RSS / Atom, or using IFRAMES, javascript blocks etc. Examples of Content Embedding are - youtube allowing you to embed an uploaded video on your own blog, website etc. This is achieved by using a simple HTML code block which can be referenced anywhere on a web page. The advantage to youtube in this case is that it promotes their brand and promotes awareness.

Another example could be a chat widget embedded in a page which shows the presence of one or more users as online or offline OR a facebook widget which shows the mini-feed of a user on his blog. It is important to make it easy for people to integrate your widgets into their websites. The easiest way is by providing clean html code blocks that can be easily inserted into their website. Note however that some blogs and content editing applications may not support insertion of html very easily.
If your application has a large number of widgets it is also important to organize them and enable the user to be able to easily find them. They should be sprinkled at the right places. For instance, in youtube, when I am watching a video, there is a link below that says “embed video”. This is the perfect place I would expect it, and also the perfect place to promote this content embedding widget.

Another interesting way to gain embed-share is to get application developers to reverse integrate your content linking and embedding widgets into their applications. For instance - a wordpress plugin, which adds a link in the “write post” section that says “Click to embed Youtube video”. On clicking it the user can simply enter the url of the video they want to embed. Wordpress can make a call to that URL passing it a special parameter which provides it the required embedding code. Content embedding can also be achieved through RSS / Atom / XML / REST / custom feed protocols. These approaches can be classified as “in-between API and direct content embedding”. While in the true sense they represent an API since the resulting data needs to be parsed and formatted by the caller application, however the returned data is not in the form of objects or standard data types. Also typically simplistic formatting of the received XML can be handled using XSLT or simple javascript and would not require a lot of effort. The call could be an AJAX call and the recvd XML can be displayed in a designated format.

Content Embedding is an important technique for achieving extensibility. It is easy to use and easy to develop. It also has an additional advantage of being able to maintain greater control over the display format of the content. It does have a few disadvantages. Content embedding widgets do not provide as much power and flexibility as a native API would for a container 3rd party application. Also, since part or all of the formatting maybe defined by you (the source application), it represents additional effort on your part as a developer, as opposed to simply exposing a direct API function. Lastly, pre-formatted content blocks when embedded into user applications or websites, may not necessarily gel with the layout and format of the container content. It therefore is important for the widget to provide certain flexibility by parameterizing layout, color, style, font etc. Offcourse this would not be required if the Content embedding is done through an XML / RSS type protocol.

When designing an application for Content Embedding you need to first determine what content do you wish to allow users to embed within other applications / websites. You also need to decide whether the container 3rd party application will be a desktop application / a website / both. If the content that requires to be embedded within another application, requires user authentication, you will need to determine a security model to ensure that the application / website within which the content is being embedded cannot compromise the security of the user in any manner. Lastly you need to determine whether you will provide formatted content blocks, an XML API or both.

UI Mashups

UI Mashups, is another technique for building extensible applications, and involves allowing the combination of multiple web services to create an entirely new service. For instance lets say a chat application returns the current online status with the IP Address of a user. The IP Address can then be fed into a geo-mapping application to figure out the location of the user which can be fed to a map application to show that location as a green dot on a map for an online user. This example uses three separate services to generate a rich mashup showing both the location and presence status of a user.

Mashups can get fairly creative. For instance, another mashup could pull your friends list from your social network / chat application, and create a world clock showing the current time where they are.

Mashups maybe generated using services of your application in combination with other applications. If you want users to be able to create mashups out of data from your application, you need to provide data that developers / users can use to create these types of mashups.

If you want users to create mashups from your own applications you should design your application as a large number of tiny loosely coupled individual apps rather than one monolithic application. For instance take the above example. If you were running a social network, you could hardcode a feature into your application which shows a local time clock for a particular friend. Alternatively you can build multiple tiny services - one which provides a list of online buddies, another which accepts a list of buddies and provides their ip addresses, another which accepts a list of ip addresses and provides a list of countries, and a final one which accepts a list of countries and creates a timeclock interface. How granular you go is entirely a business call on your part.

Mashups are a relatively new phenomenon, but the possibilities are limitless. The output of any service can be piped in as an input to another service, as long as the interchange format is understood, creating a long pipeline of intelligent services that result in useful output. Mashups also divide processing requirements across multiple server nodes, thus allowing greater scaling of the backend web services.

APIs

This is the battle-tested age-old method of making your application extensible. APIs enable developers to directly interact with the core of the application. They are easy to provide on the part of a source application and with open standards such as SOAP, XML RPC, REST etc, developing client applications to an API is quite easy. APIs offer the maximum flexibility in terms of extensibility to a potential developer. APIs can serve different purposes. Provisioning APIs are APIs which enable developers to provision a service on a platform. For instance a mail service may expose an API to facilitate direct creation of email accounts on the mail server.

Data access APIs facilitate access to data, such as profile information, statistics etc. Data modification APIs allow developers to modify data in an application.

Provisioning APIs and Data Modification APIs sometimes require significant processing at the backend, and may be batched. This means that calls to those functions maybe accepted and queued by the server to be processed subsequently. These are termed as asynchronous calls since the actual processing of the call takes place asynchronously. The outcome of the call maybe communicated back to the caller through a callback mechanism OR via an out-of-band method such as email. Even Data access APIs may at times be batched if the result to be returned contains a large amount of data. SOAP / XML-RPC based APIs return XML strings which are directly serializable into objects in most platforms. Custom APIs which return custom XML packets require custom serializers or parsers at the client end. If the API is non-standard, it typically helps to provide multi-platform client side kits, which are essentially wrappers around the API calls in different languages, that serialize the data returned into objects / native types.

Alongwith an API an application must provide the following for ease of development -

• Up-to-date documentation of all the methods, their parameters, the validation routines for those parameters, return values, format of return values, as well as all error conditions
• A sandbox environment where a developer can try out the API
• A web interface from where a developer can make calls and inspect the XML packets sent and received (incase the API is an XML API)
• Sample code which shows the implementation of a few of the API functions

Additionally, you should also setup forums or mailing lists where API users can collaborate and assist one another. The forums also serve as a repository of queries and solutions for newer developers.

APIs require careful planning from a backward compatibility standpoint. Once an API is launched, and widely adopted, it becomes very difficult to change its method signature. If there are already hundreds of applications written around the API, one cannot simply modify the interface since it would break these applications and in turn the user experience. Therefore one must plan the interface of the methods that one chooses to expose with utmost care. Always keep in mind what the API needs to look like in the future, and ensure the current architecture takes into account future updates without breaking the interface.

An ideal approach that you can use to create an API, is to ensure that you build your application as a multi-tier application from ground up. Maintain a clear separation between the user interface and the core api. In a sense your user interface becomes a client on top of your own API. A clean multi-tier design can go a long way in terms of building an application with an easy to use API

Event driven Architecture

Another technique for developing extensible applications is to create an event driven architecture. Model your application in the form of events and actions. For instance lets take facebook. Some of the events that take place within facebook are posts on your wall, private message, photo tags etc. Each event can trigger certain actions which in turn can lead to other events. An event driven architecture consists of subroutines which register for specific events. Then when that event occurs, the subroutines which had registered for the event are fired. This type of an architecture makes it easy for external applications to register for, or subscribe to events within your application, and provide unique functionality. For instance a mail server may allow applications to register for events of type “incoming mail”, “disk quota full” and so on. Client applications can register for these events. The code on the client application would be invoked when the event it registered for occurs. The client could display a toaster popup to denote a new incoming mail etc. Typically event driven architectures work using callbacks, wherein a client application registers for an event by providing the URI of a function to be called when the event takes place. The server application triggers a callback to this URI / service when the event occurs. The server may process the callbacks as a batch by queueing them and processing them using a threadpool.

Applications built using an event driven architecture can have hundreds of different events. In this case it maybe useful to classify the events into different types. This way a client can register for a set of events by registering for all events of a particular type.

Another name for an event-driven architecture is - publisher-subscriber model. Event driven architectures are push based models, wherein, a server proactively pushes a notification or message to the URI provided by a client upon an event taking place, instead of a client polling the server regularly to figure out whether an event has occurred. This results in higher efficiency, lower bandwidth utilization, low latency and a better user experience.

Protocol Translation / Protocol Bridging or Multi-Protocol Support

Protocol translation or protocol bridging is a great way to expand your client base and provide multiple ways of accessing your service. Invariably different users and different use case scenarios require different ways of accessing a service or content. For eg take a content management system. The purpose of a content management system is to store different types of douments and enable users to access them. A typical content management system may make the stored documents available through a web interface which users can use through a browser. However an extensible application would allow users to access the same content through various other protocols such as WebDAV, FTP, File share, etc.

Infact a well designed application would provide clear abstraction between the data layer and the access layer. In this manner, different access protocols can be layered on top of the core interface which accesses the data. These access plugins can be developed by anyone, including the user or a third party vendor.

Another example can be a messaging system, for eg an instant messaging system. If designed in a manner that allows third party vendors to create protocol bridges, plugins could be developed that bridge over to other instant messaging protocols or messaging systems such as mail, SMS and so on.

An application typically provides a core service. This core service is then accessed through a standard protocol such as HTTP. If the protocol and the core service are loosely coupled, one can easily layer other protocols in to access the same service.

Outbound API integration or Reverse API Integration

Reverse API integration another technique for providing extensibility through callbacks. Basically Reverse API calls or Outbound API Calls, require you, as an application vendor, to provide an interface definition for writing functions / methods, to which your application can make outbound calls to. In that sense it can be thought of as a subset of an event driven Architecture, but it does deserve its own independent mention.

For eg, lets take a financial transaction management platform, such as a payment gateway. A payment gateway facilitates B2B and B2C transactions between entities. Such a Payment Gateway platform can define an outbound call interface which can be implemented by varous accounting software, where the payment platform can make a call to an accounting system that implements such interface, to record a transaction in the books of accounts as soon as it takes place within the gateway. Since there are so many different accounting systems (eg Quicken, Oracle Financials, Tally, SAP etc) they can all implement the standard interface published by the payment gateway allowing the payment platform to connect to them. Similarily the payment system can also define a separate interface for provisioning software, whereby, when a payment is received by a vendor from a customer, then the payment platform can make a outbound call to the vendors provisioning platform with predefined parameters. The vendor provisioning system can then decide the appropriate next action such as delivering the service, creating the customer account etc. Most platforms prefer using HTTP as a protocol of choice for such outbound calls. This is because it is very easy to implement, works through firewalls, is easy to understand and easy to troubleshoot.

Plugins

A more tightly coupled form of extensibility is plugin support. A plugin is a tightly coupled extension to an application. It runs in the same process space as the application itself. It typically consists of UI widgets and associated services. It accesses the same core data and likely some of the APIs as the primary application. It extends the functionality and interface of the existing application. Some examples are - toolbars for browsers, toolbars for email clients, plugins for confluence and jira, facebook applications, Yahoo widgets, Zimbra Zimlets etc.

Plugins are generally tightly integrated into the primary application. The application usually provides a proprietary plugin development process / guide. The application provides hooks and APIs and file formats and processes that need to be used to create a workable plugin. Usually creating an elaborate plugin is relatively more complex than the other forms of extensibility we have already spoken about. However, the application vendor may make the process very simple (eg Facebook has done an amazing job of creating a point and click process to build your own integrated plugin). A plugin likely has its own UI, which blends in with the UI of the host application. Styles and standards for the UI maybe defined by the host application.

Depending on the application vendor and the functionality / flexibility desired, plugin development maybe as simple as point and click (trivial Facebook plugins) or simple XML (Zimbra zimlets) or involve complex coding (Confluence and JIRA plugins).

Creating a robust plugin framework requires considerably more preparation, thinking and planning than most other forms of extensibility. Typically a plugin development framework will provide APIs, callbacks, abillity to manipulate UI, granular data access, a security model, standards and guidelines for design, proprietary styling and markup and more.
For eg - the Facebook Development platform provides a Facebook API, no callbacks yet (to my knowledge), ability to add UI components and load external pages within the Facebook application, access to users data, a security framework where the user defines access policies of a plugin, a markup language called FBML for formatting pages, guidelines for designing the application etc. Though facebook’s development platform allows UI customization, a majority of the UI is not hosted within facebook, but rather on the plugin developers servers. This is slightly more loosely coupled than a conventional plugin model would be. This type of loose coupling has its advantages and disadvantages for Facebook inasmuch as plugin developers have to invest in infrastructural resources, but at the same time get greater flexibility and control over the application. Another robust example is the APIs, callbacks, event listeners, ability to add UI components, access to the data, etc provided by Atlassian for development of Confluence and JIRA plugins.

This sums up a short list of techniques one can use to think of their application as a Platform. Click on any of the links below to read the other two parts of this 3 part post -

PART I - Advantages of building a Platform »
PART III - Principles to keep in mind when designing a Platform »

16 Nov, 2007

Principles to keep in mind when designing a Platform (Part 3 of 3)

Posted by Bhavin Turakhia | (1) Comments

This is part 3 of a 3 part article on Building Platforms vs Building Applications. Refer the below index to review the other two parts. It is recommended that you read them in order - Part 1, Part 2 and Part 3.

Index

• PART I - Advantages of building a Platform
• PART II - How to go about building a Platform - Techniques and Concepts
• PART III - Principles to keep in mind when designing a Platform

This third part cover certain Principles to keep in mind when designing and deploying your platform. These are -

Abstraction and Loose coupling are your friends - always go n-tier

Design your software as an n-tier application. Ensure the data layer, the business logic layer and the UI layer are loosely coupled. Infact even within the business logic layer, there is opportunity to loosely couple or layer various functionality such as authentication, provisioning and so on. The more loosely coupled your architecture, the greater the possibility of providing hooks and APIs for extensibility.

Think of yourself as your own client

One easy way that I have used to get around to designing extensible applications, is to begin thinking of myself as my own client. What this means is that I divide my application into various modules, and then assume each module is being developed by a separate entity.
For instance, one can assume that separate entities are building your user interfaces and core libraries. This way not only does your design starts out as being loosely coupled but also you begin thinking from the perspective of a third party vendor. Since you will be the first consumer of your own platform, you will be able to think of the design and any pitfalls during initial development.

Identify the core foundation of your platform and build the rest as plugins

Each application has a core set of functions or core purpose. Identify this for your application. Clearly document / outline it. Then build the rest of your application as plugins that hook into the core. This is actually a very interesting way of designing an application, where your application is divided into multiple independent applications all of which plugin to or hook into the core. Infact, all successful platforms are invariably designed in this manner to start with.

Operating Systems (which in some sense are the most successful platforms) have always been designed this way. The core OS provides device access and a process space to run applications. All other features, whether bundled with the OS or installed separately, built by the same vendor or by a third party, access this core platform. Microsoft would have never gained any traction, if their OS and all bundled applications were built as a monolithic application. By separating the core OS functions from the user applications, they created a platform rather than a monolithic software.
Atlassian’s Confluence is another application that has achieved this brilliantly. At the core of their platform is a content management system, revision control system and permission management system. Everything else, all their user interfaces, including their admin interfaces hook into this core as plugins. This has resulted in a robust plugin framework that external entities can use in the same manner as atlassian, leading to a large library of plugins contributed by third party coders Facebook is a more recent example. Many of facebooks own features and applications are built as plugins to their core platform. Their core platform provides a social network, a set of privacy definitions, and a publisher-subscriber model for events. Many of their features layer on top of these core libraries, which they have now exposed through their Developer Network, enabling others to build identical extensions.

Update: If you want to take this principle one step further, then another way of breaking down your application is to have NO CORE. Think of all parts of your application as loosely coupled components of equal importance.

Leverage / Create easy-to-implement protocols - HTTP is your best friend

Noone wants to write to a CORBA API or a custom binary protocol :). While most applications will go the HTTP / SOAP / REST route without thinking twice, this is still an important element. Make your APIs as easy to understand and implement as possible. Use standard easy-to-use protocols. Ensure that the method signatures / interfaces are self explanatory. The easier it is to implement your API, the more momentum your platform will gain.

Componentize / Widgetify your UI

Do not build single, hard-coded monolithic User Interfaces. While the OOPs paradigm has taught us to think of our core code in the form of modular classes and methods, there unfortunately is no such paradigm for Web UI. Desktop UI does follow a component model to a large degree. When building a desktop app, users are forced to think of controls and components. But when designing web applications, most developers unfortunately treat a page as a unit. It is important to begin thinking of a page as a container of independent widgets. Design your Web UI such that there are no inter-dependencies between multiple controls on a single page.

Build an event driven architecture

Every application has events which result in actions. Instead of hardcoding these as a linear procedural piece of code, you can choose to build or use existing event-driven models. Wherever you find events and actions within your applications, you can build event listeners, event queues and event handlers

Build an easy discovery/installation/update mechanism for extensions / plugins

So far I have spoken about how you need to make it easy for a developer to build extensions. Another important element is how easy is it for a customer to begin using those extensions. You may have a brilliant platform, but if your users cannot find the extensions or need a phd in compsci to install them, you will not achieve any success. The two elements to keep in mind are discovery and ease of installation/updation.

Once again, at the risk of sounding like I have an agenda with them, Atlassian does a very good job of this. They have an extensive online wiki which categorizes all their plugins and extensions for Confluence and JIRA. Each extension also has its own page, containing all the relevant information about that plugin, its installation, usage etc. All-in-all it is quite well organized. Now if that wasnt enough, recently they began bundling their Confluence Repository Client within their product. So now the entire plugin directory is available from within the Admin interface of their product. This means any user who uses confluence can discover and directly install all plugins from within the application itself. This makes both the discovery and the installation/updation process fairly easy. I think this is a great strategy - ie to bundle an interface which allows plugin discovery and installation within the primary application itself. Interestingly, the interface (the Confluence Repository Client) that manages this within Confluence, is itself an extension :).

Noone however has done this better than facebook. They have not only built an very intuitive interface for installation/uninstallation/updation of plugins (point and click) - but they have linked discovery to their social network. Each user is notified about the plugins installed by their friends, creating a unparalleled viral effect, and a never-before implemented discovery model.

While it may not be possible in all applications, try and keep the process of plugin installation as simple as possible. The lesser the friction the faster the adoption. For instance a desktop app plugin which requires superuser privileges to install, has additional friction in comparison with one that doesnt.

Design a future-proof database / datastore

A sure-fire way to reverse the success of your platform is to ignore backward compatibility. The worst thing you can do, is enable third party vendors to develop plugins / extensions for your platform, and then make changes to your platform that render their extensions incompatible.

One of the common reasons for changes in an API is a poorly thought out data structure. All applications consist of code that manipulates data. Typically this information and metadata is stored in an RDBMS or other data stores. When adding new features, one needs to add new columns / attributes to an existing data definition. If the database structure was not created anticipating such future additions, then an upgrade may require a drastic change to the data model. This change in turn impacts multiple areas of the platform and its APIs.

It is important to consider a long term roadmap of your application when designing your database structure. Try and design the database in a manner that future feature additions do not impact the data structure drastically. Some other situation-specific tips are - do not try and put everything into one table, or even one database. If you build your application out such that everything is in a single monolithic linked data structure, a single change will have multiple repercussions. Many a times when using RDBMses developers are motivated to do just that to obtain referential integrity advantages even when they are not needed. IMHO Referential integrity is quite over-rated.

Build a robust Security and User Privacy model

Think through what type of access do you want to provide to a third party extensions / plugin. Design a security framework which provides various levels of access which can be controlled by you and/or the user. For instance, facebook allows a user to control what information a facebook application can access of that user. Security and User Privacy are two separate elements. While user privacy controls, define the information of a user that can be accessed by an extension, Security refers to ensuring that a plugin / extension / 3rd-party application does not obtain unauthorized access to any information, does not performs any unauthorized action, and does not compromise the security of the platform in any manner.

Take good care of your third party developer network

The success of a platform is directly proportional to the developer momentum you can create. Many successful companies go to great lengths when it comes to taking care of their third party developer network. Here are a few ideas -

• Provide detailed clear documentation. One good way of doing this is to host it on a wiki whereby the community can contribute to it. The wiki can bemoderated to prevent abuse. Your documentation should contain an implementation guide, API reference, error guide, downloadable example scripts, tutorials, videos, testing tools etc.
• Provide a sandbox environment to facilitate testing. The sandbox environment should be identical to the live environment. Developers should be able to create multiple sandbox accounts. Developers should have the capability of resetting the sandbox environment
• Many companies provide developers with additional assistance such as a free hosting environment, a dedicated bug tracker, an SVN or CVS code repository
• Organize workshops, roadshows and geek-fests for your developer following. Give them opportunity to gain recognition within peer groups and collaborate with one another face-to-face.
• Give them access to talk to and connect with your tech team, through forums / structured chats and in any other manner possible

This sums up a basic list of principles you should keep in mind when designing a platform. While I have attempted to pen down as many elements as I could think of, you need to keep in mind when designing a platform, nothing beats practical training. I would strongly reccommend checking out Facebook, Confluence, Ning, Salesforce and other such existing platforms, their implementations, APIs, documentation, strategies etc.

Click on any of the links below to read the other two parts of this 3 part post -

PART I - Advantages of building a Platform »
PART II - How to go about building a Platform - Techniques and Concepts »

5 Nov, 2007

Demystifying Storage and Building large Storage Networks - Part I

Posted by Bhavin Turakhia | (2) Comments

I have been studying storage systems over the last few months, and have been wanting to create a Training Course for it since a while now. I decided to get down to it few weeks ago and started creating a PPT for speaking at BarCampMumbai. Here are the slides which I used to give this talk. This is Part I - of what I anticipate to be a 3 part Training Course. The ppt is fairly detailed, and covers hard drives, RAID levels, and a few DAS and SAN topologies. I plan on posting a comprehensive training video of the same separately.

(Visit http://www.slideshare.net/bhavintu79/demystifying-storage for the fullscreen version)

Part II and Part III (whenever I get around to building them) will cover - Complex SAN configurations, iSCSI, NAS, Clustered Storage and other associated topics