Evolving Wikis

When I Googled Wiki, I got more than 412 million hits. While Wikis have been around for more than 10 years, their visibility has increased only in the last few years – mostly thanks to Wikipedia.

So where do wikis go from their initial role as collaborative tools? Let us look at both some current and many possible future developments.

1. Wiki Scripts
Integration of scripting into wiki lets you build wiki applications. Twiki is a great example of this. Open source products like Moin-moin and commercial products like JotSpot do this too. Client-side scripting with Javascript and server side scripting may both be supported.

2. Wiki Plug-in Modules
Plug-in modules allow new functionality to be added to the wiki through a simple component architecture. Moin-moin, Jotspot and others have a scheme for writing plug-in modules that let you extend/customize these wikis

3. Wiki Skins
Changing the skin of a wiki allows you to change the look and feel of a wiki. The initial set of wikis were bland affairs. Now there is a lot more color and style being added through custom skins. Just look at something like Wetpaint to get an idea about the trends in this direction.

4. Wiki Templates
The initial wiki had only one type of page. A title and a bunch of text with links. Nowadays most of the wikis have templates you can choose when creating a new wiki page. Here is the description about templates from Moin-moin FAQ:

if you want certain types of pages to have a similar format (similar headings, organization, etc.), you just define a page that ends in Template, and when creating pages of this type, select that template and edit it. The wiki fills in the starting content for you. Templates are editable wiki pages like any other.
To create a Template page, just create a new page called <something>Template

5. Semantic Wikis
This is the one, I am most excited about. Semantic wikis associate model based metadata with wiki pages. This increases the ability for applications to gather knowledge from wikis. Here is what Wikipedia says about the purpose of Semantic Wikis.

As Wikis often serve as CMS or knowledge management tools, Semantic Wikis try to enhance them and allow users to make their internal knowledge more explicit and more formal, so that e. g. it can be searched in better ways than just with keywords.

Some systems are aimed at Personal knowledge management, some more at knowledge management for communities. The amount of formalisation varies: existing systems range from primarily content oriented (like Semantic MediaWiki) over content oriented with strong formal background (like IkeWiki) to systems where the formal knowledge is the primary interest (like Platypus Wiki). Also, Semantic Wiki systems differ in the level of ontology support they offer. While most systems store their data as RDF, some even support various levels of ontology reasoning.

6. Attaching Documents to Wiki Pages
Several wikis allow you to attach documents to a wiki page. Some of them include adaptors that understand different document formats, index documents and allow search. This allows a small business to use wikis as low end content management systems. You can also use this capability to use Wiki as a Personal Knowledge Management tool. I had written earlier about using Wikis as backup for my Outlook email client.

7. Wiki Application Programming Interfaces
Application programming interfaces (aka APIs) allow wikis to integrate easily with other applications. This allows other applications to send data to the wikis and get data from wikis. This will also allow wiki information embedded into other, non-wiki applications.

Here is a Wiki Matrix that cover features of various wiki products. This cool resource, allows you to pick a few wikis and dynamically contruct and view a matrix of supported features.

So where do we go from here? I see a lot of exciting possibilities. Some of them may already be there:

  • Wikis as Application Platforms (saw a couple of presentations on this at MashupCamp2 with IBM leading a very ambitious effort).
  • Wikis and Microformats – Embedding microformats in wikis will increase the capability for wikis to become a canvas for building semantic webs.
  • Semantic Wikis are an exciting development. They are at a nascent stage and you will probably see more activity in this space.
  • Wikis and mashups – Imagine a mashup that takes data from a wiki and mashes it with other webservices like maps. A virtual tour guide may let you use a map as the UI, and explore historic events/places combing information from Wikipedia and Google Maps.
  • Wikis as Mashups – Wiki as a mashup canvas. This is an exciting idea with a lot of potential. There is similarity with another similar concept – Wiki as an Application Platform.
  • WebServices interface to wikis can explode the possibilities of using wiki in ways that we have not thought about. This allows wiki integration into both mashups and enterprise applications.
  • Enterprise wikis – This subject deserves a full blog of its own. I can see heavy use in HR, HelpDesk, Organizational Knowledge bases as a start. But enterprise wikis can be used whereever informal collaboration is required – product development, product rollout, marketing campaigns etc. Better than the current method of using emails and less formal than using portals.
  • Wikis and XML – The next generation of word processors all support XML as the native format. We can imagine this being the case with wikis too. The benefits of using XML as the underlying data format for wikis are too numerous to elaborate here. A few I can think of include structured content, seamless integration with the next generation document editors (aka wordprocessors), vertical wikis (wikis based on vertical XML vocabularies)
  • Wikis and RSS – RSS may act as both input and output format for wikis. Every wiki topic is a potential RSS feed. You may be able to send information into the wiki through an RSS feed too
  • Wiki and Search – Most of the wikis currently have a great search capability, but like Google, they are just based on keywords.  The possibilities for search integration are endless. You can search for data as well as meta data. Some of the ideas like page ranking take a very different meaning when applied to wikis.

I probably just scratched the surface of possibilities. I have not even touched upon multi-media wikis, wikis as game platforms, wiki-movies, wiki-previews and editing in multi-media documents and many others. I will expand on some of these ideas in future posts.

Internet Singularity

What an interesting concept! From Thinking Beyond Web 2.0: Social Computing and the Internet Singularity.

Quoting a quote from Dion’s blog:

The idea that a deeper and tighter coupling between the online and offline worlds will accelerate science, business, society, and self-actualization.” – Dr. Gary Flake

Dion talks about convergence and cross pollination of ideas from Web 2.0 to enterprises.

Here are some thoughts/wishes.

  • Mashups become as simple as writing Excel Macros
  • AJAX is not an after thought but gets absorbed into the way we build UI
  • Highlevel declarative Mashup Markup Languages spring up
  • Webservices (both lite and heavy) take root and may become the norm
  • Loosely coupled applications with thousands and even millions of components appear as different services
  • A robust platform springs up with webservices proxies and other infrastructural services
  • Data Services ( a new kind of service where database capabilities are available as webservices) absorb the complexity of storing, retrieving and scaling data access
  • Semantics take hold. Whether it is through Microformats or RDF or some fusion of these, does not really matter
  • Social Computingness (wiki-ness, blog-ness, ajax-ness, mashup-ness, openapis-ness, software-as-service-ness) become the core philosophies of development

I like the spirt of this whole thing. Just wish that is where we are going. It will not kill jobs or companies. There is so much to do.

Mashup or Shutup

Yahoo about Hack Day.

We have no idea what’s going to happen at Hack Day. We never do. We’re pretty sure we’ll be amazed. We’re still working out the final details, but we’ll be posting more here soon.

If you’re interested in participating, just fill in the form below and we’ll be in touch. Come build something! Mashup or shutup!

If you are developer in the Silicon Valley, it may be worth hanging out there. Checkout and register here. Read about this in TechCrunch.

Why I Love Python

While my feeling for Python is not exactly love, it is close. I am not a professional developer. So I dabble in Python a lot. It allows me to do a few nifty things with very little learning. And the code looks simple. So when I found the reference to Bruce’s Why I love Python presentation, I could not resist downloading it. And I hope Bruce won’t mind my reproducing the creative graphic.


I started using it, because many people I respect (like Jon Udell, Danny Ayers, Mark Pilgrim, Sam Ruby) seem to use it. I tried to find out whether anyone else’s is in love with Python, and was pleasantly surprised.

Update: 28th June 2020

Now I have a few more reasons to like it.

  • We build all our tools and products with Python. With 3.x it is even more attractive.
  • It has become one of the most popular languages for building AI, Machine Learning and Data Science applications.
  • Many educational institutions teach Python as the first language to teach programming. Since Python is a very high-level language, students can get started and build things easily.
  • We use Python to teach our courses on building WebApps (with a micro-framework called Flask), Data Apps and Machine Learning Apps.


Python Learning Material

Life is too short – You need Python

Talk: Scalable Web Architectures

How do you build a product like Flickr or LiveJournal? What are the issues of scalability? Where do you start? How do you learn to scale? This was the theme of the talk at the SDForum event on Software Architecture and Modeling SIG.

The talk was mesmerizing. For almost two hours, ideas and thoughts flowed like water, out of Cal. He would pause to answer some questions, get back to his thought stream. Almost felt as if he was thinking aloud. I think most of us did not even want to inturrupt. It was like watching a science fiction movie. Thoughts were racing in my mind. How did it start? Where did this brilliant young man learn all this from? Obviously other members of the audience had similar questions. Someone asked him what Flickr looked like when they started.

Cal talked about the humble beginnings of Flickr with a few servers in a colocation. Here is the picture of Flickr in its current configuration.


Cal’s slides are here. Cal mentioned that lots of people took inspiration from Live Journal’s back end architecture, which, incidentally is completely built out of LAMP (Linux, Apache, MySQL, Perl/PHP/Python/Ruby). A presentation on Live Journal’s backend is here.

I walked away from this talk with several thoughts:

  • This stuff is hard and takes a lot of dedication to improvement
  • Managing data(bases) seems to be one of the most critical components of scalability
  • LAMP is worth serious consideration since there seems to be so much experience in the industry in using it
  • PHP is worth looking at too. Two of the products I like most – WordPress and Flickr both are built using PHP and both seem to scale reasonably well
  • It is fascinating to observe great minds at work. There is so much intelligence in the world, so there is still hope

Using Wiki as a Store for Mail, Chats and other Documents

How about using a Wiki as a way to persist your useful maill?I used to use  Microsoft Outlook as my mail client but now mostly use Gmail. Most of the mail I receive can be classified into one of the types:

Directly related to my work. This is the highest priority and probably consumes most of my work day. This include specs, design notes, email based discussions and all the work that used to be paper before and now is transformed into email. This belongs to my worklog.

Request for Information
Requests for information from customers and partners. Normally write up some stuff or reuse existing replies. Would love to take these frequently asked questions and move them out of my email.

Information to Consume
This type of mail does not require immediate attention. It goes into a reading list, to be read at the end of the day or when I have some time away from pressing work. At the end of processing this mail (or a thread in the case of discussion groups) I either end up with a set of reference material or action items. The reference material goes into my wiki.
Other Stuff
These are bills to pay, stuff to do that you cannot escape from, meeting requests and other such trivia. Mostly ends up in a calendar or task list and discarded once completed.
After dealing with each one of these types of email, I would like to keep them away (preferably away from my mail client). One of the ways I am trying to do this is to store them in a Wiki. I use low tech methods – like copy and paste. I use a desktop wiki called moin moin desktop edition (a python based wiki). Most of the time I keep each email message as a wiki page.

Here are some benefits, I find in using the wiki as my mail store:

1. It keeps my outlook data files small. I notice that outlook is faster when you have smaller mail data files.
2. It is much easier to search the wiki than outlook. This may change in future.
3. I can add some comments to the mail page (to remember the context for future references)

4. I can link related items easily. All I need to do is to use some wiki words as tags

5. I actually have a worklog since I post both mail and IM chats to the wiki

6 . In addition to the worklog, I maintain an idealog. So when I get an idea, I need to do a few searches and get a bunch of related pages and mail messages to link to.

7. I have one place where most of the things I want to remember are stored. Makes life simple.

I am seriously thinking of building a couple of utilities (like outlook plug-ins) to automatically post from a specific folder (let us call it wikifolder) into the wiki making this process even simpler.

Even though I used Outlook as an example, you may be able to use any other mail client or wiki engine. If your wiki engine supports postings through email, it may be even simpler to post than copying and pasting.

Web Services: WS-Policy

I read the early specifications of ws-policy, almost two years ago and was pretty impressed. I felt it could be abstracted for reuse in other places (not just web-services) as well. Toufic Boubez talks about WS-Policy in this interview on why you need something like ws-policy.

Here’s the fundamental problem. In order for Web services to work, period — not to work well, not to work better, but just to work — you need more than the API. The API being the WSDL. The WSDL tells you where to send the SOAP request and what the operations are. But that’s not enough for a requestor and a service provider to actually work together. There are a lot of other things that are not API-related — security mechanisms, credentials, encryption, reliable messaging, etc. — that cannot be expressed in a WSDL. That’s the crux of the issue. How do you express those things so that you can have a requestor and a service provider actually talk to each other? WSDL is just a small, small part of it. It’s just the description of the API. It doesn’t describe anything else that is around the API.

WS-Policy, as I recall from an earlier look, is structured well. Need to take some of the XML patterns in that specification and apply it elsewhere.

Multi-core Paradigm Shift?

In the last couple of years, we have heard more and more about mult-core processors. AMD, Intel, IBM and Sun  are the more visible ones. You can get a more comprehensive list of vendors here.  However, the software tools for multi-core development has not really caught up with the hardware development. Intel is trying to fix this by funding training in several universities around the world.

“To usher in a new generation of computing technology and bring creative new products to market, it’s crucial to educate tomorrow’s software developers to architect, develop and debug the next generation of software for modern, multi-core platforms,” said Renee James, corporate vice president and general manager of Intel’s Software and Solutions Group. “The full potential of multi-core based systems to deliver great performance and expanded usages is unleashed when software is designed to take advantage of the full capabilities of the machine. Working with the world’s best universities, Intel is creating the future for performance computing.”

Here is the announcement from Intel.

Gaming systems like XBox360 from Microsoft with its three core Power PC and Sony PS3 with the IBM Cell processor seem to be some of the initial adopters of multi-core processors.

Product Capability?

Is it better to build a powerful, flexible product with lots of capability? Or is it better to build a simple product which can be used to its fullest potential by most of your users? This is a tough decision. I have not seen it said any better than this.


According to Are your users stuck in “P” mode? we have to keep asking ourselves:

  1. Are we focusing too much on the tool rather than the thing our users are trying to do with the tool
  2. Is the product just too damn hard to use even if a user does know what they want to do with it?
  3. Do we encourage/support a user community that emphasizes mastery of the thing the tool is for?
  4. Do we train our users to become better at the thing they use the tool for, in a way that helps make the need for all those other features seem obvious?

It is a great read. I definitely recommend it. The scope question is a difficult one.

1. There are rarely any clear profile of the user for software. As Matt mentioned in WordCamp2006, “Software has no single killer feature. There are hudreds or thousands of them and each user has his own set of 10 or 20″

2. Even if you start with a version of the software that is like the picture on the right above, over a period of time you tend to have a large circle (feature list) with several smaller overlapping purple circles. Each purple circle represents the needs of a single class of user.

3. We know that products get more complex before they get simple. Take Microsoft Word for example. After adding features over 5+ versions, the product team did something very different in the latest version. The simply made the access to features better. Someone mentioned in one of the Professional Developer Conference that this was because there were users asking for features that already existed in the product.

4. I think there are a spectrum of users from a casual user to a professional/expert. Their needs vary. There is enough overlap across these users/needs to make a product that looks like the figure on the left with a twist. Instead of one little purple circle, there will be many.

For example, our product InfoMinder’s use varies. About 80% of the users use about 20% of its capabilities. But there are a few, probably less than 5% that use most of the features.
What is your experience, working with products and/or building them?