Tracking Tech Trends

If you are a developer, development manager, a software company, a tech startup or an investor, you need some way of tracking short/medium/long term trends. The Google Trends introduction last week made me think about the tools available.

1. Google Trends

This is a great tool for tracking search trends. You can type a single phrase like "venture funding" and get a trend graph. In addition, you can get information about the Top 10 cities, regions and languages for the search. You can also type multiple topics like "venture funding, bootstrapping" and see the relative trends.

2. Alexa Web Search – Top 500

Alexa is an Amazon service that tracks the popularity of web sites. Their traffic rankings method is described here.

Alexa computes traffic rankings by analyzing the Web usage of millions of Alexa Toolbar users. The information is sorted, sifted, anonymized, counted, and computed, until, finally, we get the traffic rankings shown in the Alexa service.

I mostly watch the Top-100 sites, but it may be interesting to track the Top-500. Here are a few tips on how I use it. The rankings give you a sense of who the movers and shakers are. Do a bit of extrapolation based on which industries they belong to and you get some ideas on what is going on. For example, many web 2.0 companies – myspace, facebook etc. are moving up. Alexa provides RSS feeds for many of its services.

3. Blog Trends with Technorati

Type a search term, say "Java" in the search link. Then click "more" below the chart displayed on the left of the page. You will get to a page like this. Now you can refine the search a bit using the various options in "authority" and blog type. This is a pretty neat tool. Till I tried it out, I did not even know that bloggers are ranked by authority.

Technorati also publishes some interesting white papers on the emerging trends on use of blogs.

4. PubSub

Another very useful resource is PubSub. You can subscribe to keywords/phrases and receive alerts. My favorite part of this site is PubStats.

PubSub monitors millions of feeds. By generating a list of all the URLs contained in entries of each feed, it's possible to determine a site's relevance just from the number of incoming links it has. LinkRank goes one step further and calculates a score for each linking site. Sites are then scored based on the score of the sites that link to them.

5. Del.icio.us

Del.icio.us is a collaborative bookmarking service. It allows people to store their bookmarks and tag them. As a user, you can retrieve your own bookmarks, popular ones and others' by using tags. The level of activity and popularity provide you with an idea on some of the short term trends.

6. Diggdot.us

Diggdot.us combines Digg.com, Slashdot and del.icio.us to provide an integrated site for tracking the most popular technology related posts.

7. Tag Clouds

Over the past year or so, tagging is gaining popularity. Several services including Yahoo, Technorati provide tag clouds. A tag cloud is a list of tags displayed in a box. The tags used more heavily have bigger fonts. Here is an a tag cloud on Web 2.0 from technorati.

Tag clouds provide great visualization for popular tags. There are tag cloud animations that show you how the tag popularity increases over a period of time. Here is one of the most popular tag cloud animations from Jon Udell.

There are various other methods:

– Tracking mashup activity,

– Following TechCrunch and eHub,

– Tracking booklists,

– Using Google/Yahoo alerts,

– Watching user group activity

Ultimately the velocity of information in the media will give you a pretty good idea about the really short term trends.

Alexa Rankings

Some surprises in Alexa Rankings of Top-100 sites (Alexa actually does 500, but I track changes only for the Top-100).

The companies in the top 19 spots stayed pretty much the same

Microsoft's Live.com climbed to 20th position.

Xanga is happily climbing up. After overtaking The New York Times it is now at the 22nd position.

I thought no one really used Orkut much. It is right on the heals of Xanga. Need to check this out.

Rediff.com an Indian portal overtook Match.com.

Digg entered the Top-100 list (at a ranking of 100).

Did not realize that Double Click is at 99.

Alexa Rankings and now Google Trends are two good tools to use to watch some interesting trends.

XMLization of the Web

XMLization of the web is bound to happen sooner are later. Already several websites deliver content in XML format. One of the popular XML tool vendors Altova of XMLSpy fame, announced Stylevision(TM) for building XML websites and converting HTML websites to XML. In my opinion, XMLization of the web is the first step towards moving to the Semantic Web.

When content travels from a backend database to the web page, somewhere along the line the semantics are lost. This problem can be avoided by keeping the web content in XML. Also, you get a host of other advantages like the easy repurposing of content, ability to exchange rich data with other applications, etc. This ability just gets us to the first step of an immensely interoperable web.

IDC predicts that the XML tools development market will reach $395 million by 2006 at a compound annual growth rate (CAGR) of 41%. While most of the web publishing is about content, increasingly websites will merge content with applications. This movement opens up entirely new opportunities for XML components. It will be interesting watch evolution in this space.

Update – May 2016

It did happen (XML as the content format for web) and was dropped for html5. There are several reasons for this shift. XML, however, continues strong and appears in places you don’t expect (for example, as the definition of UX).