Blog Rank Monitoring – prototype
Who moved down 29869 in Technorati rank in less than three days? Who moved up 785169 points?
What blog is the “big dog” in the marketing, business, news, and entertainments market segments?
Blogs Heat Map (tier1) – tier1.pdf:
Since I started blogging and my “digital social journey” I was thinking about finding great bloggers. I was also thinking about keeping track of these excellent writers where ever they are.
As I wrote in the “Stop looking at the blog and start looking at the blogger” seriesI see a need for more monitoring around bloggers activities.
As I wrote in hereI see a business need for finding the right blogger in a target niche.
General Mission:finding the best upcoming bloggers out there in every domain of interest.
This prototype objective:monitor changes to blogs’ Technorati rank and present them visually every [day,week,month]
Motivation:knowing whom to follow now, listen to, get on the radar of, and interact with, within a specific domain of interest can help building an effective, online social media campaigns.
Value: my assumption is that using social media tools effectively leads to great returns (fame, fortune and glory).
This could be a wrong assumption. Time will tell!
The pain points:
It is easy to find the 100 top bloggers using Technorati but this list is not sorted by the blogger’s domain of interest.
It is NOT easy to find the 100 best bloggers in a category.
It is NOT easy to find the next 100, 1000 top bloggers etc…(Technorati don’t show it on their web site in this way).
I can find a great post! Actually, I can find lots of great posts.
Great posts are thrown at us all day long (especially if you are using Twitter). I keep adding feeds to my feeds aggregator and links to my favorites. There are excellent feeds aggregators out there like GoogleReader and Netvibethat make it somehow bearable, but eventually it is getting too hard to keep track what’s going on with the bloggers whom I read their great posts few days, weeks and months ago.
Preliminary results after only few days of monitoring:
Tier 2 – tier2.pdf:
Tier 3 – tier3.pdf:
Explanation for my approach:
I collected great posts over time. I “extract” the blogger from the post and built a list of bloggers to track (see list in the end).
This is the list of bloggers that I will monitor in this prototype. They may not all be the best out there but the objective is to demonstrate the suggested solution. I hope that I started with a list that is large enough to show some movements but not too large to manage taking in consideration that this is a prototype.
I then use this list to drive a simple program that I wrote to collect blog’s data from Technorati using Technorati API(see example at the end).
Collecting this data daily will allow me to compare blogs’ ranks over time.
I will show changes to these bloggers Technorati’s rank using Heat Map. .
The Heat Map will demonstrate two more blogs’ attributes: the category i.e. the domain of interest and the relative size within a domain of interest using Technorati authority (number of inboundblogs links).
I think that for this purpose Heat Map are great. There may be additional data visualization tools out there to present my findings. I’ll appreciate you pointing me at them.
Changes in blog ranks are proportional. The higher the rank the smaller the change is. Some blogs at the bottom of the rank can move up or down as much as millions. Showing all the blogs as a group will hide dramitc rank changes.
To overcome this problem I divide the losy to 3 tiers:
Moving 15 step higher in rank in the top tier is as dramatic rank move as moving 10,000 in the second tier and a 100,000 in the third tier.
Explaining, category – the domain of interest:
Most bloggers don’t write about one subject only, although the good ones do tend to focus on one. Some bloggers tend to write high percentage of their content in only few domains of interest.
It was very ambitious and sometime I even felt rude, categorizing blogs content. Please forgive me if I misjudged your blog content. I had to do it for the sake of this prototype. I will be happy to fix it per your request.
Yet, I kept it fairly wide. My objective in this demo is to show the value of monitoring and comparing within a blog grouped by category.
Please remember that thus is only a prototype. See the Beyond the prototype section for how I think that categorization should be handled in a real application.
The biggest effort of all was to collect a clean list of blogs to sample. I had to scan through feeds, favorites, twitter and social bookmarking tools. I took the blog address from the post URL. I cleaned it up from duplicates, web sites, no longer existing blogs and blogs that their blogger did not claim it in Technorati.
Some time executing the API call result with data missing one piece of information (e.g.) so I had to deal with this limitation too. At this point by either running again or going directly to Technorati.
I’m aware that this is not a scalable process – it was almost OK for this prototype.
Now I know why companies that build ETL tools makes a lot of money:)
Limitations of this solution:
Not all bloggers have a blog claimed using Technorati
Not all influential people has blogs.
Technorati only allows 500 query per day.
This solution is dependent on Technorati – they can change their policy, the API, stop supporting the API or go out of business.
Heat Map are great at visualizing large set of data but there is a limit to that option to, especially considering the huge number of bloggers out there.
Beyond this prototype
Additional data sources:
Add monitoring using more social network services like Twitter, Digg, Reddit etc…
If a service exposes API that can help find new measures and behavioral pattern it should be use to enhance dynamically building blogger profile.
Showing Heat Map for a single category with break down to subcategories.
Drill down results
Future Functionality – needs more work:
Find top by category – e.g. the 50 best blogs in video delivery market
Find by category and pattern (large move, stale) – e.g. the 50 best blogs in video delivery market that rise in rank in the last month
Alerts – e.g. a rank change is more than x%
Newsletter – e.g. blogs on the move
Blog claim – e.g. allow users to submit their blog to monitor
Solving the data problems:
Allow people to submit blogs to monitor.
Allow them to enter category and sub-category for the submitted blog.
Build a crawler to find blogs to monitor (using outbound links from great bloggers’ blogs, twitts).
Improve data clean-up.
Example of results from Technorati BlogInfo API call:
<?xml version=”1.0″ encoding=”utf-8″?>
<!– generator=”Technorati API version 1.0″ –>
<!DOCTYPE tapi PUBLIC “-//Technorati, Inc.//DTD TAPI 0.02//EN” “http://api.technorati.com/dtd/tapi-002.xml”>
<lastupdate>2008-02-11 19:00:40 GMT</lastupdate>
The blogger list to monitor in this prototype (58 blogs total):
http://www.hummingcrow.com http://batmangeek7.blogspot.com/ http://www.aliveinbaghdad.org/ http://susanreynolds.blogs.com/boobsonice/ http://mymomsblog.blogspot.com/ http://cubeme.com/ http://www.designspongeonline.com/ http://www.onlinevideowatch.com/ http://bloggernista.com/ http://outofmyhead.wordpress.com/ http://redcouch.typepad.com/ http://sethgodin.typepad.com/ http://www.socialmedia.biz/ http://andybeard.eu/ http://www.techcrunch.com http://technobabble2dot0.wordpress.com/ http://www.sixtysecondview.com/ http://www.purplecar.net/ http://daily.mahalo.com/ http://www.paulcolligan.com http://dossy.org/ http://yourinnerceo.blogspot.com/ http://www.wikinomics.com/blog/ http://blogs.zdnet.com http://www.37signals.com/svn/ http://travellperkins.com http://www.beingpeterkim.com http://www.communityguy.com http://lensblog.typepad.com http://www.business-opportunities.biz http://doughaslam.com http://www.francinekizner.com http://viva-freemania.blogspot.com http://halleyscomment.blogspot.com http://confusedofcalcutta.com http://www.successful-blog.com http://insideanalytics.blogspot.com http://darmano.typepad.com http://mediaphyter.wordpress.com http://pravdam.com http://www.megfowler.com http://learntoduck.com http://markharrison.wordpress.com http://founderresearch.blogspot.com http://www.paidcontent.org/ http://personalbrandingblog.wordpress.com http://prmeetsmarketing.wordpress.com http://remarcom.typepad.com http://www.lhotka.net/weblog http://snapshot-blg.blogspot.com http://socialhoneycomb.com http://www.socialmediaexplorer.com http://www.calacanis.com http://www.cheaprevolution.com http://thinkitservices.blogspot.com http://www.veronicabelmont.com http://www.kaushik.net/avinash/ http://www.copyblogger.com
If you made it all the way to here I really like to hear your thoughts.