Home > Method, Monitoring, Observations, Software > Blog Rank Monitoring – prototype

Blog Rank Monitoring – prototype

Who moved down 29869 in Technorati rank in less than three days? Who moved up 785169 points?

What blog is the “big dog” in the marketing, business, news, and entertainments market segments?

Blogs Heat Map (tier1) – tier1.pdf:

Tier1-sc

Prototype Description

Since I started blogging and my “digital social journey” I was thinking about finding great bloggers. I was also thinking about keeping track of these excellent writers where ever they are.

As I wrote in the “Stop looking at the blog and start looking at the blogger” seriesI see a need for more monitoring around bloggers activities.

As I wrote in hereI see a business need for finding the right blogger in a target niche.

General Mission:finding the best upcoming bloggers out there in every domain of interest.
This prototype objective:monitor changes to blogs’ Technorati rank and present them visually every [day,week,month]
Motivation:knowing whom to follow now, listen to, get on the radar of, and interact with, within a specific domain of interest can help building an effective, online social media campaigns.
Value: my assumption is that using social media tools effectively leads to great returns (fame, fortune and glory).
This could be a wrong assumption. Time will tell!

The pain points:
It is easy to find the 100 top bloggers using Technorati but this list is not sorted by the blogger’s domain of interest.
It is NOT easy to find the 100 best bloggers in a category.
It is NOT easy to find the next 100, 1000 top bloggers etc…(Technorati don’t show it on their web site in this way).

I can find a great post! Actually, I can find lots of great posts.
Great posts are thrown at us all day long (especially if you are using Twitter). I keep adding feeds to my feeds aggregator and links to my favorites. There are excellent feeds aggregators out there like GoogleReader and Netvibethat make it somehow bearable, but eventually it is getting too hard to keep track what’s going on with the bloggers whom I read their great posts few days, weeks and months ago.

Preliminary results after only few days of monitoring:

Tier 2 – tier2.pdf:

Tier2-sc

Tier 3 – tier3.pdf:

Tier3-sc

Explanation for my approach:
I collected great posts over time. I “extract” the blogger from the post and built a list of bloggers to track (see list in the end).
This is the list of bloggers that I will monitor in this prototype. They may not all be the best out there but the objective is to demonstrate the suggested solution. I hope that I started with a list that is large enough to show some movements but not too large to manage taking in consideration that this is a prototype.
I then use this list to drive a simple program that I wrote to collect blog’s data from Technorati using Technorati API(see example at the end).
Collecting this data daily will allow me to compare blogs’ ranks over time.
I will show changes to these bloggers Technorati’s rank using Heat Map.  .
The Heat Map will demonstrate two more blogs’ attributes: the category i.e. the domain of interest and the relative size within a domain of interest using Technorati authority (number of inboundblogs links).
I think that for this purpose Heat Map are great. There may be additional data visualization tools out there to present my findings. I’ll appreciate you pointing me at them.
Changes in blog ranks are proportional. The higher the rank the smaller the change is. Some blogs at the bottom of the rank can move up or down as much as millions. Showing all the blogs as a group will hide dramitc rank changes.
To overcome this problem I divide the losy to 3 tiers:
100,000>=rank
10,000>=rank<100,000
rank<10,000
Moving 15 step higher in rank in the top tier is as dramatic rank move as moving 10,000 in the second tier and a 100,000 in the third tier.

Explaining, category – the domain of interest:
Most bloggers don’t write about one subject only, although the good ones do tend to focus on one. Some bloggers tend to write high percentage of their content in only few domains of interest.
It was very ambitious and sometime I even felt rude, categorizing blogs content. Please forgive me if I misjudged your blog content. I had to do it for the sake of this prototype. I will be happy to fix it per your request.
Yet, I kept it fairly wide. My objective in this demo is to show the value of monitoring and comparing within a blog grouped by category.
Please remember that thus is only a prototype. See the Beyond the prototype section for how I think that categorization should be handled in a real application.

Data challenges:
The biggest effort of all was to collect a clean list of blogs to sample. I had to scan through feeds, favorites, twitter and social bookmarking tools. I took the blog address from the post URL. I cleaned it up from duplicates, web sites, no longer existing blogs and blogs that their blogger did not claim it in Technorati.
Some time executing the API call result with data missing one piece of information (e.g.) so I had to deal with this limitation too. At this point by either running again or going directly to Technorati.
I’m aware that this is not a scalable process – it was almost OK for this prototype.
Now I know why companies that build ETL tools makes a lot of money:)

Limitations of this solution:
Not all bloggers have a blog claimed using Technorati
Not all influential people has blogs.
Technorati only allows 500 query per day.
This solution is dependent on Technorati – they can change their policy, the API, stop supporting the API or go out of business.
Heat Map are great at visualizing large set of data but there is a limit to that option to, especially considering the huge number of bloggers out there.

Beyond this prototype

Additional data sources:
Add monitoring using more social network services like Twitter, Digg, Reddit etc…
If a service exposes API that can help find new measures and behavioral pattern it should be use to enhance dynamically building blogger profile.

Visualize results:
Showing Heat Map for a single category with break down to subcategories.
Drill down results
Other graphs

Future Functionality – needs more work:
Find top by category – e.g. the 50 best blogs in video delivery market
Find by category and pattern (large move, stale) – e.g. the 50 best blogs in video delivery market that rise in rank in the last month
Alerts – e.g. a rank change is more than x%
Newsletter – e.g. blogs on the move
Blog claim – e.g. allow users to submit their blog to monitor

Solving the data problems:
Allow people to submit blogs to monitor.
Allow them to enter category and sub-category for the submitted blog.
Build a crawler to find blogs to monitor (using outbound links from great bloggers’ blogs, twitts).
Improve data clean-up.

Appendix

Example of results from Technorati BlogInfo API call:

<?xml version=”1.0″ encoding=”utf-8″?>
<!– generator=”Technorati API version 1.0″ –>
<!DOCTYPE tapi PUBLIC “-//Technorati, Inc.//DTD TAPI 0.02//EN” “http://api.technorati.com/dtd/tapi-002.xml”>
<tapi version=”1.0″>
<document>
    <result>
        <url>http://www.designspongeonline.com/</url>
                    <weblog>
                <name>Design*Sponge </name>
                <url>http://www.designspongeonline.com</url>
                <rssurl>http://www.designspongeonline.com/feed</rssurl>
                <atomurl></atomurl>
                <inboundblogs>2802</inboundblogs>
                <inboundlinks>4726</inboundlinks>
                <lastupdate>2008-02-11 19:00:40 GMT</lastupdate>
                <rank>300</rank>
            </weblog>
                            <inboundblogs>2802</inboundblogs>
                            <inboundlinks>4726</inboundlinks>
            </result>
</document>
</tapi>

The blogger list to monitor in this prototype (58 blogs total):
http://www.hummingcrow.com http://batmangeek7.blogspot.com/ http://www.aliveinbaghdad.org/ http://susanreynolds.blogs.com/boobsonice/ http://mymomsblog.blogspot.com/ http://cubeme.com/ http://www.designspongeonline.com/ http://www.onlinevideowatch.com/ http://bloggernista.com/ http://outofmyhead.wordpress.com/ http://redcouch.typepad.com/ http://sethgodin.typepad.com/ http://www.socialmedia.biz/ http://andybeard.eu/ http://www.techcrunch.com http://technobabble2dot0.wordpress.com/ http://www.sixtysecondview.com/ http://www.purplecar.net/ http://daily.mahalo.com/ http://www.paulcolligan.com http://dossy.org/  http://yourinnerceo.blogspot.com/  http://www.wikinomics.com/blog/  http://blogs.zdnet.com  http://www.37signals.com/svn/  http://travellperkins.com  http://www.beingpeterkim.com http://www.communityguy.com  http://lensblog.typepad.com  http://www.business-opportunities.biz  http://doughaslam.com  http://www.francinekizner.com  http://viva-freemania.blogspot.com  http://halleyscomment.blogspot.com  http://confusedofcalcutta.com  http://www.successful-blog.com  http://insideanalytics.blogspot.com  http://darmano.typepad.com  http://mediaphyter.wordpress.com  http://pravdam.com  http://www.megfowler.com  http://learntoduck.com  http://markharrison.wordpress.com  http://founderresearch.blogspot.com  http://www.paidcontent.org/  http://personalbrandingblog.wordpress.com  http://prmeetsmarketing.wordpress.com  http://remarcom.typepad.com  http://www.lhotka.net/weblog  http://snapshot-blg.blogspot.com  http://socialhoneycomb.com  http://www.socialmediaexplorer.com  http://www.calacanis.com  http://www.cheaprevolution.com  http://thinkitservices.blogspot.com  http://www.veronicabelmont.com  http://www.kaushik.net/avinash/ http://www.copyblogger.com

If you made it all the way to here I really like to hear your thoughts.

Advertisements
  1. February 18, 2008 at 1:32 am

    Wow! Thanks for including me. I’m impressed that I’m even on the list and honored to be there. Interesting experiment. I’ll be watching, for sure!

  2. Keren Dagan
    February 18, 2008 at 4:04 am

    Jason,
    Your blog provide useful information and get me thinking. I’m honored by your comment.
    Thanks,
    Keren

  3. February 19, 2008 at 10:17 pm

    This is really cool! Thanks for posting it.

  4. Keren Dagan
    February 19, 2008 at 10:36 pm

    Hi Ian,

    Wow, thank you. It means a lot to me hearing this from you.

    I really think that Technorati is an excellent service and I enjoy every minute I got working on this prototype.

    Keren

  5. February 20, 2008 at 12:55 am

    Cool! Ian told me about this, I think it’s great. Somone should get you more calls for your API key. 🙂

    Keep it up!

    Dave

  6. Keren Dagan
    February 20, 2008 at 3:43 am

    Mr. Sifry,

    There is nothing more encouraging than seeing your comment on this post. I truly appreciate it, thank you.
    I hope to support Technorati and the already superb data that you provides to the blogsphere and beyond.

    Keren

  7. February 21, 2008 at 5:31 am

    Sounds cool, looking forward to the results. My two cents from what I’ve seen: Technorati’s authority scale is a far more relevant measure of blog status than the overall rank. There are so many spam blogs out there that it’s relatively easy to run up links by hitting some spam keyword like “Britney”…

    Anyway, looking forward to following along.

    Ben

  8. Keren Dagan
    February 26, 2008 at 1:56 am

    Hi Ben,
    Thanks for the tip. I plan to use both rank and authority (and maybe some more) to find great bloggers.
    Keren

  9. March 5, 2008 at 8:25 pm

    I meant to comment on this when I first saw it, but I think I was just so shocked at even being included among all those prestigious names. Anyway, thanks for including my little blog (actually, you included my portfolio site (which was made with wordpress) and not my more daily blog (blog.francinekizner.com) where I post more frequently.

    I look forward to seeing how this all turns out.

    -Francine

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: