Choosing vs Searching, … can data visualization help with information overload?

Way too much information nowadays… too many  mails, too many tasks, too many blogs,  too many projects, too many tweets, too many results in a google search, and definitely way too many books to choose from… were do we start?

Searching and browsing through endless lists and grids certainly does not make the choice easier, and unless you’ve got an infinite amount of time available… you will need to choose. Choosing is not searching. Search usually implies that we know what we are searching for… a specific mail, a specific book, a specific document.

But how can we go when we’re choosing?… choosing what to nail for the day, what to read next? Gut feeling maybe… but in the end we always need something to feel about.

Content already dominate our lives, and so will dominate content choosing, browsing, but in a much more flexible & visual way ( I hope). We’ll search and then we’ll mash it up, slice, dice, and then choose.

Amazingly, even surprisingly, we have plenty technologies already that would allow much more mashable & browsable content-anywhere and on any device-, from xml to odata to rest, json, html5 … The piece still missing is a much more open way of providing content, without the “chrome”, the much spoken but still lacking web of data (& metadata). A lot like connecting a pivot table to an olap server… but using it for finding & choosing things. Agnostic search frontends for rich data catalogs.

Somewhere between 100% dynamic PivotViewer style of data visualization and  the new Windows 8 Metro “content before chrome” mantra. I hope that search, browsing & choosing experience can become more consistent, but way more powerful. Maybe we’ll loss some variety on style & design, but the usage scenarios will be almost unlimited… fewer frontends with infinite datasets to explore, even merge together. Cloud & multi-device content delivery needs are accelerating this, that’s good.

Why this? Well, I was just trying to choose what to load next on my Kindle… as it is absolutely impossible to read the books as I buy them. And well, trying to do this with the current kindle, or kindle apps  on iphone/ipad, requires an astonishing  patience & effort to say the least. They allow us to browse a list, if you’re using kindle apps or cloud at least you have the book cover. But that’s it, search is limited, almost no classification whatsoever… The data (and I stress this, it is only data) can’t be mashed up with other feeds…

Not a nightmare if you have a small collection of books. If you have a few more (guess some people will call it information “hoarding” :) )…


Some question pop to my mind? Can I see my books by store category? Tags? What about by customer reviews or stars? And taking it further, as I use highlighting a lot , can I see from my collection what books are getting most notes & highlights, or shares on

Well, fact is… you can do it… but the effort required (as of today) is almost  unbelievably “humiliating”…  

Strangely (but predictably) I thought it would be fun to do it, so I started IE, updated a few PowerShell scripts, and in a few minutes I had my book collection updated and with stats regarding highlights & notes  ranks on kindle cloud site, book highlight ranking, public notes  ranking, and the number of my highlights.


Loaded it up into Excel, with the new stats:



then into SmartPivot PivotViewer explorer and start playing with it.

1) My top highlighted kindle books (note that this does not consider books length into account… actually not what I was expecting, I’ll probably have to double check my powershell code :) but it is familiar) :


2)Kindle highlights Top ranked books in my collection, that I didn’t started reading yet:


Side note: Switching color to amazon reviews shows some unexpected outliers for these 1000 top highlighted kindle books:


How a book with only two reviews could be ranking 831 in highlights?  in fact my data was not 100% accurate… :( some stats got mangled by my far from perfect powershell script:



So, key takeaways:

-I should start reading the bible, :) probably will take a look indeed

-I’ve still got a bug in my scripts, always reminding me that data is not reality

-This should be much more easier to do…





About Rui Quintino
Developer & Software Architect @ DevScope

2 Responses to Choosing vs Searching, … can data visualization help with information overload?

  1. Jorge says:

    Muito bom, como sempre.

  2. Pingback: Browsing & exploring a “big” #kindle library, part II, now with #Excel2013 #PowerView « Rui Quintino Blog

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: