2 Published Data Stories in #PowerBI Data Stories Gallery

Few others to follow, meantime recently published these . Be sure to check all the gallery, really amazing & inspiring data stories there.

Cortana Machine Learning Competition – Womens Health Risk Assessment – Power BI Companion

Small companion for anyone participating in the Cortana Machine Learning competition – Women’s Health Risk Assessment

The Power BI file allows for easy data profiling over the supplied training dataset, overall patterns, data quality, missing data, feature selection. Note: you’ll still have to build the models though!

Although the current version of Power BI Publish to web doesn’t allow  R visuals, they’re highly recommended (ex: density plots & others), and can easily be added to the data/column profile pages.

So what are you waiting? Smiley Happy You can be running your first submission in minutes using the online tutorial. Then it’s up to you.



The Story of Dr Ignaz Semmelweis – “Savior of Mothers”

This sample PowerBI workbook tells the real story of Dr Ignaz Semmelweis. Year is 1846, Dr Semmelweis begins his 3 year work at Vienna Hospital maternity clinic, and is immediately  disturbed  with the high incidence of puerperal fever (also known as "childbed fever") in Vienna Hospital, causing  extremely high maternal mortality between 1823-1850.

Semmelweis was one of the first data “detectives” of our time, using data and statistics to test his hypothesis and theories.

This workbook allows you to explore the data collected by Semmelweis through those years and shows typical exploratory patterns we can use to hint on important information and possible causes from observational time series data.


You can see the video walkthrough here:



A sense of Wonder… reflecting on the #PowerBI amazing journey

Few hours left in the day, been an amazing day for a lot of reasons… better late than never, they usually say right?

So… finally dumping a blog pots that formed on my mind going back several months (cof, years…) now… Anyway, today is even more relevant than then, so…
Warning: there won’t be fancy templates, crazy ideas,  strange tools, insights, snippets or new downloads on this post. in fact it will be a completely useless post for anyone reading I guess, I just need to get it out.  Don’t have a clear objective, or clear picture of what will come up. Also I won’t be slightly concerned if it’s too short, or too long, I won’t massively edit it (ok, slightly…). Fellow reader, I’m not writing for you today, I’m sorry.

Still, A-post-must-be-written.

So, how to start? How did we get here so fast? Seems only yesterday we were seeing this new Power BI Dashboards screenshots shown at WPC 2014. ~7-8 months after seeing Power BI GA then already a immensely powerful & stable Enterprise/Self-Service/even Personal BI platform, soon after that our first major project with the technology is online.

I’m not easily surprised, and yet time & time again I was completely baffled with what Power BI teams came up delivering, beyond user & community most requested features & needs, a lot others I honestly didn’t knew I was missing, and yet I can’t live without today.

Such an amazing journey… not in my wildest dreams, honestly I would say we would get here. And boy did we need it. Back in 2013-2014 remember we were getting very uncomfortable, mainly for the medium sized/enterprise BI scenarios, at least for any BI team that only works with Microsoft BI stack: powerful but completely stale OLAP engine, no clear roadmap,  failed products ("Planning" anyone?) , acquisitions that ended up bringing no value in the long term, huge number of disparate tools with poor integration, pieces that didn’t fit properly together aside from very superficial demonstrations. (we deliver solutions we hope can add value, not fancy toys that work great in demos)

The nagging feeling that just selling the latest tools was more important the adding value. I’m still a big believer that revenue is a side effect from value we bring for customers. Add value, then capture some of it so you can add more value, repeat.  You can make the other way work temporarily but it won’t be sustainable in the long term, and even if it were… that’s not the point right?   That was the  feeling honestly, yes, we had some great tools (heck PowerView  on SilverLight was great for its time, we couldn’t have Power BI today without “project crescent” right? :)  ), but still msbi kept lagging dangerously behind new vendors like Tableau, Qlick, etc in several key areas.

Something was missing, purely speculating here. Maybe that came finally with Nadella, also James Phillips strong vision, bringing the needed kind of startup mindset to Power BI vision (reminder, speculating here!).

Don’t have much visibility on Microsoft Power BI internal journey, for years know I’ve been inspired with the lean startup, customer development movement, mindset, and from the outside at least, seems a lean startup perfect case study. Hope one day some of that backstage story can be told, really.

A new or reinforced  why- building a platform that could add immense value through the use of data, none of the typical limiting constraints, an inspiring vision, a fine tuned learning machine, powered by real feedback, real needs, really hearing customers & msbi –still- strong and devoted community. And, not less amazing, brilliant , flawless execution. Incrementally, steadily, the pieces simply kept fitting together perfectly. How can these teams have such agility without never slightly compromising a stable, robust, and coherent platform down the road. I don’t know. It shouldn’t be possible. Am I dreaming?

Mission, vision, execution, brilliantly orchestrated.

To the unsung heroes, the amazing minds than could plan, design, envision such a brilliant architecture where pieces keep fitting magically together release after release… To the teams than then gradually brought it to life. Here’s to you. Know a few, but it’s pretty clear it’s only the tip of the iceberg. :)

I remember that right about Power BI GA back in ~July 2015, I was reading “Meaningful: The Story of Ideas That Fly” (truly inspiring book btw) and really felt I could be reading the story of Power BI there, the mindset, the why. Equally inspiring. heck. Even the book color reminded me of Power BI :)

From the book description…

”We don’t change the world by starting with our brilliant ideas, our dreams; we change the world by helping others to live their dreams. The story of ideas that fly is the story of the people who embrace them, love them, adopt them, care about them and share them. Successful ideas are the ones that become meaningful to others—helping them to see what’s possible for them.”

You see…? :)

Guess when you stop having customers and start having followers instead, amazing things are bound to happen.

Long post I, I warned right? Anyway, and also speaking of Azure, SQL Server,  Microsoft new support for Open Source platforms, R, Hadoop, Spark, so many others, such a powerful data platform we now have in our hands. Huge responsibility for us, data professionals, to take all this and deliver on the promise of immensely valuable data.

Should not surprise many, that I feel we’re not there yet (far from it, never been the one easily pleased with only gorgeous dashboards, sorry!). I believe we’re still capturing a tiny potential of the value hidden in all this data. But those thoughts will have to wait for another day. (hope it won’t take another year anyway…)  

Disclaimer: I don’t work for Microsoft, never have, although I work pretty much exclusively with Microsoft stack since I started my professional life, and also very sure, that I will stick with msft stack to the end of my work life. (that is, aside from my faithful Kindle and those precious things called moleskins )  

So, anyone still reading should have this in consideration. I may be biased… slightly…

Better late than never, Happy Birthday Power BI! :)


Guest interview in Power BI Developer Blog and a new Send To Power BI app for Excel

Be sure not to miss the latest blog on the official Power BI Developer Blog :) , the amazing Power BI team asked us to share some thoughts on our latest work & research working with the new Power BI APIs and capabilities. And we were deeply honored to contribute. How cool is that? :)

Read the full post here.

And celebrating the day :) also announcing that even if you can already use SmartCharts to push & sync data to the new Power BI Preview directly from Excel, you now have a dedicated app just to do that :) a brand new Send To Power BI Office App for Excel.

Give it a test drive and leave us your feedback or review. :) It will be greatly appreciated!

Download the new app from the App store.


For more info:

DevScope GitHub repository
Send To Power BI Excel App
SmartCharts Excel App
DevScope Office Apps
Rui Quintino Blog
Rui Romano Blog

Awesome day! :)


More Power BI API awesomeness, now on PowerShell & SmartCharts

Amazing week for Data lovers & Business Intelligence minds, Power BI Preview is now *globally* available! Be sure to check out the public announcement  for details. And Chris Webb’s BI blog for additional insights.

By now you know that there’s a new & strong developer story for the new Power BI,  and even if at the moment we have a simple API to test drive, it already allows to think of interesting scenarios. Particularly for real-time dashboards or  the click to insights pattern  I happen to obsess so much ….

Really, seems there will be no place for data to hide from Power BI :). When Power BI can’t get there… we will bring it to Power BI in a blink of an eye, in real time if needed, all using the brand new APIs.

Don’t have Power BI Designer? or even Excel? and need to quick load  some data to explore? Real time? Check out DevScope github  & use the awesome Power BI API Powershell modules to load your Data into PowerBI directly from the command line (for more info & a cool real time sample don’t miss  Rui Romano blog post).

Not forgetting we already have NuGet Power BI api modules for .Net, Xamarin,  iOS & Android by Sylvain Pontoreau .  Or more recently new sample web apps for Power BI api scenarios.

Adding that, as of… now :) you can also use SmartCharts to quick send & live sync to Power BI directly from Excel. :)

Start with your data as usual or the sample dataset and just click Send To Power BI.


Authenticate & authorize the app for Power BI access,  choose a Dataset and a table name :


and your data will be available in your Power BI account ready to more Power BI awesomeness

(note: if using date/time columns we also add some extra time related columns we happen to use in the app).


Your data ready to explore in Power BI:




Noting that you should be able to do all this on trial mode without any restrictions.  Got feedback and requests? let us know (there’s a feedback form in the app :) we will get back to you)

Get SmartCharts here, or checkout or other DevScope Labs  Office Apps here.

If you want to know more about the new APIs, some additional recommended reading:

More to come :) stay tuned


How #AzureML and #MachineLearning are helping managing my Inbox–Part I

In this post, how to build an AzureML model to auto suggest folders/labels for email classification & archiving. How to consume AzureML web service directly on Outlook.

note: sample AzureML experiments now published on the brand new AzureML gallery. Email Classifier-Training & Email Classifier – Scoring

Context: for years I’ve opted for an archive by folders strategy to handle my Inbox (not obsessed with inbox zero but  I try to keep it below ~50 inbox mails, and instead use inbox like a buffer – since my first contact with David Allen GTD years ago). true: once in a while I try to test the single  archive folder, but until now…always reverted back to my folders to organize so many parallel projects and threads going on

For this to work, I think one thing  is absolutely mandatory, very fast folder archiving and switching. Although have tried a few tools for this  I ended up coding some Outlook macros over the years to fit this… “peculiar” way of working and unproductive task switching…. 

So to archive a mail I just press alt-4, this window popups up, I type my search terms, enter, and it’s done. mail Archived


Or to switch context to a specific project or fast search, just press alt-3, search, enter and I’m there with the latest thread mails immediately available.


(that can admit, I obsess over searching…. “as you type” kind of search :) )

Fast forward to “the present”, #MachineLearning #DataIntelligence #AzureML era. I now have a few thousand “labeled” mails (to use machine learning terms) on several active folders. My macros were needing an improvement….  :)

So I exported my archive mails in the form:   |from | to | subject | time offset since fixed day|  format to a csv (tsv in fact)


and let AzureML do the heavy lifting of building my very personal email classifier suggestion web service, putting some multiclass classification models & also text handling /feature hashing AzureML features to work in a more useful scenario than classifying flowers :)



Sticking with the neural network model for now (default params), created and adjusted the scoring experiment & published the web service.



Used the VBA code sample from the new cool AzureML generated Excel files,


few lines of code added on my Outlook macros, based on AzureML Excel VBA code/macros


and we get a pretty impressive auto classifier ready to use & help manage our inbox, suggesting the folders where the message belongs when archiving.(note: also triggered when sending messages, archiving both sent and original message if needed.)

So for example,  if I’m disturbing Joana with another annoying mail :) about SmartCharts new features AzureML advises me:


On the other hand, if it were an help request to Romano on some stream analytics samples AzureML would opt for:


press Enter, for now it’s needed…:) ,  and that’s done. How cool is that? :)

(have to say that the accuracy is not 100% obviously , but pretty damn useful already)

All this running on a free AzureML workspace. Up & running & minutes, from training to online web service

(excluding the time to slightly adapt AzureML VBA code to call the web service & get my mail properly exported)

Training & scoring experiments are available in AzureML gallery fo you to test drive. Off course I loaded a small sample of my mail in those, so you will have to load your own to really see how it works.

…now, to be close to perfection this would benefit from automated data update & model retraining, and that will be the case for use the new AzureML training APIs.

But that will have to wait! :) (true, reduced some time processing my inbox, but not that much…!)

btw- hope to get feedback on this scenario, ex: feature engineering tips and model tuning suggestions to improve the model results



SmartCharts for Excel new features: guest post by Joana Barbosa

Sharing the first guest post on my blog (cool!) and proudly introducing my colleague Joana Barbosa (who I hope soon to have her own tek blog ;) ). She will share some of most recent feature of our tiny (but cool) dataviz App for Office, SmartCharts.

Joana works on DevScope  “multi-faceted” BI Team :)  , among several amazing contributions, she’s the main dev working on SmartCharts coolest features (namely d3/dataviz). (adding to that…. fixing my bugs, sorry Joana! :) )

ps-feedback on features or suggestions? please use the app built-in feedback form . It will reach me or Joana in a moment.

Go Joana!

Hi everyone,

So, recently SmartCharts got some new features for you to play with. And, since the 2014 FIFA Ballon d’Or winner was announced this week, I’m going to demonstrate them using a soccer dataset: Champions League – Best Scorer By Year.

First of all, we’ll load our dataset.


Our table has columns for date, player’s name, country and team as well as goals scored. But we also have three special columns: Images of the player, team and country containing links to pictures.

What can we do with this? We can enhance our charts of course!

One of SmartCharts’ new feature is binding an image column to another dimension column. For that, we need to name it according to the following rule:

– Consider our column ‘Country’. To bind an image column to this dimension, we must call the new one ‘Country_image

Applying this to our dataset, we’ll see the following result:


So, our Bubble Chart has become much nicer, with flag images on it. Also, when you go over a Bubble or over a slice in the Pie Chart, we’ll see our dimension chart over time accompanied by the dimension image we added.

Imagine now that we want to see all players from our country in the set. How could we do it?

Before, we had to alter our dataset in the spreadsheet. Now all we got to do is click on whatever you want to filter by, for instance, the Portugal bubble. This way, by changing the dimension to Player, we’ll only see the Portuguese players. That easy!


Of course, we can go even further by selecting and filtering another dimension (ex. Team: Real Madrid). Thus, we’ll only see the players from Portugal and who were also playing in Real Madrid. In this case, the single result is Cristiano Ronaldo.

Oh, if you want to print one of your charts, or take a screenshot to copy to a document but you don’t want that toolbar messing with your nice chart, just click on the small arrow icon in the upper right corner, and the toolbar will be gone, hidden until you re-click the button.


And that’s it! Our brand new features that I hope you try and enjoy.

The dataset is available for you to check it here:


So, want to build some cool fresh interactive charts?

Then, Go Smart and make your own SmartCharts!



Thoughts on the new Power BI Preview, great expectations & some wishes for 2015!

Far from a review, as there’s no need for that such is the amount of information and detailed reviews on the last Power BI preview and announcements, just to name a few:

December announcements and Power BI “vnext” preview release:

Chris Webb Blog: Thoughts On The New Power BI

Walking through the Power BI Public Preview – Jen Underwood

Unwrapping the Power BI vNext Public Preview by Jen Underwood

To be quick and to the point :) I’m looking forward for 2015! The new features announced (IMO) show a much renewed & strongly motivated state of mind from Microsoft BI teams.

Frankly speaking, be it justified or not (hope so, we’ll see!), these last weeks of deep diving in the new content & possibilities made me remind those times of SQL 2005 release, so much was new at the time, a world of possibilities, so much to learn. Complete new SSAS engine/model, DMVs, new designer, SQL 2005 man. studio (no query analyzer/ent. manager any more!).  Since then its been incremental (IMO), but that release was disruptive. *Huge* release. (~9 years ago…. )

(Kind of ironic that I “complaint” so often on the amount of constant learning & research needed to work in technology … to find out that’s precisely that learning that keeps my mind happy :) )


Power BI Preview release & features first impressions

Anyway, Power BI December announcements pack a lot of relevant news, not to be taken lightly. A lot of impressive new features & scenarios but more importantly a new roadmap/vision starts to materialize:

about the time iOS apps available and are even the first to go Live

-an hint for a Power BI now decoupled from Office 365 , even Excel (they should work even better together, but Office 365 should not be a requirement IMO and not everyone has the latest Excel 2013 for sure…. & I have to say :) why not a free tier for Power BI? –noting that the new Power BI Preview landing page features a “Use It Free “ button for signup…. just saying…)

-where one of the first Power BI brand new API samples is a Chrome extension :)

-we now have a proper PowerBI Developer Center and brand new a REST API , for someone eternally divided between Dev & Data this is heaven,

a unified HTML5 Power View  version  (new power bi portal, power bi designer, apps) , this allow (I hope) for more focus from MSFT dev teams on a single & more portable dataviz stack,,expect it to evolve on a completely different pace compared to performancepoint, reporting services, silverlight powerview, etc

finally, on-premises tabular data connector, not promised, but actually delivered, on a preview! I’m all in for the cloud dashboards/connectors/local data scenarios.

-The new Power BI if decoupled from o365/Excel, will allow for greater agility & faster release cycles from Microsoft, not so dependent on Office/O365 releases.

The new developer story (did I mention already we got a developer portal? :) ) is huge. Aside from minor extensibility points released in the past (ex: SSAS personalization extensions-remember?), and the huge extensibility of Excel, MSBI has been mostly closed to developers-for me the most annoying issue while doing BI/Analytics, it inhibits creativity. Not much we can do usually to improve end user experience (ex: web dataviz libs) for everything is locked-down, no code. I would prefer to see it as strong platform , that we can automate, extend & improve (like SharePoint, CRM, even office new app model).

-I like the “lean” mindset, lots of feedback channels (be sure to visit the Power BI support site & leave your feedback)  , MSBI teams are clearly “out of the building” and eager for feedback, doing my best!

-Have to say that not initially  impressed by the new dashboards “mashup” (the lack of  filters was a clear indication of a quick win, not a new complete dashboard designer you would expect) … but I’m now sold to the ease of use of the new dashboard mashups (specially when you use Q&A to create very slick dashboards in seconds), reminding you that these update in real time if using the new push API (pretty amazing to see actually, more on this later ;) )

would only like that tooltips/mouseover could be fixed on the final release, & would be great to at least allow for click & autofilter/highlight  like in reports (read: old dashboards). Later on, global filters, but guessing it’s not the most easy thing to do .

The roadmap ahead


Having said that, while the roadmap ahead seems very promising (and anyone close to me know, I’m not an optimist by nature…) , Microsoft will have to deliver on this promise, quickly :

-for everyone already onboard  with Power BI/ Office 365/on premises (read: paying customers) , what’s the story? migration paths? again : trust, return on investment, trust on disruptive innovation on a stable platform, not requiring constantly reformulating solutions to fit the latest version/tools

-how does this fit with the typical enterprise scenarios? where more formal application lifecycle management is needed? source control integration? deployment models?

-pricing? will Power BI have a free tier? (my strong vote. yes, even if very limited, sorry to say but basic BI is already a commodity these days, even AzureML as one free tier)

-can we expect on premises Power BI/SharePoint new releases? or instead will cloud Power  BI be the answer using embedded BI/connectors/gateways? (I’m all in for the cloud, as long as users ’can’t note the difference…)

-partner solutions? Power BI packs marketplace? will it be a reality?

-will the Power BI API allow complete access to deploy/create queries, models & reports?

-can we invest safely on this new stack? expecting it to constantly evolved in the years to come? (not be dropped to a Power BI “v3” in a recent future? starting all over again…?)


Finishing adding a few additional wishes for this year MS Data Platform


-will it be the year when we can finally get rid of having to constantly think of date tables? come on…..! boring!

-Keep a strong story for Office365 & Power BI, decoupled but better together

-Complete portability of Power BI solutions, export PBIX from Power BI, from Excel to pbix, etc

-Microsoft teams more open & able to accommodate quick wins (ex: export to excel… how difficult can this be?)

API for extending data visualizations/prebuilt analytic functions on Power BI, ex: d3js, dimple, highcharts,etc

-Powerful & even more integration with Azure, AzureML, Stream Analytics (see the note on Developer Portal regarding PowerBI/Azure Stream analytic preview) , DataFactory,etc

-hope for new breaktrough on msbi analytic models/engines, ex: a richer semantic layer, with the best of relational & multi dimensional, support for “dimensional attributes” (complex attributes types based on reusable dimensions, no extra related tables polluting the model), others: federated models, dynamic models directly on top of sql server , others, something new !

-Q&A- can we have this everywhere please? :) Excel & Power BI Designer…. guessing its not easy for local models but asking anyway

-Search everywhere in Power BI field lists, both metadata and actual model data (Q&A for sure is using indexed data , could it be a quick win? would be great to have SmartPivot kind of search everywhere)

Excel pivottables/ powerview connecting to cloud Power BI/ Office 365 models

-Multi dimensional roadmap , honestly we just need to know, if the answer is that it will end then 1) we need to have alternatives with tabular, or lose some large scale solutions  2) promises should be kept IMO: Excel Power View & new Power BI connector should be able to connect to multidimensional models , it’s a question of building/regaining trust. 

-Expect to be (pleasantly) surprised! I was for sure last December, looking forward for first release :)  and not wanting to wait another full year!

Next Post:


ps- will be fun and we’ll be taking the new Power BI API for an awesome test drive ;)

Interested in the new Power BI Preview & Roadmap? you should! :) check these additional resources.

Power BI Preview Portal

Microsoft Power BI Developer blog

Walking through the Power BI Public Preview – Jen Underwood

Unwrapping the Power BI vNext Public Preview by Jen Underwood

Power BI Designer Preview- by Adam W. Saxton

Power BI Support Blog

Adding that next Thursday 22nd we’ll be at Microsoft IT Pro Airlift: Data Platform event on the beautiful Microsoft quarters here in Portugal/Parque das Nações. I will be talking about & test driving Azure Stream Analytics (promising technology! you’ll see!), my team mate Rui Romano all about Power BI, and a lot of more sessions to see on Microsoft Data Platform: Azure, Azure Search, Azure DocumentDB, SQL Server 2014, Azure Machine Learning, Column Store, HDInsight, PowerPivot/DAX .See you there!



Take care!