5 minutes to #DeepLearning on #Azure #DataScience VM with #Docker #NVidiaDigits #Caffe–CPU version

Quick post following previous one , for anyone who can’t have a GPU for any reason you can still use the code & Docker images below to do the same, but yes it will take a lot more time to train models. Sad smile

Started from Digits CPU Docker image here (amazing work with these Docker files by Kai Arulkumaran) and just added volume configuration for digits data & jobs.

So you can use, on Docker/Ubuntu :

docker run –rm –name digits-cpu -d -v $PWD/digits-data/:/data -v $PWD/digits-jobs:/jobs -p 5000:5000 rquintino/digits-cpu && \

docker exec -it digits-cpu python -m digits.download_data mnist /data/mnist

 

On Docker/Windows:

docker run –rm –name digits-cpu -d -v "%cd:\=/%/digits-data"/:/data -v "%cd:\=/%/digits-jobs":/jobs -p 5000:5000 rquintino/digits-cpu

start http://localhost:5000

docker exec -it digits-cpu python -m digits.download_data mnist /data/mnist

One issue you’ll probably have with docker windows data volumes is performance. Think there are some known issues regarding some operations on volumes on Docker Windows, which the mnist prepare script should be triggering (60k images).

CPU Train Performance

Using the GPU on the Azure Data Science VM NC series model full train for 30 epochs is around 1-2mins, using the CPU version like above I had to wait ~19 minutes. So 5 minutes won’t be enough!

image

image

Image docker hub/github

https://hub.docker.com/r/rquintino/digits-cpu/

https://github.com/rquintino/ml-dockerfiles

Rui

Advertisements

5 minutes to #DeepLearning on #Azure #DataScience GPUs VM with #Docker #NvidiaDigits #Caffe

Shared previously on twitter but for the record, it’s really very easy to get a glimpse of Deep Learning on the new Linux GPUs powered Azure Data Science VMs. No code required aside from provided shell/nvidia-docker command line below.

image

We’ll be using Nvidia digits, that wraps Caffe deep learning framework, will do all the heavy lifting for you.

If you don’t have an Azure Subcription you can get a trial easily

image

https://azure.microsoft.com/en-us/free/

Create the DataScience Linux VM

be sure to select HDD and NC series VM (with K80 GPUs)

image

Wait a few moments until it’s available for remote connections

Connect to the Linux VM using SSH (ex: putty, mobaxterm)

you can run dvsm-more-info to get a guide of what’s available, pretty much everything you need Smile

image

Start Digits Docker container with nvidia-docker, download & prepare mnist dataset

shell code: (also available here)

sudo usermod -aG docker $USER && \

mkdir ~/data && \

mkdir ~/digits-jobs && \

nvidia-docker run –name digits -d -v ~/data:/data -v ~/digits-jobs:/jobs -p 5000:5000 nvidia/digits && \

docker exec -it digits python -m digits.download_data mnist /data/mnist

this will:

  1. add current user to docker group so you don’t need to sudo constantly to use docker
  2. create data and digits-jobs host folders for persisting data and digits jobs outside of the container
  3. run the digits container (default gpu based, exposing digits port 5000, using nvidia-docker as to use the host GPUs)
  4. executing python script inside container to download & prepare mnist dataset

God, I love Docker….! (thank you Pedro Sousa, owe you for life! Smile )

image

SSH Tunnel for digits http/5000

Now, digits has no real authentication so I suggest you configure ssh port tunnel (don’t expose this endpoint!), so that you can open http://localhost:5000 on your laptop and will open the remote digits endpoint running on docker container running on azure data science vm host Smile.

Open Nvidia digits on http://localhost:5000

You may have to wait a few seconds, and then should open on your local browser.

You can follow the guide here, no code needed: Getting Started (NVidia Digits) 

No more code required to train your first deep convolutional network with GPUs and get 99% accuracy on mnist handwritten digits dataset. Should take only a few minutes

note: path for mnist data is /data/mnist/train for use when creating the dataset, you don’t need to download the mnist data again

image

Some additional screens

Exploring dataset images

image

Training default LeNet network

clip_image001[9]

Network config

clip_image001[11]

Digits built-in network visualizer

clip_image001[13]

GPU training (1 tesla K80)

image

 

done, ~1min 37 seconds

image

testing trained model/epoch

clip_image001~

it works!

clip_image001[5]

activations, filter weights,…

clip_image001[7]

image

Persisted mnist data and digits jobs on host vm

so you can stop docker/container if needed, later start a completely new docker container and continue where you left off

image

Having shown digits nocode approach here, the really truly inspiring is getting to know how this convolutional neural networks work and how they learn the filters on their own (they learn the $”%#@#”"&% reusable filters/weights! then how to use them to build other filters…and so on). Absolutely amazing.

If you’re interested few good sessions.

Tensorflow and deep learning – without a PhD by Martin Görner 

How Deep Neural Networks Work- Brandon Rohrer

Take care!

Rui

Notes from Cortana Womens Health Risk #MachineLearning Competition-I

Another Cortana Machine Learning is now over – Womens Health Risk Assessment -, be sure to check the Winners, and most importantly  their submissions and overall approaches to the problem. (update 26/10- Power BI companion with final results uploaded & online here)
image

Must confess I hadn’t so much fun in a while working with data, stats, code & dataviz, been an amazing learning journey, really highly effective on consolidating ~more than a  year of almost exclusive deep dive/personal quest in data,stats, R, machine learning (and yes, some math involved…).

Have to say  almost as equally frustrated missing the top 10….again, 6th on public leaderboard, 13th on final rankings. Far from one of my  goals, actually winning the competition. And trust me, I gave it all, to the last mile.
(ah… and yes, prizes would be great help on research funds Winking smile )

I had got a very lucky 34th place in previous brain signals competition, but that one was completely out of reach, due to the knowledge needed on signal processing/ECoG, gave up midway really)

So, I decided to take this one very seriously as I knew it would be the only way to maximize learning, both on competition data, ml process and everything data/stats/ml related. I would only stop when there were abs nothing more I could do, and so it was.

Other goal, ensure that DevScope ml minds could maximize learning also, have some fun, hopefully get into winners, or at least top 10. Very happy to see that we’re all top 25. Smile Huge congrats Bruno & Fábio. (still couldn’t  get most brilliant mind at DevScope to enter this one though, hope better luck next time…. Winking smile )

I put so many hours… days, weeks on this one, if there hadn’t been so much learning involved  (and personal enjoyment)  I would classify this as my most unproductive period… ever.

So, I’ll publish as much as I possibly can in upcoming blog posts, what worked out, what didn’t, code, tips, insights that still intrigue me, from Azure ML, to Power BI, R, caret, xgboost, random forests, ensembles, learning with counts, even some python. Also noted as much as I can on code files, OneNote was huge help, it’s great now to look back and have all this.

Other goal was to pursue both native Azure ML & custom R tracks to compete, but preferably win with Azure ML native modules (as I knew few top competitors would be all R/python). Still believe that using AzureML modules, no code required, would be enough for top 10-20 rank. But also believe, with current azureml capabilities,  if you want to win, you’re better off using native R/python from the start.

That’s relevant, also relevant  that top 3 winners submissions use base R, xgboost,  all features, no AzureML native modules (azureml teams, some hints here…)

Still, it isn’t over yet, as now I can look back and try to understand what could I have done differently and learn from top submissions. How much randomness is involved in a competition that was so close like this one? bitten by excessive variance on my submissions? multiple comparisons? Can an ensemble of the best techniques and models from top submissions improve the score we have to date? what more insights can we get from competition data? submissions data?

Awesome journey, so much still to learn…. hope to share as much, stay tuned!

Rui

(just the tip of the iceberg really…)

online PowerBI /EDA Companion

image

clip_image001[6]

clip_image001[10]image

clip_image001[14]clip_image001[12]

clip_image001[16]

image

image

image

2 Published Data Stories in #PowerBI Data Stories Gallery

Few others to follow, meantime recently published these . Be sure to check all the gallery, really amazing & inspiring data stories there.

Cortana Machine Learning Competition – Womens Health Risk Assessment – Power BI Companion

Small companion for anyone participating in the Cortana Machine Learning competition – Women’s Health Risk Assessment

The Power BI file allows for easy data profiling over the supplied training dataset, overall patterns, data quality, missing data, feature selection. Note: you’ll still have to build the models though!

Although the current version of Power BI Publish to web doesn’t allow  R visuals, they’re highly recommended (ex: density plots & others), and can easily be added to the data/column profile pages.

So what are you waiting? Smiley Happy You can be running your first submission in minutes using the online tutorial. Then it’s up to you.

http://community.powerbi.com/t5/Data-Stories-Gallery/Cortana-Machine-Learning-Competition-Womens-Health-Risk/m-p/61588

image

The Story of Dr Ignaz Semmelweis – “Savior of Mothers”

This sample PowerBI workbook tells the real story of Dr Ignaz Semmelweis. Year is 1846, Dr Semmelweis begins his 3 year work at Vienna Hospital maternity clinic, and is immediately  disturbed  with the high incidence of puerperal fever (also known as "childbed fever") in Vienna Hospital, causing  extremely high maternal mortality between 1823-1850.

Semmelweis was one of the first data “detectives” of our time, using data and statistics to test his hypothesis and theories.

This workbook allows you to explore the data collected by Semmelweis through those years and shows typical exploratory patterns we can use to hint on important information and possible causes from observational time series data.

http://community.powerbi.com/t5/Data-Stories-Gallery/The-Story-of-Dr-Ignaz-Semmelweis-Savior-of-Mothers/m-p/62206

You can see the video walkthrough here:

https://www.youtube.com/watch?v=aDURDaFaM0I

image

How #AzureML and #MachineLearning are helping managing my Inbox–Part I

In this post, how to build an AzureML model to auto suggest folders/labels for email classification & archiving. How to consume AzureML web service directly on Outlook.

note: sample AzureML experiments now published on the brand new AzureML gallery. Email Classifier-Training & Email Classifier – Scoring

Context: for years I’ve opted for an archive by folders strategy to handle my Inbox (not obsessed with inbox zero but  I try to keep it below ~50 inbox mails, and instead use inbox like a buffer – since my first contact with David Allen GTD years ago). true: once in a while I try to test the single  archive folder, but until now…always reverted back to my folders to organize so many parallel projects and threads going on

For this to work, I think one thing  is absolutely mandatory, very fast folder archiving and switching. Although have tried a few tools for this  I ended up coding some Outlook macros over the years to fit this… “peculiar” way of working and unproductive task switching…. 

So to archive a mail I just press alt-4, this window popups up, I type my search terms, enter, and it’s done. mail Archived

image

Or to switch context to a specific project or fast search, just press alt-3, search, enter and I’m there with the latest thread mails immediately available.

image

(that can admit, I obsess over searching…. “as you type” kind of search :) )

Fast forward to “the present”, #MachineLearning #DataIntelligence #AzureML era. I now have a few thousand “labeled” mails (to use machine learning terms) on several active folders. My macros were needing an improvement….  :)

So I exported my archive mails in the form:   |from | to | subject | time offset since fixed day|  format to a csv (tsv in fact)

image

and let AzureML do the heavy lifting of building my very personal email classifier suggestion web service, putting some multiclass classification models & also text handling /feature hashing AzureML features to work in a more useful scenario than classifying flowers :)

 

image

Sticking with the neural network model for now (default params), created and adjusted the scoring experiment & published the web service.

image

image

Used the VBA code sample from the new cool AzureML generated Excel files,

image

few lines of code added on my Outlook macros, based on AzureML Excel VBA code/macros

image

and we get a pretty impressive auto classifier ready to use & help manage our inbox, suggesting the folders where the message belongs when archiving.(note: also triggered when sending messages, archiving both sent and original message if needed.)

So for example,  if I’m disturbing Joana with another annoying mail :) about SmartCharts new features AzureML advises me:

image

On the other hand, if it were an help request to Romano on some stream analytics samples AzureML would opt for:

image

press Enter, for now it’s needed…:) ,  and that’s done. How cool is that? :)

(have to say that the accuracy is not 100% obviously , but pretty damn useful already)

All this running on a free AzureML workspace. Up & running & minutes, from training to online web service

(excluding the time to slightly adapt AzureML VBA code to call the web service & get my mail properly exported)

Training & scoring experiments are available in AzureML gallery fo you to test drive. Off course I loaded a small sample of my mail in those, so you will have to load your own to really see how it works.

…now, to be close to perfection this would benefit from automated data update & model retraining, and that will be the case for use the new AzureML training APIs.

But that will have to wait! :) (true, reduced some time processing my inbox, but not that much…!)

btw- hope to get feedback on this scenario, ex: feature engineering tips and model tuning suggestions to improve the model results

Cheers

Rui

AzureML Web Service Scoring with Excel and Power Query

While drafting some samples & labs for this saturday SQLPort AzureML workshop decided to dig a little bit trying to use some regular AzureML scoring web services (request/response) directly in Excel, using Power Query. :)

So… can it be done? Sure! Advisable? Far,far from it…. but for a quick sample/test scenario it might be handy. :)

Saying again, what I’ll show here is far from a recommended web service usage for AzureML in Excel. It will be slow, probably insecure if you need to distribute your file, and there will be I’m sure better ways to do it with the batch web service mode, soon I hope (without the need to pass through the Azure marketplace),

Side notes:

    How it works

    To the point, you can use the Power Query Web.Contents function to invoke the AzureML web service api. As long as you are careful on: 1) using post and not get, 2)setting the authorization header expected by azureml, 3)setting the content type to json  4) building the json request like shown in the api help page.
    PowerQuery will handle the json response automatically..We encapsulate the web request in a function, we can then use it to build our scored table passing the score input column values and getting the score in return.

Step by Step-From AzureML to Excel Power Query Scoring

You’ll need an AzureML  scoring web service, I used a simple two class log regression model to classify the well known Fisher Iris Dataset (with only two classes, as the original as three):

image

The AzureML Experiment & Web Service Experiment, can’t get much simpler than this.

Training Experiment

image

Web Service Experiment

image

Publishing the trained model as a Web Service

AzureML-Iris Classifier Web Service API Help page

image

this JSON request will be needed for Power Query, same thing as the service key:

image

In Power Query you’ll then create a function like this (text version below):

image

image

image

Invoking the Power Query scoring function

Then you just import the original table to Power Query and add a custom column invoking the new scoring  function

image

Note: to test this it is advisable to filter the top 1/2 rows first, it is not lightning fast…. Smile

image

The result should now be visible, a little bit more Power Query expanding and renaming and that’s it!

image

image

To check that I’m getting the expected score, just used another small tool (more on this later)

image

Power Query results

image

Scoring the whole 100 rows iris dataset

The problem using the regular Power Query web.contents function  is that you will be receiving api usage threshold errors from the AzureML API very, very soon. So using another trick inspired (again) on this post by Chris Webb (amazing tip by Alejandro Lopez-Lago), to add some delay between requests.

Just remove the top filter and refresh the query. Wait between 30 secs-60 secs (this will depend on the kind of endpoint you are using, AzureML is changing this whole “stag/production web service model “ right now from what I’m seeing)

image

note: think the latest Power Query update now shows the row count progress, neat! perfect for this Smile

image

image

image

Some misses there Smile guess the model needs some additional training!

image

Download the sample here, (note: will need your own AzureML endpoint & key, you do know that AzureML has a free tier right? :) ) the full Power Query score function for this model is below.

Take care!

Rui

let

     Score= (sepal_length as any,sepal_width,petal_length,petal_width) =>

let

serviceUri="*yourserviceuri_note:ends with score*",
serviceKey="yourservicekey”,

PostContents= "

{
  ""Id"": ""score00001"",
  ""Instance"": {
    ""FeatureVector"": {
      ""sepal_length"": """&sepal_length&""",
      ""sepal_width"": """&sepal_width&""",
      ""petal_length"": """&petal_length&""",
      ""petal_width"": """&petal_width&"""
    },
    ""GlobalParameters"": {}
  }
}
        
                   
",
      //Define Wait function
    Wait = (seconds as number, action as function) =>
            if (List.Count(
             List.Generate(
              () => DateTimeZone.LocalNow() + #duration(0,0,0,seconds),
              (x) => DateTimeZone.LocalNow() < x,
              (x) => x)
              ) = 0)
               then null else action()
    //Call Wait function to wait 5 seconds
    //then return the current time again
   
,
    Source=Wait ( 0.05, () => Web.Contents(serviceUri,
    [Content=Text.ToBinary(PostContents),
    Headers=[Authorization="Bearer "&serviceKey,#"Content-Type"="application/json; charset=utf-8"]])),
    #"Imported JSON" = Json.Document(Source),
    #"Table from List" = Table.FromList(#"Imported JSON", Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Transposed Table" = Table.Transpose(#"Table from List")
   in
     #"Transposed Table"
    in Score