Month: May 2018

My self-driving car

Over the last few weeks, I built a self-driving car – which essentially is a remote control Rx car that uses a raspberry pi running Python, TensorFlow implementing a end-to-end convolution neural network (CNN)

Of course other than being  a bit geeky, I do think this is very cool to help understand and get into some of the basic constructs and mechanics around a number of things – web page design, hardware (maker things), and Artificial Intelligence principles.

There are two different models here – they do use the same ASC and controller that can be programmed. My 3D printer, did mess up a little (my supports were a little off) and which is why you see the top not clean.

The sensor and camera are quite basic, and there is provisions to add and do better over time. The Pi isn’t powerful enough to train the model – you need another machine for that (preferably a I7 core with a GPU). Once trained you can run the model on the Pi for inference.

This is the second car, which is a little different hardware, but the ESC to control the motor and actuators are the same.

The code is simple enough; below is an example of the camera (attached) to the Pi, saving the images it is seeing. Tubs is the location where the images are saved; these can then be transferred to another machine for training or inference.

import donkey as dk

#initialize the vehicle
V = dk.Vehicle()

#add a camera part
cam = dk.parts.PiCamera()
V.add(cam, outputs=['image'], threaded=True)

#add tub part to record images
tub = dk.parts.Tub(path='~/d2/gettings_started', 
                   inputs=['image'], 
                   types=['image_array'])
V.add(tub, inputs=inputs)

#start the vehicle's drive loop
V.start(max_loop_count=100)

Below you can see the car driving itself around the track, where it had to be trained first. The reason it is not driving perfectly is because during training (when I was manually driving it around), I crashed a few times and as a result the training data was messed up. Needed more time to clean that up and retrain it.

This is based on donkey car – which is an open source DIY for platform for small-scale self driving cars. I think it is also perfect to get into with those who have teenagers and a little older kids to get in and experiment. You can read up more details on how to go about building this, and the parts needed here.

Cloud and failure

Despite all the cloud talk and where I live, it is like the cloud mecca, for enterprises it is still quite new and many are just starting to think about it. A hard lesson that many of us learn (and partly how we amass our scars) is to design for failures. For those, who run things in their enterprises data center, are quite spoilt I think. Failures are rare, and if machines or state goes down, moving to another one isn’t really a big deal (of course it is a little more complex, and not to say, there isn’t any down time, or business loss, etc.).

When thinking about a cloud migration (hybrid or otherwise) – a key rule is that you are guaranteed to have failures – at many aspects, and those cannot be exceptional conditions, but rather the normal design and expected behavior. As a result, you app/services/API/whatever needs to be designed for failure. And not only how your loosely couple your architecture to be able to handle these situations, but also, how the response isn’t a binary (yay, or a fancy 404); but rather a degraded experience, where your app/service/API/whatever still performs albeit in a deprecated mode.

Things that can throw one off, and is food for thought (not exhaustive, or on any particular order):

  • Managing state (when failures is guaranteed)
  • Latency – cloud is fast, but slower than your internal data center; you know – physics. 🙂 How are your REST API’s handling latency, and are they degrading performance?
  • “Chatiness” – how talkative, are your things on the wire? And how big is the payload?
  • Rollback, or fall forward?
  • Lossy transfers (if data structure sizes are large)
  • DevOps – mashing up of Developers, and Operations (what some call SRE) – you own the stuff you build, and, responsible for it.
  • AutoScale – most think this is to scale up, but it also means to scale down when resources are not needed.
  • Physical deployments – Regional deployment vs. global ones – there isn’t a right or wrong answer, it frankly depends on the service and what you are trying to do. Personally, I would lean towards regional first.
  • Production deployment strategies – various ways to skin a cat and no one is right or wrong per se (except, please don’t do a basic deployment) – that is suicide. I am use to A/B testing, but also what is now called Blue/Green deployment. Read up more here. And of course use some kind of a deployment window (that works for your business) – this allows you and your team to watch what is going on, and take corrective actions if required.
  • Automate everything you can; yes its not free, but you recoup that investment pretty quick; and will still have hair on the scalp!
  • Instrument – if you can’t measure it, you can’t fix it.

Again, not an exhaustive list, but rather meant to get one thinking. There are also some inherent assumptions – e.g. automation and production deployment suggests, there is some automated testing in place; and a CI/CD strategy and supporting tools.

Bottom line – when it comes to cloud (or any other distributed architecture), the best way to avoid failure is to fail constantly!

From managers to leaders

Recently, a few of us went through a workshop where one of the ‘homework’ was to score oneself, on the following 7 aspects – some of these are attributes that allows one to grow from being (hopefully) good managers to great leaders.

In most enterprises, as one grows in their career, managers need to acquire new capabilities – and quickly. What they have, in terms of skills and capabilities and got her or him to this place, won’t be enough for the next step – as the scope and complexity increases it can leave executives underwhelmed. At the core, new executives need support on these seven dimensions that will help them make this transition.

    • Specialist to generalist – Understand the mental models, tools, and terms used in key business functions and develop templates for evaluating the leaders of those functions.
    • Analyst to Integrator – Integrate the collective knowledge of cross-functional teams and make appropriate trade-offs to solve complex organizational problems.
    • Tactician to Strategist – Shift fluidly between the details and the larger picture, perceive important patterns in complex environments, and anticipate and influence the reactions of key external players.
    • Bricklayer to Architect – Understand how to analyze and design organizational systems so that strategy, structure, operating models, and skill bases fit together effectively and efficiently, and harness this understanding to make needed organizational changes.
    • Problem Solver to Agenda Setter – Define the problems the organization should focus on, and spot issues that don’t fall neatly into any one function but are still important.
    • Warrior to Diplomat – Proactively shape the environment in which the business operates by influencing key external constituencies, including the government, NGOs, the media, and investors.
    • Supporting Cast Member to Lead Role – Exhibit the right behaviors as a role model for the organization and learn to communicate with and inspire large groups of people both directly and, increasingly, indirectly.

I was surprised on how few people talk about this. These come from an awesome HBR article called How Managers become Leaders, which if you haven’t read, I would highly recommend.

So, what can one do? The suggestions outlined are not rocket science, but something to think about. And fundamentally not that much different on how the armed forces trains new officers.

  • Give potential leaders:
    • Experience on cross-functional projects
    • An international assignment
    • Exposure to a broad range of business situations – accelerated growth, sustaining success, realignment, turnaround.
  • When a high potentials’ leadership promise becomes evident give them:
    • A position on a senior management team
    • Experience with external stakeholders
    • An assignment as chief of staff for an experienced enterprise leader
    • An appointment to lead an acquisition integration or a substantial restructuring
  • Just before their first leadership promotion:
    • Send them to an executive program that addresses capabilities like – organizational design, business process improvement, and transition management.
  • When promoted, place new enterprise leaders in business units:
    • That are small, distinct, and thriving
    • And are staffed with an experienced and assertive team that they can learn from.

Download Build deck and video (2018)

Just as last year, I wrote a PowerShell script using which you can download the PowerPoint decks, and, videos from Microsoft Build’s conference, instead of streaming it (or manually download it one by one). You can choose if you want the decks, or the videos, or both. For the videos you can choose the desired resolution (Low, Medium, High) – of course the higher the resolution, the more space is needed. The script also downloads the description and if there is a session image (if there is one).

A few points to note:

  • The slides only once downloaded is ~10GB and with videos (high-resolution), the size goes up to 90.5 GB. So make sure you have enough space.
  • By default the download location is “C:\build-2018\”; you can change this to whatever you want, but make sure there is a trailing backslash. Think of this as the ‘base’ folder.
  • For each session a sub-folder with the session name will be created in the ‘base’ folder setup in the previous step.
  • If a file already exists, it will be skipped.
  • As each file is downloaded, it save it in the root folder and once the download is complete, only then moves it in the relevant subfolder.
  • If a download fails for some reason to retry it, delete the ‘left over’ file(s) in the base folder and then run the script again. The script itself will ‘eat’ the exception and move on to the next file.
  • The video quality parameter is 1 for Low, 2 for Medium, and 3 for High (default).

And if you read through, the script is quite self-explanatory.

# Comments that you should read, before you kick this off. Yes, seriously. :)
# 1. Setup the folder where to download using the parameters outlined below
# 2. Loop through and get the slides first
# 3. Finally, loop through and get the videos last
 
 param (
    [string]$path = "C:\build-2018\",
    [switch]$sessionvideo = $true,
    [int][ValidateRange(1,3)]$videoquality = 3,
    [switch]$sessiondeck = $true
 )

[Environment]::CurrentDirectory=(Get-Location -PSProvider FileSystem).ProviderPath 
$rss = (new-object net.webclient)

#Filenames might get long, so keep this short!
#$downloadlocation = "D:\build-2018"

if (-not (Test-Path $path)) {
	Write-Host "Folder $fpath dosen't exist. Creating it..." 
    New-Item $path -type directory 
}

set-location $path

if($sessiondeck) {  
# Grab the Slides RSS feed - Build 2018
$slides = ($rss.downloadstring("http://s.ch9.ms/events/build/2018/rss/slides")) 

    # ********** download the decks **********
    try { 
        foreach($item in $slides.rss.channel.item) {  
            $code = $item.comments.split("/") | select -last 1     
    
            # Get the url for the pptx file
            $urlpptx = New-Object System.Uri($item.enclosure.url)
        
            # make the filename readable
            $filepptx = $code + " - " + $item.title.Replace(":", "-").Replace("?", "").Replace("/", "-").Replace("<", "").Replace("|", "").Replace('"',"").Replace("*","").Replace("’","'").Replace("â€","").Replace("'NEW SESSION'","").Replace("™","").Replace("œ","")
            $filepptx = $filepptx.substring(0, [System.Math]::Min(120, $filepptx.Length))
            $filepptx = $filepptx.trim()
            $filejpg = $filepptx
 
            $filepptx = $filepptx + ".pptx" 
            $filejpg = $filejpg + "_960.png"
 
     
            $folder = $item.title.Replace(":", "-").Replace("?", "").Replace("/", "-").Replace("<", "").Replace("|", "").Replace('"',"").Replace("*","").Replace("’","'").Replace("â€","").Replace("'NEW SESSION'","").Replace("™","").Replace("œ","")
            $folder = $folder.substring(0, [System.Math]::Min(100, $folder.Length))
            $folder = $folder.trim()
             
            if (-not (Test-Path $folder)) { 
                Write-Host "Folder $folder doesnt exist. Creating it..."  
                New-Item $folder -type directory 
            }
     
            # Make sure the PowerPoint file doesn't already exist
            if (!(test-path "$path\$folder\$filepptx")) {   
                # Echo out the file that's being downloaded
                $filepptx
                $wc = (New-Object System.Net.WebClient)  
 
                # Download the pptx file
                Invoke-WebRequest $urlpptx -OutFile $path\$filepptx
 
                # download the jpg but don't want to break if this doesn't exist; hence the nested try blocks
                try {
                    if($item.thumbnail -ne $null) {
                        $urljpg = New-Object System.Uri($item.thumbnail.url)

                        if (!(test-path "$path\$filejpg")) {    
                            $wc.DownloadFile($urljpg, "$path\$folder\$filejpg")
                        }
                    }
                }
                catch {
                    Write-Host "Image (jpeg) $filejpg doesn't exist ... eating the exception and moving on ..."
                }
         
                mv $filepptx $folder 
                #mv $filejpg $folder 
            } #endif
            else {
                Write-Host "PPTX: $filepptx exist; skipping download."  
            }

            #try and get the sessions details
            try {
                $descriptionFileName = "$($path)\$($folder)\$($Code.trim()).txt"

                if (!(test-path "$descriptionFileName")) {
                    $OutFile = New-Item -type file $descriptionFileName -Force  
                    $Content = "Title: " + $item.title.ToString().trim() + "`r`n" + "`r`n" + "Presenter: " + $item.creator + "`r`n" + "`r`n" + "Summary: " + $item.summary.ToString().trim() + "`r`n" + "`r`n" + "Link: " + $item.comments.ToString().trim() 

                    #some categories are missing; so need to eat the exception
                    #this is a hack and not very elegant
                    try {
                        if($item.category -ne $null) {
                            $Content = $Content + "`r`n" + "`r`n" + "Category: " + $item.category.ToString().trim().Replace("+"," ")
                        }
                    }
                    catch {
                        #do nothing; eat the exception
                    }

                    add-content $OutFile $Content
                }
            }
            catch {
                $ErrorMessage = $_.Exception.Message
                $FailedItem = $_.Exception.ItemName
                Write-host "\t" $ErrorMessage + "\n" + $FailedItem
            }
        } #end-loop foreach
 
        Write-host "*************** Downloading all the decks complete ***************"
    }
    catch
    {
        Write-host "Oops, could not find any slides."
        $ErrorMessage = $_.Exception.Message
        $FailedItem = $_.Exception.ItemName
        Write-host "\t" $ErrorMessage + "\n" + $FailedItem
    }
} #download session-deck 

# ********** download the videos **********
# if you don't want the video but only the slides just comment all the code below in the foreach loop
try { 
	# check for video download
	if($sessionvideo) {
        switch ($videoquality) {
            1 {$video = ($rss.downloadstring("http://s.ch9.ms/events/build/2018/rss/mp3")); break}
            2 {$video = ($rss.downloadstring("http://s.ch9.ms/events/build/2018/rss/mp4")); break}
            default {$video = ($rss.downloadstring("http://s.ch9.ms/events/build/2018/rss/mp4high")); break}
        }
 
        foreach($item in $video.rss.channel.item) {    
            # Grab the URL for the MP4 file
            $url = New-Object System.Uri($item.enclosure.url)  
     
            # Create the local file name for the video download
            $file = $item.title.Replace(":", "-").Replace("?", "").Replace("/", "-").Replace("<", "").Replace("|", "").Replace('"',"").Replace("*","").Replace("’","'").Replace("â€","").Replace("'NEW SESSION'","").Replace("™","").Replace("œ","")
            $file = $file.substring(0, [System.Math]::Min(120, $file.Length))
            $file = $file.trim()
            $file = $file + ".mp4"  
     
            $folder = $item.title.Replace(":", "-").Replace("?", "").Replace("/", "-").Replace("<", "").Replace("|", "").Replace('"',"").Replace("*","").Replace("’","'").Replace("â€","").Replace("'NEW SESSION'","").Replace("™","").Replace("œ","")
            $folder = $folder.substring(0, [System.Math]::Min(100, $folder.Length))
            $folder = $folder.trim()
     
            if (-not (Test-Path $folder)) { 
                Write-Host "Folder $folder doesn't exist. Creating it..."  
                New-Item $folder -type directory 
            }
         
     
            # Make sure the video file doesn't already exist
            if (!(test-path "$folder\$file"))     
            {   
                # Echo out the  file that's being downloaded
                $file
             
                # Download the video file
                try{
                    if (!(test-path "$path\$file"))
                    {
                        Invoke-WebRequest $url -OutFile $path\$file
             
                        #move it from the current working folder to the target
                        mv $file $folder
                    }
                    else {
                        Write-Host "Video: $file - anoter process possibly working on this; skipping download."
                    }
                }
                catch {
                    $ErrorMessage = $_.Exception.Message
                    $FailedItem = $_.Exception.ItemName
                    Write-host "\t" $ErrorMessage + "\n" + $FailedItem
                }
            }
            else {
                Write-Host "Video: $file exist; skipping download."  
            }
         
        } #end-loop foreach
    } #end - video check
}
catch
{
    Write-host "Oops, could not find any videos or some other error happened."
    $ErrorMessage = $_.Exception.Message
    $FailedItem = $_.Exception.ItemName
    Write-host "\t" $ErrorMessage + "\n" + $FailedItem
} # ********** End - download the videos section **********
 
Write-host "*************** All Done! Woot! ***************"

Certificate error with git and Donkey Car

If you were trying to pull the latest source code on your Raspberry Pi for donkeycar, and get the following error, then probably your clock is off (and I guess some nonce is failing). This can happen if your pi had been powered off for a while (as in my case), and it’s clock is off (clock drift is a real thing) :).

fatal: unable to access 'https://github.com/wroscoe/donkey/': server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none

To fix this, the following commands works. It seems the Raspberry Pi 3, by default has NTP disabled and this would enable it. I also had to check the result status with the second command, and force it with the third one.

sudo timedatectl set-ntp True
timedatectl status
sudo timedatectl set-local-rtc true

And that should do it; you might need to reboot the pi just to get it back on and then you should be able to pull the code off git and deploy your autonomous car.

AI photos–style transfer

Can #AI make me look (more) presentable? The jury is out I think. Smile

This is called style transfer, where the style/technique from a kind of painting (could be a photos too) is applied to an image, to create a new image. I took this using the built-in camera on my machine sitting at my desk and then applying the different kind of ‘styles’ on it. Each of these styles are is a separate #deeplearning model  that has learned how to apply the relevant style to a source image.

candy
Style – Candy

feathers
Style – Feathers

mosaic
Style – Mosaic

robert
Style – Robert

Specifically, this uses a Neural Network (#DeepLearning) model called VGG19, which is a 19 layer model running on TensorFlow. Of course you can export this to a ONNX model, that then can be used in most other run-times and libraries.

image

This is inspired from Cornell universities paper – Perceptual Losses for Real-Time Style Transfer and Super-Resolution. Below is a snapshot of the VGG code that.

def net(data_path, input_image):
layers = (
'conv1_1', 'relu1_1', 'conv1_2', 'relu1_2', 'pool1',
'conv2_1', 'relu2_1', 'conv2_2', 'relu2_2', 'pool2',
'conv3_1', 'relu3_1', 'conv3_2', 'relu3_2', 'conv3_3',
'relu3_3', 'conv3_4', 'relu3_4', 'pool3',
'conv4_1', 'relu4_1', 'conv4_2', 'relu4_2', 'conv4_3',
'relu4_3', 'conv4_4', 'relu4_4', 'pool4',
'conv5_1', 'relu5_1', 'conv5_2', 'relu5_2', 'conv5_3',
'relu5_3', 'conv5_4', 'relu5_4'
)
data = scipy.io.loadmat(data_path)
mean = data['normalization'][0][0][0]
mean_pixel = np.mean(mean, axis=(0, 1))
weights = data['layers'][0]

net = {}
current = input_image
for i, name in enumerate(layers):
kind = name[:4]
if kind == 'conv':
kernels, bias = weights[i][0][0][0][0]
# matconvnet: weights are [width, height, in_channels, out_channels]
# tensorflow: weights are [height, width, in_channels, out_channels]
kernels = np.transpose(kernels, (1, 0, 2, 3))
bias = bias.reshape(-1)
current = _conv_layer(current, kernels, bias)
elif kind == 'relu':
current = tf.nn.relu(current)
elif kind == 'pool':
current = _pool_layer(current)
net[name] = current

assert len(net) == len(layers)
return net

def _conv_layer(input, weights, bias):
conv = tf.nn.conv2d(input, tf.constant(weights), strides=(1, 1, 1, 1),
padding='SAME')
return tf.nn.bias_add(conv, bias)

def _pool_layer(input):
return tf.nn.max_pool(input, ksize=(1, 2, 2, 1), strides=(1, 2, 2, 1),
padding='SAME')

If you have interest to play with this, you can download the code. Personally, I like Mosaic style the best.