VMworld US 2017 – Pre-Conference Analysis

It’s VMworld US 2017 this week, starting in just a few hours, and as usual we can likely expect some huge announcements coming out of Las Vegas.

VMware have had a great year; their Q2 2018 earnings were announced a few days back, and they appear to be in great shape, with both NSX and VSAN leading the charge with great revenue growth. Their share price has definitely benefitted from this, with it now back up over $100 per share for the first time in four years (source), and this is great news at a time where there is a general outlook of doom and gloom as the march of Public Cloud continues.

Things we can no doubt expect to hear about this week:

  • VMware Cloud on AWS – this is the monster which will no doubt dominate this week; announced around a year ago, and in private beta earlier this year, it has now been made GA in the US West (Oregon) region only, this is going to provide a long list of benefits to those who can afford it. I expect we will see some cementing around details on this over the next few days, and some talk around integration with existing AWS services, which provides most of the value of this service over using a vCAN partner. The page is now live on AWS’ website, available here.
  • VMware Horizon Air on Azure – not to let Amazon get all the cloudy goodness, this new service announced back in May will no doubt be touted and elaborated on this week, or at VMworld Europe in a couple of weeks. This is going to make building a flexible Horizon estate easier than ever, and promises some additional compatibility around Microsoft’s desktop software offerings. This should be just about ready to drop according to the VMware FAQ on the service.
  • AppDefense – Gregg Robertson (@thesaffageek), has released a post about this today, but things have been bubbling up on this for some time, from talk of Project Goldilocks from around a year ago, to more recent teasings. It will be interesting to see more information on this as it develops; application security management is a growing market, as NSX has shown, and from the looks of this at the moment, it will give valuable insights into what’s running, and how it interacts. Coupled with vRealize Network Insight, and NSX, this could be a powerful set of tools.
  • VSAN – VSAN is now reaching maturity, and has a tonne of production customers if numbers are to be believed, it was talked about a long while back that VSAN is an object store under the covers, and is now able to serve iSCSI storage. I expect we will see this continue, with object and file storage being served out of VSAN in coming versions. Hopefully we will see some announcements around this in the coming days.
  • vSphere Next – it has been a year or so since vSphere 6.5 came out now, and it has proven itself to be a great release, with improved management, stability, and has revitalised an aging product. The recent announcement about the coming death of the Flex client, and the Windows vCenter install will, I would imagine, foreshadow the announcement of the final vSphere version bearing these relics this week.
  • Other Stuff – it seems some bits were announced early including:
    • Cost Insight – a tool to manage costs and make recommendations about placing of workloads across AWS, Azure, and on-premises
    • Discovery – a tool helping you discover workloads across clouds and organise them logically
    • NSX Cloud – a SaaS based offering of NSX, descibed by The Register as ‘NSX-as-a-Service’ no doubt for public cloud usage
    • Wavefront – performance monitoring for cloud native apps

I am hoping to throw more blog posts out as the week goes on, and I get some more time to check things out. In the meantime, the General Session streams can be found here, the vBrownBag/VMTN Tech Talks are available here, and there will be up-to-the minute news in all the usual places.


Becoming a Briklayer

This month saw me start a new chapter in my career, as a DevOps Sales Engineer at Rubrik. This is an exciting leap from me, moving from the channel into vendorland, and to a company which is changing the market in data management.

Rubrik’s approach to product development, being API first, and rooted in a modern distributed systems approach, is evident when using their products, and this was hugely attractive to me as a consumer. The API has truly full coverage, is simple, consistent, and well documented. Despite the number of software and hardware companies promoting themselves as modern and API first, using Rubrik in a POC I did earlier this year was truly a breath of fresh air.

This is my first foray into both a sales organisation, and to a vendor, but my feeling is that it will open my mind to the wider IT industry, and give me some great opportunities to not only see how the sausage is made, but also how to sell sausages (as a vegetarian, I should note that the sausages in this analogy are made from a soya derivative).

The last few years for me have been a fast paced journey through systems engineering and automation, product development, and learning and implementing modern software engineering and delivery processes both on-premises and in the cloud, and these skills should come in very useful in my new role helping customers and potential customers to realise the possibilities that a platform that enables end-to-end systems automation can deliver, and spreading the good word.

You can see more of the latest of what Rubrik are doing from their appearance at Cloud Field Day 2 a couple of weeks ago:

There is also a raft of the kind of DevOps things my new team work on available at the following GitHub repositories:



Goodbye to the Past

This week, in the lead up to this year’s VMworld US, VMware have announced the impending death of both vCenter on Windows, and the vSphere Web Client.

Each of these will be deprecated in the next major version of vSphere (assuming 7.0), and removed in the next major version after that (assuming 8.0). Quite when we will see these releases is still a matter of speculation, but with VMworld this week it is possible 7.0 will be slated for release in Winter 2017.

vCenter Server Appliance vs Windows

The vCenter Server Appliance, or vCSA, has been around since vSphere 5.0, and over time has become more mature, more stable, and has moved from VMware’s licensed version of SUSE Linux, to the modern and lightweight Photon OS.

These changes have turned the appliance into a performant, and reliable powerhouse; important facets for what is obviously a critical and central part of any modern vSphere datacenter.

Since vSphere 6.0, the vCSA has had at least feature parity with vCenter installed on Windows, and the added simplicity of installation that it includes – not having to install VUM separately, not having to install a MSSQL database, and not having yet another Windows instance to stroke – makes the use of vCSA over Windows a no-brainer.

As things have moved on, the migration tools too have become more and more mature, meaning that migration to vCSA 6.5 Update 1 is a really straight forward affair from most starting points.

The death of flash – long live HTML5

The vSphere Web Client has been a contentious tool since its introduction in vSphere 5.0; often unstable and poorly performing, it has been the only way to access newer features of vSphere such as VSAN, SPBM, as well as newer versions of vSphere Update Manager and Site Recovery Manager.

Added to this are the raft of security issues facing Flash, and it’s general fall from grace over the last 10 years. Adobe have now decreed that Flash will die in 2020, which is great news for desktop browsers, mobile devices, and the internet in general.

VMware responded to customer feedback on the vSphere Web Client (or Flex Client) by releasing the HTML5 Web Client under their Flings program back in March 2016, and shipped vSphere 6.5 with this installed alongside the Flex client.

While the HTML5 client still does not have feature parity at this time, the rate at which it is catching up is impressive, and we can reasonably expect it to match, and possibly, overtake in terms of features in the next 6-12 months.


The removal of both of these tools is, for me at least, great news. I have had no desire to run a Windows vCenter since probably 5.5, which was four years ago now. Likewise, the vSphere Web Client has been a necessary evil since the same time, especially when it came to using third party integrations and newer features.

Hopefully the removal of these legacy chains will mean that development is accelerated as engineering resource is freed up. It will put some burden on the engineering teams from VMware ecosystem partners, who will need to release new versions of their plugins, if they haven’t already, utilising the gorgeous HTML 5 Clarity interface, but those partners should already be moving in this direction anyway.

rbvami – Managing the vCSA VAMI using Ruby

I have been putting together a module for managing vCSA’s VAMI using Ruby. This uses the vCenter 6.5 REST API, and the long term plan is to build it out to cover the entire REST API.

My intention is to use this module as the basis for a Chef cookbook for managing VAMI configuration, and is mainly a learning rather than a practical exercise.

The project is on my GitHub site, feel free to contact me if there is functionality you would like to see added.

Cleaning up AWS OpsWorks Automate Nodes

I’ve been playing with Chef and AWS’ OpsWorks Automate product a lot in the last few weeks, one problem I had was that as I kept bootstrapping EC2 instances, using the excellent Knife EC2 tool, the nodes were not being cleaned up out of the Chef Automate portal. I’m imagining this will be a common issue for folks using ephemeral type workloads with Chef Automate in any cloud.

AWS’ documentation has some AWS CLI commands to run to remove old nodes, but this refers to AWS CLI commands which do not seem to be present in the latest version of AWS CLI (there is no ‘aws opsworks-cm’ domain now in the CLI, so no way of managing OpsWorks Automate).

I found this page in Chef’s highly recommended Learn Chef Rally training site which led me to the way to do this. The following can be run from an SSH connection into your Chef Automate server (or in my case, as I had not assigned a keypair on creation of my Automate server, through EC2 Systems Manager’s Run Command feature):

sudo automate-ctl delete-visibility-node <NODE_NAME>

If you have multiple nodes with the same name, you may receive the following response:

Multiple nodes were found matching your request. Please delete by ID using: automate-ctl delete-visibility-node-by-id NODE_UUID

Node UUID Node Name Org Name Chef Server
==================================== ========= ======== ===========
1c298e89-7c9f-4feb-b784-20b3858bfd6f webtest2 default chefautomate-1abcdefgo12abcde.eu-west-1.opsworks-cm.io
7f9b96df-7c02-4277-a5bb-879962b17136 webtest2 default chefautomate-1abcdefgo12abcde.eu-west-1.opsworks-cm.io
05f55344-2425-4764-8db6-9c0a0ef8d015 webtest2 default chefautomate-1abcdefgo12abcde.eu-west-1.opsworks-cm.io

You can delete these using the following command instead:

sudo automate-ctl delete-visibility-node-by-id <NODE_ID>

This wraps up the post, hopefully it comes in useful for people.

Deploying NX-OSv 9000 on vSphere

Cisco have recently released (1st March 2017) an updated virtual version of their Nexus 9K switch, and the good news is that this is now available as an OVA for deployment onto ESXi. We used to use VIRL in a lab, which was fine until a buggy earlier version of the virtual 9K was introduced which prevented core functionality like port channels. This new release doesn’t require the complex environment that VIRL brings, and lets you deploy a quick pair of appliances in vPC to test code against.

The download is available here, and while there are some instructions available, I did not find them particularly useful in deploying the switch to my ESXi environment. As a result, I decided to write up how I did this to hopefully save people spending time smashing their face off it.

Getting the OVA

NOTE: you will need a Cisco login to download the OVA file. My login has access to a bunch of bits so not sure exactly what the requirements are around this.

There are a few versions available from the above link, including a qcow2 (KVM) image, a .vmdk file (for rolling your own VM), a VirtualBox image (for use with VirtualBox and/or Vagrant), and an OVA (for use with Fusion, Workstation, ESXi).

Once downloaded we are ready to deploy the appliance. There are a few things to bear in mind here:

  1. This can be used to pass VM traffic between virtual machines: there are 6 connected vNICs on deployment, 1 of these simulates the mgmt0 port on the 9K, and the other 5 are able to pass VM traffic.
  2. vNICs 2-6 should not be attached to the management network (best practice)
  3. We will need to initially connect over a virtual serial port through the host, this will require opening up the ESXi host firewall temporarily

Deploying the OVA

You can deploy the OVA through the vSphere Web Client, or the new vSphere HTML5 Web Client, I’ve detailed how to do this via PowerShell here, because who’s got time for clicking buttons?


# Simulator is available at:
# https://software.cisco.com/download/release.html?mdfid=286312239&amp;amp;softwareid=282088129&amp;amp;release=7.0(3)I5(1)&amp;amp;relind=AVAILABLE&amp;amp;rellifecycle=&amp;amp;reltype=latest
# Filename: nxosv-final.7.0.3.I5.2.ova
# Documentation: http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/7-x/nx-osv/configuration/guide/b_NX-OSv_9000/b_NX-OSv_chapter_01.html

Function New-SerialPort {
  # stolen from http://access-console-port-virtual-machine.blogspot.co.uk/2013/07/add-serial-port-to-vm-through-gui-or.html
  ) #end
  $dev = New-Object VMware.Vim.VirtualDeviceConfigSpec
  $dev.operation = "add"
  $dev.device = New-Object VMware.Vim.VirtualSerialPort
  $dev.device.key = -1
  $dev.device.backing = New-Object VMware.Vim.VirtualSerialPortURIBackingInfo
  $dev.device.backing.direction = "server"
  $dev.device.backing.serviceURI = "telnet://"+$hostIP+":"+$prt
  $dev.device.connectable = New-Object VMware.Vim.VirtualDeviceConnectInfo
  $dev.device.connectable.connected = $true
  $dev.device.connectable.StartConnected = $true
  $dev.device.yieldOnPoll = $true

  $spec = New-Object VMware.Vim.VirtualMachineConfigSpec
  $spec.DeviceChange += $dev

  $vm = Get-VM -Name $vmName

# Variables - edit these...
$ovf_location = '.\nxosv-final.7.0.3.I5.1.ova'
$n9k_name = 'NXOSV-N9K-001'
$target_datastore = 'VBR_MGTESX01_Local_SSD_01'
$target_portgroup = 'vSS_Mgmt_Network'
$target_cluster = 'VBR_Mgmt_Cluster'

$vi_server = ''
$vi_user = 'administrator@vsphere.local'
$vi_pass = 'VMware1!'

# set this to $true to remove non-management network interfaces, $false to leave them where they are
$remove_additional_interfaces = $true

# Don't edit below here
Import-Module VMware.PowerCLI

Connect-VIServer $vi_server -user $vi_user -pass $vi_pass

$vmhost = $((Get-Cluster $target_cluster | Get-VMHost)[0])

$ovfconfig = Get-OvfConfiguration $ovf_location

$ovfconfig.NetworkMapping.mgmt0.Value = $target_portgroup
$ovfconfig.NetworkMapping.Ethernet1_1.Value = $target_portgroup
$ovfconfig.NetworkMapping.Ethernet1_2.Value = $target_portgroup
$ovfconfig.NetworkMapping.Ethernet1_3.Value = $target_portgroup
$ovfconfig.NetworkMapping.Ethernet1_4.Value = $target_portgroup
$ovfconfig.NetworkMapping.Ethernet1_5.Value = $target_portgroup
$ovfconfig.DeploymentOption.Value = 'default'

Import-VApp $ovf_location -OvfConfiguration $ovfconfig -VMHost $vmhost -Datastore $target_datastore -DiskStorageFormat Thin -Name $n9k_name

if ($remove_additional_interfaces) {
  Get-VM $n9k_name | Get-NetworkAdapter | ?{$_.Name -ne 'Network adapter 1'} | Remove-NetworkAdapter -Confirm:$false

New-SerialPort -vmName $n9k_name -hostIP $($vmhost | Get-VMHostNetworkAdapter -Name vmk0 | Select -ExpandProperty IP) -prt 2000

$vmhost | Get-VMHostFirewallException -Name 'VM serial port connected over network' | Set-VMHostFirewallException -Enabled $true

Get-VM $n9k_name | Start-VM

This should start the VM, we will be able to telnet into the host on port 2000 to reach the VM console, but it will not be ready for us to do that until this screen is reached:


Now when we connect we should see:


At this point we can enter ‘n’ and go through the normal Nexus 9K setup wizard. Once the management IP and SSH are configured you should be able to connect via SSH, the virtual serial port can then be removed via the vSphere Client, and the ‘VM serial port connected over network’ rule should be disabled on the host firewall.

Pimping things up

Add more NICs

Obviously here we have removed the additional NICs from the VM, which makes it only talk over the single management port. We can add a bunch more NICs and the virtual switch will let us use them to talk on. This could be an interesting use case to pass actual VM traffic through the 9K.

Set up vPC

The switch is fully vPC (Virtual Port Channel) capable, so we can spin up another virtual N9K and put them in vPC mode, this is useful to experiment with that feature.

Bring the API!

The switch is NXAPI capable, which was the main reason for me wanting to deploy it, so that I could test REST calls against it. Enable NXAPI by entering the ‘feature nxapi’ commmand.


Hopefully this post will help people struggling to deploy this OVA, or wanting to test out NXOS in a lab environment. I found the Cisco documentation a little confusing so though I would share my experiences.

Replacing the ‘All Services’ Icon in vRealize Automation

I had a conversation with Ricky El-Qasem (@rickyelqasem) on Twitter this week about the ‘All Services’ logo in vRealize Automation, and whether this could be replaced programatically.

For those which don’t know the pain of this particular element of vRA; when browsing the service catalog, groups of services are listed down the left hand side of the page with icons next to them:

Screen Shot 2017-03-17 at 17.52.42.png

These can all be changed, but until recently the top icon would remain as a blue lego brick, which can make the otherwise slick portal look unsightly. This is shown on the image below:

Screen Shot 2017-03-17 at 17.56.25.png

Now luckily, from vRA 7.1, this has been replaceable through the API, and steps have been documented in the accompanying guide here. This uses the REST API, and means you need to convert the image in PNG into Base-64 encoding in order to push it to the API, a little to manual for me!

So I quickly threw vRA 7.2 up in my home lab and got to work. I chose to script it using Python because I found that I could easily convert the image to Base-64, and I knew I could do the REST calls using the excellent ‘requests’ Python package (info available here). The code I used is available on my GitHub, and is shown below. I also created a script to delete the custom icon, and return things to vanilla state, you know, just in case 😉

Anyway, I hope this is useful for people who want to quickly and easily replace the icon.

#!/usr/bin/env python
# required packages, install with pip if not present
import requests
import json
# disable self-signed cert warnings
# replace these variables
filename = 'service.png'
vra_ip = ''
vra_user = 'administrator@vsphere.local'
vra_pass = 'VMware1!'
vra_tenant = 'vsphere.local'
# don't replace anything from here
# open file and encode it in b64
with open("./"+filename, "rb") as f:
    data = f.read()
    encoded = data.encode("base64")
encoded = encoded.replace("\r","")
encoded = encoded.replace("\n","")
# get our authorization token
uri = 'https://'+vra_ip+'/identity/api/tokens'
headers = {'Accept':'application/json','Content-Type':'application/json'}
payload = '{"username":"'+vra_user+'","password":"'+vra_pass+'","tenant":"'+vra_tenant+'"}'
r = requests.post(uri, headers=headers, verify=False, data=payload)
token = 'Bearer '+str(json.loads(r.text)["id"])
# send the new icon to the API
uri = 'https://'+vra_ip+'/catalog-service/api/icons'
headers = {'Accept':'application/json','Content-Type':'application/json','Authorization':token}
payload = '{"id":"cafe_default_icon_genericAllServices","fileName":"'+filename+'","contentType":"image/png","image":"'+encoded+'"}'
r = requests.post(uri, headers=headers, verify=False, data=payload)
if r.status_code == 201:
    print "Replacement successful"
    print "Expected return code 201, got "+r.status_code+" something went wrong"