https://vmc.vmware.com/swagger/index.html – Swagger documentation page for VMC APIs. To access this page you need to be authenticated and have the relevant permissions to be able to execute SDDC API calls.
OK, so I know I promised to condense my list of VMWorld 2017 VMC on AWS sessions down to my favourite 5-10, however since I made the aforementioned promise it seems like a flood of features has passed us by. Whilst I am getting around to selecting the finalists for a prestigious place on my list (*wink), I want take the pulse of both VMware and AWS enthusiasts to understand how you feel the service could be used in ways other than our marketers suggest.
So what the heck do you want Kev?…
True story: Recently I ran a VMC workshop that gave me a bit of insight into the resourcefulness of AWS users and how the AWS Solution Architects were often surprised by their customers creativity. Seeing as consumption of VMware Cloud on AWS is somewhat unprecedented (being the service has only just launched) I’m keen to understand how combining the technologies can lead to some unforeseen and inspiring outcomes.
So my ask? >> Please forward me any thoughts, ideas and hacks, no matter how wacky or wild and I’ll try to recreate them in my lab and detail them right here. We might even attempt to demonstrate the most requested at a local VMUG Hackathon type thingy. Reply on the post, hit me up on Twitter @_kevops, or if you’re more of a traditionalist, email@example.com. Never fear, every idea will be credited to it’s creator regardless of how absurd.
To get you thinking (and to give you a little more detail on the service in general) I’ve linked a couple of VMC sessions from AWS re:Invent specifically on “Unique Integrations” and a VMC Technical Deep Dive.
Enjoy!.. and happy holidays
VMware Cloud on AWS: Technical Deep Dive
VMware Cloud on AWS: A World of Unique Integrations Between VMware & AWS
Aside from my focus on VMware Cloud on AWS I’ve also been spending my time getting to grips with all the the various technologies supporting containerisation and DevOps. If you’ve followed my blog in the past you’ll know that I’ve was delving into modern software engineering practices, albeit from an operational perspective.
Luckily for me VMware has come a long way in creating technology that supports this theoretical understanding, which was highly evident at VMWorld 2017. There was a shedload of great sessions on Pivotal Cloud Foundry, Pivotal Container Service, vSphere Integrated Containers, vRealize Automation, Wavefront (and on and on and on…)
As I received some great feedback from my last post which consolidated some of the VMC focused sessions from VMWorld, I decided to repeat the process for VMware Cloud Native and DevOps.
Please note, I’m also going to use this as a starting point to cover Pivotal Container Service (PKS) and vSphere Integrated Containers (VIC) in more detail, before breaking it down into easily digestible chunks. In the meantime enjoy the extensive selection of CNA, DevOps & automation sessions from VMWorld 2017.
Another VMWorld done… and now for the weeks of recoding catchup on deep-dive goodness. There is plenty of detailed information to finally clear up the speculation and FUD that has been circulating on VMware Cloud on AWS over the last 12 months.
For your convenience I have collected available VMC sessions for easy consumption. I’ll follow up this post with my top 10 once I have trawled through all the below.
Over the last year (give or take a few months), VMware has been diligently tweaking a variety of its products to integrate container functionality as it becomes more prevalent in the enterprise. With this in mind, I thought I’d put together a quick post detailing three VMware tools which can be used in a simple containerised format.
Update: The below tools are intended to be run as single Docker commands rather than launching a terminal session on the container as you would with William Lam’s much more comprehensive vmware-utils Docker appliance. If @lamw‘s approach is more your bag, you can read about it here.
For me, OVFTool is a great CLI utility for migrating VM templates and ISOs to & from vCloud Air, although it’s functionality extends way beyond VCA. As a little side project I thought it would be a great idea to containerise the most recent release (v4.2), instead of installing it on my Mac and dealing with potential conflicts. To my delight this was a relatively easy task and took less than 10 mins to build, commit and push to my public repo.
Disclaimer: This image is hosted on my public Docker Hub registry, however it is not officially endorsed or supported by VMware in any way. That said, please feel free to use (but at your own risk).
To use, simply enter the below Docker command which will allow you to interactively (-i) run the skinnypin/ovftool image and execute an OVFTool command, in this case ovftool –help.
~> docker run -i skinnypin/ovftool ovftool --help
PowerCLI Core builds upon the open source Microsoft PowerShell Core and .Net Core enabling the use the of PowerCLI on non-Windows operating systems. As a Mac user, having to open up a Windows VM in VMware Fusion just to use PowerShell has been a little inconvenient. But no more…
In addition to availability on OSX and Linux, the awesomeness of PowerCLI Core can also be accessed via an offical VMware Docker image. For more info on PowerCLI Core see here.
To use, enter the below Docker command which will give you interactive (-i) access to the PowerCLI prompt.
~> docker run -i vmware/vmwarepowercli
Project Platypus is a very nice tool built by my good friend Grant Orchard (and other VMware folks) which details supported VMware product API’s and their usage. If you’ve ever tried to utilise VMware APIs by referencing official documentation, you understand why this tool is absolutely necessary. To the best of my knowledge Platypus is only available in this containerised format, so if you want the goodness you’re going to have to get familiar with Docker…
To use, enter the below Docker command which will run a detached (-d) container which is accessible from your web browser on port 8080 (-p 8080:80) using the IP address of your container host.
~> docker run -d -p 8080:80 vmware/platypus
So there you have it. Three easily accessible VMware tools that can be distributed without having to read any installation documentation (as long as you have access to a Docker environment ). As always, feedback is appreciated, especially if this is useful and you want to see other tools available in this format.
Update 2 > BONUS TOOL: I also spend some time Dockerizing VIC Machine (v0.8.0-rc3), the container host provisioning utility used with vSphere Integrated Containers. Details on VIC here.
To use, simply enter the below Docker command which will allow you to interactively (-i) run the skinnypin/vic image and execute a VICMachine command, in this case vic-machine-linux –help.
~> docker run -i skinnypin/vic /vic-machine-linux --help
Intro: So you may have been following my VCA Dissected series, but in line with the recent expansion of VMware Cloud Services my role as a Cloud Specialist has diversified to include all things VMware & Cloud. With that in mind, a series name change is in order… So VCADissected becomes VMware Cloud(s) Dissected.
All of the (VMware) Clouds…
Holy moly, it’s been a crazy few months on the road with VMWorld! So many game changing announcements delivered through keynotes, breakout sessions and group discussions. In addition to general announcements on vSphere 6.5, EUC and Cloud Native Apps, we were also introduced to several new VMware public cloud offerings and associated services. For the purposes of clarity, I’m going to give a high-level break down of each platform within VMware’s Cross Cloud Architecture (not including Cross Cloud Services) to try and illustrate where each will be most effective.
First things first. If you haven’t watched day one keynote from VMWorld Europe, I highly recommend you do so… (click on image to view the recording. If you’re not interested in the reasoning behind the vision, skip to about 30 mins in).
To summarize, P.G. talked through his predictions for cloud consumption trends in the near (and not so near) future which set the stage to announce VMware Cross-Cloud Architecture; a set of converged software services incorporating major partnerships with leaders in hyper-scale cloud. So let’s dig a little deeper.
Note: There were plenty of disclaimers and forward looking statements on tech previews in the VMWorld presentations, public FAQ’s, demos and press releases, so please understand that anything I mention here is subject to change as more information is released.
VMware Cloud (VMC) on Amazon Web Services
Boom, the cat’s finally out of the bag. As many of the talking heads have pointed out this is about as significant as any cloud partnership could be. Here are some of the highlights I have chosen from the recent VMWorld VMC sessions.
The big stuff…
The VMware SDDC stack (vSphere 6.5, VSAN & NSX) available within AWS Datacenter’s, on AWS infrastructure dedicated to this service.
VMC procurement, provisioning and lifecycle is via the VMC customer portal.
VMC upgrades, maintenance and billing are exclusively managed by VMware.
Non-VMC services are still billed and managed by AWS directly.
VMC can be consumed as a standalone platform on AWS, as a hybrid cloud through vCenter Enhanced Linked Mode, or (in the future) cloud-to-cloud between AWS regions/availability zones through the same mechanism.
Continuous upgrades of the SDDC components (including vCenter) on AWS will be scheduled and executed by VMware.
Billed by the hour, or procured for a reduced price over 12 or 36 months in a similar commercial model to AWS reserved instances. Customers will also be able to leverage their existing investments in VMware licenses through VMware customer loyalty programs.
The technical stuff…
Initial deployment of between 4 to 64 hosts, which can be scaled through manual process or by;
Elastic Distributed Resource Scheduler (Elastic DRS) which dynamically adds and removes physical hosts based on predefined EDRS rules.
Enhanced Linked Mode enables inventory management, content library synchronization, etc. of AWS VMC hosts from on-prem vCenter.
Each tenancy uses the AWS VPC construct for logical isolation.
Edge/perimeter services are provided by NSX Edge Services Gateway, not AWS VPC network services.
Full VMC integration with AWS Direct Connect.
VMC and AWS user accounts are linked, but separate interfaces and authentication is required for services unique to each vendor.
Administrators have direct access to vCenter UI and REST APIs.
VMware defined RBAC limits the install of untested third party software with custom VIB’s.
Simply put, industry leading SDDC platform on an industry leading hyper-scale public cloud. Truly the best of both worlds.
The ability to easily integrate and extend our VMware IaaS platform to incorporate AWS storage, data, application and automation specific services.
Intra region/availability zone efficiency through low latency connectivity to AWS services, avoiding costs incurred when data and network traffic leaves the AWS region.
Zero downtime workload migration to VMC-on-AWS through Cross vCenter Server vMotion.
Maintenance and upgrade of SDDC platform components managed entirely by VMware.
There’s not a whole bunch of detailed information on VMC right now as it’s early days, but Frank Denneman’s blog and the AWS blog is a good place to start. Note, during the ‘Closer Look’ VMWorld breakout session it was also acknowledged that a number of announcements are still to be revealed at AWS Re:Invent at the end of November.
VMware Cloud Foundation (VCF) on IBM Softlayer
VMware Cloud Foundation is the same SDDC stack (vSphere, VSAN, NSX) as VMC but with VMware SDDC Manager as the overlay software which handles platform deployment, configuration and ongoing SDDC lifecycle tasks for specific use cases. What makes VCF different from VMC (other than the obvious partnerships), is that Cloud Foundation can be deployed privately within our own datacenters in addition to public cloud.
The global partnership with IBM was announced at VMWorld Las Vegas and they will be the first global cloud service provider to offer Cloud Foundation. vCloud Air will also join IBM in the near future in addition to other numerous VCAN providers throughout 2017.
Note, I’m not really going into any detail about VCF as this is a public cloud breakdown. I would recommend a read of Ray Heffer’s fantastic official VMware blog digging deeper into VCF’s underlying architecture.
In addition to the numerous benefits of VCF architecture here are some of the notes I have taken around the IBM partnership.
The big stuff…
Fully automated deployment of the VCF stack (vSphere 6.5, VSAN & NSX) on IBM Softlayer dedicated infrastructure.
All services are billed directly by IBM.
VCF can be consumed as a standalone platform on IBM, as a hybrid cloud through vCenter Enhanced Linked Mode, or cloud-to-cloud between IBM regions through the same mechanism.
vCenter-as-a-Service can be also procured as a subscription through IBM, but customers also have the option to procure perpetual licensing if non-VCF license ownership is desired.
Availability; before the end of 2017 for IBM, early 2017 for VCA. Other VCAN partners TBA.
The technical stuff…
SDDC Manager will not be directly accessible as it abstracted through the Softlayer Customer Portal. Provisioning, lifecycle tasks, patch management and upgrades are delivered through this portal.
NSX completely removes the constraint of IBM Softlayer internal networking (3-4 VLANs).
Integrated snapshot based backups of management layer components.
VCF best practice single management layer governing multiple IBM Softlayer regions.
Linking Cloud Foundation environments is achieved through vCenter Enhanced Linked Mode, not via SDDC Manager.
Minimum deployment of four hosts (converged management and workload domains).
The value stuff…
BYO-Cloud and consume the full VCF stack on a monthly basis.
Low latency access to IBM Cloud services (Object Storage, Bluemix, Watson, etc.)
Zero cost private datacenter interconnects between IBM Softlayer Regions.
True BYO public cloud for those who require full access to all SDDC functions, including the upgrade and patching of individual SDDC components which is maintained by the customer, not VMware (or IBM without additional services).
Ability to build and manage identical SDDC components both on-prem and in public cloud.
Note that VCF is not the only way to consume VMware on IBM Softlayer as IBM customers have previously been able to select individual VMware technologies and deploy them on IBM Softlayer bare metal. This also allows customers to bring their existing licensing to IBM Cloud, which can be a real bonus when migrating from, or replacing an existing datacenter. Note, as an example of how much complexity is actually involved with deploying an entire SDDC platform independently on IBM Softlayer I would suggest a read of the extremely comprehensive reference architecture here.
vCloud Air (non-VCF services)
Contrary to a number of blogs and articles I have read recently, vCloud Air is here to stay, albeit with a renewed focus to address specific VMware hybrid-cloud challenges. I’m not going to cover the existing vCloud Air service here as it has been available for a while now and we should all know it back to front, right? 🙂
In addition to VCF on vCloud Air, there were numerous announcements including;
Enhancements to Hybrid Cloud Manager with the full release of version 2.0, including;
Zero downtime Cross-Cloud vMotion utilizing fully integrated WAN opto, proximity routing. Note: This has no dependency on vSphere 6.x and can be used with vSphere 5.5 today.
NSX policy migration.
New services for Enterprise DR, Hybrid DMZ and DMZ lite.
Enhanced Integrated Identity & Access Management.
Increased DPC host memory capacity (up to 1TB per host)
Today, vCloud Air is still the only way to subscribe to a fully managed VMware cloud service and take full advantage of Hybrid Cloud Manager. As an added benefit, the entry point for Dedicated Private Cloud (as a direct comparison) is only a single N+1 host meaning the overall initial commitment is not as significant as the other services.
Although the these individual cloud offerings may seem to overlap they each address a different set of challenges by integrating with key partners who are market leaders in a specific hybrid/public cloud capabilities. This puts VMware customers in a unique position of having a choice of multiple clouds depending on individual requirements.
In addition to the above, VMware also has 4000+ vCloud Air Network partners who all offer unique services with VMware software at the core. If I even began to try and break down the breadth of services covered through these partners this blog would turn into War & Peace…
I have only covered a very small amount of high-level info here as I hope to flesh out each service as more information is released. Comments, opinions and feedback in general is always welcome. If your attending vForum Australia 2016 I will also be presenting a couple of sessions on VMware Cross-Cloud Architecture and demoing VMC on AWS, so come and say hello and give me your take on this new world…
If you’ve followed my blog or seen me presenting in the last six months you may have noticed I have developed a keen interest in Cloud Native Apps and DevOps in general. I was lucky enough to present a combined CNA/vCloud Air session at VMWorld this year which was a little different from the hybrid cloud talks I usually give.
In addition to the ‘what-why-how’, I also ran a live demo showing the provisioning and decommissioning of a remotely accessible VCA Docker host, complete with NAT and firewall configuration using two simple commands. Since Las Vegas I have been meaning to post how I constructed the demo, so here it is.
Note: some prior knowledge of basic vCloud Air administration and Docker functionality is assumed…
Docker Machine Driver for vCloud Air
In my previous post I talked about VM’s and containers living side by side as decomposing (or building alongside) monolithic apps can take an extended period of time, or may not be possible at all. To support this notion, VMware has made great strides in the containers space to provide technology that allows organisations to run containers natively on vSphere (through VIC) or Photon Platform depending on operational requirements and overall maturity with the cloud native apps.
However there is one aspect of the VMware CNA vision that is often overlooked, namely vCloud Air. This may be because vCloud Air does not have a native container offering (at the time of writing this post), but it does have support for Docker Machine which is an essential part of the Docker workflow if using Docker Toolbox for administration.
What do we need?
In order to use the Docker Machine Driver for vCloud Air we will need to have a VCA subscription (either Virtual or Dedicated Private Cloud) and a user account with network & compute administrator permissions assigned. With this we can go ahead and create a private template which Docker Machine will use to create our container host. Note, if not specified in our docker-machine create command, Docker Machine will use Ubuntu Server 12.04 LTS from the VCA Public Catalogue by default.
Quick Tip: To create a quick template I used the Ubuntu Server 12.04 LTS image from the VCA Public Catalogue as it already has VMware tools installed. After I ran my usual VCA linux template prep, (root pw change, network config, ssh config, apt-get upgrade, apt-get update, etc) I renamed vchs.list to vchs.list.old found in /etc/apt/sources.list.d/. Now I did this because when Docker Machine runs through the provisioner process it uses apt-get to retrieve VCA packages from packages.vmware.com, which can sometimes be a little slow to respond. This occasionally results in the provisioner process timing out (as it did in my demo at VMWorld….grrr). Note, post initial template creation it is not necessary to have the packages.vmware.com repo available for the docker provisioning process.
Provided we have a access to a VCA routable network and an available public IP address, we can go ahead and run a relatively simple shell script to execute the entire provisioning process. It should be noted that I created this script to be easily distributed to anyone needing quick access to a docker environment, provided they had the correct VCA permissions. It also avoids storing your VCA password in clear text.
Note, this is a minimal subset of commands for basic VCA container host provisioning. I have also changed the VDC ID, Edge Gateway ID and public IP in the example script for obvious reasons. A full list of Docker Machine Driver for vCloud Air commands can be found on the Docker website here.
Once the provisioner process is complete, we should have an internet accessible container host configured with 1 vCPU, 2GB of memory with Docker installed, running and listening for client commands on the configured public IP we specified earlier.
To natively connect to this environment from our Docker client we simply enter the following…
That was easy, right? Well… it’s not quite that simple.
The above will create a relatively insecure Docker environment as the edge firewall rules are not locked down at all (as shown below).
This can be handy for testing internet facing containers quickly as we do not need to explicitly define and lock down the ports needed for external access. However if this Docker host is intended to become a even a little more permanent, we can use VCA-CLI or the VCA web/VCD user interface to alter the rules (at a minimum port 2376 needs to be open from a trusted source address for client-server communications, and whatever ports are needed to access containers directly from the internet).
Assuming our environment is temporary, we can also tear it down quickly using:
So there you have it. The entire build provisioning process takes less than 5 mins (once you have set up a template) and decommissioning takes less then 2 mins! In addition to simple tasks I’ve outlined here we can also use a similar process to create a Docker Swarm cluster, which I will cover in my next post.
As always, if you have any questions or feedback feel free to leave a comment or hit me up on Twitter.