CNA for the Ops Guy: VM’s and Containers Living Together in Harmony

Intro: OK. So you may have noticed that I have been absent from the blogosphere for a few months (probably not)… In any case, where have I been? 

Simply put, I took a break so I could take a step back to take a glance at the big picture, assess ‘who’s who’ in my areas of interest and return with renewed focus. Now you might think this sounds a little conceited, but I had written one too many blogs that never made it to published status because I wasn’t entirely certain of their content or more importantly, relevance. But part of stepping away is stepping back in again… and with that said, let the blogging begin!

Note: This is not a technical tutorial on Docker or vSphere Integrated Containers, rather my views on the philosophy and gradual integration of containers into our existing VMware ecosystem.

Container will replace VM’s… Eventually… Maybe…

I recently presented a session at the Melbourne VMUG called “Can VM’s and Containers happily coexist?”.  Though somewhat rhetorical the title was born out of the regular bashing that the VM’s take from container fanboys. To condense a session overview into a single sentence, we tackled a brief history of containers, Dockers rise to fame and the inherent issues with this rise to fame. Despite the age of containers, the fresh faced vendors have yet to prove their worth as a full replacement for virtualisation.

In my first post in this series I described the basic tenets behind Cloud Native Applications, one of which is the 12 Factor App. This framework has arguably become the unofficial guideline to creating applications suitable for a microservices architecture, but it also lends itself perfectly to illustrating why a vast majority of existing monolithic & layered applications are not suitable. With this in mind it’s clear that VM’s still have to a large part to play in the evolution of application architecture.

Taking the above into account, the rush to try and scale / orchestrate containerised services becomes less relevant than current marketing would have us believe. It’s far more important to have a product that has enough demand to require scaling, than to worry about scale before you have the demand. Of course this doesn’t mean that scaling should be an afterthought, but it shouldn’t be the primary focus.

Evolution of the workload…

We’ve all heard the pets vs livestock analogy many times (we’ve stopped picking on cattle :)), but a recent focus on serverless computing and the art of economics through mass autoscaling has introduced the ‘Organism’: A computing entity that is minuscule from both a footprint and lifespan perspective, has little impact as an individual, but when combined with other organisms the ‘grouping’ becomes highly dynamic and resilient. Where can we find these organism type of workloads being used to great effect? Think Google, Facebook et al.

Screen Shot 2016-06-03 at 10.30.56 AM

Allow me digress to make a point. A large part of my role is identifying how to modernise datacenter practices to incorporate cloud technology in whatever format aligns best with a strategic outcome. I only mention this because I believe marketing is well beyond the point where use of container technology can be useful for a business that has come from traditional mode 1 operations.

In my opinion most organisations are still finding the balance between pets and livestock through evolved lifecycle practices, but that doesn’t mean they can’t incorporate the numerous benefits of containers other than those more commonly found in a microservices architecture.

Note: As an aside, I recently watched “Containerization for the Virtualisation Admin” posted on the Docker blog where the misconception of containers only supporting microservices was dismissed, something I have long been arguing against. Nice one guys.

Sneaking containers to operations…

For ops, the most common first encounter with containers is likely to occur when Dev’s request a large Linux/Windows VM that will eventually become a container host unbeknownst to the the ops team. In almost all cases this means that operations lose visibility of what’s running within the container host(s). Not ideal for monitoring, security, performance troubleshooting and so forth.

In this  scenario our Devs interaction with the rogue Docker hosts may look something like below:

Screen Shot 2016-06-09 at 3.34.44 PM

This approach leaves a lot to be desired from an operational perspective and therefore rules out almost all production scenarios. A better approach is to try to evolve the compute construct to suit existing practices within a business. In VMware’s world, this means treating VM’s and containers as one and the same (at least from an evolutionary perspective).

A box within a box!?.

As an atomic unit, do developers care if a container runs in a VM, natively on a public cloud or on bare metal? Well… it depends on the developer, but generally the answer is no. As long as they have free range access to a build/hosting environment with the required characteristics to support the application, all is good. For operations this is an entirely different story. So like in any good relationship, we need to compromise…

Arguably, there is an easy way to get around this issue: Create a 1:1 mapping of containers to VM’s (I can already hear the container fanboys groaning). Yes, we do lose some of the rapid provisioning benefits (from microseconds to seconds) and some of the broader Docker ecosystem features, but we don’t have to forklift an environment that we have spent years (and lots of $$$) refining. Anecdotally, it seems we have spent so long trying to create monster VM’s that we have forgotten the power of the hypervisors ability to balance and isolate numerous tiny VM’s.

Admittedly, for some organisations having the bottleneck of provisioning individual VM’s is still a very real headache for the development team…

*Fanfare… Enter vSphere Integrated Containers!

vSphere Integrated Containers (aka VIC) provides our Dev’s with a way to transparently work with containers in a vSphere environment, therefore reducing a lot of the friction traditionally found with operations having to create VM’s.

The premise behind VIC is a single container per VM (aka, pico VM, Micro VM, Just Enough VM), but with all the characteristics of a container (isolated, lightweight, portable, etc). This has a numerous benefits for operations around resource control/distribution, monitoring and security using mechanisms are already well established (and more importantly, well understood) by network and security teams.

Screen Shot 2016-06-09 at 3.34.18 PM

So we can visualise the above using a familiar interface, when I run a basic command like <$docker run hello-world> from my Docker client to the VCH, vCenter launches a PicoVM with our container running inside. From a vCenter perspective we can see the the container running in the same vApp where the VCH and Instant Clone Template exists.

Screen Shot 2016-06-28 at 12.02.20 PM

Note: The version of VIC I used for this screen shot is based on Project Bonneville (detailed here) to show the use of the command <docker ps -a> which displays both running and exited containers. At the time of writing (0.3.0), the VIC beta (available here) did not support certain docker commands, including ps. Based on user feedback there have been some changes to the overall architecture to better align with real world requirements. More to follow soon…

The result is vSphere admins have can enforce control directly from the parent resource pool. We can route, monitor, shape and secure network traffic on the port group assigned to the VCH as the docker bridge network. We can set CPU/memory shares, reservations and limits to ensure we don’t compromise other workloads…  and our devs get access to a Docker environment that operations fully comprehend and have existing operational policies and procedures that can be adapted.

Conclusion.

Before the container/microservices fanboys get up in arms, this post was not intended to show the use of containers for isolated projects or startups, but rather the integration of a new construct into an existing VMware enterprise. IMO, traditional organisations value the portability, ubiquity and flexibility of Docker across disparate operational platforms more than rapid provisioning and scaling… and us Ops folk need to learn to walk before we can sprint…

In the next post of this series we will start to tackle the challenge of scaling using the same philosophies detailed in this post. See you next time.

Author:@Kev_McCloud

Cloud Native Apps for the Ops Guy – Container Basics

Welcome back All. In part 1 we covered the basic understanding of Cloud Native Applications (CNA) and more importantly its relevance to todays IT Operations Teams. Let’s start with a quick recap of what we’ve already covered:

  • CNA is an evolution in software development focused on speed and agility, born out of removing the challenges of the traditional SDLC
  • Ops Teams are struggling to fully operationalise modern web-based applications, i.e. build and/or maintain operational practices for hosting cloud native applications
  • There is a vast array of CNA architectures, platforms and tools, all of which are in their relative infancy and require a degree of rationalisation to be useful in the enterprise

I also covered breaking down my understanding of CNA into two areas of research; Foundational Concepts and CNA Enablers, the later of which we’ll cover in this post.

How can CNA fit into existing IT Operations?..

To see where CNA Enablers might fit, I took a look at the responsibilities of a modern IT Team for application delivery. At a high-level our priorities might cover:

Development Team: Application architecture / resiliency / stability / performance, deployment, version control, data management, UX/UI.

Operations Team: Platform automation / orchestration / availability, scaling, security, authentication, network access, monitoring.

Note, this is a very generic view of the average IT Team dichotomy, but it does illustrate that there is virtually no crossover. More importantly, this shows that the core of operational tasks are still aligned with keeping hosting platform(s) alive, secure and running efficiently. So with this mind, how do we go about bringing operations and development closer together? Where will we start to see some overlap in responsibilities?

Introducing Containers…

There has been a lot of noise around containers (and by association, microservices) as the genesis of everything cloud native, however Linux containers have existed for a long time (and don’t the Linux bigots love letting us know it😉 ). If we filter the noise a little, it’s clear to see that containers have become essential because they address the lack of standardisation and consistency across development and operations environments, which has become more prevalent with the growing adoption of public clouds like AWS.

So what is all the fuss about? To begin to describe the simple beauty of containers, I like to think of them as a physical box where our developers take care of what’s inside the box, whilst operations ensure that the box is available, wherever it needs to be available. The box becomes the only component that both teams need to manipulate.

Screen Shot 2016-01-25 at 10.57.23 AM

To overlay this onto the real world, our dev’s have to deal with multiple programming languages and frameworks, whilst we (as ops) have numerous platforms to maintain, which are often comprised of drastically different performance and security characteristics. If we introduce a container based architecture, the “box” reduces friction by providing a layer of consistency between both teams.

Architecture basics…

Now for me it was also important to understand that containers are not the only way to deploy cloud native applications (please refer to this excellent post from my VMware colleague @mreferre), but also to acknowledge that they are important for a number of reasons, namely:

  • They provide a portable, consistent runtime across multiple platforms (desktop, bare metal, private & public cloud)
  • They have a much smaller, more dynamic resource footprint
  • They can be manipulated entirely via API
  • They start in milliseconds, not seconds or minutes
  • They strip away some layers which could be considered to add significant ‘bloat’ to a traditional deployment

If we compare a “traditional” VM architecture to a cloud native architecture (diagram below), it’s evident that gen 2 (i.e. monolithic / single code base) apps often have a larger resource overhead because of their reliance on vertical scaling and the tight coupling of their constituent parts. If we have to move (or redeploy) a gen 2 app, we need to move (or redeploy) everything northbound of the VM layer, which can be considerable if we are moving app data as well.

Screen Shot 2016-01-29 at 10.33.13 AM

Note: The above diagram is not intended to show a refactoring from gen 2 to gen 3, but instead how the same applications might look if architected differently from scratch.

From an operational perspective, gen 3 (ie. cloud native) apps which leverage containers have a greater focus on horizontal scaling, whilst greatly increasing consolidation of supporting platform resources.

As a comparison, when moving gen 3 apps between environments we only have to push the updated app code and supporting binaries/libraries not included in the base OS. This means we have a much smaller package to move (or redeploy) as the VM, guest OS and other supporting components already exist at the destination. Deployment therefore becomes far more rapid with far less dependency.

Now this is all very exciting, but in reality gen 2 and gen 3 will have to coexist for some time yet, therefore it’s probably best to have a strategy that supports both worlds. For this reason, I am researching the synergies between the two constructs which is where I believe many IT shops will thrive in the near term.

Where do we begin?..

If we start with a minimal platform, all we really need to be able to build a containerised application is; a host, an OS which supports a container runtime and a client for access. It’s entirely possible to build containerised applications in this way, but obviously we are severely limited in scalability. Once we go beyond a single host platform, management becomes far more complex and therefore requires greater sophistication in our control plane. But I guess we should try to walk before we run…

Let’s take a closer look at some of the layers of abstraction we will be working with. Note: So as not to confuse myself with too many technologies, I’ve focused my research on VMware’s Photon (for obvious reasons) and Docker, which I believe has firmly established itself as the leader in container and container management software.

Container Engine / Runtime – This is the software layer responsible for running multiple, isolated systems (i.e. containers) by providing a virtual environment that has its own CPU, memory, block I/O, network, cgroups and namespaces within a single host. It is also responsible for scheduling critical container functions (create, start, stop, destroy) in much the same way a hypervisor does.

In the case of Docker, it’s also the runtime that manages tasks from the Docker Daemon which is the interface that exposes the Docker API for client-server interaction (through socket or REST API).

Container OS – A container OS (as the name would suggest) is an operating system which provides all the binaries and libraries needed to run our code. It also enables the container engine interact with the underlying host by providing the hardware interfacing operations and other critical OS services.

Photon is VMware’s open source Linux operating system, optimised for containers. in addition to Docker, Photon also supports Rkt and Garden meaning we are not limited to a single container engine. It’s also fully supported on vSphere (and therefore vCloud Air) and it has no problems running on AWS, Azure and Google Cloud Engine (though it may be fully supported by these service providers at the time of writing).

Note: If you feel like having a play around with Photon (ISO), it can be downloaded from here, deployed directly from the public catalogue in vCloud Air, or if you want to build your own Photon image you can also fork it directly from GitHub.

Host – Our operating system still needs somewhere to run. I believe that for most of us, virtual machines are still best used here because of the sophistication in security, management and monitoring capabilities. In the short term it means we can run our containers and VM’s side by side, but it should be noted that we can also run our container OS on bare metal and schedule container operations through the control plane.

Platform – A platform in the context of operations is simply a hosting environment. This could be a laptop with AppCatalyst or Fusion, vSphere and / or private and public cloud, really any environment that is capable of hosting a container OS and the ecosystem of tools needed to manage our containers.

Basic Container Usage…

In order to make this an effective approach for our dev’s, they need self-service access to deploy code and consume resources as they see fit. The simplest approach for our dev’s is to deploy in an environment where they have full control over the resources, like their laptop.

It’s pretty common for a dev to build artefacts in this way, then push to an environment where the service is eventually destined to exist. If their laptop is running Linux as it’s base OS then they can deploy the runtime directly on the OS, but I’ve always used a Mac or Windows OS meaning that I need another layer of abstraction to run my containers. For me this is VMware AppCatalyst (originally I used Fusion).

As Docker is based on a client-server architecture, our dev’s need access to an exposed Docker API. At a very high level, they might interact with ‘Docker on VMware’ as illustrated in the below diagram. Note: I recommend reading through Understand the Architecture from Docker to understand the basics of  the client, host and registry components.

Screen Shot 2016-01-22 at 11.19.08 AM

Once we go beyond the dev’s laptop, our platforms might include on-premises virtual infrastructure, bare metal and public cloud. The platform itself is not really that important to our dev’s provided it has the capabilities needed to support the application. So ops really need to concentrate on transparently supporting our dev’s ability to operate at scale. With that comes operational changes, which might include:

  • Secure access to the container runtime (via a container scheduling interface, which we’ll cover in the next post)
  • Internal network communications to support containerised services to function at scale, including virtual routing/switching, distributed firewall, load balancing, message queuing, etc
  • Secure internet and/or production network communications for application front end network traffic
  • Support for auto-scaling and infrastructure lifecycle management, including configuration management, asset management, service discovery, etc
  • Authentication across the entire stack defined through identity management and role based access controls (RBAC)
  • Monitoring throughout the entire infrastructure stack (including the containers!)
  • Patching container OS / runtime and all supporting platforms

Now I realise this is only scratching the surface, but if we listed all of the operational changes needed to incorporate this mode of delivery we would be here all day. For this reason I’m ignoring CI/CD and automation tools for the time being. Don’t get me wrong, they are absolutely critical to building a reliable self-service capability for our dev’s, but for now they are just adding a layer of complexity which is not going to aid our understanding. We’ll break it down in a later post.

So there you have it. In looking at the simple benefits that containers provide, we quickly begin to realise why so many organisation are developing cloud native capability. In the next post we’ll start to look at some of the realities of introducing a cloud native capability to our operations when working at scale.

 

References and credits:

 

Author: Kev_McCloud

 

 

#TechRant: Utility is the Dirtiest Word in IaaS

It is amazing what a well positioned marketing campaign can do…

Let’s rewind back a few years (enter the wavy lines ≈ ≈ ≈). Cloud was a commonly misunderstood venture that promised the rise to glory for any CIO who dared to venture past the boundaries of the data centre! “Drastically reduce the cost of IT, enable disruption through web-scale software, sack all of your SAN admins and run like Netflix”, and so on… Anybody with half a brain instantly saw through the BS, however the notion that cloud is cheap stuck firmly in peoples minds.

For the most part, this was driven by the wild west days of AWS, MS Azure and Google very publicly hacking away at pricing for cloud resources. To further compound the effect, AWS’s marketing around meter-by-the-minute or utility consumption promised to save big $$$ for everyone and anyone, regardless of the use case.

Fast forward to now: I’m genuinely surprised by just how many conversations I am still having with customers and peers around this misconception. Every time I hear the word utility in the context of IaaS, it sends me into an uncontrollable rant… (true story)

Can a leopard change it’s spots?..

I recently blogged on the importance of understanding Cloud Native Applications (aka CNA) in a previous post. One of the foundations of CNA architecture is the principal that individual constituents can scale rapidly. In a basic IaaS model, scaling involves standing up / tearing down workloads and platform services to support load (in AWS terms think EC2, EBS, Cloud Watch, Auto-Scale, ELB, etc) aka horizontal scaling.

Herein lies the crux of the issue. To achieve the “utility” consumption that AWS so frequently spruiks, our application needs to scale it’s constituent workloads horizontally… Ergo, utility is achieved by multiplying compute instances.

Screen Shot 2015-12-17 at 11.16.30 AM

Steady-state workloads, dare I say ‘Gen 2’ applications, where VM’s are powered on 24x7x365 in a monolithic stack does not fit this mould at all. Most of these application stacks support non-frontline services (ie. email, CRM, payroll, etc) which are not regularly updated and have no semblance of a rapid development lifecycle.

More importantly, a vast majority of these apps cannot be refactored (ie. changing the source code to suit a new architecture without altering the service) to take advantage of horizontal scaling. Figuratively, it’s like trying to hammer a large square peg through a number of tiny round holes… There is a small chance may achieve it, but the applications end-state will have irrevocably changed to suit the destination (which is not necessarily related so I’ll cover this some other time).

Screen Shot 2015-12-17 at 11.31.57 AM

Leaving the lights on…

The reality is most pre-cloud businesses have a multitude of gen 2 apps like this and will continue to support them until a successor is put in place (if ever). Until that time, traditional methods of scaling must remain. Most commonly this involves statically adding / removing compute resources within a runtime in order to support the required load, aka vertical scaling.

In most cases dynamic vertical scaling is not possible, so the common practice is to size individual workloads to support peak utilisation. In the context of public cloud, we are paying for the total resource allocation, not what is consumed within the workload. The result is the consumer pays for the total compute whether utilised or not. Not at all efficient from a utility perspective…

Screen Shot 2015-12-17 at 11.18.29 AM

I am not saying there is anything wrong with this mode of operation. In fact this has been a common practice for decades in both virtual and physical environments. My beef is with a simple misrepresentation: That somehow cloud compute is cheaper when it’s metered by the minute. That my friends, is a completely false economy unless you have deployed your applications in a manner that supports this model. Kinda like leaving all the house lights on just in case you need to enter a room at some point.

As a final word before y’all AWS fanboys (or girls) get up in arms, I’m not bashing AWS for their technical capabilities. However I do believe their marketing team has a lot to answer for (as is the case with most vendors😉 ). Keep fighting the FUD and stay informed.

 

Author: @Kev_McCloud

Cloud Native Apps for the Ops Guy – What the hell is CNA anyway?

Intro: DevOps, containers and every other buzzword bingo concept seem to be severely muddying the waters of understanding the practicality of web-scale application development and it’s relevance to the everyday Operations Team. I’m hoping this series explains the basics of Cloud Native Applications from the perspective of an ops guy by joining some of the dots across a commonly misunderstood topic. 

Are we “failing fast”, or just plain failing?..

Cloud Native Applications (CNA) has to be one of the most enigmatic concepts in operations today, but it is widely accepted as the future of software development. New school, software centric businesses like NetFlix brought CNA and open source software into sharp focus with their recent success, showing both can be heavily utilised to support a multi billion dollar business. Couple that with the fact traditional application scaling and resiliency is now considered to be slow, inefficient and expensive and it’s of little wonder that CNA is gaining such rampant popularity.

With that said, I’m always a little alarmed when I have a conversation with a fellow ops engineer that generates grumblings along the lines of…

Our devs just build stuff in the cloud, then dump it on us with little or no information on how to operationalise it…

I hear this quibble (or variation of) surprisingly often. You would almost think it was standard practice for all dev > ops hand overs, right? Question is; Do most devs actually hand over with poor context, or is it a lack of knowledge / unwillingness to learn the basics of modern web application architecture that holds us ops fellas back?

Screen Shot 2015-11-02 at 5.28.08 PM

If you believe the later, then clearly it is critical for us to understand how our development teams function (what with them being the future n’ all). The depth of our understanding however is something most of us will have struggled with.

DevOps today is just tomorrows operations…

Let’s go back to our recent past (like yesterday). Does weekly CAB meetings, architectural review boards, problem management, excessive documentation all sound familiar? This is the kind of rigid ITSM doctrine that has been pushed for many years to bring stability to complex environments. The problem is that it’s costly and moves at a near glacial pace. Pretty much the opposite of agile…

Don’t get me wrong, stability and resiliency are critical for enterprise, a net result of having to baby sit a whole raft of buggy software. This is why most ops guys view devs as cowboys. From our perspective it’s buggy code that stresses the supporting platform, causes outages and so on (the age old blame game). The impact of ‘bugginess’ depends on how far reaching the impacted code is within a service. Monolithic, layered applications tend to be the worst affected because they are comprised of only a few services which are tightly coupled (ie. close dependencies with other services).

Screen Shot 2015-11-08 at 9.32.56 AM

And therein lies the beauty of CNA: Applications comprised of numerous, loosely coupled, highly distributed services where the impact of faults (read: bad code, potentially😉 ) is drastically reduced. Sound good? I think so…

The reality is most software developers have already had this epiphany and are working (or experimenting) with platforms that support CNA today. The adoption of this practice has forced infrastructure teams to be more forward thinking in accomodating what our devs are building. Make no mistake, they are leading us into the future. What’s more important right now is that we have a opportunity to control and shape the platform(s) used to host, deploy and secure cloud native applications.

Where to start?..

To paraphrase from some self-awareness reading I did recently; first seek to understand, then to be understood. The initial problem I had was that CNA is more of a broad conceptual practice than a prescriptive framework. So the first port of call was to gain a high level understanding of CNA basics so at the very least I could have a constructive conversation with software development teams. I’m not saying this will work for everyone, but it did for me…

First off, I split my research into two areas of understanding; Foundational Concepts and CNA Enablers. This is in no way official, it’s just my way of breaking down a broad topic into chunks that I could easily consume.

Disclaimer: I am pretty far from being any kind of authority on CNA, so rather than try to cover everything or take credit for others great work I have also included a bunch of links which gave me a high level understanding of CNA. These blogs, posts and articles are not intended to show how to implement or architect. This is purely an exercise in understanding…

Foundational Concepts – The following concepts / components are constituents of a CNA practice. This should help us to understand a modern software developers challenges.

Twelve-Factor Applications – The 12-factor app is a widely accepted framework of best practices for deploying scalable applications on any platform. There are litterally hundreds of blogs detailing this and it’s a lot of info to digest for the uninitiated, but the awesome 12-Factor Apps in Plain English helped me to understand this concept with relative ease.

Microservices – I like to use the terrible analogy of a legacy Windows OS as a starting point for understanding microservices (note: it’s just for common ground for explaining a construct and completely unrelated to CNA in general). Hundreds of single instance, single function, fragile services combined to run a basic operating system. I say fragile because historically any one of approx 50% of these services could cause a critical failure in the overall OS, and did, regularly. Imagine if all these services actually ran independently and in a fault tolerant, balanced state so any issue could be isolated and remediated with little or no service degradation!

Essentially microservices are just that. Loosely coupled, lightweight, easily scaleable components that have a single essential function within in a lager application or service. What are microservices explains this in basic detail, but Microservices – Not a free lunch brings a dose of reality by helping us to realise the level of technical maturity required for effective usage. If you do want to get deeper into microservices @martinfowler is a great chap to follow and his blog has a plethora of information to get you pretty far down the rabbit hole, so to speak.

API Collaboration – Microservices are of little power unless they can interact with other microservices. This collaboration is achieved almost exclusively via addressable APIs (typically a web service protocol such as REST). I don’t want to delve to deeply, so A Beginner’s Guide to Creating a REST API is great place to start for understanding what the hell RESTful API’s are in the first place. I also like Build a reusable REST API back end as it identifies some of the common pitfalls likely to be experienced in an enterprise .

Anti-Fragility – This is another concept pioneered by Netflix and popularised through their use of Chaos Monkey in addition to other anti-fragility tools. It’s ‘simply’ a practice wherein services are designed to be highly resilient (self-healing) to abnormal states or stressors (faults, load spikes, etc). In addition, application architects learn how to strengthen the composite service(s) through deliberately inflicting the aforementioned anomalies and then remediating and reinforcing through a constant learning cycle.

What’s interesting here is that anti-fragility requires a level of creativity and strategically defensive architecture that is well outside the normal realms of traditional operations resiliency. DevOps Anti-fragility and the Borg Collective summarises this in a blog that borrows from Star Trek to articulate the beauty in this principal.

CNA Enablers – This is where our focus lies in operations. These are the mechanisms, platforms and practices that will enable us to standardise and control the deployment of highly-distributed (web-scale) applications.

Containers – Containers are an ideal construct for CNA and CNA is arguably the best use case for containers. They provide a consistent runtime across multiple platforms, built with common toolsets, common API and they support extremely rapid lightweight provisioning. From an operational perspective, when building services utilising containers (or any other form of portable construct) it also means that we are not locked into a proprietary PaaS offering which can act as an anchor in the longterm. I believe this is one of the key challenges to successful cloud adoption and a potentially serious lock-in issue that our developers are not aware of.

VMware is focusing on bringing easily consumable, open source container technology to the enterprise through Photon (Machine & Controller) and vSphere Integrated Containers (VIC aka. Project Bonneville). You can be sure I’ll be covering this in great detail in future posts.

Self-Service Infrastructure – CNA’s effectiveness is reliant upon having rapid, on-demand access to platform resources. Containers can provide this but they are not commonplace in the enterprise which is why public cloud tends to be the landing ground for these CNA type apps. Not only do Dev’s need instantaneous access to PaaS resources, but also the ability to rapidly deploy and configure IaaS resources in a structured and repeatable manner (ie. infrastructure as code).

Immutable infrastructure – As an extension of self-service infrastructure, immutable infrastructure is simply the concept of deploying infrastructure once with zero alteration once deployed. If a change needs to be made (or a fault rectified), simple destroy the instance and roll out a new one in it’s place. This is a fantastic way to ensure service integrity through version control. The impact of this is significant and well explained in An Introduction to Immutable Infrastructure.

In summary, I think it’s important to understand that this is still a relatively new mode of operations in the enterprise realm. Unless you are working for a business with software at it’s core it is unlikely that it will be your entire focus for quite some time, however it is always good to be prepared.

 

References and credits:

 

Author:@Kev_McCloud

vCloud Air @VMWorld 2015 pt.2

Wow. It’s been a crazy couple of months for VMware, what with the talking heads spewing speculation and unwarranted opinion on the Dell/EMC acquisition (and therefore the downstream VMware) and vCloud Air spinning out into VirtuStream. It’s been a real shame that all the “news” was incredibly distracting during the fantastic events that are VMWorld Barcelona and vForum Australia.

For those of you that missed the announcements here is a breakdown of the recent vCloud Air futures in addition to those from my first post.


VMware vSphere Integrated Containers (VIC) – This is not specifically a vCloud Air capability however it will be critical to the future of the platform. VIC builds upon the existing support for Photon OS , enabling support for containerised applications on vCloud Air. This is probably the most interesting innovation in VMware’s portfolio and one that I am spending a lot of time delving into to get a better understanding of what’s to come.

Have a gander at the below clip for an introduction to VIC (aka Project Bonneville)


VMware vCloud Air Monitoring Insight – Stripped down, SaaS/subscription delivery model of vROPs like monitoring. Expect operational metrics, event logs, and user-defined alarms that provide analytics into cloud service operations. These metrics provide information on the infrastructure, application health and platform performance and are also exposed via API. To be honest, this is an essential cloud operations requirement rather than an innovation, but definitely one of the most requested platform enhancements and a most welcome addition.


Enhanced Identity Access Management – Finally we can federate our Active Directory and other SAML 2.0-compliant identity services with vCloud Air. Again, table stakes in the public cloud game, but critical to automated application deployment and cloud security.


Project Michigan – This extends the capabilities of Advanced Networking Services and Hybrid Cloud Manager to all vCloud Air services. Currently HCM and ANS are only available on Dedicated Private Cloud due to the fact that this was the first platform which supported integrated NSX. Having the tools available across all platforms will be a complete game changer for VMware and cannot come soon enough…


On a side note, I had taken a taken a brief hiatus from blogging whilst I figured out what all the recent noise meant for VMware and my current role as a Cloud Specialist within this awesome company. Now the proverbial cat is out of the bag I am hoping just to get on with what I do best: Soapboxing on the technology I am passionate about and ranting about the state of IT…

 

Author: @Kev_McCloud

vCloud Air @VMWorld 2015 pt.1

VMWorld was bigger than ever this year with 23,000+ attendees. Announcements came in droves and vCloud Air certainly didn’t miss out on its fair share. I will be covering all of these in lot more detail in coming posts, but for now here’s high level summary.

Some of these announcements have been doing the rounds for a while, whilst others came out of the blue. Here is a breakdown of what got me childishly clapping at the screen whilst I watched remotely.


Hybrid Cloud Manager: is the replacement for the vCloud Air vSphere Web Client Plugin and vCloud Connector. It also adds advanced hybrid network capabilities based on NSX technology.

Enhanced Hybrid Cloud Management: Greatly improved administration and management of vCloud Air workloads from vCenter.

Enhanced Virtual Machine Migration: Replication-based migration of VMs over Direct Connect and VPN with automated cutover (ASAP, scheduled and optimised) and finally an exposed API. Oh, and the all important addition of WAN optimisation!

Infrastructure extension: Stretch multiple L2 segments (up to 200!) into vCloud Air over Direct Connect or VPN. This replaces vCloud Connector Stretch Deploy which is a very welcome addition if you have ever tried to use Stretch Deploy in any anger…

Project Skyscraper was also announced during the keynote on Day One of VMWorld and got all the ‘ooohs’ and ‘aahhs’ of a world class fireworks display. It has been a long time coming but unfortunately is still in technology preview (*sad face). It includes;

Cross-Cloud vMotion: This had to be one of the coolest demos of the entire event. vSphere vMotion into vCloud Air (and back again). Zero downtime for workload migration to public cloud. That is an absolute first! You can guarantee I will be covering this in great detail!

Cross-Cloud Content Syncsynchronise VM templates, vApps, ISOs, and scripts with their content catalog from vSphere to vCloud Air.


Advanced Networking Services: The long-awaited NSX integration into vCloud Air. There’s way too much here to go into any detail. High level enhancements include;

Enhanced Edge Services: Current vCA Edge capabilities + support for dynamic routing (eBGP/OSPF), SSL-VPN, enhanced load balancer, multi-interface trunking (True L2 Stretch), enhanced stateful edge firewall + more.

Trust groups: Stateful in-kernel L2/3 firewall (micro-segmentation), Object based rules, 5-tuples mapped to apps + more.

Compliance and operations: Enhanced API, Distributed firewall logs, Load balancer health checks + more.


Site Recovery Manager Air (SRM Air): In terms of revision, DR2C (or Disaster Recovery to Cloud) is in it’s second incarnation. v2.0 gave us reverse replication, fail-back, multi point-in-time recovery in addition to other small improvements. Enhanced Disaster Recovery aka SRM Air (or DR2C 3.0 if you will) adds SRM capabilities in a SaaS delivery model. Expect application-centric automated workflows that orchestrate testing, failover and failback of VMs to and from vCloud Air.

DR2C will also move away from a subscription model to OnDemand, metered by the minute pricing model where you only pay for the storage you consume + a per VM monthly subscription. Watch this space!


EMC & Google Cloud Object Storage: Object storage has been a cornerstone of public cloud for many years now. VMware has partnered with EMC and Google Cloud to provide the following services integrated directly into vCloud Air.

Google Cloud Object Storage: Available in three flavours. Standard, Durable Reduced Availability and Nearline. I think these services are all pretty self explanatory but for more info see here.

EMC Object Storage: Geo-protected distribution, located in the same data centers as vCloud Air, S3 compatible API. Currently only available in the US, but coming to other global vCA data centers in the near future.


vCloud Air SQL (DBaaS): Again relatively self explanatory. vCA SQL is a managed database service initially based on, you guessed it, Microsoft SQL. This was announced last year but has made it into the fold this year.


Backend-as-a-ServicevCloud Air’s first foray into the PaaS world outside of the partnership with federation brethren, Pivotal. Most impressive in my opinion is Kinvey, a vCloud Air integrated mobile development platform with complete backend services for end-to-end delivery of mobile applications: identity management, data services, business logic, analytics + more.

Apparently you can build and deploy a complete mobile application without writing a single line of code! I find this difficult to believe but I look forward to giving it a go.


There were a number of other smaller announcements which weren’t as exciting (to me at least). If you want any more info about any of the above feel free to give me a shout.

Author: @Kev_McCloud

Fusion 8 – vCloud Air Integration

True story… I spend a lot of my daytime hours working with VMware Fusion Pro for BAU and local lab tasks. Fusion has been a mainstay for me since switching to a Macbook Pro for my main workhorse a number of years ago. I test a lot of beta software, build & break stuff for training. Fusion has always had me covered.

I’ve also gotten used to working with a variety of interfaces when testing various cloud platforms, but recently most of my time has been spent in vCloud Air (obvz). So when I fired up Fusion 8 Pro this morning and found the <VMWARE VCLOUD AIR> header in my navigation pane, I was understandably excited.

vCA Login

A simple click onto the header reveals login fields and a couple of contextual links to vCloud Air (bonus points to the product team for adding <Remember Password> checkbox).

Screen Shot 2015-08-26 at 7.13.32 am

I enter my login credentials and *BAAM*. Within a couple of seconds, full access to all of my Subscriptions and VPC OnDemand. I will admit that I was surprised by how rapidly Fusion was able to display and allow interaction with my full vCA inventory across multiple SID’s / VDC’s.

I’m hoping to see a more advanced federated ID type service integrated into Fusion in the near future, but this will do for now.

VM Remote Console (VMRC)

Hands down one the best features of this Fusion release is VMRC access to vCA workloads. No messing with firewall and NAT rules. Just plain VMRC…

Screen Shot 2015-08-26 at 10.33.50 am

The result is Operations can continue to administer the vCA platform and assign access to VDC’s based on login. Developers (or users) who have no delegated administrative control in vCA can login via Fusion and get access to the resources the need. No additional URL’s to send out. No training users on the multiple interfaces. They just continue to use Fusion in the same way they always have…

Transport to vCA

As for workload transport, we can still export to OVF/OVA and upload to our Private Catalogue in vCA…

Screen Shot 2015-08-26 at 10.35.16 am

…but why would we when we can now drag ‘n drop our local VM’s into vCloud Air! Select the VM, drag to our VDC of choice, rename if required and click <Upload> to confirm. Simple.

Screen Shot 2015-08-26 at 11.37.01 am

Note: One small gotcha (why is there always a gotcha…). In order to use this method of migration we need to update Fusion tools to the latest version, and then downgrade the hardware version to a maximum of v10. The VM can be upgraded again post migration which is a small hassle (edit:when supported), but in general this method rocks!

Screen Shot 2015-08-26 at 11.48.44 am

And that’s it… A ~5GB VM took just under 10 minutes to move from Fusion to vCA (AU South). Of course the network still needs to be setup on the vCA side to support intra-VDC and external network communication, but if the VM is standalone then nothing else is required.

Screen Shot 2015-08-26 at 12.14.23 pm

More detail to come in the near future. Happy Fusion’ing😉

 

Author: @Kev_McCloud