VVol and HDS Part 2: With and Without VVols

Bye Bye VMFS

At its simplest, VVol removes the layer of abstraction that exists between a virtual machine and it’s data. As one of my colleagues has said, it’s akin to providing an a raw-device mapped LUN for every VMDK file without the operational implications of RDM.

It is important to understand that the layer of abstraction that will be removed is the VMFS filesystem, and this has been the middleman between the virtual machine and it’s data. This is now removed and this has many implications.

For starters, it means there is no local filesystem instantiation in vSphere. This has an impact on snapshots, for example, which now cannot be completed without hardware offload.

Why ? If you think about it there is nowhere “local” where vSphere can store the snapshot. And there are some neat benefits of snapshots from a placement, fault isolation and containment perspective. I’ll cover them in later posts …. the concept of a datastore filling up and stopping all I/O to all resident VMs will become a thing of the past.

As I mentioned in my last post here <VVol and HDS: An Introduction> there have been consequences from an operational perspective such as the way mapped blocks (used space on disk) are managed separately by the hypervisor and the storage array. This is why operationally there have been problems with the traditional LUN presentation approach to virtual infrastructure.

Without VVols

Consider the diagram:

Screen Shot 2015-08-30 at 11.06.50The Impact:

This approach relies on either the predictive schema (discrete datastore for individual apps, VMs, or VMDKs) or the adaptive schema where many VMs share the same datastore and if a problem arises you move the individual VMDKs to another datastore. So most folks “suck it an see” in the adaptive schema.

That’s not to say you can’t empirically size datastores – You can – but most people don’t know how or don’t do it. And problems many vendor engineers don’t know how either.

Some of the operational challenges today are:

  • If you want to use the fastest application consistent backup method (hardware snapshot), the snapshot has to snap the entire datastore.
    • If the entire contents are being backed up then the period of exposure can be a long time to capture all the objects.
    • Creation of these snapshots involves coordinating the point in time a datastore and LUN are snapped to ensure consistency.
    • This is not a simple engineering problem and can lead to problems when it stops working. It also is highly dependent on a proven interoperability configuration which is also tricky.
  • People feeling that have to adopt single size fits all approach
    • Set all datastores the same size for simplicity
    • Present datastores to all clusters (despite this being not a good practice or a good idea)
  • The VMware consumer has no visibility regarding the capabilities of the datastore and is entirely reliant on the storage administrator to ensure the correct datastore is presented etc.
    • He also has no idea where they are. Wouldn’t it be great if this metadata was exposed ?
  • Typically more datastores are pre-allocated than are required, normally to multiple clusters “just in case”. This is not a good idea and leads to too large a fault domain.

With VVols 

Screen Shot 2015-08-30 at 11.08.00

The impact:

Now consider the removal of VMFS from the picture. Now we map every VM disk (including config files and swap files) to a separate object on the storage.
ASIDE: One of my NAS colleagues quite funnily refers to block storage being able to finally replicate the same simple features of NAS. It’s not quite that simple but he has a point. Later on I’ll show how we still think the we have same structure (folders) on the storage due to the way the Web Client abstracts the Virtual Machine objects. This is quite interesting too !

Now we have an improved operational picture:

  • Data services are provided at virtual machine disk level.
  • No longer do we need to compromise on making everything the same for the sake of administrative overhead.
    • So “Let’s make all datastores 1TB or 2TB …….”
  • Abstraction of back-end storage, pools and containers from a virtual machine.
    • Now we just place a disk somewhere like “SQL Server Production”, “Dublin Business Critical”. Now you don’t need to know where your data resides. Why do you care ? Once your requirements are met, let VM storage profiles help with data placement.
  • Use VM storage profiles as the logical container in which virtual machines are stored.
    • This is a paradigm shift. What happens now when you change the location of where a VM disk is stored ? We will address that question later but it’s interesting and one you should consider !.

In the next post I will describe the VASA provider and Protocol Endpoints in more detail. Until then …. think about some of these ideas !




VMware VVol and HDS: An Introduction

Over the last few weeks I’ve been delivering presentations with colleagues as part of internal and external training sessions on VMware VVol technology, and more specifically, it’s application when using HDS storage & software.

On August 18th VMware VVol was formally supported on HDS G1000 Storage, our first Block storage array to support VVol.

This post is the first in a series to share some of what I learned as well as how VVol will be realised with HDS technology. I hope that you will read on – this will not be a case of whether HDS implementation is better or worse than other – I will try to keep the series neutral but will describe HDS specific considerations for our customers.

We at HDS expect a phased adoption of VVol technology at least in terms of the planning and evaluation side, prior to migration. This will be due to the ramifications VVol brings to storage and virtual architectures, and more importantly the operational processes that need to be put in place to support VVol. There are significant opportunities to enhance operations with VVol which we are sure will be achieved.

Several layers of interoperability must be considered, not withstanding HDS or VMware features. So we are advising customers to consider this a number of months before deployment.

Start at the beginning

When will you deploy vSphere 6 ?  

That is the first question that needs to be considered and one we ask customers first.

This is not a trivial concern due to the fundamental changes vSphere 6 brings to the landscape. Now is the ideal time to test drive the technology and get hands on experience which will inform better decision making. That depends on what equipment you have, but this is the only real way to “get your head around it”.

The ultimate deliverable and unit of consumption of VVol will be Virtual Machine Storage Policies,   consumed by applications and VMs. That must not be forgotten !. Therefore, a VM Storage Policy is a fundamental construct in the VVol ecosystem.

Operational Benefits are key

No matter how you look at it, the operational benefits of VVol are clear, and even the most ardent debates ended up with audiences agreeing this point. That alone justifies adoption of the technology.

These benefits can be due to legacy constructs that exist within the presentation of storage (LUNs/VMFS) to virtual infrastructure which clearly needed a new framework. Zero Page Reclaim and SCSI UNMAP were highlighted by many on the VMware Administration side of the house as causing significant administrative overhead.

These will be a thing of the past with the adoption of VVol.

Not a question of If

In the sessions so far it is not a question of If, but rather How and When VVol can be implemented.

So in this series I will cover many of the key topics within what I am calling a foundational technology for the future of the virtual datacenter (if you’re a VMware customer).

Benefits to Cloud & “Storage As A Service”

If you are planning to consume storage at scale by provisioning large sets of virtual machines using a Cloud Management Platform (e.g. for DevOps), VVol will make possible what is otherwise very difficult from an operational perspective.

Consider creation of 400 VMs using lazyzeroedthick block allocation onto a thin pool, and the difficulties this incurs. This makes traditional storage unsuited to the rigours of flexible cloud consumption.

Now consider how the VVol disks of a Virtual Machine can be created and destroyed on-demand, via the VASA Provider. So control has moved to the hypervisor management layer.  This is a KEY concept to grasp and will be a key benefit to cloud consumption models.

VVol allows consumption and capacity provisioning to be truly separate activities. This is a paradigm shift that allows a VM to be more easily offered as a service via service portal, including storage without the previous risks that went along with that.

The starting position for our sessions is always the organisational considerations. We believe VVol can help heal fractures between the Storage and Virtual Infrastructure teams in the same way it can bring closely-aligned teams even closer together.

in the next post I will start to describe the concepts from a storage and vCenter perspective. We will slowly build the picture up layer-by-layer to prevent confusion which can be common with VVol ……

Thoughts from vExpert Session July14th -> Scaling Converged vs Hyperconverged

First published on July 24, 2015 11:23:00 AM at HDS VMware Community

On July 14th we held a private webinar for the vExpert Community. Thanks to the always helpful Corey Romero, vExpert community manager for hosting it and giving us the opportunity to share some of the coolest converged technology out there.

The session was all about the power of the Unified Compute Platform API to allow orchestration of servers, storage, networking via API, the same way you can do so directly within the vCenter UI.

As I mentioned on the webex how about the following service catalog items in vRA !!

Continue reading Thoughts from vExpert Session July14th -> Scaling Converged vs Hyperconverged

HDS customers and fans join our new VMware Community

HDS recently launched the new VMware Community over at VMware Community. On this site you will find many of our product managers and senior VMware Consultants contributing deep-dive knowledge. This is perfect timing with VMworld fast approaching.

This is a lot more than just  storage, so you can find awesome resources about VMware VVol, EVO:RAIL, UCP Director for VMware and there will lots of other resources added…coming soon !

The purpose of the site is to share knowledge for when you put HDS and VMware together and shows the incredibly strong relationship that exists today between Hitachi Data Systems and VMware.

For anyone who needs information, now you have a shortcut where you can go and talk to the people who really know. There is also huge support internally for this initiative so over time many more internal experts will be getting involved directly helping customers….As usual we are all busy so sometimes we need to poke and prod them first :-)

Screen Shot 2015-07-27 at 17.38.28

Join up today !!

My VMworld 2015 USA Session Schedule

This year is my first time to attend VMworld USA. Will be great to get the chance to meet up with some great community comrades and also see the bigger scale. For me this is also the first time to be doing some presentations on behalf of the company. Probably booth-based in our booth theatre but let’s see what happens.

When I’m not at customer meetings or booth duty I wanted to share  my current session list. For me there are a few main areas of focus

  • Automation (agnostic)
  • vRealize Automation Suite
  • Service Catalog Design and Automation
  • Microservices / Docker / Applications in the 21 Century and how to automate them
  • vSphere advanced topics – Performance, Certificate Management, VCSA, NSX deep dive
  • vCloud Air as part of a coherent DR strategy
  • Openstack (VIO)

Continue reading My VMworld 2015 USA Session Schedule

Attention vExperts: Join vExpert HDS session on July 14th 4pm GMT

To all my esteemed 2015 vExperts in the VMware community, I really hope you can join the HDS team on a vExpert briefing next week to talk about why we believe HDS super-converged offering …. (I just made that term up) …… is the best fit today for scaling your Private Cloud environment.


Continue reading Attention vExperts: Join vExpert HDS session on July 14th 4pm GMT

Come say hello at Frankfurt VMUG !!!


On Wednesday this week I will be travelling to the Frankfurt VMware User Group (VMUG). The event is on at the Marriott downtown Frankfurt.

I’m filling in for one of my colleagues on the desk at the conference and have taken the opportunity to come visit Germany.

I am bringing my laptop so anyone who wants to come along and talk to me about why I believe Hitachi Converged platforms are the best out there can catch a glimpse of Unified Compute Platform Director software running inside vCenter orchestrating servers, network and storage. No parlour tricks !!!.

Continue reading Come say hello at Frankfurt VMUG !!!

SDDC, I am your father, and I am called SDI


Listening to one of my favourite albums, by one of my favourite bands, Alt-J, I happened to glance across at the song title “Choice Kingdom”. It reminded me of what is going on in the ICT Industry today. We are in a world where some might say it’s All-or-Nothing. That’s their job, to disrupt, destroy and create doubt in the mind of buyers so they question everything they ever knew. To say it’s cheaper, faster, better; nothing can ever go wrong with our product. Right?

This is not new and has been covered before as a concept in the book Accidental Empires by Robert X. Cringely, describing Silicon Valley in the days of Jobs and others, where he likens the serial entrepreneurs as commandos who have a singular purpose. That is to create as much mayhem as possible in the enemy’s camps and ranks. It’s all about disruption.

Continue reading SDDC, I am your father, and I am called SDI