Vmware on Neale's Corner

Setting vSphere Round Robin multipath policy with PowerCLI

Mon, 02 Aug 2021 00:00:00 +0000

Last week I was adding new datastores to a cluster for a provider whose best practices recommended a Round Robin multipath policy, changing paths with every iop. There is a VMware knowledge base article on how to do this - https://kb.vmware.com/s/article/2069356 - which details the CLI commands that need to be run on every host in the cluster for every datastore that needs to be configured. Attached to the KB article is a list of the PowerCLI commands that will also implement the same configuration.

When pressed for a choice, either - enabling SSH on each host, logging in, running the commands, logging out, disabling SSH - or - running PowerCLI commands remotely against the vCenter Server - the easiest option seems obvious.

So, I’ve written a script that will do this for multiple datastores against all hosts in a cluster. The latest version is available at https://github.com/nelons/psscripts/blob/master/vsphere/Set-ESXRoundRobinForDatastore.ps1, or the version at the time of writing this article is displayed below.

# Configuration variables.
# The FQDN (or IP) of your vCenter Server.
$vCenterServer = "vcenter.dnsname";
$ClusterName = "cluster";

# An array of datastore names that you want to convert.
$datastores = @("Datastore Name");

# When should the path be switched ?
$iops_switch = 1;

Connect-VIServer $vCenterServer | out-null;
$hosts = Get-VMHost -Location $ClusterName | Where-object {($_.ConnectionState -like "Connected")} | Sort-Object Name

# This will get the datastores from the array at the top of the file. 
$ds_id = $datastores | ForEach-Object { Get-Datastore -Name $_ } | ForEach-Object { return $_.ExtensionData.Info.Vmfs.Extent.DiskName; }

# Uncomment this line if you want to ignore the datastore name array
# and instead filter on the start of the datastore name.
# Remember to update the part of the line that starts with "start of datastore name" !
# Remember to comment out the datastore array line above !
#$ds_id = $datastores | ForEach-Object { Get-Datastore | { $_.Name.StartsWith("Start of datastore name"); } } | ForEach-Object { return $_.ExtensionData.Info.Vmfs.Extent.DiskName; }

$hosts | ForEach-Object { 
    $esx = $_;
    Write-Output "$($esx.Name): Checking LUNs.";

    $ds_id | ForEach-Object {
        $lun = $esx | Get-SCSILun -LunType Disk -CanonicalName $_;
        if ($null -ne $lun) {
            # Make sure it is RoundRobin
            if ($($lun.MultipathPolicy) -notlike â€œRoundRobinâ€) {
                Write-Output "$($esx.Name), $($_): Setting multipath policy to RoundRobin.";
                $lun | Set-ScsiLun -MultipathPolicy RoundRobin | out-null;
            }

            if ($($lun.CommandsToSwitchPath) -ne $iops_switch) {
                # Set the IOPS switch.
                Write-Output "$($esx.Name), $($_): Settings commands to switch path parameter to 1.";
                $lun | Set-ScsiLun -CommandsToSwitchPath $iops_switch | out-null;
            }
        }
    }
}

Disconnect-VIServer -Force -Confirm:$False;

Get VMHost Swap datastore using PowerCLI

Mon, 23 Mar 2020 00:00:00 +0000

This is a quick one-liner to get the configured swap datastore on a host/hosts:

`1`	`Get-VMHost \| Sort Name \| select Name, VMSwapfileDatastore`

If we want to make the name look a little nicer we could do:

`1`	`Get-VMHost \| Sort Name \| select Name, @{L='Swap Datastore';E={ ($_.VMSwapfileDatastore).Name } }`

This would get the results for all the esxi hosts for the target that you are connected to, however if you wanted to reduce the scope to a cluster or a datacenter you could change the start to:

`1`	`Get-VMHost -Location <Datacenter / Cluster Name> \| ...`

Or you could pipe in the location or array of hosts:

`1`	`Get-Cluster <cluster name> \| Get-VMHost \| ...`

or:

`1`	`$array_of_host_objects \| ...`

Enabling SSH on ESXi with vRO

Mon, 19 Aug 2019 00:00:00 +0000

I think it is a good rule of thumb that the security maturity of the organisation can be measured by whether SSH is left enabled (with warnings muted) or disabled on ESXi Hosts. As there is a warning on the host if the service is enabled, and the hardening guides recommend that it is disabled, there is little doubt that this disabled state is VMware’s preference and a security best practice.

So how do we enable SSH on ESXi ? I am assuming that the only accessible interface is the Web Client, as access to the console is likely restricted too - perhaps the hosts are in lockdown mode ? We have a few options..

Web Client

This is a little bit convoluted. We need to follow the steps:

Select the host on which to enable/disable SSH.
Go to the Configure tab.
Select the Security Profile menu option.
*Optional: Minimise one or both Firewall sections (Incoming Connections and Outgoing Connections).
*Optional: Scroll down to the Services section.
Press the Edit button in the Services section.
Select the SSH entry in the list.
Press Start or Stop.

The * noted items are for vCenter 6.5 or earlier. In vCenter 6.7 the Services panel is moved to the top of the Security Profile section.

The menu to manipulate the service looks as below:

In summary, this is quite a bit of work to enable the service on only one host. Next we’ll look at how to do the same on the HTML5 Client.

HTML5 Client

The steps required to enable/disable SSH on a host using the HTML5 client are as follows:

Select the host on which to enable/disable SSH.
Go to the Configure tab.
Select the Services menu option.
Select the SSH entry in the list.
Press the Start button

There are fewer steps in the list compared to the Flex client, however this is still onerous when having to work with multiple hosts. The section for changing service state in this client looks as follows:

vRealize Orchestrator

Ah, so we’ve got to the real reason of why I’m writing this post ! With past experience I’ve needed to use SSH to:

troubleshoot on a single host
perform maintenance/change on every host in the cluster.
disable SSH on host as someone else left it enabled and the timeout value hasn’t been set properly.

This identifies that we need workflows to enable and disable the service for a host, and for every host in a cluster.

However, thinking about this, if a host typically has SSH disabled then the shell/ssh enabled warning will not be suppressed - so you will know whether SSH is enabled or not. So, we will also have a workflow that toggles the service between enabled/disabled on a host - purely as a way of presenting fewer options to the end user so the vRealize Orchestrator submenu is kept as trim as necessary :)

Our approach to the cluster can re-use some of this - in a best case scenario the service is in the opposite of the desired state for every host in the cluster - i.e. if we want to enable, then the state of every host’s service is disabled. However we must also account for the possibility that the service is enabled on some hosts and disabled on others. Conversely, if the SSH service has been left enabled on a few hosts in a big cluster and we want to disable them, it could be easier to do in a single cluster workflow rather than running a workflow per host. A toggle workflow will not work here - we want to explicitly set the state of the service for all hosts in the cluster - whether enabled or disabled - otherwise we could end up reversing the state. From the point of the user - they know what end state they want, let’s get them there !

This leaves us with one last architecture decision between:

A workflow for enable and a workflow for disable.
A single workflow with a boolean parameter that defines enable/disable.

Honestly, I don’t think there is much between the two, both are good and reasonable choices. I decided to go with the top option for the reasons:

The workflow can be invoked remotely with less parameters.
Flex WebClient integration will allow the workflow to be immediately run with less clicks.

Building the workflow

The following code block will enumerate the services available on the host, and if the SSH service is not running, start it.

var services = host.configManager.serviceSystem.serviceInfo.service;

for each (var svc in services)
{
	if (svc.key == "TSM-SSH")
	{
		var current_state = (svc.running == true) ? true : false;	
		
		if (current_state == false)
		{
			host.configManager.serviceSystem.startService("TSM-SSH");
		}	
		
		break;
	}
}

Conversely, to stop the service, most the above is identical, except for the innermost conditional block:

		if (current_state == true)
		{
			host.configManager.serviceSystem.stopService("TSM-SSH");
		}

Our toggle code can be written as:

		if (current_state == false)
		{
			host.configManager.serviceSystem.startService("TSM-SSH");
		}
		else if (current_state == true)
		{
			host.configManager.serviceSystem.stopService("TSM-SSH");
		}		
		
		break;

The workflows for the cluster follow a typical pattern:

Get all the hosts in the cluster
Call the appropriate workflow for each host

The workflow schema looks as follows:

As can be seen, we use a built-in action to get all the hosts in a cluster, and the builtin workflow loop object to run a workflow for every host in the array.

In the workflows I also check the connection state of the host - there is no point trying to change the service state on a host that is disconnected !

Download

If your browser renames the file, you are free to change the file extension to “.package” (the import may still work if you do not). The file is an exported vRO package (from version 7.5) and is ready for import onto your server.

Click this link to download the package.

Reporting on vSphere Replication transfer sizes

Mon, 15 Apr 2019 00:00:00 +0000

vSphere Replication is really a wonderful product - I’ve been using it for a few years now across a few employers and I’ve never had a really serious problem with it - it just seems to work.

Sure, there are a few feature requests we could make of it, for example setting a schedule of when to replicate and ignoring RPO violations outside of that, and bandwidth limiting on the traffic coming out of the appliance, but they really aren’t necessary for what it is designed for.

Something I like exists in the Flash/Flex UI, where we can find out the size of the last data replication was (“Last sync size” in the image below). However this only exists in this one place and I haven’t found that this can be reported on anywhere else. It’s good to know the rate of change for your servers for judging bandwidth (yes we should have monitoring on the lines, but this won’t break transfers down into individual servers)

Finding what to report upon

As I was asked to provide change rates for some servers I started in the usual place (your search engine of choice) but couldn’t find a way of extracting this information. I then looked in the Events section of a VM being replicated and found that something was being logged - hurrah !

Yes I’m still using the Flash GUI. Come at me :p

Looking at a completed event I found:

We have a number of bytes transferred - we might be able to work with this. Time to switch to PowerCLI And these are the last two events to be logged with the VM:

If we have a look at the Arguments field we find (I’ve assigned $e to the first event in the above image because laziness !):

This information should give us enough to generate a report:

Filter on “EventTypeId” and matching to “hbr.primary.DeltaCompletedEvent”.
Using the Arguments.Value field to report on the size of the transfer.

Let’s give this a try:

Get-VM $name | get-vievent -MaxSamples ([int]::MaxValue) | ? { $_.EventTypeId -eq "hbr.primary.DeltaCompletedEvent" } | Select CreatedTime, @{L="MB Transferred";E={  [math]::Round($_.Arguments.Value/1MB, 2)}}

This will display the following - perfect !

Determining accurate numbers

Let’s remind ourselves of the UI report for the last transfer size - 40.64 MB:

So we can see that the numbers don’t quite add up - the GUI reports 40.64 MB and the event reports 40.78 MB.

Time to venture onto the Internet and see if we can find out how to get from 42761392 bytes to 40.64 MB (as I’m going to assume that the VMware engineers are clever than I and I’ve made a mistake somewhere). Using Google’s inbuilt calculator we find that 42761392 bytes are 42.76 MB.

If we use gbmb.org we get a bit more distinction - it is 42.76 MB if we take 1 MB to mean 1000000 (decimal, megabytes), and it is 40.78 MB if 1 MB is equal to 1048576 bytes (binary, mebibytes):

So I honestly have no idea how or why the UI reports the size as being 40.64 - from the above it should be either 40.78 or 42.76 depending on the value of 1 MB in bytes. So this leaves us with the following options:

The Arguments.Value is incorrect.
We are using Arguments.Value incorrectly.
The UI is incorrect.

For this report, we are going to accept the inaccuracy and use 1048576 as the value of 1 MB, as this will get us closer to the UI number.

Bringing it all together

Our script to generate the report is now:

Param([Parameter(Mandatory=$true)] $VMName,
      [DateTime] $Start,
      [DateTime] $Finish,
      [string] $OutputFile);

$vm = Get-VM $VMName;
if ($null -ne $vm) {
    $EventStart = $null;
    if ($null -eq $Start) {
        $EventStart = $(Get-Date).AddDays(-7);
    } else {
        Write-Host $Start.ToString();
        $EventStart = $Start;
    }

    $EventEnd = $null;
    if ($null -eq $Finish) {
        $EventEnd = Get-Date;
    } else {
        Write-Host $Finish.ToString();
        $EventEnd = $EventEnd;
    }

    $events = get-vievent -MaxSamples ([int]::MaxValue) -Start $EventStart -Finish $EventEnd | ? { $_.EventTypeId -eq "hbr.primary.DeltaCompletedEvent" } | Select CreatedTime, @{L="MB Transferred";E={ [math]::Round($_.Arguments.Value/1MB, 2)}}
    if ($null -ne $events) {
        if ($null -ne $OutputFile -And $OutputFile.Length -gt 0) {
            $events | Export-Csv -NoTypeInformation -Path $OutputFile;
        } else {
            $events;
        }
    } else {
        Write-Host "Could not find any replication events for the vm $VMName" -Foreground Red;
    }

} else {
    Write-Host "Could not find a VM with the name $VMName" -Foreground Red;
}

From here, we can output a CSV file, import that into Excel and make some fun graphs !

As far as my limited search engine searches have returned, there is no way to find out which VMs are protected by vSphere Replication with PowerCLI. Given that there is a vRO plugin, I think that if this needs to be used if you need further information.

I’ve uploaded the script to github :)

As usual, please let me know if I’ve got something wrong or you found it useful !

Managing a DRS group with vRealize Orchestrator

Mon, 25 Mar 2019 00:00:00 +0000

The use case for this challenge was a small (8-10 ESXi hosts) cluster, that only a subset of servers were ‘assigned’ with Windows datacenter licenses - so in order to maintain compliance all the Windows VMs would need to run on those hosts. The natural choice for ensuring this happens is a “must run” DRS rule between a group of VMs and a group of hosts and the automation step for this is ensuring that the VM group is accurate by only containing VMs that are still in the cluster and contains all Windows VMs in the cluster. Thus, the plan is use vRealize Orchestrator to manage the DRS group.

This technique (and workflow) will be left open enough for it to be used with any type of Operating System, after all I believe Red Hat also license on a per-hypervisor basis.

Core

The schema of the workflow is pictured below.

The flow is one of the simplest we can make ! The steps are:

Retrieve the configuration variables from the ConfigurationElement.
Get the VMs from the cluster (exit the workflow if it has none).
Get the DRS Group from the name (exit if it does not exist).
Loop through the VMs and add/remove them to the DRS group as necessary.
Send an email report if changes are made.

It has a single input of a ConfigurationElement, with the following variables. The description should make it apparent what each variable is used for.

The magic of the workflow is the updating of the DRS rule, which the below script covers:

Where next

As it stands now, we can schedule the workflow to run at the end of the day and allow DRS to place the VMs correctly.

However, to really ensure that the VM is running on the correct host we would invoke this workflow on every creation and addition to the cluster. This is possible when using LogInsight - configured as a destination for the ESXi hosts’ syslogs - to call Orchestrator via a Webhook when the appropriate events are received.

Now that we can dynamically change a DRS rule, this is starting to open up possibilities. Stay tuned :)

Summary

You can download the workflow package using the button below. Please let me know if you found this useful !

Download package file

Updating a HP iLO using Rest API and vRealize Orchestrator

Mon, 18 Mar 2019 00:00:00 +0000

A security vulnerability against Hewlett Packard iLOs was announced last year (link - https://www.rapid7.com/db/modules/auxiliary/admin/hp/hp_ilo_create_admin_account), along with the required firmware to patch. No doubt there are some amazing tools by HP that will allow firmware to be patched across an estate, but as I don’t have access to them I need to find another way to get firmware out to 20+ servers.

There is an API (documentation at https://hewlettpackard.github.io/ilo-rest-api-docs/ilo4/) which is quite straight forward to use, so we’ll have a look at using Postman to update an iLO and then use vRealize Orchestrator to implement a workflow.

Using Postman

Throughout this I will be using the headers below, though the second one is only required when updating the firmware.

To start, we’ll ask the iLO for a Manager interface, which will provide links to the services it offers. Authentication is handled by the headers on a per-request basis. We won’t need to manage session or tokens :)

The initial request is a GET to https://<FQDN/IP>/redfish/v1/managers/ as below:

The data returned contains an array with the name “Members” that will point to the URL for the manager for the iLO. We’re going to use that to get the details for the manager - there is a lot of data here so I’m not going to post it all. There are two points of interest for us - firstly the current firmware version:

Secondly the links to services - we are interested in the UpdateService.

We will use the location provided to access the service - which will allow the current firmware status to be retrieved and an update to be triggered.

The documentation has some reference on what to do - https://hewlettpackard.github.io/ilo-rest-api-docs/ilo4/#links-updateuri-extref-and-links-updateuri-extref - which is pictured below.

However, other documentation at http://h22208.www2.hpe.com/eginfolib/servers/docs/HPRestfultool/iLo4/data_model_reference.html#HpiLOFirmwareUpdate details the following:

This explains we’ll need to include an “Action” parameter in our request. To update the iLO we will need to host the firmware on a webserver that is accessible from the iLO address - this blog entry isn’t going to cover how to do this (I’m assuming that you’ve got a webserver to hand and you’re allowing the extension/content-type to be downloaded. Remember to test it manually before trying to automate, especially if your environment is subject to network controls)

Our POST request body will look as follows:

The iLO will restart when complete - there is no need to manually trigger this, though it is generally a good idea to ensure it has completed. To do this, we will call the GET request to the UpdateService/ URL and check the “State” value - it will change to the following values:

UPDATING
COMPLETED
(The request will fail when the iLO automatically restarts)
IDLE

So if we get a value of COMPLETED or IDLE then we can check the iLO version. However, given that a value of COMPLETED will follow with a period of silence it is more prudent to wait for the IDLE - which I believe we will also receive in the result of failure.

Using vRealize Orchestrator

Now to find a way to put the above into Orchestrator - the workflow to upgrade an iLO is:

The workflow has the following inputs:

The check version value will, if set to true, retrieve the version from the iLO and compare it to the values in the new_major and new_minor fields. If the firmware version on the iLO is newer then the firmware installation will not proceed. This check is dependent on human accuracy as there is no way of reliably retrieving the firmware version from the file, thus if the incorrect version is entered then installation will fail.

The heart of using vRO to drive this process is dynamically creating REST hosts in code, sending the requests and dealing with the response. Taking a look in the vRO workflow “Invoke a dynamic REST operation” we can see how VMware have done it:

The important things to note are:

We need to create a REST Host object, and then use it to create a transient host.
We need to specify the base url and authentication on the transient host object.

However if we invoke a REST call over HTTPS without importing the certificate it will fail, so we will need to handle this failure by importing the certificate and re-running the REST invocation.

This is handled by the Import Cert ?/Import a certificate/Log Cert Import elements. This is done by catching the exception that is thrown.

We now need to can connect to the manager address and find out the location of the UpdateService service - which we did earlier in Postman. Doing this in some javascript looks like the following:

The main part of this workflow is upgrading the firmware - we need to POST the update:

And then loop until we reach the IDLE state. When we exit the loop we can check if the version is different from before we started the process. This will give us our success/failure result to return to the user. The loop is as below:

Wrapping it Up !

Updating multiple iLOs

This is quite straight forward - there are two approaches.

The first one will duplicate the workflow and then adjust the input to take an array of addresses (This does however mean that the credentials to access the iLO must be the same). The schema will be changed to loop through the addresses.

The second approach is the one I favour - we create a new workflow which has duplicate inputs, with the exception of an addition for an array of addresses. The workflow loops through the addresses and calls our original workflow with the other inputs. This method means that we only have one workflow which contains the code that upgrades the iLO, so there is less to manage/maintain.

The workflow in this case will look like this:

with the following inputs:

Updating an ESXi Cluster using a custom attribute

To take this a step further, each ESXi host has a custom attribute with the address of its iLO (work Host so address is blanked):

A workflow is created that will get all the hosts in the cluster, and for each host retrieve the custom attribute. If a value exists, then the original workflow is called to upgrade the firmware. This workflow will only upgrade one physical server at a time, even though there is (or should not be) any impact on the running workloads. It’s just less risky, and isn’t that what our job is really about ? ;)

The challenge here will be ensuring that hosts have a completed (and valid) address for the iLO.

vSphere Client Integration

Still using the Flash Client because it allows you to call vRealize Orchestrator workflows ? Yep, me too. We can export the workflows we’ve created here to make them available on the context menu for hosts and clusters - thus making the upgrade even easier !

Further Automation / Improvements

Taking this further would be to wrap this in an interface (custom HTTP, ansible, etc) that allows end users to provide the firmware file and the host(s) or cluster(s) to upgrade.

This is beyond the scope of this post, but could be provisioned through your DevOps platform of choice. Let me know your thoughts on this one !

Summary

The workflow(s) and associated actions/etc are all available to download via the button below.

Download package file

I hope you’ve found this useful, please let me know your feedback and whether it helped. Thanks for reading :)

Managing Snapshots with vRO

Mon, 25 Feb 2019 00:00:00 +0000

One of the main issues that face departments on their virtualisation journey is how they are managing snapshots:

Who can take them
Who can delete them
How long do they stick around for
Name/Description formatting policy

I consider them ’low hanging fruit’ on the management tree - how an organisation approaches this is defined by policy and there shouldn’t be anything too contentious involved. There are also options to have the snapshots deleted automatically, with most management software providing the function. If you don’t have this, then a vSphere alarm can be created to warn on snapshots sizes.

There was no option to automatically remove snapshots that was in accordance with the policy that was agreed at a previous role, so it was time to automate something :). One of the items revolved around communication to the snapshot owner - it is very important to inform them that the snapshot will be automatically removed and when it has been. Another item was where some snapshots that might be automatically cleared up would need to be held for longer - perhaps as a part of a Root Cause Analysis investigation.

Requirements

From the above, the requirements are this:

Inform the user who took the snapshot that it will be deleted in X days, daily until deleted. (X should be configurable).
Inform the user that the snapshot has been deleted.
Report to a team/email address details on all snapshots present and estimate the space use.
Exclude Virtual Machines from having their snapshots automatically deleted.

These will mean that:

The workflow will always produce a report. Therefore our input values should include an email address.
A means of identifying a Virtual Machine as being excluded. We could supply a list, or a naming format to match, a tag or a custom attribute.
Given that we can retrieve the username of the person who created the snapshot (it is contained within the event), we still need a means of identifying the email address of the user.

My preference for the second item above is to use a tag as we can retrieve VMs easier from a management perspective.

The identity source for the vCenter is assumed to be Active Directory, thus we would have the further assumption that no local accounts are allowed (or only in exceptional circumstances) and will be taking snapshots. We can therefore use the Active Directory plugin to query for the user and retrieve the email address. However we should also handle the edge case that the user does not have an email address set, or it is not valid - therefore we will issue warning emails to an appropriate team that the end user was not/cannot be warned of the snapshot expiry.

In summary, our input values are:

# Days when start to warn end users of deletion.
# Days when the snapshot should be deleted.
Email address in case of errors.
Email address for the report.
Tag name for VMs to be excluded from snapshot management.
A ‘WhatIf’ to control whether action is taken.

Our vRO server will also need to have the Active Directory plugin active and configured.

We also need to deal with another assumption - there can be more than one snapshot on a VM, and multiple snapshots may need to be deleted on each day.

\[Hindsight\]

After creating the workflows I added another couple of inputs:

Minimum age of VM snapshot to appear on the report.
Subject of the report.

Workflows

I write vRO workflows in a decentralised manner - for example if I am to do something with a Virtual Machine I have a workflow for:

The Virtual Machine
The Cluster
(Sometimes) The Datacenter
vCenter Server

This can add up to a lot of workflows, but it allows for testing and iterating changes easily. I will often put a “WhatIf” input on the Virtual Machine workflow so that I can test the workflows that function at a higher level - this ties in with the identified inputs earlier. So in this case, the workflow design mirrors the above (but without a datacenter script).

Additionally, when communicating to people the details of a VM with a snapshot I include Custom Attributes and Tags. So the workflows below will contain some tagging actions that I really need to release ;)

Virtual Machine

This workflow will need to do the following:

Get the snapshot information for a VM.
Calculate snapshot size (if possible).
Get the tags for a VM.
Add information to the report.
Check the tag to see if the VM will have it’s snapshots automatically managed.

If the VM is to be automatically managed, it will need to do:

If the snapshot is older than the warning age, send a warning to the snapshot creator, or the errors address.
If the snapshot is older than the deletion age, send a notification to the snapshot creator, or the errors address.

I’ve separate the above into two lists as I think the logic flows naturally in this way - the first list will apply to every VM that we examine whereas the second will only apply to those VMs with snapshots that we will manage. Thus we can break the logic up into two workflows.

Disclaimer: I’m assuming some familiarity with Orchestrator here, and there are plenty of other awesome blogs introducing the tool, so I’m going to dive in to some completed workflows.

The schema for the first flow is below:

Some notable steps:

Excluded from snaps - This is where the tag is checked.
Automatic Management - This is where the WhatIf is handled.
Manage Snapshots - We call the workflow to manage them. This sits inside a loop that will examine all the snapshots on a VM.

Some scripts are included to help with generating the report :)

The schema for Managing Snapshots is here:

The notable steps for this workflow are:

getEmailAddressForUser - this will query Active Directory for the mail attribute for the user identified earlier.
Remove Snapshot - deletes the snapshot and sets a task object so the workflows/entire process operates synchronously.
Customise Email - these write the warning and notification emails to be sent to the end user.

Cluster Workflow

This workflow retrieves all the VMs in the cluster and runs the main VM workflow against it. Nothing special here :)

Clusters / vCenter Servers Workflow

This is fairly standard workflow that goes through all the clusters on a vCenter Server and runs the Cluster workflow. It follows a standard pattern of retrieving a set of items and iterating through them. The end of the schema is devoted to sending the report email.

Configuration Element

As I wrote this a while ago, I put all the configuration inside a Configuration Element so that configuration variables could be updated once a workflow was scheduled. Please excuse my naming format, it does the job but the names made sense at the time I promise !

These names match the inputs that the workflows take:

Emails

Here is an example of the report email running on a live environment:

An example of the email notification a user receives about their snapshot (some field values have been removed):

Download

The workflows, actions and configuration element is available to download here. The package includes the following workflows

Once the package is imported, run one of the workflows and let me know what you think ! You’ll be able to run the cluster and VM workflows without having to configure a Configuration Element.

Download package file

Improvements

The most obvious improvement to this workflow would be to made the notifications optional and controlled via an input. Maybe something for a rainy day ;)

Let me know if this was useful or interesting to you !

Detach old ESXi installation devices

Thu, 22 Mar 2018 00:00:00 +0000

When clearing up host storage, you might sometimes encounter an error when you come to detach a device. The cause of this can be an old ESXi installation and you’ll need to erase the ESXi partitions in order to detach old ESXi installation devices.

The typical use case for this is reclaiming SAN boot LUNs that are no longer required since we’ve moved to SD card / Local disk / AutoDeploy. Reclaiming unused storage is always a good idea ;)

The first step in the process is unmounting any datastores that exist on the device:

When Unmount is selected, we’ll need to confirm the hosts to unmount from:

The UI should update to show that it is now unmounted.

If this was a normal device we would then detach it - let’s do this now.

Aha, we get an error message:

This means that an ESXi install has been found on the device, even if it is old and inactive. In order to get past this, we will make sure this isn’t an active installation and then we will erase the partitions on the device.

A summary of the partitions is displayed and confirmation of erasure is required:

The device can be successfully detached now that the ESXi installation has been erased.

That’s it :) I’m sure there is a way to automate this, but I only had a few to do so I took the lazy way out this time !

vRealise Log Insight and Orchestrator

Fri, 15 Sep 2017 00:00:00 +0000

I’m a big fan of the vRealize suite as it really adds value to a vSphere deployment. In relation to this post, LogInsight is a great tool for providing log analysis and Orchestrator manages workflows to automate and orchestrate most/all of the infrastructure.

A series of postsÂ on vmware.com - “Self Healing Datacenter” - have examined how to connect Operations Manager to Orchestrator using a shim - the first part of the series is here.Â The shim is also capable of taking input from LogInsight, though this isn’t covered by the articles.

That webhooks are offered for LogInsight is great, but it is sad that there is no official tool to send events/data from one product in the vRealize suite to another. I’m interested in how and why things work (and also because it doesn’t appear to work quite right), so instead of using the shim presented in the series above I’m going to write my own in node.js so we can look at how this all works.

Environment

In this environment we’ll be using vRealize LogInsight 4.3 and Orchestrator 7.2, both which have a straight forward and simple deployment - both are single servers (no clusters !) and both are configured to a vSphere 6.5 deployment.

In vSphere we create a nested host to test against, and have put it in its own cluster. I’ve done this because some of the actions I want to take involve shutting the host down (not always gracefully) and I don’t have the spare physical hosts to do this (e.g. the auto-esxi VMs below are the hosts in the Nested cluster).

LogInsight has been set up with vSphere integration - all hosts (including the nested host) have been configured to send syslogs to the LogInsight address. A look at the “Hosts” page in the LogInsight Administration section shows that logs are being successfully received:

Integration

It isn’t actually possible for LogInsight to start Orchestrator workflows, the closest we can come is by using the “webhook” functionality when setting up an alert - this will make an HTTP request to an URL with some information when something happens (below is an example of our LogInsight query for a host going into maintenance mode).

It is because the vRealize Orchestrator API requires some additional parameters that we can’t start Workflows, and when we look at the schema on the swagger API reference page (https://

:8281/vco/api/docs/index.html - where

is the FQDN/IP of your Orchestrator server), we can see what is required:

WebHook Shims

The shim referenced in the “Self Healing Datacenter” series can be found as the “log insight webhook-shims” on githubÂ (this also works with vROPs). To test this I deployed a Photon OS instance and installed it via docker:

systemctl enable docker
systemctl start docker
docker pull vmware/webhook-shims
docker run -it -p 5001:5001 –name lishims vmware/webhook-shims bash

The public page for this docker container shows how to set it up -Â https://hub.docker.com/r/vmware/webhook-shims/ - and after this we need to configure the vRealize Orchestrator connection in the container. At the command line, enter the following:

vi loginsightwebhookdemo/vrealizeorchestrator.py:

When this file is saved we can start the server within the container:

./runserver.py

As per the INFO message above, if we browse to http://

:5001 (where

is the FQDN/IP of the server running the shim) we get a guidance screen on how to use the software - scroll down to view the advice on how to integrate with vRealize Orchestrator.

Now we can return to our LogInsight query and set up an alert based upon it. Select the Alert button on the UI and the following window will appear:

Select the Webhook notify option and put in the formatted address of the webhook server:

http://address : port / endpoint / vro / workflow-id

Aha, we don’t have a workflow set up so we don’t know what worfklow-id to type in here - let’s do this now. Open up the vRealize Orchestrator client and create a new workflow - we’re going to keep this one simple and just write something in the log, the schema for which is below:

When we look at the details of the workflow we see the ID to use the alert:

So now we have a workflow id we can create our LogInsight alert. Go back to the LogInsight alert configuration page, update the webhook and URL and save.

Workflow problems

Time to put a host (that is configured to send syslog to our LogInsight server) into Maintenance Mode - which I did I 0753 in the screenshot below:

We see the following workflow implementations:

Woah - we only put the host into maintenance mode once and the workflow was started 10 times - what could have happened ? The sequence of what happens is:

LogInsight matches the query and notifies shim server via webhook.
Shim server receives request and formats a vRealize Orchestrator API call. Sends to vRealize Orchestrator
vRealize Orchestrator starts the workflow.

If you were watching the console of the Photon OS container then you would have seen that for every workflow invocation a new request was sent by the webhook shim code. So this excludes vRealize Orchestrator from causing this, but is the shim code receiving a new notification for every workflow ?

I did toy with the idea of setting up port mirroring via a distributed switch and using Wireshark to check only one request was sent, but this seems unnecessarily complex. Â The is another way :)

Node.js

We can spin up a very simple node.js server that logs out every request that is made to it. Our server code looks as follows:

// include the http module
var http = require('http');

// create a webserver
http.createServer(function (req, res) {

    // respond to any incoming http request
    res.writeHead(200, {'Content-Type': 'text/plain'});
    res.end();
    console.log("Request Received.\n");

}).listen(5001, '127.0.0.1');

// log what that we started listening 
console.log('Server running at 127.0.0.1:5001');

When we put a host into maintenance mode now we can see that we only receive one email and that our notification monitor only receives one request from Log Insight. So the problem is somewhere in the shim server. But in the spirit of creating stuff and learning something new, let’s write our own shim server !

The main body of our server function is:

var responseCode = 500;

// Get Body	
var request_body_array = new Array();
req.on('data', function(chunk) {
	request_body_array.push(chunk);
});
	
req.on('end', function() {
	var request_body = "";
	if (request_body_array.length > 0) {
		request_body = request_body_array.join('');
	}

	responseCode = handle_request(req.url, request_body);
	res.writeHead(responseCode, {'Content-Type': 'text/plain'});
	res.end();
});

This is really straight forward and will receive the full request and call handle_request (which I’m not going to detail Â here), which will then invoke the call_vro function with appropriate parameters.

In our call_vro function we will need to set up the parameters for the Orchestrator call:

var creds_base64 = new Buffer(vro_username + ":" + vro_password).toString('base64');
var messages_value = new Buffer(request_body).toString('base64');

var request_body_json = JSON.parse(request_body);

var vro_execution_context = {
	parameters: [
		{
			name: 'messages',
			description: 'Base64 encoded JSON of LogInsight messages',
			scope: 'local',
			type: 'string',
			value: {
				string: {
					value: messages_value
				}
			}
		},
		{
			name: 'alertName',
			scope: 'local',
			type: 'string',
			value: {
				string: {
					value: request_body_json.AlertName
				}
			}
		},
		{
			name: 'hitCount',
			scope: 'local',
			type: 'number',
			value: {
				number: {
					value: request_body_json.HitCount
				}
			}
		}
	]
};

var workflow_request_body = JSON.stringify(vro_execution_context);

The code above sets up the basic authentication string and defines the parameter arrayÂ for the request. We are going to provide three parameters for the workflow:

messages - the messages passed by LogInsight. These will be Base64 encoded so Orchestrator will need to decode them.
alertName - the name of the alert that was configured in LogInsight.
hitCount - the number of times this alert was triggered.

As our alert should be triggered on every invocation we won’t worry about hitCount for now - but we should make a note to test this later.

After the parameters have been set up, they are convered to a JSON string for the request. We can see below in the request header set up that we are declaring our content is JSON.

var vro_auth_options = {
	host: vro_server,
	port: vro_port,
	path: '/vco/api/workflows/' + workflow_id + '/executions',
	method: 'POST',
	rejectUnauthorized: false,
	headers:  {
		'Accept': 'application/json',
		'Content-Type': 'application/json;v=5.1.1',
		'Content-Length': workflow_request_body.length,
		'Authorization': 'Basic ' + creds_base64
	}
};

var auth_data_array = new Array();
var callback = function(response) {
	response.on('data', function(chunk) {
		auth_data_array.push(chunk);
	});

	response.on('error', function(chunk) {
		console.log("ERROR = " + err.message);
	});

	response.on('end', function() {
		var auth_data = auth_data_array.join('');

		if (response.statusCode == 202) {
			if (auth_data.length > 0) {				
				var res = JSON.parse(auth_data);
			}

			console.log("Workflow " + workflow_id + " successfully started");

		} else {
			console.log("Workflow execution response: " + response.statusCode);
		}
	});
}

const req = https.request(vro_auth_options, callback);
req.write(workflow_request_body);
req.end();

This code sets up the connection, defines the functions for what happens when the request errors or is complete and then makes the connection (https.request at the bottom). If we receive an error code of 202 then all is good !

Mental note: current status codes for starting a workflow:

Trying it out

When we put a host into maintenance mode now we can see that we only get one Orchestrator workflow generated for each alert. But if we look at Log Insight, the alert is examined every 5 minutes:

So what happens if we put two hosts into maintenance mode before the alert is examined ?

This is the console of the server:

Looking at the request that has been passed we can see that there are messages for both hosts, but we only have one workflow started:

We need to look at how we pull out the information contained in the ‘messages’ input variable. First, we need to download and install a plugin that will allow us to decode base 64 strings. This can be obtained from Â https://github.com/vmware/o11n-plugin-crypto and installation is straight forward from the Control Center. After installation, the vCO service needs to be restarted (not the server) - the message stating the service needs to be restarted doesn’t disappear immediately, but it does go :)

Then we need to modify our workflow to accept these parameters:

I’ve also added an attribute to the workflow with the name “hosts” and type “Array/string” - but you can omit this and comment out the lines that reference this attribute. We can now add a scriptable task with the following:

// Decode the string and parse into a JSON objecct.
var decoded_str = CryptoEncoding.base64Decode(messages);
var li_obj = JSON.parse(decoded_str);

// Create the array to hold our hosts detail.
hosts = new Array();
for each (var item in li_obj.messages)
{
	// Get the Host name
	var hostname = "";
	
	for each (var field in item.fields)
	{
		if (field.name == "__li_source_path")
		{
			hostname = field.content;
			break;
		}
	}
	
	// Debugging: Write out the host name.
	System.log(hostname + " has entered maintenance mode.");
	
	// Add to the array.
	hosts.push(hostname);
}

This is what we see now in the Logs section for the workflow when we put two hosts into maintenance mode for the same alert interval - we can now handle more than one host, and identify which host has gone into maintenance mode.

Summary

So we can now decode the messages we receive from LogInsight and work with multiple messages - I’ve left the node.js console printing out the contents of the request (line 34) in order to help identify useful fields of information. This needs some tidying up to be useful (and a solution for storing username/passwords in the node.js file should be found) but hopefully is a good proof of concept of what is capable.

As usual, the code is available on github - https://github.com/nelons/loginsight_webhook_shim. Feel free to use/abuse in any way you like, let me know if I’ve made a mistake or if you found this useful.

Detaching Devices from a Cluster

Mon, 14 Aug 2017 00:00:00 +0000

When working with block storage we occasionally need to remove a storage device from the hosts it is connected to. When using the Web Client we can detach devices on a per-host basis, but there is nothing to detach from all its attached hosts.

What we do have is a helpful KB article on how to remove/detach a datastore -Â https://kb.vmware.com/kb/2004605Â and a PowerCLI file that contains some functions. However the functions revolve around managing actions on datastores as opposed to devices that are connected and could be used for RDMs.

Most LUNs are masked to all the hosts in a cluster so that, whether the device is an RDM or has a VMFS datastore on it, VMs can run on any host in the cluster. To deal with this use case, I use a script that will detach a device from every host in a cluster. If the device does not exist on Â a host then an error is displayed and the script moves on to the next device/host.

Before we can detach a device we need to ensure the following:

It is not used for a datastore.
It is not in use as a RDM.
The perennially reserved flag is not set for the device on the host.

The last point comes from the KB article above - if the flag is set (and it should be for any RDMs) then it could prevent the device from being unmounted successfully. So although clearing the flag is not required, clearing it ensures the best chance of success.

So, our script will start off with the following - we will accept a cluster to unmount/detach the device(s) from, an array of device identifiers, and a switch that will allow us to do checks without actually detaching anything.

param(
	[Parameter(Mandatory=$true,ValueFromPipeline)][VMware.VimAutomation.ViCore.Impl.V1.Inventory.ClusterImpl] $Cluster,
	[Parameter(Mandatory=$true)][String[]] $LUNIdentifiers,
	[switch]$CheckUsage = $false
)

For the sake of time, we will make the assumption that all devices are masked to all hosts in the cluster - so when checking what the device is used for we will only check on the first host in the cluster. The following code will get the VMs in the cluster (for checking RDM usage in the next step) and the host that we will use for checking.

$vms = $Cluster | Get-VM;
$cluster_host = $Cluster | Get-VMHost | sort Name | select -First 1

$LUNs_To_Detach = @();
$LUNIdentifiers | % {
	$lunid = $_;
$lun = get-scsilun -VMHost $cluster_host | ? { $_.CanonicalName -eq $lunid }

Finally we will use the Get-ESXCLI cmdlet to check if the perennially reserved flag is set.

`1`	`$myesxcli = get-esxcli -VMHost ESXhost`

When all our checks are complete, we can loop through the hosts in the cluster and attempt to detach the LUN.

	$Cluster | get-vmhost | sort Name | % {
		write-host "Detaching LUNs from $($_.Name)" -Foreground Yellow
		$esx = $_;
		
		$storSys = Get-View $_.Extensiondata.ConfigManager.StorageSystem
		
		foreach ($lunid in $LUNs_To_Detach){				
			$lun = Get-ScsiLun -VmHost $esx | ? { $_.CanonicalName -eq $lunid }
			if ($lun -ne $null) {
				# Ensure the LUN is attached 
				if ($lun.ExtensionData.OperationalState -eq "ok") {
					write-host "Detaching LUN $lun" # from $($esx.Name)"
					$storSys.DetachScsiLun($lun.ExtensionData.Uuid)
					write-host "Detach Complete" -Foreground Green
					
				} elseif ($lun.ExtensionData.OperationalState -eq "off") {
					Write-Host "LUN is already unmounted on this host." -Foreground Green;
					
				} else {
					Write-Host "OperationalState is $($lun.ExtensionData.OperationalState)";
					
				}
				
			} else {
				Write-Host "Could not find LUN $lun on host $esx.Name";
			}
		}
}

The entire script is available from my github here. As always, feel free to fork and make pull requests if you find a mistake.

vSphere REST API and C++ (Part 2)

Tue, 13 Jun 2017 00:00:00 +0000

This is the second part in an ongoing series looking at using the vSphere REST API and C++ with the cpp rest sdk (formerly known as “Casablanca”). In the first part we authenticated against SSO and got some information from the vCenter.

In this part we are going to refine our methods so that we can reuse our code. We will introduce some structs and functions that allow us to work with results.

Connecting

The first thing we are going to introduce is a struct that will manage our connection data. It is as follows:

struct vc_info {
  std::wstring server; // The address of the server
  std::wstring token; // The SSO token.

  http_client_config config; // The connection configuration
  credentials server_credentials; // Authentication credentials for the connection

  int last_status_code; // The last HTTP status code we received.
  std::string last_exception; // A text explanation of the last exception
};

All the members are annotated with how they will be used.

We will now assemble a ConnectAsync function which will return a task (and a pointer to a vc_info object). The code looks as follows:

task<shared_ptr<vc_info>> ConnectAsync(const wchar_t* server, const wchar_t* username, const wchar_t* password)
{
	std::shared_ptr<vc_info> pInfo = make_shared<vc_info>();
	pInfo->config.set_validate_certificates(false);
	pInfo->server_credentials = credentials(username, password);
	pInfo->config.set_credentials(pInfo->server_credentials);
	pInfo->server = server;

	http_client client(server, pInfo->config);
	http_request request(methods::POST);
	request.set_request_uri(U("/rest/com/vmware/cis/session"));

	return concurrency::create_task(client.request(request)).then([pInfo](http_response response) {
		// if status is OK then extract json.
		pInfo->last_status_code = response.status_code();
		return response.extract_json();

	}).then([pInfo](web::json::value v) {
		// get and set the token
		std::shared_ptr<vc_object> data = CheckJson(U(""), v);
		if (data) {
			std::shared_ptr<vc_property> value = get_object_property(data, L"value");
			if (value) {
				pInfo->token = value->s;
			}
		}

	}).then([pInfo](task<void> t) {
		// exception catching.
		try {
			t.get();

		}
		catch (const std::exception& e) {
			pInfo->last_exception = e.what();
			printf("Exception: %s\n", e.what());

		}

		return pInfo;
	});
}

As you can see,this function will wrap the call toÂ /rest/com/vmware/cis/session and provision our object to hold information about the “connection”.

It takes a server name, username and password and attempts to connect, returninf a vc_info object with completed information whether it is successful or not. The task chain is similar to what we wrote in part 1, with the exception that we now store the SSO token in the vc_info object. We will use a function “CheckJson” to format the result into an object hierarchy and we will be creating this in the next section.

This is how our ConnectAsync function can be used in a synchronous fashion.

`1`	`std::shared_ptr<vc_info> pInfo = ConnectAsync("my-vsphere.fqdn", L"administrator@vsphere.local", L"password").get();`

Handling JSON

As most calls will return information in a JSON format, we will use the following two structs to store and organise information.The CPP Rest SDK contains some JSON classes, so when we call

`1`	`return response.extract_json();`

on a response, we get a

`1`	`web::json::value`

object that we can use as we desire. We will transform this into our own objects -Â vc_property and vc_object, which are defined below.

struct vc_property {
	std::wstring name;
	int type;

	// unrestricted unions not working on MSVC mean s cannot be a member :(
	union {
		bool b;
		int i;
		double d;
	};

	std::wstring s;
};

struct vc_object {
	std::vector<std::shared_ptr<vc_property>> properties;

	std::shared_ptr<vc_object> parent;
	std::vector<std::shared_ptr<vc_object>> children;
};

As noted, as unrestricted unions were/are not supported on the version of Visual Studio I wrote this on, we exclude the variable “s” from the union. Possibly fixed or working in another complier thoughI haven’t checked.

We now need a function that we will call to extrapolate our JSON and structure it so we can use it. Because objects can contain other objects we may need to call this function recursively. The function takes the name of the key/value pair, the value and the parent object.

// Properties are automatically added to parent. Don't need to be returned.
// Objects are returned to be added to the children ..
std::shared_ptr<vc_object> CheckJson(utility::string_t name, web::json::value v, std::shared_ptr<vc_object> parent = nullptr)
{
	std::shared_ptr<vc_object> result = nullptr;

	if (v.is_array()) {
		for (unsigned int i = 0; i < v.size(); ++i) {
			std::shared_ptr<vc_object> obj = CheckJson(U(""), v[i], parent);
		}

	}
	else if (v.is_object()) {
		result = make_shared<vc_object>();

		web::json::object o = v.as_object();
		std::for_each(begin(o), end(o), [&result](std::pair<utility::string_t, web::json::value> item) {
			CheckJson(item.first, item.second, result);
		});

		if (parent != nullptr)
			parent->children.push_back(result);

	}
	else {
		std::shared_ptr<vc_property> prop = std::make_shared<vc_property>();
		prop->name = name;

		if (v.is_string()) {
			prop->s = v.as_string();

		}
		else if (v.is_integer()) {
			prop->i = v.as_integer();

		}
		else if (v.is_boolean()) {
			prop->b = v.as_bool();

		}
		else if (v.is_double()) {
			prop->d = v.as_double();

		}

		if (parent)
			parent->properties.push_back(prop);
	}

	return result;
}

Note: I believe the array code won’t work at this time - I haven’t worked with any arrays to test yet - we’ll take a look at this later. It will however, handle objects as children and fill in the properties as required.

As an example, when authenticating with ConnectAsync the following information is returned:

We deal with this data with the following code:

std::shared_ptr<vc_object> data = CheckJson(U(""), data);
if (data) {
	std::shared_ptr<vc_property> value = get_object_property(data, L"value");
	if (value) {
		pInfo->token = value->s;
	}
}

The code above should be fairly straight forward - the code will transform the result to a vc_object object called data, and we get the value of the key/value pair with the key of “value” and assign it to the pInfo->token member property.

We are still missing a “get_object_property” function to make the code work. As the code to walk object properties for a specific entry will be commonly used it has been pulled out into it’s own function. This is straight forward and below:

std::shared_ptr<vc_property> get_object_property(std::shared_ptr<vc_object>& obj, const wchar_t* name)
{
	std::shared_ptr<vc_property> result = nullptr;

	if (obj != nullptr && name != 0 && wcslen(name) > 0) {
		std::vector<std::shared_ptr<vc_property>>::iterator find_result = std::find_if(begin(obj->properties), end(obj->properties), [name](std::shared_ptr<vc_property> prop) {
			return (_wcsicmp(prop->name.c_str(), name) == 0);
		});

		if (find_result != end(obj->properties)) {
			result = *find_result;
		}
	}

	return result;
}

Tasks

To do things with the API I’ve written a generic function that can send requests - this is very basic at the moment. One of the ways we can improve this later is to allow for the passing of values in a POST request. The body of the function is below:

task<std::shared_ptr<vc_task>> SendRequestAsync(web::http::method method, std::shared_ptr<vc_info> pInfo, const wchar_t* uri)
{
	std::shared_ptr<vc_task> result = make_shared<vc_task>();
	result->pServer = pInfo;

	return concurrency::create_task([pInfo, uri, method]() {

		http_client client(pInfo->server, pInfo->config);
		http_request request(method);
		request.set_request_uri(uri);

		// We need to pass the token with the request.
		request.headers().add(U("vmware-api-session-id"), pInfo->token.c_str());

		return client.request(request);

	}).then([pInfo, result](http_response response) {
		pInfo->last_status_code = response.status_code();
		result->status_code = pInfo->last_status_code;
		return response.extract_json();

	}).then([result](web::json::value v) {
		// parse the result and store.
		result->data = CheckJson(U(""), v);

	}).then([result](task<void> t) -> std::shared_ptr<vc_task> {
		try {
			t.get();

		}
		catch (const std::exception& e) {
			printf("Get exception: %s.\n", e.what());
		}

		return result;
	});
}

The function takes:

The type of request - i.e. GET, POST, PUT, DELETE
The vc_info object for the server.
The URI to send the request to.

It will return a vc_task object that will contain the status code and the parsed JSON from the response body. The vc_task struct is defined as:

struct vc_task {
	std::weak_ptr<vc_info> pServer;
	int status_code;
	std::string exception;
	std::shared_ptr<vc_object> data;
};

Example

Now we’ve done all of this, we can put this together and try it out - we’ll get information about the hosts. This is the pertinent section of our code:

std::shared_ptr<vc_info> pInfo = ConnectAsync(vcenter_url.c_str(), admin_username.c_str(), admin_password.c_str()).get();
if (pInfo && pInfo->last_status_code == 200) {
	wprintf(L"Connected successfully to %s.\n", vcenter_url.c_str());

	// Get hosts.
	shared_ptr<vc_task> task_results = SendRequestAsync(methods::GET, pInfo, L"/rest/vcenter/host").get();
	if (task_results->status_code == 200) {
		// Do something with the hosts ...
		if (task_results->data) {
			printf("There are %d results.\n", task_results->data->children.size());

			for_each(begin(task_results->data->children), end(task_results->data->children), [](std::shared_ptr<vc_object> host) {
				std::shared_ptr<vc_property> host_name = get_object_property(host, L"name");
				if (host_name) {
					wprintf(L"Host discovered: %s\n", host_name->s.c_str());
				}
			});
		}
	}

} else {
	// Could not connect.
	if (pInfo) {
		printf("ConnectAsync request returned code %d.\n", pInfo->last_status_code);

	} else {
		printf("Unknown error - ConnectAsync did not return server information object.\n");

	}
}

When running this against my home lab (please ignore the different DNS domains :p) I get the following output:

The code is available at the github repo - github.com/nelons/vSphereRestApi - please try it out and give me any feedback you have :)

Next Time

Now we’ve tidied up our structure to allow reuse we’ll look at tags and getting relevant objects.

Using Orchestrator to automatically upgrade Tools

Wed, 08 Mar 2017 00:00:00 +0000

One of the main challenges in every (non-small) place I’ve worked is keeping VMware Tools updated, and with Change Management processes this becomes a little bit more complicated. There is a setting on each VM to upgrade Tools when the VM is booted and this is a good way to keep Tools updated on those VMs that aren’t mission critical and require manual intervention.

In this post we’ll look at creating an action and a workflow to configure this setting, and pushing it as a context setting in the Flex Web Client.

This leads to the issue of how this setting is enabled - firstly as part of the initial push (you don’t want to enable this manually for 100’s (or more) VMs) and with newly created VMs. Although this can be set on a VM template or “Gold Build” (or equivalent terminology), this can still lead to human error so we’ll also look creating a workflow to set this for a cluster and scheduling this using the Flex Web Client.

Upgrade Tools Action

To configure a VM to upgrade Tools we need to create a new configuration specification and set the tools configuration appropriately. Our actionÂ will return the Task for the configuration (note: this can be null) and will take two inputs - the VM to configure and whether we are automatically upgrading Tools. Here is a screenshot of the action configuration:

The script of the action is as follows:

task = null;

if (vm != null)
{
	var toolsconfig = new VcToolsConfigInfo();
	
	if (autoupgrade == true) {
		toolsconfig.toolsUpgradePolicy = "upgradeAtPowerCycle";
	} else {
		toolsconfig.toolsUpgradePolicy = "manual";
	}
	
	var vmconfig = new VcVirtualMachineConfigSpec();
	vmconfig.tools = toolsconfig;

	task = vm.reconfigVM_Task(vmconfig);
}

return task;

In the code snippet above, we ensure that the “vm” variable contains a valid VM. We create a new ToolsConfigInfo object which will contain the tools settings that we wish to apply and then create a new VirtualMachineConfigSpec object and assign the ToolsConfigInfo object to it. Thus when we call the reconfigVM_Task() function with the configuration object, the VM is only reconfigured with the settings we have provided.

Now we have an action, we can create a workflow that will invoke it and then wait for the task to complete. This is the schema diagram:

The workflow (and action) can be downloadedÂ from the file at the end of this post.

Cluster Workflow

Now we have an action to make the changes, we can put together a workflow that will look at all VMs in the cluster and configure them. The schema is as follows:

The workflow will get all VMs in the cluster and for each one:

Only upgrade Windows VMs (“Upgrade VM ?” decision)
Check if the configuration setting currently matches what we want it to. (“Decide setting” script).
Change the setting.

The meat of the process is in the “Decide setting” script item - we will look at what the setting should be, and if the current setting does not match then we will change it. This also means that if we wish the VM to update Tools manually and it is set to automatically update at power on, the setting will be changed to update manually.

Setting as a context action

To set this as a context action on the cluster within the Flex Web Client, we need to look at the Orchestrator pages in the Web Client. The server page will have a Manage screen and a Context Actions page:

Next we need to add a new one - select the workflow to add and the type of object to associate it with - in this case we’re adding the workflow to set the upgrade for Tools for a single virtual machine, so we select the Virtual Machine object:

Press OK and all is done. You’ll need to reload the Flex Web Client in order to see it - it appears at the bottom of the actions menu for the object - under the “vRealize Orchestrator context actions” title.

That’s it !Â This workflow a prime candidate for being scheduled to ensure all your VMs are kept up to date.

The workflows (and action) for this are available at my github repo - https://github.com/nelons/vco-content.

The monitor_control.restrict_backdoor setting

Mon, 20 Feb 2017 00:00:00 +0000

TheÂ monitor_control.restrict_backdoor advanced setting is a strange one that I’ve encountered recently. It was set on a VM that had been (allegedly) security hardened, and researching this setting in a search engine indicated that this was to prevent the backdoor port from being accessed from anything other than ring 0 (source: https://communities.vmware.com/thread/464535).Â Most of the search results utilise this to hide a guest OS from knowing it is a VM - thus allowing a nested ESXi configuration.

According to the source above, VMware Tools runs on ring 3, so this has the side effect of preventing not only VMware Tools from being installed, but also from running. This is how I came to find out about this setting ;)

This is the error message you’ll see if you try and install VMware Tools:

“The VMware Tools should only be installed inside a virtual machine.”

This is what you’ll see (whether VMware Tools is installed or not) in the management view (Note the “Not running”):

When this setting is removed from the VMX file, everything is awesome again !:

http://www.vmware.com/uk/security/hardening-guides.htmlÂ contains the latest hardening guides from VMware, and “monitor_control.restrict_backdoor” is not mentioned in them for any of the listed versions. I believe this information negates the argument that this setting should be applied for security reasons.

So without any other good reason I can think or find, this setting should not be applied on a VM, especially since it is not needed for nested ESXi (anymore).

Hope this helps someone !

vSphere REST API and C++ (Part 1)

Fri, 27 Jan 2017 00:00:00 +0000

One of the new features in vSphere 6.5 is the vSphere REST API. I’ve usedÂ the Web Services SDK before and it has never appeared as elegant as other automation APIs I’ve worked with, so the introduction of this held a lot of promise.

I rarely see examples for REST services in C++ and as it’s my chosen language and I’ve occasionally used “cpprestsdk” (previously known as Casablanca) I wondered if I could put something together. This is the first article in a series playing around with what is possible.

We can browse the API by connecting to our local vCenter via the URL https:///apiexplorer/ and see what we can do.Â An overview of the process of a session is:

Send a POST request to /rest/com/vmware/cis/session and respond to the basic authentication request.
The responseÂ contains an authentication token which represents the session id.
For future requests pass the token in the headers as the value “vmware-api-session-id”.

The SDK and the reference guide for the REST API are available atÂ https://developercenter.vmware.com/web/sdk/65/vsphere-automation-rest.

The Code

The aim of this article is to show the authentication process and make a sample request.

This makes the authentication request:

web::http::client::http_client_config config;
config.set_validate_certificates(false);
credentials cred(U("administrator@vsphere.local"), "password");
config.set_credentials(cred);

http_client client(vcenter_url.c_str(), config);
uri_builder builder(U("/rest/com/vmware/cis/session"));

std::wstring token = L"";
create_task(client.request(methods::POST, builder.to_string())).then([](http_response response) {
  printf("SSO Request Response Status Code: %d\n", response.status_code());
  return response.extract_json();

}).then([&token](web::json::value v) {

  if (v.has_field(U("value"))) {
    json::value f = v.at(U("value"));
    token = f.as_string();
    wprintf(L"SSO Token value: %s.\n", f.as_string().c_str());
  }

}).then([](task t) {
  try {
    t.get();

  }
  catch (const std::exception& e) {
    printf("Exception: %s\n", e.what());

  }
}).wait();

if (token.length() > 0) {
  printf("Successfully authenticated against SSO.\n");
}

Most of the code above are is about how we use the cpprestsdk. The basic authentication response is handled by creating a credentials object and assigning it the configuration for our http_client. We also disable SSL certificate verification as we are using self signed certificates. If you are writing something for a secure environment then it’s probably a good idea not to do this ;)

The request is made at line 10 and the task returns the result in a json format (line 12). We then look for an entry with the name “value” and make a note of what it is set to - this is our token. The last part of the concurrency task chain is for handling any exceptions.

The code below will is an example request to vCenter that will use the newly retrieved session token.

web::http::client::http_client_config vc_config;
vc_config.set_validate_certificates(false);
http_client vc_client(vcenter_url.c_str(), vc_config);

uri_builder vc_vm_uri(U("/rest/vcenter/host"));

http_request vc_req(methods::GET);
vc_req.headers().add(U("vmware-api-session-id"), token.c_str());
vc_req.set_request_uri(vc_vm_uri.to_string());

create_task(vc_client.request(vc_req)).then([](http_response response) {
  printf("/rest/vcenter/host Request Status Code: %d\n", response.status_code());
  return response.extract_string();

}).then([](utility::string_t text) {
  printf("The response was:\n");
  wprintf(text.c_str());
  printf("\n");

}).wait();

The code above will create a request and pass the token as the “vmware-api-session-id” value. As can be seen, we are still disabling SSL certificate verification but we are not passing credentials - the session id is enough.

The request will be sent to “/rest/vcenter/host” (line 5), which retrieves information about the hosts added to the vCenter. Once the request is made, the output is printed out to the screen in the JSON format.

We could add a final task segment to catch any exceptions that occur in the chain - there is no particular reason for why it is missing.

An Example

Below is a screenshot of the output of the program - I’ve blanked out the address and password of my home lab, though you can see that after a successful authentication information about two hosts is returned.

Full Source

The source is available on GitHub - https://github.com/nelons/vSphereRestApi/tree/master/part1. It was written in Visual Studio 2015 Community Edition and uses the cpprestsdk nuget package (and has UNICODE enabled).

It is not optimised or efficient code, but is meant to be straight forward to read and understand for near beginners (as some understanding of concurrency task code is required).

Comments, suggestions or errors - please let me know !

Setting VM SDRS automation levels

Thu, 11 Jun 2015 00:00:00 +0000

I learnt recently that when you Storage vMotion a VM within a datastore cluster and you check the box to select which datastore you explicitly want the VM to be migrated to, it sets the SDRS automation level for the VM to disabled.

Finding out which VMs have an automation level of disabled can be achieved with the following line:

`1`	`Get-DatastoreCluster \| % { $_.ExtensionData.PodStorageDrsEntry.StorageDrsConfig.VmConfig } \| ? { $_.Enabled -eq $False } \| % { Get-View $_.VM \| select Name }`

Since I like to have VMs set to the cluster default we need a bit of PowerCLI to change the VMs back. The following code will look at all the datastore clusters on the vCenter, and all the VMs with a disabled automation level therein.

Credit goes to LucD whose posts in the VMware Communities forum formed the base of this script. The threads are here and here.

This script is designed to be used with PowerCLI 6, as per the “Import-Module” statements at the top. For previous versions of PowerCLI the lines should be replaced with adding the appropriate snapin.

param([Parameter(Mandatory=$true)][String] $VIServer)

Import-Module VMware.VimAutomation.Core;
Import-Module VMware.VimAutomation.Storage;

Connect-VIServer $VIServer

$storMgr = Get-View StorageResourceManager;

Get-DatastoreCluster |
	% { $vms = $_.ExtensionData.PodStorageDrsEntry.StorageDrsConfig.VmConfig;
		$spec = New-Object VMware.Vim.StorageDrsConfigSpec;
		$vms | ? { $_.Enabled -eq $False } | % {
			$entry = New-Object VMware.Vim.StorageDrsVmConfigSpec;
			$entry.Operation = "edit";
			$entry.Info = $_;
			$entry.Info.Enabled = $true;
			$spec.vmConfigSpec += $entry;
		}
		
		$storMgr.ConfigureStorageDrsForPod($_.ExtensionData.MoRef, $spec, $true);
	}
	
Disconnect-VIServer -Confirm:$False

If needed, this script can easily be expanded to check for Tags or Custom Attributes assigned to the VM, and only to change the automation level given the presence or absence of a tag or the value of a custom attribute.

VMware vCheck NTP and Syslog Plugin Amendments

Tue, 09 Jun 2015 00:00:00 +0000

I’m a big fan of the vCheck script by the very talented Alan Renouf and community, but a couple of the plugins don’t work as well as I’d like with our environment, where we have multiple NTP and Syslog servers and these settings can differ depending on the cluster within a single vCenter. So I’ve made a few changes to the vCheck NTP and Syslog plugins.

NTP Plugin

The file is called “31 NTP Name and Service.ps1” and kept in the “30 Host” folder.

I’m going to cut out the bottom part of the plugin script as there would only be cosmetic changes made there.

# Start of Settings
# The NTP server which should be set on your hosts
$ntpserver = "ntp-server1.cluster1 ntp-server2.cluster1", "ntp-server1.cluster2"
# End of Settings

$Result = @($VMH | Where {$_.Connectionstate -ne "Disconnected"} | Select Name, @{N="NTPServer";E={"$($_ | Get-VMHostNtpServer)"}}, @{N="ServiceRunning";E={(Get-VmHostService -VMHost $_ | Where-Object {$_.key -eq "ntpd"}).Running}} | Where {$_.ServiceRunning -eq $false -o $ntpserver -notcontains $_.NTPServer})
$Result

There are two changes here:

The line “$($_ | Get-VMHostNtpServer)” has been enclosed in quotation marks to expand the array into a string.
The check at the end has been changed so we are now checking if the NTP server values are in the provided array.

I’m fairly sure the list of NTP servers are ordered (but have been known to be wrong before), so I’m leaving the list of servers in the order that PowerCLI decides to return them. So when specifying a list of NTP servers, do it in the order they should be used/configured.

Syslog Servers

The syslog plugin file is called “51 Syslog Name.ps1” and it is also in the “30 Host” folder.

I don’t believe that the syslog hosts are ordered (they certainly shouldn’t be), so in the code below we’re looking at the list of syslog hosts sorted alphabetically.

The code is now:

# Start of Settings 
# The Syslog server which should be set on your hosts
# Multiple syslog entries per host must be space separated and in alphabetical order.
# Multiple possible entries per vCenter should be comma separated so an array is created.
$SyslogServer = "udp://syslog-server1 udp://yet-another-syslog-server"
# End of Settings

@($VMH | Where-Object {$_.ExtensionData.Summary.Config.Product.Name -eq 'VMware ESXi'} | Select-Object Name,@{Name='SyslogServer';Expression = {"$(($_ | Get-VMHostSysLogServer).Host | sort Host)"}} | Where-Object {$SyslogServer -notcontains $_.SyslogServer})

There’s a few changes:

The ‘SyslogSettings’ section, supporting ESXi 4.x has been removed.
The line “$(($_ | Get-VMHostSysLogServer).Host | sort Host)” sorts the names and expands the array into a string.
The comparison check at the end now checks for presence in the array.

Summary

This is very much a lazy fix to a problem I had, and there will be a better solution. For instance, it’s not going to check that hosts in a certain cluster have particular settings - just that all hosts have a valid setting permissable for the vCenter. But it’ll do the job, and for everything else there should be host profiles - and there is vCheck plugin that can be used unaltered to check on those ;)

Viewing VM CPU Masks via PowerCLI

Fri, 05 Jun 2015 00:00:00 +0000

This post was updated on 9th May 2017 to fix an error condition in the script.

Support for Windows Server 2012 was announced starting with ESXi 5.0 Update 2, but there was a little known flaw in this support - in that without a CPU mask, a VM could blue screen at any time. The support article KB2060019 was released which explained everything and the issue was fixed for Update 3.

It might be useful to try and track down which 2012 VM’s (it can be adjusted for Windows 8) had their CPU masked during this time, in order to remove. The following PowerCLI reports on all Windows 2012 (and R2) VM’s and displays any CPU masking they might have. You’ll then have to manually check the results and look for a 0 in the 5th most significant bit on the edx register, but I’m sure this could be automated further.

get-vm | ? { $_.ExtensionData.Config.GuestFullName -like "*2012*" -And $_.ExtensionData.Config.cpuFeatureMask -ne $null } | select Name, @{N="CPU";E={$_.ExtensionData.Config.cpuFeatureMask}} | select -Property Name -ExpandProperty CPU | Sort Name | fl

Installing ESXi 6 onto USB via Workstation 9

Sat, 28 Mar 2015 00:00:00 +0000

This is going to one of those blog posts that is mostly pictures ;) When installing ESXi 6 I decided to install onto (and therefore boot from) USB instead of installing and booting from local disk. This would allow my SSDs to be used for playing with Flash Cache. I had Workstation 9 available to me, so tried to use it (spoiler alert: it worked).

To begin, create a new Virtual Machine with the latest compatibility level.

Select to install VMware ESXi 5.

It needs to have a minimum of 2 processors.

As per ESXi 5.5 it requires 4 GB of RAM.

Workstation 9 forces you to configure a drive. Add a small drive and delete afterwards.

Follow the wizard through to the end to create the VM. Then edit the settings and delete the hard disk. I don’t know if it is important, but I made sure the VM had the same number of ethernet adapters as the target host.

We can now boot the VM and go through the standard installation procedure. Select the USB drive to install from, though make sure it is larger than 910 MB:

If we are installing to another usb drive (for another host) after this one completes we need to change the MAC address of the VM’s network adapter, or we will create another installation with the same MAC address for the management adapter. Which will make one host unusable. Go to the Virtual Machine settings, select the Network Adapter and then press the Advanced button.

Press the Generate button to create a new MAC address.

We’re now set to install ESXi again ;)

vSphere Web Client and SSH Warnings

Fri, 20 Mar 2015 00:00:00 +0000

I started using the vSphere C# client back when the product was called Virtual Infrastructure 3, and despite its many flaws it has become second nature to me. I’ve used the Web Client in 5.0 and 5.5 but never really gotten into it (for many reasons, speed being the main one).

I typically use Host Profiles to configure new hosts through the C# client, so have never been through the process manually with the Web Client before. Starting with 6.0 I decided to try that and found a pleasant surprise to the usual yellow warnings I had seen previously.

I always enable SSH as when there are problems I like to be able to see esxtop and get information first hand from hosts, and I typically enable SSH via the DCUI after configuring the management network settings. Following this I’d join the host to the cluster and apply the appropriate host profile (containing the advanced setting “UserVars.SuppressShellWarning”), so the warning that SSH is enabled is purely transitory.

When I join the host to the cluster in the Web Client, I see the warning icon against the host and select the host to see what it is:

The “Suppress Warning” link caught my eye, and when clicking on it I got this message:

After that the warning disappeared ! Looking in the Advanced Settings we can see that this sets the value for the “UserVars.SuppressShellWarning” setting to 1.

I know that this quality of life improvement was also present in the 5.5 version of the Web Client (and maybe before) but it’s a nice little touch for those small deployments where you can clear the alerts on your host with a couple of clicks instead of having to lookup the setting. Especially when you’re not licensed for host profiles !

Unmounted NFS datastore after ESXi boots

Fri, 17 Jan 2014 00:00:00 +0000

In my home lab I recently found a problem where an NFS share I had mounted on my ESXi 5.5 device was unmounted after it had been turned on/restarted. My home lab is not that unusual, I have a N54L HP Microserver for shared storage - where I am using ZFS to carve up my 4x1TB disks - and a couple of Intel NUC’s for compute. All of this is joined together by a Cisco SG300-10 gigabit switch/router. I’ve been putting together some articles for this blog about a home lab and was curious as to the performance difference between running Windows Server 2012 Standard and CentOS 6.5 with ZFS for the shared storage. So I’ve been adding and deleting a lot of NFS and ISCSI datastores on my ESXi hosts. But since moving to CentOS to try out ZFS I’ve had a problem where one of my two compute devices will refuse to mount any shared VM exports. It will mount my ISO repository (which has slightly different permissions to allow anonymous SMB access), but nothing else. My other ESXi unit will mount everything.

So, knowing that one device has no problems, the permissions difference between the ISO repository and the shared vm’s can be eliminated. We can also discount the NFS permissions for root and read/write access as I am able to manually mount and use the datastores, it’s only on boot they are unmounted:

First thing I tried was looking at the /var/log/messages log on the storage. This is what I would see:

The ESXi device is not attempting to mount the share - confirming there is no problem with the storage server. So, turning to a recent purchase “Troubleshooting vSphere Storage” by Mike Preston (it was on offer at the start of the year and is a very good read), the section about troubleshooting NFS datastores tells us to look in the hostd logs. Let’s take a look:

“lb-fs-01” was the name of my Windows 2012 server (and is the hostname of the current storage server), but it’s not in DNS. In addition, all the datastores I’ve set up with the CentOS box have been defined by IP address:

So, it is strange that ESXi is trying to resolve “lb-fs-01”. For the answer, we’re going to need to look at the configuration - /etc/vmware/esx.conf. This is the point in the article where I wish I had a screenshot of the offending entries but I didn’t realise that ESXi purges /etc/vmware of all additional files on boot. So, sadly, there is no picture of what was causing the problem.

To find out what NFS storage entries your ESXi has, type the following from the console/ssh:

/etc/vmware # cat esx.conf | grep "/nas/"

This will display all your NFS datastore entries (and a few other lines). A working NFS entry looks like this:

There are four lines to define the storage - however there were only lines pertaining to lb-fs-01 and I’m only sure of “host” and “share” being present. I would imagine “enabled” would be the line that was missing.

Unlike me, you should take a backup of the file via WinSCP or FastSCP and not take a copy to the local file system. Once you’ve got a backup, delete the lines for the non-existent export/datastore and restart ("# reboot"). When the host came back up, I added one shared VM datastore, restarted again and whilst booting this is what I saw on the storage server:

Success :). After logging into the vSphere Client, I could confirm:

I have no idea why those entries where still in my esx.conf file. I deleted unused and non-existent datastores through the vSphere Client in the normal way and don’t recall any errors. It’s very strange as to how or why this occured.

That being said there is a main lesson - when backing up from ESXi, use WinSCP or FastSCP :)

Listing Virtual Machines and Portgroups

Mon, 20 May 2013 00:00:00 +0000

vSphere 5.0 - If you need a list of Virtual Machines and the portgroups they are attached to:

`1`	`Get-VM -Location "somewhere" \| ft Name, @{Expression={$_.NetworkAdapters.NetworkName};Label="Network Name"}`

If you need to export this to a CSV:

`1`	`Get-VM -Location "somewhere" \| select-object Name, @{Expression={$_.NetworkAdapters.NetworkName};Label="Network Name"} \| export-csv "vm-networks.csv"`

Getting a vSphere VM's Network Adapters

Tue, 26 Mar 2013 00:00:00 +0000

Getting all the network adapters for a VM is an easy task:

`1`	`get-networkadapter -vm [vm name]`

When you remove a network adapter, vSphere will remove it’s entry in the .vmx file. In the .vmx file, the adapter configuration options are prepended by an “ethernetX” string, where X is a number between 0 and 9. Removing the adapter, removes all the appropriate “ethernetX” entries from the file. Adding an adapter to a VM will use the first available number, so if in a 5 NIC VM “ethernet2” is not present, this is where the adapter will be placed. This has the effect of calling the NIC “Network Adapter 3”.

Why does this matter ? Because we want to get our adapters in a reliable order, regardless of the number of NICs present.

`1`	`get-networkadapter -vm [vm name] \| sort Name`

The above line will work for all VM’s with less than 10 NICs. When we have 10 NICs we need to do a little bit of regex on the sort code:

`1`	`get-networkadapter -vm [vm name] \| sort { if ($_.Name -match '(\d+)') { [int] $matches[1] } }`

This strips out the number from the name of the adapter, converts it to an int and sorts based upon that.

This was all done with PowerCLI 5.0.0.3501.

Sidenote: I really need to find a decent plugin for code highlighting :-/

Swapfile Datastores

Wed, 16 Jan 2013 00:00:00 +0000

If you have more than one swap file datastore for your hosts, then you might want to see which hosts are connected to what datastores.

`1`	`get-vmhost -location "<cluster/datacenter name>" \| sort-object Name \| ft Name, VMSwapfileDatastore`

Quick PowerCLI #1

Mon, 17 Dec 2012 00:00:00 +0000

I may or may not make this a series ;) PowerShell with VMware is very useful. Ignoring the Connect-VIServer and Disconnect-VIServer commands, here are some tips.

Getting the Swap File Datastore for all hosts in a cluster

If you store VM swap files away from the VM folder (faster disk I would hope) and have more than one datastore for doing this then you’ll want to separate the datastores used for each host.

`1`	`get-vmhost -location "(cluster name)" \| sort Name \| ft Name, VMSwapfileDatastore`

Getting the Tools Version for a VM

I prefer the first way because it’s slightly less to type ;). With get-vm (the second query) you can target containers that you can’t with get-vmguest. So it’s all a matter of choice !

1
2

get-vmguest "(name/match query)" | ft VM, @{Name="ToolsVersion";Expression={$_.ExtensionData.ToolsVersion}}
get-vm "(name/match query)" | ft Name, @{Name="ToolsVersion";Expression={$_.ExtensionData.Config.Tools.ToolsVersion}}

VMware Orchestrator workflow failing

Mon, 20 Aug 2012 00:00:00 +0000

Had an interesting issue the other week involving VMware Orchestrator. Workflows have been set up to use Configuration elements, and one of the attributes in the element was a VC:VmFolder. This would be set, but the workflow would fail, and when I went back to look at the attribute it was “not set”.

This had started happening after upgrading to vCenter 5, which includes Orchestrator by default. There was a relevant error in the logs:

ERROR
\[VimSession\]
getOrCreateVimObject() –> Cannot create com.vmware.vmo.plugin.vi4.model.VimVmFolder

Luckily the first link in a Google Search found the answer - - this was that the default vCenter plugin for vCO has an issue and needed to be updated.

After updating, the scheduled workflows complete without any problems … but the logs do still say “vi4” …

VCP5

Sat, 03 Mar 2012 00:00:00 +0000

I passed the VMware VCP510 exam earlier this week, so am now waiting on VMware to confirm it

\[and send the Workstation 8 license !\]

. It was a tough exam, seemed to be a fair test of experience with the product and the time allocated was balanced to the questions quite well.

Many sites offer advice for revision (read the VCP510 blueprint !) and what I found most useful were various mock exams on other sites (including VMware’s own) and Forbes Guthries’ vSphere 5 Reference Card.