If you have vCenter Server (Appliance) you have full licensing and can use the confusing update manager built-in.  However, this doesn’t automatically schedule updates of the servers.  To properly update the servers they should have each VM shut down and the host should be in maintenance mode.  I haven’t investigated yet whether this can be automated.  I just do it manually every so often.

The process:

1. Log into your ESXi host using whatever your preferred method is
2. Gracefully shut down all your VM’s
3. Place the host server into the management mode
4. Connect to the host with SSH (you may need to turn this on as it’s off by default) and log in as root user
5. Open a web browser to:  VMware ESXi Patch Tracker

At this point you need to determine which version you currently have in order to know what you’re updating to.

From the SSH command prompt:

vmware -vl

You should see a response like this:

Obviously I have already updated my host to 6.5 but the command works on any ESXi host from version 4 and up.

In the past I have always gone from 5.x to 6.0 to 6.5 standard (GA release) then patched 6.5 to whatever the current level is.  However, it turns out this extra step isn’t actually necessary – I jumped from 5.5 directly to 6.5 GA without incident.*

One other thing I have found is the SWAP memory settings are often lost in the conversion.  I’ll detail how to update that below.

Ok – so now you are ready to update.  From the SSH command prompt, enter the following:

esxcli network firewall ruleset set -e true -r httpClient
esxcli software profile update -p ESXi-6.5.0-4564106-standard \
-d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml

After a while it will seem like nothing is happening but in fact the ISO of the build you specified is being downloaded and installed.  Eventually you’ll see this:

Update Result

Message: The update completed successfully, but the system needs to be rebooted for the changes to take effect
Reboot Required:  true

Type “reboot” at the prompt and give the system a few minutes to reboot.

I always start with the GA initial release because as VMware updates the builds they insert new drivers and other code that may not be included with every build release.  I review each build release on the patches page (link above) to see what exactly each build is updating.  If I see a release where only the esx-base, vsan, vsanhealth is updated, I skip the build until I see things like ehci-ehci-hcd, esx-ui, esx-tboot, misc-drivers, etc…  This ensures that I am getting everything included to the host server as I go along.  This could mean several installs and reboots, but it’s better to be thorough than try to figure out why your host isn’t behaving as expected with a NIC driver, or RAID array management.

The syntax for the update is always the same:

esxcli software profile update -p <<build goes here>> -d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml

When done you can optionally turn the firewall back on to restrict httpClient access from the host:

esxcli network firewall ruleset set -e false -r httpClient

When all updates are completed, log into the UI now provided by the host (https://esxihost/ui) and exit maintenance mode.  I usually just reboot to stat up the VM’s but if you don’t have auto-start enabled you’ll have to start up each VM separately.

The fun doesn’t stop there.  Once the host has been fully updated all of the VM’s will need their VMware Tools installs updated.  If you edit the VM and set the Tools to upgrade automatically it *should* just work.  The old Tools install should upgrade and force a reboot of the VM.  This usually requires some patience on your part because it takes a while for each VM to upgrade.

I have found that the tools don’t always upgrade though, even if set to automatically upgrade.  One common issue is no CD-DVD ROM virtual drive has been installed (or was removed for some reason).  Without it, the ISO image of the VMware Tools software cannot mount and therefore cannot be run.  Check each VM to ensure you have a working CD-DVD ROM set to connect on startup.  

Another item to note is that the Virtual Machine Hardware Version doesn’t automatically update with each VM either.  This also must be done manually and you should pay special care to ensure the VMware Virtual Machine Hardware Version is compatible with your VM.  I haven’t yet found an instance where the version had to be kept back to a old version but do your homework.  The version of the drivers (video, network, drives, audio, etc.) depends on it.

To update the Hardware Version log into the UI and click on the VM you want to update.  Make sure it’s powered down, then click on Actions and click on Upgrade VM Compatibility.

Select the appropriate version for your VM and click on Upgrade.

This will cause the VMware Tools to go through the process of upgrading again, which in turn will cause your VM to reboot again.  If you know in advance what you want to set the compatibility to, do this while the host server is still in maintenance mode and before you power up each VM for the first time after the host updates are completed (which is also a good time to check the status of your CD-DVD ROM for each machine).

*NOTE – If you jump from 5.x to 6.x you MUST obtain a new license. If it’s a free license just register again. If not you will need to use the VMware license manager to upgrade your licensing to the current version (runs a conversion process).  You need to log into https://my.vmware.com to do this.  Don’t forget to update the license!

The post VMware – Updating the ESXi host and VM’s appeared first on Wiredwolf Canada.

For example, I recently built a server with 6 NIC’s.  The default configuration assigned all six physical network adapters to “VM Network” which included the Management interface.  Well, I figured since I have this many NIC’s in a server, with a limited number of VM’s, I could afford to set two dedicated Management NIC’s and have four to service the VM’s.

After a lot of exploring I discovered that removing “uplinks” to the pre-existing virtual switch was simple enough.  I created a new Port Group and Virtual Switch easily enough.  But then adding those newly released NIC’s to the new Virtual Switch I created turned into a nightmare.  Only through sheer perseverance did I determine that the “Add Uplink” button provided in the list view of the Virtual Switches is completely broken.  You have to open the Virtual Switch itself, then edit it’s properties, whereupon you find another “Add Uplink” button.  This button works. 

Another issue I found was when creating a new VM, the default hardware model (Compatibility) is set to ESXi 6.5 in the menu drop-down, but the other two selection (Guest OS family and Guest OS version) remain blank.  Until you switch Compatibility to something else, then back, you cannot select either of these values.  And until you do select those values, you cannot create a VM. 

Perhaps it’s just me, or my browser, because I tried to duplicate these issues on a client site and failed to do so. 

The post VMware ESXi 6.5 – Web Interface Woes appeared first on Wiredwolf Canada.

For some reason, the ESXi host always wants to assign the Intel E1000 virtual NIC.  Yet if you do any reading on this, you’ll quickly find that actually using this driver is frowned upon. Exactly why, I can’t remember.  Just don’t use it.

Instead you are supposed to select the VMXNet 3 driver.  This is a fully feature-packed driver and in my experience it does operate very well in Windows environments… but not without some tweaking.

The biggest issue seems to be related to some of the Offload features that are always on by default:

• IPv4 Checksum Offload – set to Disable
• Large Send Offload V2 (IPv4) – set to Disable
• Large Send Offload V2 (IPv6) – set to Disable
• Receive Side Scaling – set to Disable
• Recv Segment Coalescing (IPv4) – set to Disable
• Recv Segment Coalescing (IPv6) – set to Disable
• TCP Checksum Offload (IPv4) – set to Disable
• TCP Checksum Offload (IPv6) – set to Disable
• UPD Checksum Offload (IPv4) – set to Disable
• UDP Checksum Offload (IPv4) – set to Disable

The other item that can cause issues is the Receive Side Scaling.  Based on what I read, Receive Side Scaling is for when you want to distribute your network connectivity CPU load across all CPU cores.  In theory this should work great, but in practice I have found it doesn’t work very well.  I always turn it off.

I have found not all features for the VMXNET3 adapter are present on older Windows Server installations.  Server 2012 – 2016 have the Receive Side Scaling and Recv Segment Coalescing features where Server 2008 R2 doesn’t.

There are other Offload features that you shouldn’t touch or should ensure they are turned on:

• Offload IP Options
• Offload Tagged traffic
• Offload TCP Options

Particularly with Active Directory, some of these features are actually required for the server to process correctly.

Hope this helps someone else out there who’s struggled constantly with ESXi network efficiency issues. 

• edited Thursday June 14, 2018 (missing Recv Segment Coalescing and Receive Side Scaling) – encountered abysmal network connectivity on two separate host server VM’s because I missed this the first time around – cost me many hours of painful file copying until I resolved it (again)

The post VMware ESXi – Network Interfaces appeared first on Wiredwolf Canada.

When you pay for VMware vCenter Server you automatically get a Web Console UI that appears to be replacing the older VMware vSphere Client.  But when you use the free ESXi Host Hypervisor you don’t get the Web Console UI by default.

There is a very simple way to add the Web Console UI:

• SSH to the host (make sure SSH is enabled)
• Log in as ROOT
• run this simple command:

esxcli software vib install -v http://download3.vmware.com/software/vmw-tools/esxui/esxui_signed.vib

Wait for the install to complete and you are given the message:  Operation finished successfully.

Reboot is not required.

Now using the IP of the host, navigate your favorite browser to:  https://hostname_or_IP_of_host/ui

Log in as you would with the client.  The interface is apparently somewhat crippled due to the free license but as near as I can tell you can still do pretty much everything you could with the client without having to actually install the client. 

As a bonus, now that you have a Web Console UI, you can set up firewall rules for remote management.  This isn’t recommended unless you know how to properly secure the connection but it is feasible.

The post VMware ESXi – Web Console appeared first on Wiredwolf Canada.

VMware really could try to make this easier.

The first steps is to download the software you need.
1. MegaRAID Storage Manager – here
2. Latest SMIS Provider (VMware 5.5 & 6.0) – here

Basically it’s the same place.  When I did the search I got 527 results.  You have to sift through to find what you’re looking for.

The install of the MegaRAID Storage Manager on Windows is standard.  The only difference is you configure the discovery to pick out ESXi hosts specifically.

For the ESXi host to respond to requests this is where the SMIS Provider has to be installed.  And this can be the tricky part.

I suggest creating a folder in your datastore that you can keep stuff like this.  Call it whatever you like as long as you can identify this folder from among your VM’s.  Such as ‘drivers’.

Upload the file you got from Broadcom to this folder using WinSCP then open an SSH session (I use Putty).  Log in as root.

This is where I got a lot of push-back from ESXi.  The command is pretty straightforward:

esxcli software vib install -d /path/to/filename.zip

However, I kept getting errors about the filename.zip?index.xml is missing.

1. The path to the filename must be exact:  /vmfs/volumes/datastore1/drivers//VMW-ESX-5.5.0
   -lsiprovider-500.04.V0.61-0007-offline_bundle-4555634.zip
2. Sometimes ‘esxcli’ will fail no matter what you try – switch to ‘localcli’

I actually unpacked the original ZIP file so I didn’t have to try to remember (or type out) these ridiculously long file names but you can skip unzipping the files by separating the target with a ‘/’.

esxcli software vib install -d /vmfs/volumes/datastore1/VMW-ESX-5.5.0-lsiprovider-500.04.V0.61-0007-4
555634.zip/VMW-ESX-5.5.0-lsiprovider-500.04.V0.61-0007-offline_bundle-4555634.zip

You should put the ESXi host in maintenance mode prior to installing the SMIS Provider but I haven’t found that it’s strictly necessary.  A reboot after the install is necessary though.

Once the VIB is installed you can then install the MegaRAID Storage Manager to your Windows Server.  It should be a server on the ESXi host you want to link to the controller with but I found that’s not strictly necessary.

First update your hosts file:

notepad c:\windows\system32\drivers\etc\hosts

Add an entry that looks like this:  192.168.1.10 esxi-host.local esxi-host  (obviously substitute with the IP and name of your ESXi host)

Open the MegaRAID Storage Manager and click on Configure Host
Select “Display all the ESXi-CIMOM server in the network of local server.”
Click on Save Settings

Click on Discover Host – your ESXi host server IP shold now show up showing an Unknown operating system and health status of “Needs Login”.
Select your host server and use your VMware (root or equivalent) credentials to log in.

Be patient.  I found this rarely loads in under a minute.  A window will open and a big circle will swirl around for quite a while before the controller is contacted and all the relevant data is pumped back into the browser.

The post Installing Offline Bundles appeared first on Wiredwolf Canada.

Yes this happens.  Typically on networks where the VM in question is the DC and the only DNS source.  My agents drop offline because they cannot resolve anything and can no longer find my server.  If they can’t connect to me – I can’t connect to them.

Anyway, as a back door I usually do enable SSH to the ESXi host (I know it’s not recommended) and connect that way to get done what I need.

Once connected to the ESXi host via SSH:

esxcli vm process list

This command lists all of the VM’s running.  In particular I am interested in the World ID.

Once I have the World ID:

esxcli vm process kill -t [soft|hard|force] -w WorldID

Give it a few seconds then check the state of the VM by rerunning the process list command again. The VM should no longer be showing.

Now you may need to power up the VM.  You’ll need the vmid to power on the system.

vim-cmd vmsvc/getallvms

The VM’s will all be listed and have a number such as “1” or “2”:

Vmid               Name                                                    File                                             Guest OS         Version                                                                                                                                                                                                                                                                                                                                                                             Annotation
1      SERVER1-DC                   [datastore2] SERVER1-DC/SERVER1-DC.vmx                                   windows8Server64Guest   vmx-08
2      SERVER2-SQL                  [datastore2] SERVER2-SQL/SERVER2-SQL.vmx                                 windows8Server64Guest   vmx-08

Now get the power state of the VM:

vim-cmd vmsvc/power.getstate <vmid>

You’ll see a response like this:

~ # vim-cmd vmsvc/power.getstate 1
Retrieved runtime info
Powered off

Power on the VM:

vim-cmd vmsvc/power.on <vmid>

Check the power state again.  It should show as “Powered on”.

The post VMware ESXi – Shut Down Hung VM appeared first on Wiredwolf Canada.