The monitor_control.restrict_backdoor advanced setting is a strange one that I’ve encountered recently. It was set on a VM that had been (allegedly) security hardened, and researching this setting in a search engine indicated that this was to prevent the backdoor port from being accessed from anything other than ring 0 (source: https://communities.vmware.com/thread/464535). Most of the search results utilise this to hide a guest OS from knowing it is a VM – thus allowing a nested ESXi configuration.
According to the source above, VMware Tools runs on ring 3, so this has the side effect of preventing not only VMware Tools from being installed, but also from running. This is how I came to find out about this setting 😉
This is the error message you’ll see if you try and install VMware Tools:
“The VMware Tools should only be installed inside a virtual machine.”
This is what you’ll see (whether VMware Tools is installed or not) in the management view (Note the “Not running”):
When this setting is removed from the VMX file, everything is awesome again !:
http://www.vmware.com/uk/security/hardening-guides.html contains the latest hardening guides from VMware, and “monitor_control.restrict_backdoor” is not mentioned in them for any of the listed versions. I believe this information negates the argument that this setting should be applied for security reasons.
So without any other good reason I can think or find, this setting should not be applied on a VM, especially since it is not needed for nested ESXi (anymore).
Hope this helps someone !
