ESX Regenerate Self-Signed Certificate - 夜莺博客

ESX Regenerate Self-Signed Certificate

原文:ESX Regenerate Self-Signed Certificate — theDXT (Daniel Keer)

During the installation of VMware ESX, you’re never prompted to set a hostname or domain name. As a result, ESX defaults to the hostname localhost and the domain localdomain. Due to this when ESXi generates a self-signed certificate, it is for localhost.localdomain.

In this post, I will show you step by step how to regenerate the self-signed certificate for ESX.

Why the ESXi Certificate Matters

The certificate is not cosmetic. ESXi uses it to terminate HTTPS for the Host Client, to identify itself to vCenter, and to secure the reverse proxy that fronts the whole management stack. When the subject name does not match how you actually reach the host, you get a steady stream of consequences:

  • Browsers refuse to load the Host Client, or require a manual exception on every session.
  • esxcli and PowerCLI sessions fail when certificate verification is enabled.
  • vCenter shows the host with a certificate warning in the cluster summary, and some workflows — certificate-based authentication, for example — will not work at all.
  • Backup, monitoring and automation tooling that validates TLS thumbprints breaks or must be reconfigured.

The most common trigger is exactly the one in the opening paragraph: the host was renamed after installation, so the certificate still says localhost.localdomain while the host answers on esxi01.lab.local. Regenerating is a two-minute fix that removes an irritating, recurring problem.

Understanding the ESXi Certificate Files

Before touching anything, it helps to know what you are rewriting. All of the Host Client TLS material lives in a single directory:

/etc/vmware/ssl/rui.crt    # the public certificate
/etc/vmware/ssl/rui.key    # the matching private key

rui stands for Remote User Interface — the original name for what is now the ESXi Host Client. Several services read these two files:

  • hostd — the main management daemon, which serves the Host Client and the API.
  • vpxa — the vCenter agent, the process that lets vCenter manage the host.
  • rhttpproxy — the Envoy-based reverse proxy that sits in front of everything on port 443 and 902.

Every one of them caches the certificate at start-up, which is the single most important fact about this procedure and the reason the official documentation is incomplete: regenerating the files on disk changes nothing until each of those services is restarted. That is the step most people miss, and the reason they conclude the procedure "did not work".

Prerequisites

  • The ESXi hostname has been changed.

For more information about changing the ESXi hostname, my blog post, ESXi changing the host name goes into detail.

Beyond the hostname, gather these before you start:

  • DNS and reverse DNS resolving correctly for the new hostname. A certificate for a name nothing can resolve is no better than the old one.
  • SSH access enabled and a maintenance window agreed. The service restarts interrupt the management plane for a few seconds; running virtual machines are unaffected, but vCenter will briefly lose contact with the host.
  • A current backup of the host configuration (or at least a copy of rui.crt and rui.key) so you can roll back if the new certificate is issued with a name you did not intend.
  • Time and date synchronised (NTP). A certificate generated with a badly skewed system clock is rejected outright by validating clients.

The Process

  • Connect to your ESXi host.
  • Right-click on the ESXi host and select Services > Enable Secure Shell (SSH).

Image 2

  • SSH into the ESXi host.
  • To view the current certificate installed on your ESXi host, run the following command openssl x509 -noout -subject -in /etc/vmware/ssl/rui.crt

Image 3

If you want to view all the details about your ESXi certificate, run the following command openssl x509 -noout -in /etc/vmware/ssl/rui.crt -text

Image 4

If your output still shows CN = localhost.localdomain, you have confirmed the problem. Take a note of the dates while you are here — an expired certificate produces a different error path than a name mismatch, even though the regeneration fix is the same one.

  • Run the following command to regenerate the ESXi self-signed certificate /sbin/generate-certificates

Image 5

generate-certificates reads the host's fully qualified domain name and issues a brand-new 2048-bit key pair and self-signed certificate, overwriting rui.crt and rui.key. There are no prompts and no confirmation — if the hostname is wrong at this moment, you have just issued another wrong certificate and will need to fix the hostname and run it again.

  • Restart the ESXi management agents by running the following command /etc/init.d/hostd restart && /etc/init.d/vpxa restart

Image 6

Although the official documentation states the process is complete, this is not entirely true. ESXi runs the Envoy proxy Discovery Service, also known as rhttpproxy, which is a reverse proxy for ESXi. To complete the certificate change, restart the rhttpproxy service or reboot the ESXi host.

  • Restart the ESXi Envoy reverse proxy by running the following command /etc/init.d/rhttpproxy restart

Image 7

Because rhttpproxy is what actually accepts the connection on port 443, skipping it means your browser keeps being served the old certificate even though the file on disk is new. Restart all three services, or reboot the host — there is no half-way option.

Verifying the New Certificate

  • To view your new certificate, run the following command openssl x509 -noout -subject -in /etc/vmware/ssl/rui.crt

Image 8

The subject should now show your hostname instead of localhost.localdomain. Confirm the rest of the chain of evidence with a few more checks:

# full details of the live file on disk
openssl x509 -noout -in /etc/vmware/ssl/rui.crt -text

# what the reverse proxy is actually presenting on the wire
openssl s_client -connect esxi01.lab.local:443 -showcerts < /dev/null | openssl x509 -noout -subject -dates

# service state
/etc/init.d/hostd status
/etc/init.d/rhttpproxy status

The second command is the one that settles the argument. Checking rui.crt on disk only proves the file was written; querying port 443 proves the running proxy has picked it up. If those two disagree, the proxy has not been restarted.

Then open the Host Client in a fresh browser session (clear the TLS state first — browsers cache certificate decisions aggressively), re-connect the host in vCenter, and confirm the certificate warning has gone. In vCenter, right-click the host and use Connection > Refresh Certificates so the inventory is not still holding a cached thumbprint.

Common Problems and Fixes

Host Client still shows the old certificate. Almost always the missing rhttpproxy restart, or a browser that cached the old certificate decision. Try a private window before assuming anything else.

The certificate name is still wrong. The hostname was not fully qualified when generate-certificates ran. Fix the FQDN, then run /sbin/generate-certificates again followed by the three service restarts.

vCenter shows the host as disconnected. Give it a minute — vpxa restarting means vCenter loses its agent briefly, and reconnection is automatic. If it persists, check /var/log/vpxa.log and confirm DNS resolution of the new hostname from the vCenter server itself.

The certificate expired rather than the name being wrong. Same fix. Self-signed ESXi certificates carry a long but finite lifetime, and a rebuilt host that was never renamed will still one day present an expired certificate.

Automation broke after the change. Anything pinning the old thumbprint — a script, a monitoring check, a backup product — needs updating. This is expected and is usually the price of the fix.

Replacing the Self-Signed Certificate with a CA-Signed One

If the environment has an internal CA or a public certificate is required, generate-certificates is only the first half of the job. The supported path is to generate a CSR, sign it, and install the result:

# create a key and certificate signing request
openssl req -new -newkey rsa:2048 -nodes \
  -keyout /etc/vmware/ssl/rui.key \
  -out /tmp/rui.csr \
  -subj "/CN=esxi01.lab.local"

# view the CSR before sending it to your CA
openssl req -noout -text -in /tmp/rui.csr

# install the signed certificate returned by the CA
cp /tmp/esxi01.crt /etc/vmware/ssl/rui.crt

# restart the full stack and verify on the wire
/etc/init.d/hostd restart && /etc/init.d/vpxa restart
/etc/init.d/rhttpproxy restart
openssl s_client -connect esxi01.lab.local:443 -showcerts < /dev/null | openssl x509 -noout -subject -issuer -dates

Keep the key file permissions as they were — rui.key should be readable only by root, since a leaked private key means anyone can impersonate the host's management interface.

Rollback and Version Notes

If the new certificate turns out to be wrong — a typo in the hostname, the wrong domain suffix, or a change that broke more than it fixed — the rollback is simply to restore the previous pair of files and restart the same three services. This is why taking a copy of rui.crt and rui.key before running generate-certificates is worth the ten seconds:

# before the change
cp /etc/vmware/ssl/rui.crt /tmp/rui.crt.bak
cp /etc/vmware/ssl/rui.key /tmp/rui.key.bak

# to roll back
cp /tmp/rui.crt.bak /etc/vmware/ssl/rui.crt
cp /tmp/rui.key.bak /etc/vmware/ssl/rui.key
/sbin/generate-certificates --help   # confirm the binary still behaves as expected
/etc/init.d/hostd restart && /etc/init.d/vpxa restart
/etc/init.d/rhttpproxy restart

The same procedure applies on ESXi 6.x, 7.x and 8.x. What changed across those releases is the surrounding tooling, not the certificate files: ESXi 7 and later tightened TLS protocol support and removed the old Flash-based client entirely, so the certificate now protects the Host Client and the API and nothing else. On ESXi 8, vCenter additionally validates the host thumbprint during add host, which is why a host with a stale localhost.localdomain certificate can refuse to be added at all rather than merely producing a warning.

Finally, if the environment is being rebuilt anyway, setting the hostname correctly before the first power-on — or at minimum before the first vCenter enrolment — avoids the whole procedure. The certificate is generated once during installation from whatever the hostname happens to be at that moment, and every subsequent rename is what creates the mismatch.

That’s all it takes to regenerate the self-signed certificate on ESX. If you want to read more about regenerating the ESXi self-signed certificates, here is the Broadcom documentation.

Related ESXi and Certificate Content on This Site

Back up the host configuration before you start with the ESXi config backup procedure, and read up on manual ESXi updates since certificate files can be regenerated automatically as part of an upgrade. For trust between components, see VMware vCenter OVA certificate trust, and for diagnosing chain and verification errors more generally, TLS certificate chain problems with OpenSSL.