VMware vCenter OVA Certificate Trust

原文：VMware vCenter OVA Certificate Trust — theDXT (Daniel Keer)

Typically, importing an OVA or OVF file is straightforward. However, after VMware vCenter version 7.0 Update 2, a new message began appearing when importing an OVA or OVF file. The message would say that the certificate is not trusted.

Technically speaking, you could click ignore and keep going. However, I didn’t want to do that.

In this post, I will show you step-by-step how to resolve the certificate is not trusted warning without clicking ignore.

Why vCenter Reports an Untrusted OVA Certificate

Starting with VMware vCenter 7.0 Update 2, vCenter validates the digital signature of an OVF or OVA package before it deploys the appliance. Every OVA that a vendor signs ships with a code-signing certificate and the chain of certificates behind it. When you finish the deploy wizard, vCenter verifies that chain against the trusted root certificates held in its own trust store. If the root certificate that ultimately signed the package is not in that store, the deployment stops with a message telling you the certificate is not trusted.

The important thing to understand is that this is usually neither a broken appliance nor a broken vCenter. It is a case of the appliance being signed by a public certificate authority that your particular vCenter build has never heard of. Vendors rotate signing providers and certificate authorities over time, and a vCenter that was installed before that authority published its root certificate simply has no reason to trust it yet.

You can click Ignore and let the deployment continue, but that throws away the only integrity check you get on the package. The signature exists so that vCenter can prove two things: that the appliance really was produced by the vendor named in the OVF descriptor, and that nothing inside the archive was modified after it was signed. If you bypass the check, a tampered OVA with an injected root account or a modified post-deploy script in it would deploy exactly as happily as the genuine one. That is why it is worth spending ten minutes to add the missing certificate authority rather than clicking through the warning.

How OVA Certificate Chains Work

An OVA file is a tar archive that has been given the .ova extension, which is why a compression tool such as 7-Zip can open it and why the archive can be unpacked on a Linux host with a single tar command. Inside a signed package you will typically find the following members:

• The .ovf descriptor, an XML file that describes the virtual hardware, the virtual disks and the deployment options.
• One or more .vmdk files holding the appliance disks, often compressed.
• A .mf manifest file containing SHA digests of every other file in the package.
• A .cert file holding the code-signing certificate of the vendor that produced the appliance.

Each file listed in the manifest is hashed, and the manifest itself is signed with the vendor private key. The matching public key is in the certificate in the .cert file. That certificate is almost never self-signed, so it carries a chain of trust: a leaf code-signing certificate, one or more intermediates, and a root certificate authority. A public authority such as DigiCert or Sectigo issues the intermediates and keeps the root certificate in the operating system and application trust stores, so that a client verifying the signature can build an unbroken path from the leaf certificate up to a root it already trusts.

When an OVA leaves out the root certificate, which is deliberately common, the verifying client has to supply that root from its own store. That is exactly what vCenter's Trusted Root certificate store is for. The store is used to validate OVF and OVA signing certificates, and unlike your Windows or macOS trust store it is entirely separate and does not inherit anything from the workstation you run the vSphere Client on.

Prerequisites and Safety Checks

Before you change anything, make sure the following are in place:

• vCenter Server 7.0 Update 2 or later. The Trusted Root tab described below was introduced in that release.
• An account with administrator rights on the vCenter Server, not just on an ESXi host.
• A compression tool such as 7-Zip on the workstation you use for the vSphere Client.
• An independent verification of the package you are about to deploy. Download the SHA checksum published by the vendor and compare it against a locally computed hash before you extract anything.
• If you have more than one vCenter Server, or a vCenter in Enhanced Linked Mode, plan to add the certificates to every instance. Trust is per appliance.
• A vCenter file-based backup or a snapshot of the appliance taken shortly before the change, so you can roll back quickly if something goes wrong.

Most administrators run this procedure once per vendor and never touch it again, because the certificates added stay in the trust store through later vCenter updates. Treat it as a documented change, record which authority you added and why, and keep the exported certificate files with your build documentation. If you are still standing up the management plane, then Install VMware vCenter covers the deployment and first-boot steps that have to happen before any of this is relevant.

The Process

Make sure you only follow this process on an OVA file you’ve received from a trusted source and that you’ve verified its authenticity.

• Open the OVA file with a program like 7-Zip.

An OVA is technically a tar archive, which is why we can view its contents with most zip programs.

• Extract the .cert file from the OVA file.

• Rename the .cert file to .cer.

• Click Yes to confirm the file name extension rename.

• Open the .cer file.

• Click on the Certification Path tab.

• Click on the root certificate, then click View Certificate.

In my example, the first certificate is the root certificate, which is the DigiCert Trusted Root G4 certificate.

• In the new certificate window, click the Details tab, then click Copy to File.

• Click Nexton the Certificate Export Wizard.

• Select Base-64 encoded X.509 (.CER) and click Next.

• Select a location to export the first certificate to, then click Next.

• Review the certificate export details and click Finish.

• Click Ok to confirm that the root certificate was exported successfully.

• Close the Certificate window for the root certificate.
• On the Certification Path tab for the main certificate, select the next certificate in the chain, then click View Certificate.

In my example, the next certificate is the intermediate certificate, which is the DigiCert Trusted G4 Code Signing RSA4096 SHA384 2021 CA1 certificate.

• In the new certificate window, click the Detailstab, then click Copy to File.

• Click Nexton the Certificate Export Wizard.

• Select Base-64 encoded X.509 (.CER) and click Next.

• Select a location to export the next certificate, then click Next.

• Review the certificate export details and click Finish.

• Click Ok to confirm that the intermediate certificate was exported successfully.

• Close the Certificate box for the intermediate certificate.

You’ll need to repeat this process for each intermediate certificate.

• Log in to your VMware vCenter.

• Open the vCenter menu, then click Administration.

• In the Certificatessection, click onCertificate Management.

• Click on the Trusted Root tab.

• Click on Add Trusted Root Certificate.

• Browse to the certificates you extracted (you need to do one at a time), then click Add.

• Repeat the process for each certificate you extracted.

• Once completed, you will see the certificates you added in Certificate Management.

Reading the Certificate Chain Without Windows

The Windows certificate viewer described above is the quickest route when you are already working from a workstation, but it is not the only one. If you prefer to inspect and export the chain from a Linux or macOS host, an OVA is a tar archive and the standard OpenSSL toolset can do everything the wizard does. The commands below work on the appliance package itself and make no changes to vCenter.

# List the members of the OVA without extracting it
tar -tvf appliance.ova

# Extract the package (an OVA is a tar archive)
tar -xvf appliance.ova

# Show the signing certificate: subject, issuer, validity, key usage
openssl x509 -inform DER -in appliance.cert -text -noout

# If the .cert file is a PKCS#7 bundle rather than a single certificate,
# unwrap the whole chain to PEM first
openssl pkcs7 -inform DER -in appliance.cert -print_certs -out ova-chain.pem

# Split a PEM bundle into one file per certificate
awk 'BEGIN{n=1} /BEGIN CERTIFICATE/{f="chain-" n++ ".pem"} {print > f}' ova-chain.pem

# Print subject and issuer for each certificate so you can order the chain
for f in chain-*.pem; do echo "== $f"; openssl x509 -in "$f" -noout -subject -issuer -dates; done

Reading the subject and issuer of each certificate gives you the shape of the chain immediately. The certificate whose subject is not mentioned as an issuer anywhere else is the root, and the one whose issuer is not the subject of any other file is the leaf. Once you have separated them, you can verify the chain offline exactly as vCenter does:

# Verify leaf against the intermediates and the root
openssl verify -CAfile root-ca.pem -untrusted intermediates.pem leaf.pem

When the verification prints leaf.pem: OK, the chain is complete and internally consistent, and the only reason vCenter refuses it is that the root is missing from the trust store. That is the case this procedure fixes. If verification fails instead, the package is damaged or the files were exported incorrectly, and adding the certificates to vCenter will not help. If you regularly diagnose chain problems, TLS Certificate Chain Problems: Diagnosing with OpenSSL covers the same verification logic from the TLS side.

Verifying the Trusted Root Certificates

Once the certificates are in vCenter, it is worth confirming that the store holds the certificate you think it holds. The Certificate Management page lists each trusted root with its issuer, subject, validity period and thumbprint, so you can compare it against the file you exported.

On Windows, certutil prints the same details for a local copy of the certificate, including the SHA-1 hash that vCenter displays alongside it:

certutil -dump root-ca.cer

A useful habit is to compare the thumbprint rather than the subject name. Two different certificate authorities can share a very similar subject string, and a renamed file proves nothing. If the thumbprint in the vSphere Client matches the thumbprint of the certificate you exported from the OVA, the trust store contains the right root.

On the appliance itself you can also confirm the state of the vCenter services after the change. Connect to the vCenter Server Appliance shell over SSH and use the service control utility:

service-control --status --all
service-control --restart vmware-vpxd

Restarting vmware-vpxd is not normally required, because the trust store is read when a deployment starts, but it is a quick way to force vCenter to pick up a newly added certificate if an import still reports the old result.

Troubleshooting Common Certificate Trust Problems

Most of the problems that survive the procedure fall into a handful of categories.

Only the root certificate was added

Adding the root without the intermediates does not build a chain. Go back to the Certification Path tab and export every certificate in the list, not just the one at the top, then add them all. vCenter will not fill in intermediates on your behalf.

The import still warns about an untrusted certificate

Check whether the appliance you are importing is the same package you inspected. Vendors rebuild appliances regularly, and a rebuild can be signed by a different intermediate certificate. Re-extract the .cert file from the package that is actually failing.

The Trusted Root tab does not exist

The tab was introduced in vCenter 7.0 Update 2. On earlier releases there is no supported way to add an OVF signing certificate from the client. Do not hand-edit the vCenter trust store or the Java keystores on the appliance: the change is unsupported, and a later update will overwrite it. Upgrade the appliance, or verify the package by another means before deploying.

The certificate was added but another administrator still sees the warning

Confirm that you added the certificate to the vCenter they are connected to and not to a different instance in the environment. In Enhanced Linked Mode each vCenter keeps its own trust store even though the inventories are joined.

The warning appears during an update rather than a deployment

Vendor appliances such as the Omnissa ones are OVAs too, so a Horizon edge deployment like Deploy Omnissa UAG on VMware vCenter goes through the same signature check. Adding the signing certificates once covers all of these cases.

Security Considerations

• Only trust a root certificate authority that you recognise and have a reason to trust. The trust store is global to the vCenter instance, so anything you add applies to every future deployment and every administrator using that instance.
• Never add the self-signed certificate of an unverified appliance just to make the warning disappear. That removes the value of the check entirely and turns signature verification into a formality.
• Verify the package checksum against the value published by the vendor before you extract and inspect it. If the checksum does not match, stop and obtain the package again from the source.
• Keep the exported certificate files, the thumbprints and the reason for the change in your build documentation. The next administrator to import an appliance will want to know why a given authority is trusted.
• Root certificates do expire, even the long-lived ones. Add the certificates you inserted here to the same renewal monitoring that already watches your own TLS certificates, so a deployment does not suddenly start failing years from now.

Frequently Asked Questions

Is it safe to click Ignore on the certificate warning?

It will deploy the appliance, but you lose the integrity guarantee. For a lab or a throwaway environment that is a reasonable trade, though adding the certificates takes only a few minutes and removes the doubt permanently.

Do I need to add the certificates to every host in the cluster?

No. The check is performed by vCenter, not by ESXi, so the trust store on the vCenter Server covers every deployment in that inventory.

Will a vCenter upgrade remove the certificates I added?

No. Certificates in the Trusted Root store are treated as data and are preserved across updates, which is why this is a one-time job per vendor rather than a task for every upgrade.

Does this affect the vCenter certificate itself?

No. The Trusted Root store is unrelated to the machine certificate that vCenter presents to clients. If you need to regenerate a host certificate instead, ESX Regenerate Self-Signed Certificate walks through that separate procedure.

Summary

The next time you import an OVA containing those certificates in the certificate chain, it will show as a trusted certificate, and you won’t need to click ignore.



That’s all it takes to trust an OVA certificate chain to resolve the certificate is not trusted warning message.

If you want to read more about adding trusted root certificates to VMware vCenter, here is the VMware by Broadcom documentation about it.

The procedure itself does not change anything about the appliance, the hosts or the networks. It simply teaches vCenter which certificate authority signs the appliances you deploy, so that the integrity check it performs can succeed rather than be bypassed.

• Export every certificate in the signing chain, root included, from the OVA using the Certificate Export Wizard or OpenSSL.
• Add each certificate to vCenter under Administration, Certificate Management, Trusted Root.
• Verify the thumbprints match the files you exported, and document what you trusted and why.
• Repeat the work on every vCenter Server in the environment, and add the certificates to your renewal monitoring.