HashiCorp Developer AI Private beta now available Sign up today
Vault
Vault Home

Overview

The Vault 1.16.x upgrade guide contains information on deprecations, important or breaking changes, and remediation recommendations for anyone upgrading from Vault 1.15. Please read carefully.

Plugin environment variables

Starting in 1.16.0, plugin-specific environment variables take precedence when they conflict with the main Vault process' environment.

For example, if an external plugin is registered with SOURCE=child in the env parameter, but the main Vault process has SOURCE=parent in its environment, versions prior to 1.16.0 will start the plugin process with SOURCE=parent. 1.16.0+ will start the plugin process with SOURCE=child.

For any users relying on the old behavior, you can opt out of this change by setting VAULT_PLUGIN_USE_LEGACY_ENV_LAYERING=true for the main Vault process. If this flag is detected, Vault will continue to prioritize Vault process environment variables when there is a conflict. Additionally, during unseal Vault will enumerate all plugins in the catalog and print a warning for any plugins whose environment conflicts with the main process. If the flag is set and you see an informational log that there are no conflicts, the flag can safely be removed.

Note that containerized plugins do not inherit any environment variables from the main Vault process, so are unaffected. Builtin plugins and external plugins that do not specify additional environment variables are also unaffected.

See plugin management for more details on plugin environment variables.

Known issues and workarounds

Existing clusters do not show the current Vault version in UI by default

Affected versions

  • 1.16+

Issue

Previous versions of the Vault UI fetched version information from un-authenticated endpoints like sys/health and sys/seal-status. Since introducing the redact_version TCP listener parameter, version information may no longer be available under some configurations. As a result, the Vault UI now uses a new, authenticated endpoint (sys/internal/ui/version) to fetch version information.

By default, all new Vault servers created with v1.16+ include the following rule, in the automatically-generated policy:

# Allow a token to look up the Vault version. This path is not subject to
# redaction like the unauthenticated endpoints that provide the Vault version.
path "sys/internal/ui/version" {
    capabilities = ["read"]
}

However, the default policy for existing Vault servers does not update automatically during the upgrade. You must updated the policy manually in order for the Vault version to be displayed in the Vault UI.

No other functionality in the Vault UI is affected be this issue.

You can use the Vault CLI to update the default policy and allow the Vault IU to query the Vault server for version information:

$ vault policy read default | cat - <<< '
# Allow a token to look up the Vault version. This path is not subject to
# redaction like the unauthenticated endpoints that provide the Vault version.
path "sys/internal/ui/version" {
    capabilities = ["read"]
}
' > default-policy.hcl
$ vault policy write default ./default-policy.hcl

Default lease count quota enabled when upgrading from Vault versions before 1.9

Affected versions

  • 1.16+

Issue

Vault began tracking version history as of version 1.9. As of version 1.16, new Vault installs automatically enable a lease count quota by consulting the version history. If the version history is empty on upgrade, Vault treats the upgrade as a new install and automatically enables a default lease count quota.

Before you upgrade Vault from a version prior to 1.9 to versions 1.16+, you should check the current number of unexpired leases via the vault.expire.num_leases metric.

If the number of unexpired leases is below the default lease count quota, value of 300000 no extra steps are required.

If the number of unexpired leases is greater than the default threshold of 300000, there is a two step workaround to safely upgrade without the default lease count quota:

  1. Upgrade to any Vault version prior to 1.16 (between 1.9 and 1.15) to populate the version store.
  2. Upgrade to Vault version 1.16+.

You can review, modify, and delete the global default quota at any point with the /sys/quotas/lease-count/default endpoint:

$ vault read sys/quotas/lease-count/default
$ vault delete sys/quotas/lease-count/default
$ vault write sys/quotas/lease-count/default max_leases=<# of max leases>

Refer to Protecting Vault with Resource Quotas for a step-by-step tutorial on quota tuning.

Refer to Lease Explosions for more information on lease management.

PKI OCSP GET requests can return HTTP redirect responses

If a base64 encoded OCSP request contains consecutive '/' characters, the GET request will return a 301 permanent redirect response. If the redirection is followed, the request will not decode as it will not be a properly base64 encoded request.

As a workaround, OCSP POST requests can be used which are unaffected.

Impacted versions

Affects all current versions of 1.12.x, 1.13.x, 1.14.x, 1.15.x, 1.16.x

Edit this page on GitHub