What is HashiCorp Vault and What Can I Do With It?

What is HashiCorp Vault?

HashiCorp Vault is a tool that allows you to safely manage secrets. By secrets, we mean sensitive information like digital certificates, database credentials, passwords, and API encryption keys. HashiCorp Vault gives you a way to store these secrets and then authenticates, validates, and authorises before it grants access to clients and users.

The most logical thing one might ask is, But why would I need an encryption service like this? Isn’t this basically trading one key for another? And if the vault gets compromised, doesn’t that give an attacker access to all the credentials I’m storing? 

That’s a great question. Let’s unpack HashiCorp Vault to find out. 

Why do we need centralised key management systems?

• Complex infrastructure

The need for sophisticated encryption management systems arose when organisations started moving part or all of their infrastructure to cloud. With the move to more shared, distributed service-oriented infrastructures, the need for orchestration, automation, and integration increased as well. Plus, our modern applications and microservices will often communicate with APIs or services outside of an organisation’s data centre.

As the Vault website explains, “Most enterprises today have credentials sprawled across their organisations. …  Because these credentials live everywhere, the sprawl can make it difficult and daunting to really know who has access and authorisation to what.” 

Workloads have also become “more and more ephemeral and short-lived,” and therefore, “long-lived static credentials pose a big security threat vector.” 

All of these and other factors make safeguarding digital keys tricky. Covering all your bases from a security standpoint becomes tedious. 

• Toward better security and compliance

Today it’s no longer enough to engineer tight controls in the name of prevention. It’s also about how effectively your company can monitor and how quickly and your company can detect and respond to breaches. The right tools, configured right for your organisation, can help. And when it comes to encrypting keys, HashiCorp Vault is a great choice.

HashiCorp Vault is open source, self-hosted, and cloud agnostic and was specifically designed to make storing, generating, encrypting, and transmitting secrets a whole lot more safe and simple—without adding new vulnerabilities or expanding the attack surface. In fact, it reduces the attack surface and, with built-in traceability, aids forensics in the event of a compromise. 

Features and benefits: What can I do with HashiCorp Vault? 

In very general terms, Vault’s use cases include (1) general secret storage in the form of both static and dynamic secrets, (2) data encryption, (3) identity-based access, and (4) key management.

Let’s dive into some of Vault’s main functionalities. 

• Vault can store arbitrary key/value secrets. It encrypts this information before it stores them. This protects the plaintext version and means that gaining access to the storage doesn’t actually expose the secrets. 
• Vault offers encryption as a service. With its “transit” secrets engine, Vault can handle cryptographic functions on data-in-transit without storing it. Essentially, this is encryption as a service, where Vault acts as a pass-through. That way, Vault is responsible for encryption/decryption instead of your application developers, and you can then write that encrypted data to another location, like an SQL database. 

This is really helpful in use cases where, say, a web application handles sensitive customer data (like credit card information) and is backed by a database. 

• Vault can create dynamic secrets. This is where HashiCorp Vault differentiates itself from other encrypted key value stores. It can generate dynamic secrets, in other words, secrets “on demand.” 

Here’s what that means.

A dynamic secret doesn’t get created until it is read. It will be unique to a client, and the credential is automatically destroyed when the lease (e.g., one month, three days, one hour—or whatever you’ve configured) expires. 

You can also set it up so that the dynamic secret is revoked right after it’s used. The benefit here is that you can reduce the lifespan of that credential to just the moment in time that it’s used.

The ability to generate secrets on demand or “on the fly” is helpful because it gives you security in an area that is notorious for leaking secrets in more ways than you can probably think of: applications. 

Applications will often leave secrets in log files or logging systems. Secrets can also be “captured in exception tracebacks or crash reports sent to external monitoring systems” or are “leaked via debugging endpoints and diagnostic pages after hitting an error,” says Armon Dadgar, co-founder of HashiCorp. The list of potential vulnerabilities goes on.

Dynamic secret creation removes the problem of multiple clients/users sharing the same credential. It also allows you to rotate those credentials manually or set up automatic credential rotation. Having automatic credential rotation in place makes it easy to meet regulatory or compliance requirements. 

• Vault supports secret revocation. It allows you to revoke secrets before lease expiry and lets you get specific. Whether you need to revoke single secrets or a “tree of secrets” (grouped by user, type, or application, etc.), Vault has built-in support for revocation. 

This makes key rolling easier. And since you can get granular, this lets you respond to a breach effectively while reducing or even eliminating downtime.

• Vault keeps a detailed log of all requests and responses.

In Vault, every request and response, every process, is audited and authenticated. This means you can trace everything. According to Vault, every operation “is an API request/response, … the audit log contains every authenticated interaction with Vault, including errors.”

In addition, you can enable “multiple audit devices” instead of a single audit device within Vault. As such, you can have redundant copies and, more importantly, are able to “check for data tampering in the logs themselves.” 

Audit trails are essential to meeting compliance requirements in highly regulated industries. 

How well does HashiCorp Vault integrate with providers and platforms?

HashiCorp Vault is open source and cloud agnostic. You can run its Secrets Engines on various cloud platforms, including AWS, Azure, and Google Cloud. You can integrate it with existing workflows.

HashiCorp also has its own cloud platform offering Vault as a service. Think of authorization methods and secrets engines within Vault as plugins, making this a versatile tool.

Vault offers various database capabilities that you can deploy on Cassandra, Couchbase, Elasticsearch, MongoDB and MongoDB Atlas, MSSQL, MySQL, Oracle, Snowlake, and more. The Vault website shows a helpful chart outlining database capabilities.

You can also deploy Vault on GitHub, Kubernetes, Microsoft SQL Server EKM provider, and ServiceNow. 

Which major companies are using HashiCorp Vault?

HashiCorp Vault has solved secrets encryption and storage challenges for many enterprise-level organisations, such as:

• Intel
• Ubisoft
• Adobe
• Starbucks
• Bank of America
• Citigroup
• Q2
• Wayfair
• eBay
• Autodesk

Since digital credentials are an essential part of interacting with the digital world, you can imagine Vault being used in every industry—from gaming to e-commerce to education to banking.

Starbucks deploys Vault

As a case example, Starbucks used Vault to solve the unique challenges of their edge environment (which uses a network of data centres placed in hubs to reduce latency for end users)—a complex type of computing common for retail at scale. 

In a video case study Andrew McCormick, Lead Systems Engineer at Starbucks, explains, “We went all the way down the rabbit hole with Terraform and Helm to implement everything as code, from the infrastructure and network all the way down to Vault policy and identity.” This was the first large-scale platform they had built, and they did it with HashiCorp Terraform’s open-source infrastructure-as-code software on Kubernetes.

In the video, McCormick says the two features within Vault Enterprise that were “critical for resiliency and scalability” were performance replica and disaster recovery clusters. “Vault replica clusters were a significant aspect of our scalability design for three main reasons: (1) they provide a read-only cluster for edge devices to connect to without exposing the primary Vault cluster, (2) they provide horizontal scaling and regional load balancing, and (3) replication filtering enables physical isolation between tenants.”  

To conclude, HashiCorp Vault, as a secrets management tool, can reduce operational complexity for enterprises that work with orchestrated architecture and operate across multiple cloud providers.

How can I learn more?

This article is a part of a greater series centred around the technologies and themes found within the first edition of the TechRadar by Devoteam. To read further into these topics, please download TechRadar by Devoteam