Protect Secrets and Passwords with Ansible Vault: A Practical Guide with Examples

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • MyrinNew
    Senior Member
    • Feb 2024
    • 5168

    #1

    Protect Secrets and Passwords with Ansible Vault: A Practical Guide with Examples

    Configuration as Code helps teams manage infrastructure efficiently by automating repetitive tasks and improving reliability. However, it also brings new challenges—managing secrets securely is one of the most critical. Without proper handling, sensitive data like API keys, passwords, and certificates can be exposed, creating security risks.


    Protecting secrets while maintaining automation benefits is essential. For instance, expecting operators to manually input secrets during an automated process is both impractical and error-prone. In this post, we explore Ansible Vault, a powerful tool that secures sensitive data without disrupting DevOps workflows.


    What is Ansible Vault

    Ansible Vault is an encryption tool included with Ansible that protects secrets while enabling DevOps workflows. In this article, we will take a comprehensive look at Ansible Vault to understand its use cases and features.


    Ansible Vault is a utility included in Ansible that can encrypt and decrypt arbitrary data using a password. It can encrypt a variety of data using AES256 encryption, including:
    • Structured YAML files, such as variable files or even entire playbooks
    • Configuration files with sensitive information
    • Individual variables in Ansible playbooks


    Most importantly, Ansible Vault integrates transparently with other Ansible commands, such as ansible and ansible-playbook. These commands can automatically detect and decrypt encrypted data to use in standard Ansible workflows. 


    For example, an Ansible playbook can reference variables stored in an encrypted variable file. These files will automatically be decrypted at runtime using the appropriate password.


    Password Protection and Encryption

    Creating an Encrypted Files

    At its most basic level, Ansible Vault can encrypt entire files with a password. For example, consider a situation where you want to store an API key as a variable in a YAML file. You can create the initial vault encrypted file using ansible-vault create:




    $ ansible-vault create vars.yaml
    New Vault password:
    Confirm New Vault password:





    The file will launch in your shell’s default editor, as controlled by the EDITOR environment variable. If no editor is set, it will default to vi. Create the file like any other text file and save it:




    api_key: SuperS3cretP@ssword!





    Now, when you try to view the file, notice that it is an encrypted blob:




    $ cat vars.yaml
    $ANSIBLE_VAULT;1.1;AES256 62303264313066366365353436353866356436313038663534 32383461636137646265373565303762356238343532373335 31336530623863356530376563650a35663866633863303639 63313230613562323238333330343130313465623531396636 37663261646430353535663232373365373937656338346665 313932370a3930353764336166366238396539306538616565 32663164643863306264646661303461366662396636326436 36356133633134323766613062643266333231




    Viewing and Editing Encrypted Files

    You will frequently need to view the contents of vault-encrypted files or edit them directly. You can do this with the view and edit commands.


    The view command displays the contents of the encrypted file:



    $ ansible-vault view vars.yaml
    Vault password:
    api_key: SuperS3cretP@ssword!





    The edit command launches an editor to modify the encrypted file using the shell’s default editor:




    $ ansible-vault edit vars.yaml
    Vault password:




    Decrypting an Encrypted File

    Sometimes, you may want to fully decrypt a file. For example, you may determine that the data is no longer sensitive and doesn’t need to be protected as a secret.


    You can fully decrypt a file using the decrypt command. This command fully removes the encrypted file and leaves only the decrypted file in place:



    $ ansible-vault decrypt vars.yaml
    Vault password:
    Decryption successful

    $ cat vars.yaml
    api_key: SuperS3cretP@ssword!




    Encrypting an Existing File

    Configuration as Code is frequently used to transfer sensitive configuration files to remote hosts. Ansible Vault can fully encrypt an existing file. For example, consider a configuration file with sensitive information in it:



    $ cat config.yaml
    auth_host: auth.example.com
    auth_username: admin
    auth_password: Sens1tiveP@ssw0rd123!





    Ansible Vault can encrypt this entire file using a password with the encrypt command:




    $ ansible-vault encrypt config.yaml
    New Vault password:
    Confirm New Vault password:
    Encryption successful
    $ cat config.yaml
    $ANSIBLE_VAULT;1.1;AES256
    61323230636236303664306230323961336133366161303766 30616663353261613234366564653035653362643264346366 62366565326564353134316630660a37376339383937633833 34346338363464383136666336636362653762373863303363 64396331636365633138393537376538613032333635386364 636334630a6431613333653866363639633166663361323836 30643232383231313737353834346266393032313231386432 63343333373963396636303364336463376134386631626133 34306336303838623962383039613661336137386138343565 35396231383637616664356162393135623563336566623135 65643465633661626530646532363239396263366437363636 34333131323163396534313765633533623830326161653065 61393833376331646139373634373865343064353635




    Changing the Encryption Key

    It’s a security best practice to regularly rotate encryption material. This principle also applies to the passwords used to protect files encrypted with Ansible Vault.


    Ansible Vault makes it easy to rotate the encryption key for a file using the rekey command. Simply provide it with the existing password and a new password:



    $ ansible-vault rekey config.yaml
    Vault password:
    New Vault password:
    Confirm New Vault password:
    Rekey successful




    Encrypting Variables in a Playbook

    The examples we have looked at so far encrypt entire files. This is the most common way to use Ansible Vault. However, there are also situations where you want to encrypt data within a playbook while leaving the rest of the playbook unencrypted.


    Ansible Vault enables this pattern with the encrypt_string command. You can use encrypt_string to encrypt the contents of an arbitrary string and then place these contents in a playbook.


    For example, consider a playbook that makes an HTTP request to a password-protected API endpoint:



    $ cat playbook.yaml
    ---
    - hosts: all
    tasks:
    - name: Make HTTP request
    ansible.builtin.uri:
    url: https://api.example.com
    user: apiUser
    password: S3cretK3y123
    method: POST





    We want to protect this password using Ansible Vault, but we don’t want to encrypt the entire file. You can use encrypt_string to encrypt the API key:




    $ ansible-vault encrypt_string
    New Vault password:
    Confirm New Vault password:
    Reading plaintext input from stdin. (ctrl-d to end input, twice if your content does not already have a newline)
    S3cretK3y123
    Encryption successful
    !vault |
    $ANSIBLE_VAULT;1.1;AES256
    32383139623739323166616363333766356461386639323463 61366630643437666466343835303761353035366362656133 31383131343562343165643364380a38623563663939383836 32356335626564643966323738356665363434656130643162 36303433356138343036303531323763366362316561653331 306162330a3964396430373662623365346434393335356430 336333346262623964633638





    Notice that encrypt_string allows you to directly input the string you want to encrypt into the terminal. You end the string with CTRL+D, not with a newline character. This is important to remember, as any newlines you enter into the terminal will become part of the encrypted string.


    Finally, you can insert this encrypted string directly into the playbook:




    $ cat playbook.yaml
    ---
    - hosts: all
    tasks:
    - name: Make HTTP request
    ansible.builtin.uri:
    url: https://api.example.com
    user: apiUser
    password: !vault |
    $ANSIBLE_VAULT;1.1;AES256
    32383139623739323166616363333766356461386639323463 61366630643437666466343835303761353035366362656133 31383131343562343165643364380a38623563663939383836 32356335626564643966323738356665363434656130643162 36303433356138343036303531323763366362316561653331 306162330a3964396430373662623365346434393335356430 336333346262623964633638
    method: POST





    This approach is useful for very basic playbooks, but it has limitations. You must encrypt each string individually, which can be tedious and time-consuming. Additionally, there is no way to easily rekey all of the encrypted strings in a file. Instead, you must re-encrypt each string.


    A better approach in most scenarios is to use a fully encrypted variable file and limit the use of encrypt_string. However, using encrypt_string can be helpful in very simple playbooks that don’t require the overhead of fully encrypted variable files.


    Running Ansible Plays with Encrypted Files

    By now, you should have a good understanding of how to use Ansible Vault to create and manipulate encrypted files and data. Ansible integrates transparently with Ansible Vault and allows you to use encrypted files and variables within your plays. Ansible automatically decrypts the encrypted data using the password that you provide.


    To illustrate these principles, you can use a playbook that contains both encrypted variables and fully encrypted files:




    $ cat playbook.yaml
    ---
    - hosts: localhost
    vars:
    api_user: serviceAccountUser
    api_key: !vault |
    $ANSIBLE_VAULT;1.1;AES256
    61383863303864313933353063306633303431623936626362 65393061306464313136646431323966393534656639326264 32636239383331623631643833370a39623134323133613236 38323537613330613236373839623233633361663235376437 38323739323766653831313063626336656535613135663364 386336360a6231396632393237316264633037643635376163 37373166373764366634373633396565356532343765663935 36383231373432656462383461376162623035
    tasks:
    - name: Make HTTP request
    ansible.builtin.uri:
    url: http://localhost:3000
    user: "{{ api_user }}"
    password: "{{ api_key }}"
    method: POST

    - name: Transfer encrypted configuration file
    ansible.builtin.copy:
    src: config.yaml
    dest: /etc/application/config.yaml
    owner: root
    group: root
    mode: 0600





    This playbook performs two tasks:

    1. Makes an HTTP request using the encrypted api_key variable
    2. Copies an encrypted configuration file to the remote host. This configuration file is created using ansible-vault create


    Next, it’s time to run the playbook. Ansible will automatically recognize data encrypted by Ansible Vault and decrypt it at the appropriate time. Ansible relies on a password to make this work. There are three main methods for providing Ansible with a password for decrypting protected data:

    1. Provide the password manually via the command line using --ask-vault-pass. This is inappropriate for automated scenarios but works well when testing.
    2. Reference a password file that contains the password with --vault-password-file. This is the most common approach. This file should be carefully locked down to prevent exposing the password.
    3. Look up the password using a script specified by --vault-password-file. This is an advanced approach that is very useful when you want Ansible to interact with an external secret storage system, such as AWS Secrets Manager.


    For this example, we will use a password file:




    $ cat password.txt
    demo





    With everything in place, it’s time to run Ansible:




    $ ansible-playbook playbook.yaml --vault-password-file password.txt
    [WARNING]: No inventory was parsed, only implicit localhost is available
    [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit
    localhost does not match 'all'

    PLAY [localhost] ************************************************** *********************************

    TASK [Gathering Facts] ************************************************** ***************************
    ok: [localhost]

    TASK [Make HTTP request] ************************************************** *************************
    ok: [localhost]

    TASK [Transfer encrypted configuration file] ************************************************** *****
    changed: [localhost]

    PLAY RECAP ************************************************** ***************************************
    localhost : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0





    Notice that the ansible-playbook command transparently decrypts the necessary files and variables. This makes it easy to begin using encrypted secrets without disrupting existing automation workflows.


    Advanced Use Cases

    The basic features we have covered so far are enough for most scenarios. However, Ansible Vault also features several advanced usage patterns that are helpful for more complex environments.


    Managing Multiple Vaults

    Large environments frequently use multiple secrets with different permission levels. For example, a different set of secrets may be used for staging and production infrastructure. Operators may also choose to encrypt different files and variables using separate passwords.


    Ansible Vault allows operators to work with multiple vaults, each uniquely identified by a vault ID. Vault IDs provide a “hint” to indicate the correct password to use when decrypting a vault file.


    For example, consider a situation where you want to encrypt two different configuration files with different passwords. Ansible Vault enables this with the --vault-id flag. This flag takes its argument in the format of ID@SOURCE, where “ID” is the vault ID to use, and “SOURCE” is the location to find the vault password.


    For example, you can encrypt two different files with different passwords provided via the command line prompt:




    $ ansible-vault create --vault-id config1@prompt config_file_1.yaml
    New vault password (config1):
    Confirm new vault password (config1):

    $ ansible-vault create --vault-id config2@prompt config_file_2.yaml
    New vault password (config2):
    Confirm new vault password (config2):





    Next, you can tell ansible-playbook about the appropriate place to obtain the password for each vault using the --vault-id flag. Notice that the password for the vault with ID “config1” is given at the prompt, while the password for the vault with ID “config2” is provided through a password file:




    $ ansible-playbook playbook.yaml --vault-id config1@prompt --vault-id config2@vault-2-password.txt
    Vault password (config1):
    [WARNING]: No inventory was parsed, only implicit localhost is available
    [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit
    localhost does not match 'all'

    PLAY [localhost] ************************************************** *********************************

    TASK [Gathering Facts] ************************************************** ***************************
    ok: [localhost]

    TASK [Copy config file 1] ************************************************** ************************
    changed: [localhost]

    TASK [Copy config file 2] ************************************************** ************************
    changed: [localhost]

    PLAY RECAP ************************************************** ***************************************
    localhost : ok=3 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0





    This approach provides great flexibility when using multiple passwords. However, it can quickly become complicated. You should still try to maintain simplicity when designing your password strategy.


    It’s important to understand that vault IDs are just hints. Vault ID matching is not strictly enforced by Ansible unless you set the [.codeDEFAULT_VAULT_ID_MATCH[.code] environment variable. Ansible will try all provided passwords with all provided vaults until it succeeds or fails to decrypt a vault.


    Integrating with a Secrets Manager

    Storing an Ansible Vault password in a text file or entering it in the command line is appropriate for basic use cases, but advanced environments will typically use an external secrets storage solution. For example, your environment might use HashiCorp Vault, Amazon Web Service Secrets Manager, or your in-house solution.


    Ansible makes it very easy to look up the decryption password for a vault with a client script. Client scripts can perform whatever logic is necessary to look up a vault’s password, including interacting with external secret managers. The script then prints the password to standard output, and Ansible uses this password to decrypt the Vault.


    The example below uses the aws-secrets-manager-client.sh script to look up a vault password. The actual content and logic of this script isn’t important; all that matters is that the script prints the password to standard output for Ansible to use:




    $ ansible-playbook playbook.yaml --extra-vars @vars.yaml --vault-password-file aws-secrets-manager-client.sh
    [WARNING]: No inventory was parsed, only implicit localhost is available
    [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit
    localhost does not match 'all'

    PLAY [localhost] ************************************************** *********************************

    TASK [Gathering Facts] ************************************************** ***************************
    ok: [localhost]

    TASK [Print encrypted variable] ************************************************** ******************
    ok: [localhost] => {
    "msg": "Test123"
    }

    PLAY RECAP ************************************************** ***************************************
    localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0





    Using a client script provides the best of both worlds: You can encrypt files and variables in your Ansible playbooks and Configuration as Code repositories, and also integrate with external secrets managers to keep your Ansible Vault passwords safe.


    Practical Tips

    Version Control

    A significant benefit of the Ansible Vault approach is the ability to store encrypted files and variables directly in your version control system. This avoids the need to store data in multiple places, providing you with a single source of truth for your Configuration as Code.


    However, you must still ensure that any sensitive information, such as the password file for Ansible Vault, is stored outside of the version control system.


    Environment Variables and Ansible Configuration

    There are several Ansible configuration directives related to Ansible Vault. While you don’t need to know all of them, it is helpful to mention the most common directives that you will encounter:
    • DEFAULT_VAULT_ID_MATCH: This environment variable controls vault ID matching. By default, Ansible will not enforce strict ID matching and will try every password with all vaults. Set this variable to “True” to change this behavior.
    • DEFAULT_VAULT_PASSWORD_FILE: This environment variable specifies the default vault password file.
    • DEFAULT_VAULT_IDENTITY_LIST: This environment variable is equivalent to specifying multiple --vault-id flags, and it can be useful for shortening the length of your ansible-playbook commands.


    Understanding When Files Are Decrypted

    Ansible will decrypt files as needed when running plays, and the files will remain encrypted at rest once the play has been completed. However, files will be decrypted at rest on a target host when they are used as the src argument to the copy, template, unarchive, script, or assemble module.


    This is intended and desirable behavior. It allows you to decrypt a file and place it on a remote host. For example, you can reference an encrypted configuration file in Ansible’s copy module and it will be decrypted and placed onto a remote host.


    While this behavior may seem obvious, it's important to understand the scenarios when Ansible will leave a file decrypted at rest.


    Integrating Ansible with env0

    env0 includes native support for Ansible, enabling you to use your existing playbooks alongside its infrastructure lifecycle management capabilities. With Ansible templates, you can consistently deploy environments while leveraging env0's features like controlled access, cost estimation, and automated deployment flows. Learn more here


    Final Thoughts

    Protecting sensitive data while still enabling Configuration as Code best practices is a challenge for DevOps teams of any size. It is one of the earliest challenges faced by organizations when they automate their configuration management practices, and it continues to challenge mature teams. A robust approach must be flexible enough to preserve velocity without compromising on security.


    Ansible Vault is an ideal solution for teams that already leverage Ansible in their automation workflows. It has a very low entry barrier, and its basic and advanced features make it appropriate for a variety of scenarios. Simple environments can benefit from encrypted vaults with basic password authentication. Advanced environments with complex needs can tier their vaults using vault IDs and externally stored vault passwords with frequent rotation.


    Ansible Vault is a simple utility that offers robust secrets protection in a variety of scenarios. It is an important component of any Ansible environment’s security posture.


    Frequently Asked Questions

    Q. What is Ansible Vault?

    Ansible Vault is a utility for encrypting secrets. Secrets can include variables inside Ansible playbooks, external variable files, or even arbitrary data. Ansible Vault integrates transparently with other Ansible tools, such as the ansible-playbook command. These Ansible tools can automatically decrypt and use secrets in playbooks and Ansible commands.


    Q. Is Ansible Vault just for encrypting Ansible playbooks?

    No. Ansible Vault can also encrypt arbitrary files, such as sensitive configuration files. Ansible can automatically decrypt these files as necessary, such as when they are transferred to a remote host.


    Q. How is Ansible Vault different from an external secrets manager, such as AWS Secrets Manager?

    Ansible Vault is built directly into Ansible and doesn’t require any additional modules or external infrastructure to work. It directly encrypts files using a shared key. External secrets managers exist outside of Ansible and require their own configuration, tooling, and modules to work with Ansible.


    Q. Can you integrate Ansible Vault with an external secrets manager?

    Yes. Ansible Vault uses a shared secret password to encrypt and decrypt secrets. This password can be stored in an external secrets manager. Ansible can look up this password in an external secrets manager at runtime using a client script.




    More...
Working...