Using Git-crypt to Protect Sensitive Data

The advent of the EU General Data Protection Regulation (GDPR) permitted to highlight the need to protect sensitive information from leakage.

It is of course even more important in the context of git repositories, whether public or private, since the disposal of a working copy of the repository enable the access to the full history of commits, in particular the ones eventually done by mistake (git commit -a) that used to include sensitive files. That’s where git-crypt comes for help. It is an open source, command line utility that empowers developers to protect specific files within a git repository.

git-crypt enables transparent encryption and decryption of files in a git repository. Files which you choose to protect are encrypted when committed, and decrypted when checked out. git-crypt lets you freely share a repository containing a mix of public and private content. git-crypt gracefully degrades, so developers without the secret key can still clone and commit to a repository with encrypted files. This lets you store your secret material (such as keys or passwords) in the same repository as your code, without requiring you to lock down your entire repository.

The biggest advantage of git-crypt is that private data and public data can live in the same location.

Note: there are alternatives tools/approaches you can use to protect/encrypt data within a Git repository, listed at the end of this post

Table of Content

Pre-requisites

To use git-crypt, you need a working Git and GPG environnment

1
2
3
4
5

[user]
    name   = Sebastien Varrette
    email  = Sebastien.Varrette@uni.lu
    helper = osxkeychain           # ONLY for Mac OS
    signingkey = 5D08BCDD4F156AD7
  • For GPG: Gnu Privacy Guard, see this tutorial To reach this state:
1
2
3
4
5
6
7
8
9
10

# List GPG keys for which you have both a public and private key.
$> gpg --list-secret-keys --keyid-format LONG
sec   rsa4096/5D08BCDD4F156AD7 2017-03-01 [C] [expires: 2019-08-27]
[...]
uid                 [ultimate] Sebastien Varrette <Sebastien.Varrette@uni.lu>
uid                 [ultimate] Sebastien Varrette (Falkor) <Sebastien.Varrette@gmail.com>
uid                 [ultimate] [jpeg image of size 3075]
[...]
#  Set your GPG signing key in Git
$> git config --global user.signingkey 5D08BCDD4F156AD7

GPG Key Management

General recommendations / Best practices

  • create a 4096bit RSA key, with the sha512 hashing algorithm
  • Use the concept of GPG key subpairs
    • your primary key is only meant for certification / authentication purposes (in particular not for signing or encrypting).
  • Expiration date should be within less than two years.
    • You can always extend the key expiration as long as you still have access to the key, even after it has expired

This applies for your personnal GPG keyring on your laptop. You may be reluctant to transfer or share your primary key pair over a remote [computing] system, such as an HPC facility. To handle your GPG keys on such platform (for instance the UL HPC clusters, you have two alternatives:

  1. create a new key pair proper to each cluster, that you will sign with your primary key.
  2. create a subkey you will export on the remote facility.

Installation

On your local machine:

If you’re running Mac OS X – and assuming Homebrew is installed:

1

$> brew install git-crypt

If you’re running Linux:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

## Pre-requisites for compilation
# Debian/Ubuntu
$> apt-get install build-essential
$> apt-get install libssl-dev
$> apt-get install xsltproc
#
# CentOS 7
$> yum groupinstall "Development tools"
$> yum install openssl-devel
$> yum install libxslt       # required to build man pages
#
## Collect sources
$> cd /usr/local/src
$> wget https://www.agwa.name/projects/git-crypt/downloads/git-crypt-0.6.0.tar.gz
$> wget https://www.agwa.name/projects/git-crypt/downloads/git-crypt-0.6.0.tar.gz.asc
#
## Check file signature
# Import PGP key -- see https://www.agwa.name/about/pgp.page
$> gpg --recv-key 0xEF5D84C1838F2EB6D8968C0410378EFC2080080C
# If above fail for firewall filtering reason, target a key servers answering to port 80
$> gpg --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-key 0xEF5D84C1838F2EB6D8968C0410378EFC2080080C
# Verify the signature file
$> gpg --verify git-crypt-0.6.0.tar.gz.asc git-crypt-0.6.0.tar.gz
#
## uncompress, compile and install
$> tar xf git-crypt-0.6.0.tar.gz
$> cd git-crypt-0.6.0/
$> make ENABLE_MAN=yes          # requires xsltproc
$> make ENABLE_MAN=yes install

Note: git-crypt is installed on the UL HPC platform

Initial Repository Setup and Configuration

Configure/setup a repository to use git-crypt as follows:

1

$> git-crypt init

This will generate a symmetric key for encrypting your files (stored in .git/git-crypt/keys/default).

Then there are a couple of actions to perform, detailed below:

  1. claim ownership of the git-crypt vault
  2. bootstrap a .gitattributes file at the root of the repository defining the encryption policy for the files of the repository
  3. enable a custom git pre-commit hook (see doc to avoid accidentally adding unencrypted files – see issue #45.
  4. share this key with allowed collaborators through a commited version of its encrypted version using their respective GPG key (see git-crypt add-gpg-user)

Note you need of course to have imported the corresponding GPG key ID into your keyring

1
2
3

$> gpg --search-keys <GPG_ID>    # Search and Import
# If above fail for firewall filtering reason, target a key servers answering to port 80
$> gpg --keyserver hkp://p80.pool.sks-keyservers.net:80 --search-keys <GPG_ID>

.gitattributes setup

Create and commit at the root of your repository a new file named .gitattributes with the following content:

1
2
3
4
5
6
7
8
9
10

# -*- mode: conf -*-
#
# specify which files to encrypt using [git-crypt](https://www.agwa.name/projects/git-crypt/)
#
#
# Certificate private keys
# *.key               filter=git-crypt diff=git-crypt
# Private document/folders
# *PRIVATE*           filter=git-crypt diff=git-crypt
# subfolder/**/*      filter=git-crypt diff=git-crypt

Note: you can find this template file on Github.

To automate the process from online sources:

1
2
3
4

$> cd /path/to/repo
$> wget https://raw.githubusercontent.com/Falkor/falkorlib/devel/templates/git-crypt/.gitattributes
$> git add .gitattributes
$> git commit -s -m 'Initialize .gitattributes for git-crypt' .gitattributes

Git pre-commit hook

You need also to setup a Pre-commit hook to avoid accidentally adding unencrypted files with git-crypt – see issue #45. You can find it as a gist:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

#!/bin/bash
################################################################################
# See <https://gist.github.com/Falkor/848c82daa63710b6c132bb42029b30ef>
# Pre-commit hook to avoid accidentally adding unencrypted files with [git-crypt](https://www.agwa.name/projects/git-crypt/)
# Fix to [Issue #45](https://github.com/AGWA/git-crypt/issues/45)
#
# Usage:
#    $> cd /path/to/repository
#    $> git-crypt init
#    $> curl <url/to/this/raw/gist> -o .git/hooks/pre-commit
#    $> chmod +x .git/hooks/pre-commit
#
# Otherwise, you might want to add it as a git submodule, using:
#    $> git submodule add https://gist.github.com/848c82daa63710b6c132bb42029b30ef.git config/hooks/pre-commit.git-crypt
#    $> cd .git/hooks
#    $> ln -s ../../config/hooks/pre-commit.git-crypt/pre-commit.git-crypt.sh pre-commit
#
if [ -d .git-crypt ]; then
    STAGED_FILES=$(git diff --cached --name-status | awk '$1 != "D" { print $2 }' | xargs echo)
    if [ -n "${STAGED_FILES}" ]; then
        git-crypt status ${STAGED_FILES} &>/dev/null
        if [[ $? -ne 0  ]]; then
            git-crypt status -e ${STAGED_FILES}
            echo '/!\ You should have first unlocked your repository BEFORE staging the above file(s)'
            echo '/!\ Proceed now as follows:'
            echo -e "\t git unstage ${STAGED_FILES}"
            echo -e "\t git crypt unlock"
            echo -e "\t git add ${STAGED_FILES}"
            exit 1
        fi
    fi
fi

Recommended way to automate the installation (leaving the pre-commit hook script in a dedicated directory config/hooks/):

1
2
3
4
5
6
7
8

$> cd /path/to/repo
$> mkdir -p config/hooks
$> curl https://gist.githubusercontent.com/Falkor/848c82daa63710b6c132bb42029b30ef/raw/610bac85ca512171d04b19d668098bd2678559a7/pre-commit.git-crypt.sh -o config/hooks/pre-commit.git-crypt.sh
$> chmod +x config/hooks/pre-commit.git-crypt.sh
$> git add  config/hooks/pre-commit.git-crypt.sh
$> git commit -s -m "pre-commit hook for git-crypt" config/hooks/pre-commit.git-crypt.sh
# bootstrapping special Git pre-commit hook for git-crypt
$> ln -s ../../config/hooks/pre-commit.git-crypt.sh .git/hooks/pre-commit

(optional) Multiple key support

In addition to the implicit default key, git-crypt supports alternative keys which can be used to encrypt specific files and can be shared with specific GPG users. This is useful if you want to grant different collaborators access to different sets of files.

To generate an alternative key named <KEYNAME> and/or share it with a GPG user, pass the -k <KEYNAME> option to git-crypt { init | add-gpg-user} as follows:

1
2
3

$> git-crypt init -k <KEYNAME>
# Share it with a GPG user
$> git-crypt add-gpg-user -k <KEYNAME> <GPG_ID>

To encrypt a file with an alternative key, use the git-crypt-<KEYNAME> filter in .gitattributes as follows:

1

secretfile filter=git-crypt-KEYNAME diff=git-crypt-KEYNAME

git-crypt Usage

Unlock/lock the git-crypt vault

You can unlock the vault i.e. decrypt the encryption key using your personnal GPG ID by running

1

$> git-crypt unlock

You can lock back the vault by running

1

$> git-crypt lock

/!\ IMPORTANT thanks to the above configured Git pre-commit hook, you avoid having sensitive files (as filtered within the .gitattributes file) commited in cleartext while the git-crypt vault is locked.

Adding data sensitive file to the repository

  1. First you need to unlock the vault (if not yet done) with git-crypt unlock.
  2. Then specify files/wildcard patterns to encrypt by commpleting the .gitattributes file at the root of the repository filter=git-crypt diff=git-crypt
  3. commit the changes to the .gitattributes file.
  4. add and commit your file

Example of specifications within the .gitattributes file:

1
2
3
4
5
6
7
8
9
10

# global wildcard pattern
*.key               filter=git-crypt diff=git-crypt
# Private document/folders
# *PRIVATE*         filter=git-crypt diff=git-crypt
#
# ALL files under a certain sub directory
subfolder/**/*      filter=git-crypt diff=git-crypt
#
# A specific file
subdir/secretfile   filter=git-crypt diff=git-crypt

For instance at step 4, assuming you plan to add a *.key file (thus expected to be encrypted as per above .gitattributes policy), proceed as follows:

1
2
3
4
5

# Eventually unlock the repository
$> git-crypt unlock
$> echo 'secret' > secret.key
$> git add secret.key
$> git commit -s -m "add secret.key (encrypted) file" secret.key

Note that thanks to the pre-commit hook, in case you have forgotten to unlock the repository, the above commit command would fail as follows:

1
2
3
4
5

$> git commit -s -m "add secret.key (encrypted) file" secret.key
    encrypted: secret.key *** WARNING: staged/committed version is NOT ENCRYPTED! ***
Warning: one or more files is marked for encryption via .gitattributes but
was staged and/or committed before the .gitattributes file was in effect.
Run 'git-crypt status' with the '-f' option to stage an encrypted version.

So assuming you did well, you can commit the file and check that the content is indeed encrypted:

1
2
3
4
5
6
7
8
9
10
11

$> git commit -s -m "add secret.key (encrypted) file" secret.key
[master 9ae570f] add secret.key (encrypted) file
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 secret.key
$> cat secret.key
secret
#
# Lock back the repository
$> git-crypt lock
$> cat secret.key
GITCRYPT<XXXXXXXX>

Adding new collaborator to the vault

To grant access to the encrypted files stored in the repository to a collaborator, you first need to collect his GPG ID. You have several options at this level:

  1. query and import the GPG ID from the official GPG servers and carefully check it (assuming you do not have yet import it in your keyring)
  2. as distributing GPG keys can be cumbersome, rely on the keybase.io service to collect certified GPG ID from their username – see tutorial
1
2
3
4
5
6
7
8
9
10
11
12
13

# Option 1 - traditional query and import the GPG ID
$> gpg --search-keys <email>  # Search & Import
# If above fail for firewall filtering reason, target a key servers answering to port 80
$> gpg --keyserver hkp://p80.pool.sks-keyservers.net:80 --search-keys <email>
#
## Option 2: using Keybase.io
# curl + gpg pro tip: import svarrette's keys
$> curl https://keybase.io/svarrette/pgp_keys.asc | gpg --import
# the Keybase app can push to gpg keychain, too
$> keybase pgp pull svarrette
#
# Now get the GPG ID
$> gpg --list-key <email> | grep pub

Now you can share the repository with this GPG ID:

1
2
3
4
5
6
7
8
9
10

# Check associated (imported) GPG identity
$> gpg --list-key <email> | grep pub
#
# Add new git-crypt collaborator i.e. sign the
#     encryption key with this GPG key and store it into '.git-crypt/'
$> git crypt add-gpg-user <GPG_ID>
[master (root-commit) a967527] Add 1 git-crypt collaborator
 2 files changed, 4 insertions(+)
 create mode 100644 .git-crypt/.gitattributes
 create mode 100644 .git-crypt/keys/default/0/<FINGERPRINT>.gpg

By default, git-crypt add-gpg-user will fail if there is no assurance that the key belongs to the named user. If you trust the key you imported (but did not commit this entitlement within your keyring by actually signing this key), you can use the --trusted option to enforce the operation to succeed:

1

$> git crypt add-gpg-user --trusted <GPG_ID>

You can add as many collaborators as you wish.

Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

# Define the list of GPG ID of the collaborators
# Ex: the Uni.lu HPC Team
$> GPGKEYS="0x5D08BCDD4F156AD7
0x07FEA8BA69203C2D
0x37183CEF550DF40B
0x3F3242C5B34D98C2
0x6429C2514EBC4737"
#
# Check the associated primary identity
$> parallel -j 1  echo '--- {} ---'\; gpg --list-key {} '|' grep uid '|' head -n 1 ::: $GPGKEYS
--- 0x5D08BCDD4F156AD7 ---
uid                   [ultimate] Sebastien Varrette <Sebastien.Varrette@uni.lu>
--- 0x07FEA8BA69203C2D ---
uid                   [  full  ] Clement Parisot <Clement.Parisot@uni.lu>
--- 0x37183CEF550DF40B ---
uid                   [  full  ] Hyacinthe Cartiaux <hyacinthe.cartiaux@uni.lu>
--- 0x3F3242C5B34D98C2 ---
uid                   [  full  ] Valentin Plugaru <Valentin.Plugaru@uni.lu>
--- 0x6429C2514EBC4737 ---
uid                   [  full  ] Sarah DIEHL <sarah.diehl@uni.lu>
#
# Allow them to access the vault...
$> parallel -j 1  echo '--- {} ---'\; git-crypt add-gpg-user {} ::: $GPGKEYS

Removing a collaborator from the vault

(Update Sept 19, 2019) That’s a tricky part, an open bug is running since 2015 on that subject. What is sure is that it is not sufficient to remove from the repository the .git-crypt/keys/default/0/<GPG-Key-to-remove-fingerprint>.gpg. It is requried to re-initialize git-crypt for the repository with a new key and re-add all keys except the one requested for removal.

Note: You still need to change all your secrets to fully protect yourself. Removing a user will prevent them from reading future changes but they will still have a copy of the data up to the point of their removal.

There is no consensus on the appropriate way to handle it but you can find a convenient script that makes the job in this gist

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

# Get the script and place it in ~/bin/
mkdir -p git/gist
cd git/gist
git clone https://gist.github.com/7b29f16f5f79404fe41476be0d992783.git git-crypt-remove-gpg-user.sh
cd git-crypt-remove-gpg-user.sh
ln -s $(pwd)/remove-gpg-user.sh ~/bin/git-crypt-remove-gpg-user.sh
# Now you should have a script ready for usage as '~/bin/git-crypt-remove-gpg-user.sh'
#
# To remove a collaborator key from a git-crypt protected repository: 
cd /path/to/repo
git checkout -b  git-crypt-remove       # Prefer to work in a separate branch (see below)
~/bin/git-crypt-remove-gpg-user.sh -l   # List curently configured GPG keys for git-crypt 
~/bin/git-crypt-remove-gpg-user.sh -r <KEYID>   # Reinitialize git-crypt, restore legitimate GPG keys and reencrypt the protected files 
#
# The script will create multiple commits to your repo. Feel free to squash them
# all down to one.
# [...]
git push origin git-crypt-remove

IMPORTANT Before merging the changes made in the git-crypt-remove branch into yoru main branch and promote (push) these commits, you have to ensure that your collaborator have locked the repository before pulling or they will end in a conflicting state quite annoying to recover.

1
2
3
4
5

# Updating the repo
cd /path/to/repo
git-crypt lock    # VERY IMPORTANT 
git fetch -va
git merge origin/git-crypt-remove

git-crypt alternatives

Password Management with pass

Another nice git-based approach that team nicely with GPG relies on pass, the standard unix password manager. Password are stored inside GPG encrypted files inside a simple directory tree, meant to become a password repository.

Then pass is an utility to insert, display or copy to clipboard passwords stored into this git repository. It is not mandatory to use it, but it eases password management.

Assuming you have set the environnment variables PASSWORD_STORE_{DIR,SIGNING_KEY}, the pass CLI usage can be summarized below:

1
2
3
4
5
6
7

$> pass help      # Pass usage
$> pass git pull  # Fetch latest passwords
$> pass           # List passwords
$> pass twitter/<accountname>         # Display a password
$> pass -c twitter/<accountname>      # Copy a password to clipboard
$> pass insert google/<accountname>   # Insert a new password
$> pass git push                      # Push your changes

Notes: Git commit is done automatically by the pass utility. If you need to add comments in addition to the password, use the -m option to insert extra lines. A dedicated tutorial page will be made available for this tool.

Now if you are allergic to GnuPG and/or by extension git-crypt, here are a few other alternatives you can use to protect your sensitive data in a repository.

EncFS / GocryptFS / eCryptFS / Cryptomator / securefs / CryFS

All these open-source file encryption solutions for Linux (and thus Mac OS) are available. In contrast to disk-encryption software that operate on whole disks (TrueCrypt, dm-crypt etc), file encryption operates on individual files that can be backed up or synchronised easily, especially within a Git repository.

  • Comparison matrix
    • gocryptfs, aspiring successor of EncFS written in Go
    • EncFS, mature with known security issues
    • eCryptFS, integrated into the Linux kernel
    • Cryptomator, strong cross-platform support through Java and WebDAV
    • securefs, a cross-platform project implemented in C++.
    • CryFS, result of a master thesis at the KIT University that uses chunked storage to obfuscate file sizes.

Assuming your working copy is stored in /path/to/repo, your workflow (mentionned below for EncFS, but it can be adpated to all the other tools) operated on encrypted vaults and would be as follows:

  • you ignore the mounting directory (ex: vault/*) in the root .gitignore of the repository
    • this ensures neither you nor a collaborator will commit any unencrypted version of a file by mistake
  • you commit only the EncFS / GocryptFS / eCryptFS / Cryptomator / securefs / CryFS raw directory (ex: .crypt/) in your repository.
    • thus only encrypted form or your files are commited
  • You create the EncFS / GocryptFS / eCryptFS / Cryptomator / securefs / CryFS encrypted vault
  • You prepare macros/scripts/Makefile/Rakefile tasks to lock/unlock the vault on demand

Here are for instance a few example of these operations in live (for EncFS, adapt accordingly)

1
2
3
4
5
6
7
8
9
10
11
12
13

$> cd /path/to/repo
$> rawdir=.crypt      # /!\ ADAPT accordingly
$> mountdir=vault     # /!\ ADAPT accordingly
#
# Ignore the mount dir
$> echo $mountdir >> .gitignore
#
# Creation of an EncFS vault (only once)
$> encfs --standard $rawdir $mountdir
#
# OR
# Creation of a GoCryptFS vault (only once)
$> gocryptfs -init $rawdir
Tool OS Opening/Unlocking the vault Closing/locking the vault
EncFS Linux encfs -o nonempty --idle=60 $rawdir $mountdir fusermount -u $mountdir
EncFS Mac OS encfs --idle=60 $rawdir $mountdir umount $mountdir
GocryptFS   gocryptfs $rawdir $mountdir as above

Note: In a Puppet control repository relying on hiera, you can use the hiera-eyaml format.

File Encryption using SSH [RSA] Key Pairs

If you encrypt/decrypt files or messages on more than a one-off occasion, you should really use GnuPGP as that is a much better suited tool for this kind of operations. But if you already have someone’s public SSH key, it can be convenient to use it, and it is safe.

The below notes assumes you have a (potentially big) file you want to send encrypted to a collaborator, typically on a remote server where your SSH public key is allowed (i.e. your id_rsa.pub key is added to the remote ~/.ssh/authorized_keys file). We also assume that you own a copy of the SSH public key of your collaborator (denoted by id_dst_rsa.pub) in the sequel.

Note: as a reminder, you can generate a strong RSA key pair (4096 bits) using

  ssh-keygen -t rsa -b 4096 -a 100 [-f <name>]

This will produce the key files <name> and <name>.pub, where <name> is ~/.ssh/id_rsa by default. Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

# Eventually ask your collaborator to generate a dedicated SSH key pair -- classical RSA keys (4096 bits)
$> ssh-keygen -t rsa -b 4096 -a 100 -f ~/.ssh/id_$(whoami)_rsa
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/vagrant/.ssh/id_vagrant_rsa.
Your public key has been saved in /home/vagrant/.ssh/id_vagrant_rsa.pub.
The key fingerprint is:
SHA256:tf04p/tYXfjVdbqbYgWVHHfuawKlTi8q6zAyqKWel8I vagrant@adminfront.vagrant.dev
The key randomart image is:
+---[RSA 4096]----+
|             ..oo|
|              +o.|
|          .  o  +|
|         . o+  ++|
|        S .+..o =|
|  .       o oo.++|
|...o.o     oo+=oo|
|.E.oo o.  . +*oo |
|+oo   .oo. .+++  |
+----[SHA256]-----+

/!\ IMPORTANT the below instructions are NOT compliant with the new OpenSSH format which is used for storing encrypted (or unencrypted) RSA, EcDSA and Ed25519 keys (among others) when you use the -o option of ssh-keygen. You can recognize these keys by the fact that the private SSH key ~/.ssh/id_rsa starts with -----BEGIN OPENSSH PRIVATE KEY-----

Encrypt a file using a public SSH key

(eventually) SSH RSA public key conversion to PEM PKCS8

OpenSSL encryption/decryption operations performed using the RSA algorithm relies on keys following the PEM format 1 (ideally in the PKCS#8 format). It is possible to convert OpenSSH public keys (private ones are already compliant) to the PEM PKCS8 format (a more secure format). For that he can either use the ssh-keygen or the openssl commands, the first one being recommended.

1
2
3
4

# Convert the public key of your collaborator to the PEM PKCS8 format (a more secure format)
$> ssh-keygen -f id_dst_rsa.pub -e -m pkcs8 > id_dst_rsa.pkcs8.pub
# OR use OpenSSL for that...
$> openssl rsa -in id_dst_rsa -pubout -outform PKCS8 > id_dst_rsa.pkcs8.pub

Note that you don’t actually need to save the PKCS#8 version of his public key file – the below command will make this conversion on demand.

Generate a 256 bit (32 byte) random symmetric key

There is a limit to the maximum length of a message i.e. size of a file that can be encrypted using asymmetric RSA public key encryption keys (which is what SSH keys are). For this reason, you should better rely on a 256 bit key to use for symmetric AES encryption and then encrypt/decrypt that symmetric AES key with the asymmetric RSA keys This is how encrypted connections usually work, by the way.

Generate the unique symmetric key key.bin of 32 bytes (i.e. 256 bit) as follows:

1

$> openssl rand -base64 32 -out key.bin

You should only use this key once. If you send something else to the recipient at another time, you should regenerate another key.

Encrypt the (potentially big) file with the symmetric key

1

$> openssl enc -aes-256-cbc -salt -in bigdata.dat -out bigdata.dat.enc  -pass file:./key.bin

Note: for your tests, you can quickly generate random files of 1 GiB size as follows:

1
2
3
4

# Random generation of a 1GiB file
$> dd if=/dev/urandom of=bigfile_1GiB.dat  bs=64M count=16  iflag=fullblock
# Random generation of a 1GiB file
$> dd if=/dev/urandom of=bigfile_10GiB.dat bs=64M count=160 iflag=fullblock

An indicated encryption time taken for the above random file is proposed in the below table, using

     openssl enc -aes-256-cbc -salt -in bigfile_<N>GiB.dat -out bigfile_<N>GiB.dat.enc  -pass file:./key.bin
File size Encryption time
bigfile_1GiB.dat 1 GiB 0m5.395s
bigfile_10GiB.dat 10 GiB 2m50.214s

Encrypt the symmetric key, using your collaborator public SSH key in PKCS8 format:

1
2
3

$> openssl rsautl -encrypt -pubin -inkey <(ssh-keygen -e -m PKCS8 -f id_dst_rsa.pub) -in key.bin -out key.bin.enc
# OR, if you have a copy of the PKCS#8 version of his public key
$> openssl rsautl -encrypt -pubin -inkey  id_dst_rsa.pkcs8.pub -in key.bin -out key.bin.enc

Delete the unencrypted symmetric key as you don’t need it any more (and you should not use it anymore)

1

$> rm key.bin

Now you can transfer the *.enc files i.e. send the (potentially big) encrypted file <file>.enc and the encrypted symmetric key (i.e. key.bin.enc ) to the recipient _i.e. your collaborator. If you’re allowed to, transfer them by SSH to an agreed remote server. It is even safe to upload the files to a public file sharing service and tell the recipient to download them from there.

Decrypt a file encrypted with a public SSH key

First decrypt the symmetric key using the SSH private counterpart:

1
2
3

# Decrypt the key -- /!\ ADAPT the path to the private SSH key
$> openssl rsautl -decrypt -inkey ~/.ssh/id_rsa -in key.bin.enc -out key.bin
Enter pass phrase for ~/.ssh/id_rsa:

Now the (potentially big) file can be decrypted, using the symmetric key:

1

$> openssl enc -d -aes-256-cbc -in bigdata.dat.enc -out bigdata.dat -pass file:./key.bin

Misc

For a ‘quick and dirty’ encryption/decryption of small files:

1
2
3
4

# Encrypt
$>  openssl rsautl -encrypt -inkey <(ssh-keygen -e -m PKCS8 -f ~/.ssh/id_rsa.pub) -pubin -in <cleartext_file>.dat -out <encrypted_file>.dat.enc
# Decrypt
$> openssl rsautl -decrypt -inkey ~/.ssh/id_rsa -in <encrypted_file>.dat.enc -out <cleartext_file>.dat

  1. Defined in RFCs 1421 through 1424, is a container format for public/private keys or certificates used preferentially by open-source software such as OpenSSL. The name is from Privacy Enhanced Mail (PEM) (a failed method for secure email, but the container format it used lives on, and is a base64 translation of the x509 ASN.1 keys. ↩