Some repos use git-crypt to encrypt secrets; they are committed in encrypted form, and decrypted locally, auto-magically, using GPG keys.
This magic is, shall we say, simple until it's not -- in "interesting" ways.This is written from the point-of-view of someone inheriting an existing config - which broke, due to multiple keys.
In the hope of saving someone else their sanity, here are a few learnings.
(Which are hopefully even correct.)
We'll start with the easy stuff; then, well, Buckle up...
How to get started as a new collaborator:
- brew install git-crypt
- install the "GPG Suite"
- generate a key pair
- upload your new public key to the interwebs
- give public key to an existing collaborator, who must:
- add the new user to their GPG keychain
- sign the new user's key
git-crypt add-gpg-user --trusted USER_ID- Use '--trusted' to avoid dependency of public "Web of Trust"
- (Yes; this is potentially less secure.)
- The USER_ID above is usually the email address that the user configured their GPG keypair with
- add-gpg-user should result in output like this:
[master 30babf07] Add 1 git-crypt collaborator
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 .git-crypt/keys/default/0/72E278AE2FB3...8F90BB21B36FD67.gpg
How was git-crypt set up in the first place?
(See above for a bit more detail on some of these steps, such as expected output.)- brew install git-crypt
- navigate to the repo you want to use git-crypt with
git-crypt init- Note: This creates a symmetric key.
- add the first GPG user:
git-crypt add-gpg-user --trusted ADMIN-USER_ID- This user must exit already in GPG.
- You might consider this the "admin" user; they'll be the only one to be able to decrypt secrets, add more users, etc. - until other users are added.
- Why yes; it would be a good idea to add more people - say, if this person leaves the organization.
- unlock, using new GPG key (will prompt for that key's passphrase):
git-crypt unlock - config a
.gitattributesfile with contents likesecretfile* filter=git-crypt diff=git-crypt- the
.gitattributesfile defines which files are to be encrypted - And must be in place BEFORE adding a file that must be encrypted.
- the
git add .gitattributes- commit and push:
git commit -m 'your comment here'; git push
A few notes:
- what causes the encryption to actually take place?
- See the notes on the
.gitattributesfile, above.
- See the notes on the
- check encryption status (for
encrypted files, GITCRYPTshows at the top): git-crypt status -e | awk '{print $2}' | while read thePath; do echo $thePath\: $(cat $thePath | xxd -l 9); done- Warning: You could push the change and confirm it's encrypted in the web UI - except if it's not, that secret is now forever* ensconced in your repo. (*How to remove secrets from a repo - AKA: It's too late.)
- GPG items in MacOS keychain, are not named with "GPG", but with "GnuPG"
- we're using GPG here (not PGP); it makes little difference to the procedure (ex: a GPG fingerprint is not for GPG only)
- if freshly cloned, need to
git-crypt unlockagain (default state is locked) - who's got access?
ls -l .git-crypt/keys- each filename contains a user's fingerprint (look that up, on a keyserver)
- Trouble getting file to actually encrypt?
git-crypt status -fgit-crypt lock --force- "Touching the file and re-committing it forces git-crypt to re-encrypt it"
- Seeing errs like: "
still unencrypted even after staging" OR "encrypted file has been tampered with" OR "Warning: one or more files is marked for encryption via .gitattributes but was staged and/or committed before the .gitattributes file was in effect" ?- unstage (ex:
git reset HEAD secrets.yml) - redo the dance with
git-crypt status -fandgit-crypt lock --force - maybe start from a fresh clone - and save aside, any files that are unencrypted
- be certain the file is really encrypted before using
git add filename
- unstage (ex:
- But
git-crypt status -esays the files are encrypted! - NO; it's only saying that those files are configured to be encrypted
- check the contents to confirm if it's actually encrypted (see "check encryption status" above)
- getting an err like ERROR! Unexpected Exception: 'utf8' codec can't decode byte 0xd0 in position 11: invalid continuation byte ?
- Check your git-crypt config; if that's OK, reclone the repo (the git-crypt status may be hosed.)
How to reset the encryption on a repo:
Here be dragons; this should be avoided, but if you have to...- list files that have been encrypted:
git-crypt status -e | awk '{print $2}' > encrypted-files - make sure repo is in UNlocked state:
git-crypt unlock - save decrypted copies of all encrypted files; ex:
git-crypt unlock; tar czf ../saved.tgz ./ - if you don't have unencrypted copies anymore?
- get them from another collaborator, old unlocked copy of the repo, ...
- there is no known way to recover them otherwise
- remove all encrypted files; ex:
cat encrypted-files | xargs -t -n1 rm - remove all git-crypt files:
rm -rf .git-crypt .git/git-crypt - ? may be necessary to save & remove the
.gitattributesfile too? (doubtful) - commit:
git commit -a -m 'your comment here' - re-config git-crypt:
git-crypt init- Note that this creates a new symmetric key, stranding files encrypted with any other key.
- add the first AKA "admin" user:
git-crypt add-gpg-user --trusted ADMIN-USER_ID - unlock, using new GPG key:
git-crypt unlock - add any addtl users:
git-crypt add-gpg-user --trusted ONCE-PER-ADDTL-USER_ID - if you removed the
.gitattributesfile above, copy it (or its contents) back- the
.gitattributesfile must be in place BEFORE adding a file that must be encrypted.
- the
- copy decrypted files back in
- confirm files are decrypted:
git-crypt status -e | awk '{print $2}' | while read thePath; do echo $thePath\: $(cat $thePath | xxd -l 9); done- It may be helpful to make the files different (ex: add a comment) to help force encryption with new key...
- some extra git-crypt magic:
git-crypt status -f - make sure repo is in UNlocked state:
git-crypt unlock - force encryption:
git-crypt lock --force - CONFIRM FILES ARE ENCRYPTED (they'll show
GITCRYPT): git-crypt status -e | awk '{print $2}' | while read thePath; do echo $thePath\: $(cat $thePath | xxd -l 9); done- if not, see notes above - it will be messy if you add (or worse commit) unencrypted info
for each of the encrypted files: git add ...- commit & push:
git commit -a -m 'your comment here'; git push - you probably want to unlock again:
git-crypt unlock - after keys are reset, a possible solution to: checkout (ex: of a branch) fails with "encrypted file has been tampered with":
- make a fresh clone of the repo
- leave it locked
- checkout branch (ex: keys were reset on master, but old keys are left on your branch)
- cherry-pick the commits for the new keys & newly-encrypted files (in chron order?)
- then unlock
- a fresh clone is best; otherwise, something like this might help:
git-crypt lock --force; git stash; git pull
WHY would you ever want to reset the encryption on a repo??
- You somehow got secrets committed, with multiple symmetric keys (ex: ran
git crypt initmore than once). - You want to be safe, after a collaborator has left the project.
