2018/10/26

git-crypt works smoothly - until it doesn't

Some repos use git-crypt to encrypt secrets; they are committed in encrypted form, and decrypted locally, auto-magically, using GPG keys.

This magic is, shall we say, simple until it's not -- in "interesting" ways.

This is written from the point-of-view of someone inheriting an existing config - which broke, due to multiple keys.

In the hope of saving someone else their sanity, here are a few learnings.
(Which are hopefully even correct.)

We'll start with the easy stuff; then, well, Buckle up...


How to get started as a new collaborator:

  • brew install git-crypt
  • install the "GPG Suite"
  • generate a key pair
  • upload your new public key to the interwebs
  • give public key to an existing collaborator, who must:
    • add the new user to their GPG keychain
    • sign the new user's key
    • git-crypt add-gpg-user --trusted USER_ID
      • Use '--trusted' to avoid dependency of public "Web of Trust"
        • (Yes; this is potentially less secure.)
      • The USER_ID above is usually the email address that the user configured their GPG keypair with
      • add-gpg-user should result in output like this:
[master 30babf07] Add 1 git-crypt collaborator
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 .git-crypt/keys/default/0/72E278AE2FB3...8F90BB21B36FD67.gpg

How was git-crypt set up in the first place?

(See above for a bit more detail on some of these steps, such as expected output.)
  • brew install git-crypt
  • navigate to the repo you want to use git-crypt with
  • git-crypt init
    • Note: This creates a symmetric key.
  • add the first GPG user: git-crypt add-gpg-user --trusted ADMIN-USER_ID
    • This user must exit already in GPG.
    • You might consider this the "admin" user; they'll be the only one to be able to decrypt secrets, add more users, etc. - until other users are added.
      • Why yes; it would be a good idea to add more people - say, if this person leaves the organization.
  • unlock, using new GPG key (will prompt for that key's passphrase): git-crypt unlock
  • config a .gitattributes file with contents like secretfile* filter=git-crypt diff=git-crypt
    • the .gitattributes file defines which files are to be encrypted
      • And  must be in place BEFORE adding a file that must be encrypted.
  • git add .gitattributes
  • commit and push: git commit -m 'your comment here'; git push

A few notes:

  • what causes the encryption to actually take place?
    • See the notes on the .gitattributes file, above.
  • check encryption status (for encrypted files, GITCRYPT shows at the top):
    • git-crypt status -e | awk '{print $2}' | while read thePath; do echo $thePath\: $(cat $thePath | xxd -l 9); done
    • Warning: You could push the change and confirm it's encrypted in the web UI - except if it's not, that secret is now forever* ensconced in your repo. (*How to remove secrets from a repo - AKA: It's too late.)
  • GPG items in MacOS keychain, are not named with "GPG", but with "GnuPG"
  • we're using GPG here (not PGP); it makes little difference to the procedure (ex: a GPG fingerprint is not for GPG only)
  • if freshly cloned, need to git-crypt unlock again (default state is locked)
  • who's got access?
    • ls -l .git-crypt/keys
    • each filename contains a user's fingerprint (look that up, on a keyserver)
  • Trouble getting file to actually encrypt?
  • Seeing errs like: "still unencrypted even after staging" OR "encrypted file has been tampered with" OR "Warning: one or more files is marked for encryption via .gitattributes but was staged and/or committed before the .gitattributes file was in effect" ?
    • unstage (ex: git reset HEAD secrets.yml)
    • redo the dance with git-crypt status -f and git-crypt lock --force
    • maybe start from a fresh clone - and save aside, any files that are unencrypted
    • be certain the file is really encrypted before using git add filename
  • But git-crypt status -e says the files are encrypted!
    • NO; it's only saying that those files are configured to be encrypted
    • check the contents to confirm if it's actually encrypted (see "check encryption status" above)
  • getting an err like ERROR! Unexpected Exception: 'utf8' codec can't decode byte 0xd0 in position 11: invalid continuation byte ?
    • Check your git-crypt config; if that's OK, reclone the repo (the git-crypt status may be hosed.)

How to reset the encryption on a repo:

Here be dragons; this should be avoided, but if you have to...
  • list files that have been encrypted: git-crypt status -e | awk '{print $2}' > encrypted-files
  • make sure repo is in UNlocked state: git-crypt unlock
  • save decrypted copies of all encrypted files; ex: git-crypt unlock; tar czf ../saved.tgz ./
    • if you don't have unencrypted copies anymore?
    • get them from another collaborator, old unlocked copy of the repo, ...
    • there is no known way to recover them otherwise
  • remove all encrypted files; ex: cat encrypted-files | xargs -t -n1 rm
  • remove all git-crypt files: rm -rf .git-crypt .git/git-crypt
  • ? may be necessary to save & remove the .gitattributes file too? (doubtful)
  • commit: git commit -a -m 'your comment here'
  • re-config git-crypt: git-crypt init
    • Note that this creates a new symmetric key, stranding files encrypted with any other key.
  • add the first AKA "admin" user: git-crypt add-gpg-user --trusted ADMIN-USER_ID
  • unlock, using new GPG key: git-crypt unlock
  • add any addtl users: git-crypt add-gpg-user --trusted ONCE-PER-ADDTL-USER_ID
  • if you removed the .gitattributes file above, copy it (or its contents) back
    • the .gitattributes file must be in place BEFORE adding a file that must be encrypted.
  • copy decrypted files back in
  • confirm files are decrypted: git-crypt status -e | awk '{print $2}' | while read thePath; do echo $thePath\: $(cat $thePath | xxd -l 9); done
    • It may be helpful to make the files different (ex: add a comment) to help force encryption with new key...
  • some extra git-crypt magic: git-crypt status -f
  • make sure repo is in UNlocked state: git-crypt unlock
  • force encryption: git-crypt lock --force
  • CONFIRM FILES ARE ENCRYPTED (they'll show GITCRYPT):
    • git-crypt status -e | awk '{print $2}' | while read thePath; do echo $thePath\: $(cat $thePath | xxd -l 9); done
  • if not, see notes above - it will be messy if you add (or worse commit) unencrypted info
  • for each of the encrypted files: git add ...
  • commit & push: git commit -a -m 'your comment here'; git push
  • you probably want to unlock again: git-crypt unlock
  • after keys are reset, a possible solution to: checkout (ex: of a branch) fails with "encrypted file has been tampered with":
    • make a fresh clone of the repo
    • leave it locked
    • checkout branch (ex: keys were reset on master, but old keys are left on your branch)
    • cherry-pick the commits for the new keys & newly-encrypted files (in chron order?)
    • then unlock
  • a fresh clone is best; otherwise, something like this might help: git-crypt lock --force; git stash; git pull

WHY would you ever want to reset the encryption on a repo??

  • You somehow got secrets committed, with multiple symmetric keys (ex: ran git crypt init more than once).
  • You want to be safe, after a collaborator has left the project.

References:

No comments: