2021/12/31

Moving off Google

Please join me at my new blog.

(Why give any more to Google - home of "Do(n't) be evil".)

2021/07/30

Config macOS to boot headless - with some user accounts protected by FileVault

I recently wrote about some challenges using FileVault on a headless Mac.

And later experimented with a way of getting:

  • both headless (including a full boot, headless)
  • and FileVault
Which amounts to this:
  • Configure the boot volume without FileVault.
  • Set up a 2nd volume (APFS makes this easy).
  • Encrypt the 2nd volume.
  • Create a new "secure" user - whose homedir is on the 2nd, encrypted, volume.
Why? That new user's homedir is now better protected - if for example, the storage were removed.

Notes:
  • The "secure" user, can't log in til that 2nd volume is mounted.
  • The encrypted volume won't auto-mount (which would kinda defeat the point).
  • If you want to be a bit less secure, you could config a user whose homedir is on the unencrypted volume, to use the Keychain to mount the encrypted volume when they log in - in which case, you'd still have to take that manual step (of logging in to the "primary account"), though that would immediately allow the "secure" user to login.

Curious if anyone else is interested in doing anything like this, or if this prompts any thoughts.

2021/07/24

Open app using Rosetta, via command-line, on an M1 Mac

Apple has done so many architecture transitions (amazingly well, it's worth noting) that there's an old utility to help with this:

arch -arch x86_64 /System/Applications/Utilities/Terminal.app/Contents/MacOS/Terminal

On an M1 Mac, this has the same effect as clicking the "Open using Rosetta" checkbox, in an app's "Get Info" window, except it's:

  • for this run only; you likely want to generally run Terminal native
  • via command-line - which is why you're in Terminal anyway 😏

Why would you want to do this? Let's say you're using brew and not ready to try the M1 version - this then requires doing basic things like `brew upgrade` using Rosetta.

(BTW: For anyone not familiar with this sort of thing, it will open a second instance of the app; it doesn't affect the currently-running instance.)

2021/07/21

Unable to get IntelliJ to successfully use BBEdit as an External Editor?

Here's what I saw, after I set that up in IntelliJ:

You must allow `bbedit` to send events to the BBEdit application.

Use `tccutil reset AppleEvents` to reset the system's permissions, and try again.

bbedit: error: -1743

I don't want to "reset the system's permissions", even if only for AppleEvents.

So here's what I did instead:
  1. Within IntelliJ, open a Terminal pane / tab.
  2. At that terminal prompt, paste something like this:
    • osascript -e 'tell application "BBEdit" to set position of every window to {0, 44}'
      • (Yes that's ugly - it was a handy "innocuous" AppleEvent -- which did the job.)
    • And execute it. (ex: Press Return.)
  3. Click "OK" to allow IntelliJ to send AppleEvents to BBEdit.

To see the effect this has had:
  1. Open System Preferences.
  2. Select Security & Privacy.
  3. Scroll down & select Automation.
  4. IntelliJ is now listed, with the permission to control the BBEdit app.
    • Which you also have likely also allowed, for the Terminal app.
Background:

2021/07/10

Mac Bluetooth keyboard sluggish - and/or repeats?

Symptoms
  • Press a key – no response.
    • Or a several-second time lag, before any output.
  • Press a key (even briefly / lightly) and several characters are output.

When observed
  • After some time (hours to days).
  • Usually, when the screen is locked or asleep
    • Which makes entering the password, functionally impossible.

Equipment
  • MacBook Air (M1, 2020)
  • Apple Magic Keyboard (on Bluetooth)
  • macOS Big Sur (V11.4 (20F71))

Workaround
  1. Hold Shift and Option, then click the Bluetooth icon in the menubar.
    • Yes; challenging when having KB trouble. 😕
      • Though at least possible, since these are keys meant to be held down.
      • If even this doesn't work, hopefully you have a wired KB handy…
  2. Select "Reset the Bluetooth module".
    • Hmm; maybe this is possible to trigger with no KB use…
  3. There is no step three 😏
Shoutout to the good folks at OSXDaily.

2021/05/05

I just dismissed the screensaver - why is every keypress ignored?

You just authenticated to your Mac, and the screensaver cleared, to put you right back to where you were - almost.

This is why every keypress is ignored - aside from beeping at you.

Look more closely… Why are the window controls grayed out, as if it's not the active window?

Yeah; it's not exactly right back where you left off - the keyboard focus has not returned to your window; apparently, a side effect of the screensaver. 

Solution: Click where you want the focus (your document, terminal window, whatever), and off you go.

Note: This also affects at least some global key equivalents (I recently whipped up something to really Skip a track in iTunes) - and in such cases the solution is the same: Click somewhere, and they'll work.

Weird.

2021/04/17

Music (was iTunes) and Skipped vs Played - ish

Turns out, Apple's macOS Music app, uses some arcane rules to determine if and when a track is logged as a Skip, or Played.

Which doesn't much matter - unless you want to use that info in a Smart Playlist. 😕


I want to listen to something different, so I use that metadata, to filter out tracks either recently listened to, or recently skipped - which shakes things up nicely.


For awhile now, it's been nagging at me, that I seemed to be hearing tracks I should't - a bit of research led to the link above. I frequently hit the FF button within the first few seconds, so those don't count as Skips. 😕


I wrote a quick AppleScript to use, instead of the default FF button, and force a Skip (by first moving playback to the 6s mark); I use it on a Quicksilver Trigger (see p. 34 of the Quicksilver manual) - though you can use this code however you like:


tell application "Music"

if player state is playing then

set player position to 6

next track

end if

end tell


I'm no expert at Quicksilver Triggers, however here's how I set it up:

  • Open Quicksilver and navigate to Triggers (icon in top menu).
  • Press dot (period), in the top pane, to be able to paste text.
  • Paste the code above (newlines and all).
  • The 2nd pane may already be set to the proper kind of code execution: "Run as AppleScript".
  • If not, set that.
  • If it's not available, you may have to enable that Action.
  • Click "Save".
  • Click into the "Trigger" column (right), to assign a hotkey.

2021/03/07

iCloud Drive: some files refuse to upload

I recently moved a large group of files (many directories) to iCloud Drive - so they'd be available on my other devices - and watched as they all uploaded...

Then did some testing to confirm - only to find that many files never did upload. (I tested over the course of _weeks_; this was not an issue of bandwidth.)

I compared (using diff) the state of the Mac on which the files were first moved into iCloud, versus the files on another Mac which was freshly set up with iCloud, so got them only via iCloud sync. (I also spot-checked, using the web interface for iCloud Drive.)

So I started trying to determine why. It's a somewhat lively source of discussion - among those who've actually noticed the problem -- which is almost entirely invisible, unless you go looking for it. :/

Much of the discussion was the expected (unfortunately) reflexive "have you tried turning it off and on again?" - about killing processes, deleting various files (ex: ~/Library/Application Support/CloudDocs) and rebooting - and, as expected, none of that helped.

However, the clue I needed was in this Ask Different post: iCloud Drive stuck on “Waiting to upload”

(BTW: The only way to see that "Waiting to upload" message, is to first already know that there's a file in that state; then hover over the tiny dotted line cloud icon next to it, in Finder. Nearly-invisible errors are a terrible thing. And, unfortunately, info about iCloud Drive status icons is hard to find, and somewhat limited. Note also: Contrary to the info in the above link, the icons are for iCloud Drive, regardless whether the sub-feature of Desktop and Documents is specifically enabled - in my case, that's disabled.)

With the info gained above, I see that in all the cases that I'm dealing with - of a file stuck in "Waiting to upload" - each of them has an extended attribute com.apple.metadata:kMDItemWhereFroms which is _very large_.

It does not contain, as expected, a URL of where the document came from - but it actually apparently contains another copy of the full document. :/

This metadata is also apparently unexpected / dysfunctional, since even though it shows via ls or xattr, the mdls command ignores it.

So now I'm even more sure, that this extended attribute is disposable. One last question: Does removing it, update the "modified" timestamp (which is useful info that I want to preserve)? Thankfully, some testing shows that the timestamp is unchanged.

So what I did (standard disclaimers apply - use at your own risk) to find and fix all such files in my iCloud Drive directory, was something like this:

find ~/Library/Mobile\ Documents/com\~apple\~CloudDocs \
    -type f \
    -xattrname com.apple.metadata:kMDItemWhereFroms \
    -print | while read -r thePath; do
        testVal=$(xattr -lp com.apple.metadata:kMDItemWhereFroms "$thePath" 2>/dev/null | \
            head -5 | tail -4 | sed 's/^.* |//;s/|$//' | tr -d '\n')
        if [ $(echo "$testVal" | grep -E -c 'data:(application|attachment)') -gt 0 ] ; then
            echo 'BEFORE:'
            ls -l@ "$thePath"
            xattr -d com.apple.metadata:kMDItemWhereFroms "$thePath"
            echo 'AFTER:'
            ls -l@ "$thePath"
        fi
        done

Quite soon after doing the above, bird noticed and sync'ed them all. Nice.

(BTW: Check out brctl - fun to watch that _really_ verbose log - though didn't seem help in my case.)

2021/02/03

macos term file "could not be opened because it is corrupt"

A ".term" file is a useful way to wrap up some command-line stuff, in a neat little package which requires no command-line knowledge, no luck in typing it perfectly, etc. - for some discussion, see these old Hints articles: Run shell scripts from GUI and Use partial .term files for quick… (note that the ExecutionString attribute can be anything you want…).

I was working on a new .term file, having started by duplicating a perfectly usable one - but getting this unhelpful error:

The file “test.term” could not be opened because it is corrupt.

It may have been incompletely copied from another machine.

And then _finally_ remembered that the file is XML, and ampersands have literal meaning, so must be "escaped", as in HTML, like so:

&

Hopefully, noting that here, saves someone (like myself) a bit of hair-tearing next time.

2020/12/17

FileVault Mac, in headless mode, notifies on boot

Oh; fun!

I'm lucky enough to be experimenting with a new M1 Mac mini, and followed through the prompts (on a kinda autopilot, from setting up my own MacBook) to set it up with FileVault 2 "full-disk encryption", even though this will be a server.

So I found out the hard way, that this then requires (purposely; for security) entering a password _very_ early in the boot process - before it even gets an IP address.

Which also means no remote connection; not via Screen Sharing, or ssh - no headless.

(Note: There is a way to avoid this - for a single boot.)

But; back to the cool part; I rebooted (to apply a software update) before I remembered the above, and a few moments later, while I was trying to debug why I couldn't reach it, the mini (which was indeed headless), started playing the "Find My..." sound effect - that is a nice touch!

2020/11/14

Touch ID becomes an annoyance, when using a MacBook in clamshell mode

For instance: It's a very cool feature, to store an auth token, which allows decryption of passcode-encrypted iWorks docs, via Touch ID.

Except if you've switched to using your MacBook in clamshell mode (i.e.: using an external display, with the lid closed) - in which case, the apps will prompt for the decryption passcode (which, optimally, would be different for each doc) -- even though, in the days before Touch ID, it would store a token accessible via KeyChain.

So, to make the switch back, to using KeyChain instead of Touch ID:

Delete all fingerprints from the Touch ID preference pane, in System Preferences. (Which, since you've switched to clamshell mode, you're rarely using anyway - if ever.)

In retrospect, kinda obvious - though this annoyance has been on my To Do list for awhile. Hopefully this tip saves someone else some trouble.

2020/10/08

can't unlock Mac with Apple Watch anymore?

Problems with "Use your Apple Watch to unlock apps and your Mac" function of macOS?

 (In the "Security & Privacy" section of System Preferences.)


Seeing an error message like "Your Mac was unable to communicate with your Apple Watch"?


Maybe it was working before, and suddenly isn't?


Maybe you recently reset you Apple Watch? (That was the precipitating event for me.)


It took me some digging to find, so hopefully these extra breadcrumbs here, help more folks to this signal boost, for the good folks at macrumors.com, who solved the problem.

2020/02/16

Quicksilver, Spotlight and indexing content

I've noticed that the info I care about (messages, tasks, notes, code, whatever), is spreading to quite a few documents, apps and systems - both local to the device I'm using, and remote. (Maybe in a web system - which may or may not be sync'd / cached on my local device.)

And it's increasingly common, when I'm looking for something, that I can't immediately recall which system it's in, so where to start looking.

Which of course means I now have to put aside what I was trying to do, to focus instead, on how to find the info. And then remember what I was trying to do in the first place, to resume that task - hopefully…

This kind of context-switching sometimes does result in losing the primary task. It's certainly a waste of time and energy.

So, since I use Quicksilver extensively, I started to see this new issue, as a nail, for my Quicksilver hammer…

In the back of my head, I had a recollection that, while I'd always found Spotlight limited and cumbersome, it did a great job indexing the content of apps like Notes and Reminders - two of the systems I wanted to be able to reach into.

People who use Spotlight, may be inclined to observe how well it works for them, and how mystified they are, that some of us don't use it. It's possible, that those of us not using it, have a good reason - there may also be some inertia…

Thinking of Spotlight prompted another memory, that Quicksilver might… yes; it does have a Spotlight plugin - however I got confusing results: It was not showing content (which I knew was there) from Notes or Reminders… It seems to be limited to showing results in a Finder window (vs. the main Spotlight window) - which apparently does not show "Other" content such as Notes or Reminders. :/

I do virtually all searches (including across the web) from Quicksilver. And while I'd prefer to avoid the cognitive cost of deciding how/where to search, then launching a separate search system, and only then specifying what I wanted to find (this is backwards, for most humans) Spotlight does appear to be a better tool, for finding content.

Quicksilver is oriented more to files - and performing actions - than finding content. (Perhaps someone will point out that this too is a misunderstanding. :) )

Another plus for Spotlight: It's available (by default) on iOS as well as macOS, so as I move among those devices, Spotlight is right there.

Time to at least consider, that there are different kinds of nails, and different hammers - some of which, we thought we knew.


2020/01/04

hdiutil Error 110: image not recognized

A script to keep a disk image (AKA dmg) mounted, was suddenly failing - silently.

(Why create such a script? In this use-case, the image / volume should only be mounted when the app using it, is frontmost and might need it; otherwise it should be dismounted - so that it may be safely mounted by another Mac. Since there's no file-locking in this scenario, mounting it on multiple Macs simultaneously, can result in conflicts. Since this is a sparsebundle, conflicts result in multiple bands, which is usually (somewhat) straightforward to sort out - but manually, and who wants that?)

I ran the script in a debug / trace mode (bash -x) and saw nothing interesting - except then I noticed:

hdiutil attach -quiet -noautoopen /path/here

Ah yes; after debugging the previous issue, I'd set -quiet, to make the output cleaner.

(BTW: The -noautoopen option is handy, since without it, the Finder otherwise can open a window for the top level of the volume - right in front of whatever it was you were working on.)

So I ran that command with -verbose instead - and got this error message:

Error 110 (image not recognized).

Which didn't get me far; I couldn't find any references that helped, especially since the image mounted OK from the GUI - at which point I realized the problem:

It worked in the GUI, because /path/here was an alias - however the script did not know how to resolve an alias; it needed a symlink instead - which the GUI happily also recognizes. A symlink just isn't be able to auto-update, as an alias can, if the target is moved. (Side note: symlinks can be _much_ smaller; handy on a small filesystem.)

2019/11/26

Moving tabs between windows, in macOS Safari

Great feature; drag a tab from one window to another - works like a charm!

(No tabs showing, to drag? Create a new tab - at which point, you'll see a tab, for your original, to drag.)

Drag not working? You always end up with a separate window? Likely because (interestingly): It's disallowed to drag tags between a "private" window, and a normal window - looks like the private-ness is per-window.

2019/11/22

Recover lost Safari windows

You know how macOS Safari has that great feature (I believe credit goes to other browsers, for getting it earlier), where when you quit the app (or restart or it crashes), and when you start it up again, all your windows are right back where they were? And what a huge weight off your mind that is?

My Mac just had a Kernel Panic, and when I restarted, all the windows were gone - what a feeling of letdown and loss.

Luckily, I remembered another feature - which is apparently built on the same saved data:

History -> Reopen Last Closed Window

(If you like command-key shortcuts: shift - command - T)

And - faith restored: It saves a _bunch_ of them; just keep hitting it, to open the N last closed windows.

I don't know what the max of N is, but I did about 30, no sweat.

Phew!

2019/02/07

more about delight in the Apple Human Interface

Cool; I just used the "Back" function in Apple's Safari web browser - and happened to be paying attention enough, to see that what it actually did, was what I _wanted_, rather than what I asked for:

I happened to be in a tab created by a command-click of a URL in the previous tab; there was no "back" in this case - so it did _exactly_ the right thing - it closed the tab and returned me to the previous tab.

This is precisely the thoughtful design that Apple is known for:

  • Some of us (developers) know it directly, and know how much work it takes.
  • And the rest of us who use a Mac, iPhone, iPad, etc., know it simply because, when the interface surprises us, it's usually a pleasant surprise.

A delight if you will - which is actually an explicit part of the official Apple Human Interface Guidelines.

That's a lot of hard work, which is usually (by design!) unnoticed - the point is not the device, but what _you_ want to do with it.

Although, every once in awhile, it's nice to appreciate - thanks, folks!

2019/01/03

Delving Into Outlook / Exchange Server-Side Rules

Got a lot of rules?

Tired of paging through the GUI, trying to find just which rule is (mis)handling those emails?

Here's how to get the "code" for the rules, as text:

Use OWA - something like this, modified for your realm:

https://outlook.office.com/owa/?realm=YOUR-REALM-HERE&path=/options/inboxrules

Manual navigation (if the above URL doesn't work):
  • InBox Rules (under Options > Mail > Automatic Processing - or use Help)
  • At page bottom: "click here to generate a diagnostic report"
The resulting email contain your rules - now you can at least grep / search through them!

The file most people will be interested in, is "InBoxRules.txt"; here are some highlights from one:
  • Name="the name you gave your rule"
  • ExecutionSequence="101" <-- important; rules are executed in order
  • StateFlags="Enabled, ExitAfterExecution" <-- no further rules will be used
  • <Restriction
    • SubType="RecipientRestriction" <-- this rule is based on recipient
  • <Action…
    • Type="OP_MOVE"
  • FolderName="Deleted Items" <-- and the action is to delete the msg

Notes:
  • An XML browser helps a lot; I use BBEdit.
  • Yes; this is read-only - you do have to go back to the GUI to make any changes.
  • Since this web UI shows the contents of rules as text, it may be enough to use it directly (ex: using the browser's "Find" function) - with the added benefit of being able to make any desired changes directly. It is however, necessary to click through each rule, to see the full definition...
  • My previous post may also be helpful: How-To recover / export Outlook for Mac "Smart Folder" AKA "Saved Search" configs

2018/12/10

How-To recover / export Outlook for Mac "Smart Folder" AKA "Saved Search" configs

Apparently there's no way to export this part of the Outlook config directly. :/

However there are some strings to pull on, which can help:

There are "links" to the config data, in a sqlite DB, which can be seen like so:
sqlite3 ~/'Library/Group Containers/UBF8T346G9.Office/Outlook/Outlook 15 Profiles/Main Profile/Data/Outlook.sqlite' 'select * from SavedSpotlightSearch'
(The "UBF8T346G9" above, may vary?)


Even better, the "Smart Folder" configs (including, apparently, those that have been deleted?) are stored in this directory (for recent versions of Outlook for Mac):
~/'Library/Group Containers/UBF8T346G9.Office/Outlook/Outlook 15 Profiles/Main Profile/Data/Saved Searches/'

As a cheap hack, here's what I did, to recover the searches from one Mac, and implement on another:

  1. Dump the data; ex:
    find ~/'Library/Group Containers/UBF8T346G9.Office/Outlook/Outlook 15 Profiles/Main Profile/Data/Saved Searches' -type f | while read -r thePath; do echo; echo; ls -ld "$thePath"; cat "$thePath" | tr -d '\000' | tr -cs "[:print:]" '\n' | egrep -v '^[[:blank:]]*$' | sed 's/\([^ (]\)(/\1\'$'\n(''/g' | sed 's/)\([^ )]\)/)\'$'\n''\1/g'; done
    (Removes NULLs; translates non-printing chars to LFs; removes extraneous LFs; puts "sections" on separate lines - a bit hacky, since there's no docs that I can find, for this file format.)
  2. Interpret the needed bits; more info below.
Some tips, on decoding the data:
  • The data is apparently in two-byte characters - the above is a cheap hack which works in ASCII anyway. :/
  • Near the top, is some definitory info, like the com_microsoft_outlook_folderID to search in. (You may be able to deduce the correct folders, without having to figure out how to determine which folder corresponds to which ID.)
  • Then the "Smart Folder" name.
  • Then, finally, the search / query itself - there's a bit "encoding" here too, so not quite "cut and paste". :/
And since (on a Mac) these are stored in the "Raw Query" format, which uses the underlying Spotlight metadata, here's the official Apple doc on the File Metadata Query Expression Syntax.

Interesting note: This exposes the "Raw Query" syntax (leveraging Spotlight / mdfind), which can be a learning opportunity, for creating more interesting searches (since Raw Query docs are a wee bit sparse).

2018/10/26

git-crypt works smoothly - until it doesn't

Some repos use git-crypt to encrypt secrets; they are committed in encrypted form, and decrypted locally, auto-magically, using GPG keys.

This magic is, shall we say, simple until it's not -- in "interesting" ways.

This is written from the point-of-view of someone inheriting an existing config - which broke, due to multiple keys.

In the hope of saving someone else their sanity, here are a few learnings.
(Which are hopefully even correct.)

We'll start with the easy stuff; then, well, Buckle up...


How to get started as a new collaborator:

  • brew install git-crypt
  • install the "GPG Suite"
  • generate a key pair
  • upload your new public key to the interwebs
  • give public key to an existing collaborator, who must:
    • add the new user to their GPG keychain
    • sign the new user's key
    • git-crypt add-gpg-user --trusted USER_ID
      • Use '--trusted' to avoid dependency of public "Web of Trust"
        • (Yes; this is potentially less secure.)
      • The USER_ID above is usually the email address that the user configured their GPG keypair with
      • add-gpg-user should result in output like this:
[master 30babf07] Add 1 git-crypt collaborator
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 .git-crypt/keys/default/0/72E278AE2FB3...8F90BB21B36FD67.gpg

How was git-crypt set up in the first place?

(See above for a bit more detail on some of these steps, such as expected output.)
  • brew install git-crypt
  • navigate to the repo you want to use git-crypt with
  • git-crypt init
    • Note: This creates a symmetric key.
  • add the first GPG user: git-crypt add-gpg-user --trusted ADMIN-USER_ID
    • This user must exit already in GPG.
    • You might consider this the "admin" user; they'll be the only one to be able to decrypt secrets, add more users, etc. - until other users are added.
      • Why yes; it would be a good idea to add more people - say, if this person leaves the organization.
  • unlock, using new GPG key (will prompt for that key's passphrase): git-crypt unlock
  • config a .gitattributes file with contents like secretfile* filter=git-crypt diff=git-crypt
    • the .gitattributes file defines which files are to be encrypted
      • And  must be in place BEFORE adding a file that must be encrypted.
  • git add .gitattributes
  • commit and push: git commit -m 'your comment here'; git push

A few notes:

  • what causes the encryption to actually take place?
    • See the notes on the .gitattributes file, above.
  • check encryption status (for encrypted files, GITCRYPT shows at the top):
    • git-crypt status -e | awk '{print $2}' | while read thePath; do echo $thePath\: $(cat $thePath | xxd -l 9); done
    • Warning: You could push the change and confirm it's encrypted in the web UI - except if it's not, that secret is now forever* ensconced in your repo. (*How to remove secrets from a repo - AKA: It's too late.)
  • GPG items in MacOS keychain, are not named with "GPG", but with "GnuPG"
  • we're using GPG here (not PGP); it makes little difference to the procedure (ex: a GPG fingerprint is not for GPG only)
  • if freshly cloned, need to git-crypt unlock again (default state is locked)
  • who's got access?
    • ls -l .git-crypt/keys
    • each filename contains a user's fingerprint (look that up, on a keyserver)
  • Trouble getting file to actually encrypt?
  • Seeing errs like: "still unencrypted even after staging" OR "encrypted file has been tampered with" OR "Warning: one or more files is marked for encryption via .gitattributes but was staged and/or committed before the .gitattributes file was in effect" ?
    • unstage (ex: git reset HEAD secrets.yml)
    • redo the dance with git-crypt status -f and git-crypt lock --force
    • maybe start from a fresh clone - and save aside, any files that are unencrypted
    • be certain the file is really encrypted before using git add filename
  • But git-crypt status -e says the files are encrypted!
    • NO; it's only saying that those files are configured to be encrypted
    • check the contents to confirm if it's actually encrypted (see "check encryption status" above)
  • getting an err like ERROR! Unexpected Exception: 'utf8' codec can't decode byte 0xd0 in position 11: invalid continuation byte ?
    • Check your git-crypt config; if that's OK, reclone the repo (the git-crypt status may be hosed.)

How to reset the encryption on a repo:

Here be dragons; this should be avoided, but if you have to...
  • list files that have been encrypted: git-crypt status -e | awk '{print $2}' > encrypted-files
  • make sure repo is in UNlocked state: git-crypt unlock
  • save decrypted copies of all encrypted files; ex: git-crypt unlock; tar czf ../saved.tgz ./
    • if you don't have unencrypted copies anymore?
    • get them from another collaborator, old unlocked copy of the repo, ...
    • there is no known way to recover them otherwise
  • remove all encrypted files; ex: cat encrypted-files | xargs -t -n1 rm
  • remove all git-crypt files: rm -rf .git-crypt .git/git-crypt
  • ? may be necessary to save & remove the .gitattributes file too? (doubtful)
  • commit: git commit -a -m 'your comment here'
  • re-config git-crypt: git-crypt init
    • Note that this creates a new symmetric key, stranding files encrypted with any other key.
  • add the first AKA "admin" user: git-crypt add-gpg-user --trusted ADMIN-USER_ID
  • unlock, using new GPG key: git-crypt unlock
  • add any addtl users: git-crypt add-gpg-user --trusted ONCE-PER-ADDTL-USER_ID
  • if you removed the .gitattributes file above, copy it (or its contents) back
    • the .gitattributes file must be in place BEFORE adding a file that must be encrypted.
  • copy decrypted files back in
  • confirm files are decrypted: git-crypt status -e | awk '{print $2}' | while read thePath; do echo $thePath\: $(cat $thePath | xxd -l 9); done
    • It may be helpful to make the files different (ex: add a comment) to help force encryption with new key...
  • some extra git-crypt magic: git-crypt status -f
  • make sure repo is in UNlocked state: git-crypt unlock
  • force encryption: git-crypt lock --force
  • CONFIRM FILES ARE ENCRYPTED (they'll show GITCRYPT):
    • git-crypt status -e | awk '{print $2}' | while read thePath; do echo $thePath\: $(cat $thePath | xxd -l 9); done
  • if not, see notes above - it will be messy if you add (or worse commit) unencrypted info
  • for each of the encrypted files: git add ...
  • commit & push: git commit -a -m 'your comment here'; git push
  • you probably want to unlock again: git-crypt unlock
  • after keys are reset, a possible solution to: checkout (ex: of a branch) fails with "encrypted file has been tampered with":
    • make a fresh clone of the repo
    • leave it locked
    • checkout branch (ex: keys were reset on master, but old keys are left on your branch)
    • cherry-pick the commits for the new keys & newly-encrypted files (in chron order?)
    • then unlock
  • a fresh clone is best; otherwise, something like this might help: git-crypt lock --force; git stash; git pull

WHY would you ever want to reset the encryption on a repo??

  • You somehow got secrets committed, with multiple symmetric keys (ex: ran git crypt init more than once).
  • You want to be safe, after a collaborator has left the project.

References: