There was plenty of idle, though it was enough of an irritant - especially since it had the highest cumulative CPU time of all processes - that I wanted to look into it.
(BTW: Nice concise man page for coreaudiod. :)
Interestingly, CPU usage dropped down under 2% as soon as I plugged in headphones, and jumped back up to 10% when I unplugged the headphones - very consistently, each time.
I profiled coreaudiod with Instruments (part of the Apple Developer Tools).
It was spending a lot of time in DspFuncHelper::process_IIR_xmm_LR(). The interesting symbols in the symbol stack under that: DspFuncEQ, IOAudioEngineUserClient, IOA_HWDevice, IOA_SingleDevice, IOA_Device, CAPThread.
I tried turning the iTunes EQ off. And then toggled it a few more times. No change. I tried quitting iTunes. Nothing I tried made a difference. I don't have any other audio apps (*) so I dropped it there.
There's some chatter online about it, though none seemed useful. (Mostly the standard voodoo of "Have you tried turning it off and on again?" (ref: The IT Crowd)
However there was one suggestion by a user named "bompi" in the Apple Discussions, to use launchctl to stop the process gracefully:
sudo launchctl stop com.apple.audio.coreaudiod
It automatically restarts - and even if you're listening to something at the time, there's only a very brief interruption.
I'd still like to figure out what happened, why, and how to prevent it, however this at least is a temporary fix.
*Correction: Obviously there are other apps that use audio, though no significant ones came to mind at the time. However, I do use Skype and that can obviously use audio - and I'd been conferencing about 12 hours earlier -- with some audio trouble (on my end only; the other side reported that it was fine). FWIW, it was Skype v184.108.40.20680 and I see that v220.127.116.1194 is available so I'll try that and see if that makes any difference...