I generally notice every little noise and it drives me nuts. I've never noticed any buzz from CFLs.

I get the impression that your CFLs are kinda bare. You might try putting some sort of shade around them to see if that helps the harshness. I'd just wrap a piece of wax paper around it for a few minutes to see if you think that improves things.