I'd definitely pick the PSU fan as the culprit.
The Shuttle SN45G takes a standard-sized 1U rackmount PSU, so I swapped it out for a Seasonic SS-250SU. It's definitely quieter but, more importantly, the noise is also a different pitch, so it's not as annoying. I think the most important thing, though, is that it's temperature controlled.
According to "sensors", it's running at 38C, which I'm happy with.
Using a borrowed power meter, it's drawing 16W, as compared to the current server's 130W or so.
I'm going to look at swapping out the CPU fan the next time I have more than £4 worth of stuff to buy.
The only other things making any noise are the internal hard disk -- I might look at a CF/IDE adapter (unless anyone makes PATA SSD drives) -- and the video card fan (but passively-cooled AGP cards are kinda hard to find these days).
Currently, it's sitting on the floor (I removed the metal feet and replaced them with rubber ones), with a 1TB WD MyBook hanging out of the back.