Straight from the on MP3/Tag Studio's page:

Q: Will I be able to totally remove all the advert related files from my computer as soon as I register the program?
A: Yes! As soon as the program is registered, all the advert related files can safely be deleted from your system. You will get detailed instructions about how to do this together with the other registration info. Once you have registered the program on your computer, a special file will be created. In the future you can then download special totally advert-free versions of the program from the website that will work only if you have such a file.

Q: It's not enough for me to be able to remove the advert components after registering the program, I don't even want to install an adware-enabled program on my computer. If I register, can I have a totally adware-free install package?
A: Sure! Just send me a mail right after registering the program, and you will get instructions on how to acquire such an install package.

(Hmm... should I slap Tony for not reading a FAQ entry? Or is he going to complain that the FAQ wasn't easy enough to find, since it's under "Contact info"? Sorry, Tony... just couldn't resist!)

Also, as for whether the files stick around... I have the registered version and Ad-aware, so I can't say for certain, but I believe they are removed completely. Magnus is pretty good about respecting people's wishes.