I've been using the Garmin GPSIII+ for a couple of years. mostly for fishing. It's pre-loaded with the major interstates and most highays. I upload topographical maps to it with separate Garmin software. It can be found for around $300. Most all models on the market nowdays are good enough for basic navigation, the newer, higher end units will have DGPS and WAAS for even more accuracy.
I've never done the geocaching, but I've thought about it many times. There are lots around here in Austin on the geocaching.com site.