I started off with a Garmin Etrex Venture, which I now occasionally still use when debugging gpsapp on my laptop. In my car I wired one of the trimble SV6 receivers that were sold by www.bgmicro.com for something like $25 a while back. Some people are using Motorola Oncore, which were also sold by bgmicro for about $15 without the active antenna, or modified palm-III delorme earthmates that went for a similarily ridiculous low price when Staples decided to drop all palm III accessories.

I don't know of any current deals of this kind, and all of these low-cost solutions do require some additional parts and soldering to get power and RS232 level serial signals. But basically any receiver that spits out the industry standard NMEA sequences over a serial line should just work fine.