Sounds good so far except that you still gotta shift that heat with a suitable heat sink otherwise the device will fry.

Correct. The only right way to do this is in combination with a water cooled setup.

BUT...Though the idea of peltier cooling is nice, I really don't like it.
This is because most of the times, peltier cooling is not worth the effort. You've got to use a really powerful peltier to really make a difference in temperature. (a difference of 2-3 degrees is not worth it IMO)
But, this has so many downsides that I really don't know why anybody would consider it.

1) biggest downside : peltier elements really DRAIN power. I mean REALLY drain a lot of juice. In practice, you'll always end up having to put a secondary PSU into your PC, solely for the purpose of powering that peltier.

It's not uncommon that one Peltier element alone will drain a lot more power than any other device in the PC!

2) not every PSU is a good choice. Most peltiers require AT LEAST a PSU that's rated at 16 ampères, some even need 24 ampères. Those PSU's don't come cheap.

3) The heat effciency for a peltier is really low : only 5-8%. Al the other energy is just lost. That's why it drains that much power.

In practice you'll need a pretty hefty peltier to reach temperatures below 10 °C.

Conclusion : it's not really worth it. If you really want to try die-hard cooling, you'l have to consider phase-change cooling. Great solutions in that area are Vapochill and Prometeia cases. (basically putting a mini fridge in your PC)

I've also added a pic of my watercooling setup when I was bleeding it. At this point it hasn't been built in to the case yet. (a safety period of 24hrs at least is needed to check for leaks) . On the picture you can see the pump with reservoir, the CPU block, the VGA block and the radiator. The bottle next to the circuit is a bottle of demineralised water.