After reading the whole Extensions-applicable parts of the How To Use Joomla! Report, here's summary of the important points:

• It's a good idea to install a local web server (on your computer). If you keep it as a mirror image of your remote site, you'll be able to install and test extensions you're interested in first. You can get a feel for them without cluttering up your main site. Just keep in mind that the environment will not be exactly the same, so it's not foolproof. Here's the link again for a local server setup
  
• Where-ever you do install (local or remote) new extensions for evaluation, uninstall them again if you're not going to use them
• Extensions DON'T always co-exist peacefully, beware
• Extensions can eat up a lot of time tracking down the best in the category.  Watch our Newsletter (you can join on the front page of the UseJoomla.com site) and the site in general for recommendations.