+1
When an application/game is installed, it scatters files on your hard drive as it sees fit. Most of it goes into its own folder in program files, but the installation may commonly put entries into the registry. I've seen some applications also put bits and pieces in the Windows folder, etc.
It's recommended to go through Add/Remove Programs (or if the program has an uninstall on the Start menu). This way, the uninstallation goes back to a file that was used during the original installation...there it will find all the places that the program got scattered and then properly uninstall each piece.
If one or more pieces are missing/corrupted when it tries to uninstall...the uninstall process may error out before completeing it's job. Manually removing each piece can take time just to find the pieces..plus you might leave a little bit behind (which is likely harmless, but could leave an error message when you boot up or close down the PC...depends highly on the program).
Another trick is to fully reinstall the program..many partially installed programs will successfully do this, (maybe reboot in between...depends on the program). Then you may be able to successfully uninstall the program through Add/Remove Programs.
Since each program is different, the exact steps will vary greatly.
Interestingly, many spyware/adware type apps are just programs like this, but their programmers simply leave out the steps to tell Windows where everything was placed.