Always make sure your hard disk is defragmented. Fragmentation affects the performance of any hard disk severely. A defragmentation utility is included with Windows. See your computer manuals for details.
When you use Purge Segments in the Pool, it will be evident which files are no longer used in a Song (the files not used will have no Segments at all). This gives you a way of finding out which files can be deleted. But please remember that an audio file can be used in more than one Song.