To permanently delete a Quote (note that this cannot be undone), from the main menu go to Quote > View. 

You should see a page similar to

      
Click on Delete next to the Quote you wish to permanently delete.  Note that only Administrators can delete Quotes.