It doesn't matter if you have not received any google warning for manual webspam but if your site has witnessed a significant drop in organic search traffic lately or post google penguin update, there is a high possibility of some serious issues with the backlinks pointing to your site and that those contradict with google's ranking algorithms. Your site's back links may have anchor-text issues, look artificial or were not naturally developed, are from low quality spammy seo directories etc.