You shouldn’t take action to clean up your Twitter following base until you have about 500 or so followers. Twitter operates in a reciprocal sphere. Initially to get people to follow you, you need to follow them. It just works that way in the real world.
However as your account grows and you get more serious about using Twitter and not just dabbling with it, you want to cut out people from your list that are sapping your SocialRank. That means chopping out followers who are not following you back. Don’t cut out people who have merit or who are high profile, but cut the people out who initially followed you to build their fan base and then chopped you. Additionally you want to cut out people who are just spamming your fan base with their repetitive tweets or get rich quick schemes.
To make this easy, I have found a free tool that allows me to cull through my fan base and with one click drop non- followers. I use www.JustUnfollow.com to clean up my list. (They are not paying me – I just like the online application.)
First login to your Twitter account then visit JustUnfollow.com and allow it to connect with your Twitter account. You will be able to unfollow people right from that screen. As your account and following base grows it is important to keep a differential between the number of people you follow and the people who follow you. Just Unfollow makes it easy to do.