Social media expands the ways people communicate with each other. On a popular social media website, a user typically has hundreds of contacts (or friends) on average. As a person's social network grows, friend management is increasingly important for effective communications. Often, one can only afford to maintain close friendship in a small scale due to limited time and other resources. In other words, the majority of one's connections are so-so friends and do not hold strong influence on the user. One approach resorts to network denoising, by which unimportant connections are removed as noise. We study the challenges of network denoising in social media and how we can leverage a variety of social media information to denoise the links. We formulate the network denoising task as an optimization problem, and show the efficacy of our network denoising approach and its scalability experimentally in the domain of behavior inference.