Online users generate tremendous amounts of textual information by participating in different online activities. This data provides opportunities for researchers and business partners to understand individuals. However, this user-generated textual data not only can reveal the identity of the user but also may contain individual's private attribute information. Publishing the textual data thus compromises the privacy of users. It is challenging to design effective anonymization techniques for textual information which minimize the chances of re-identification and does not contain private information while retaining the textual semantic meaning. In this paper, we study this problem and propose a novel double privacy preserving text representation learning framework, DPText. We show the effectiveness of DPText in preserving privacy and utility.