Authenticating users in online social networks (OSNs) is different from traditional authentication, because the participants in the authentication may not share any prior secret information. In this paper, we propose a decentralized authentication framework to help users authenticate unknown users in an OSN. In our framework, a user requests certificates from other trusted users to prove his identity in authentication. However, collecting certificates is constrained by the fact that trust is usually attritted with the length of a trust chain. Considering this constraint, our framework utilizes a decentralized online learning approach to help users collect more certificates. We further prove that the number of the collected certificates by each user using our protocols is close to optimum.