It is a common practice for programmers to leave annotations during program development. Most of the annotated documentations are predominantly being used as the archive of the coding events for limited developers. We hypothesize that these annotations captured mass amount of valuable information which can be utilized to identify similar codes or to examine code quality. However, due to the annotating behaviors vary and the language composition can be complex, this work sets out to investigate a systematic method to examine the annotation semantics and their relations with codes. We designed a semantic parser to extract concepts from codes and the corresponding annotations. Additionally, text mining techniques are applied to summarize linguistic features from the annotations. We then build models to predict concepts in programming code annotations. Results show that the proposed semantic modeling method achieved a higher performance compared to a random guessed baseline.