The National Institute of Standards and Technology (NIST) has identified natural language policies as the preferred expression of policy and implicitly called for an automated translation of ABAC natural language access control policy (NLACP) to a machine-readable form. An essential step towards this automation is to automate the extraction of ABAC attributes from NLACPs, which is the focus of this paper. We, therefore, raise the question of: how can we automate the task of attributes extraction from natural language documents? Our proposed solution to this question is built upon the recent advancements in natural language processing and machine learning techniques. For such a solution, the lack of appropriate data often poses a bottleneck. Therefore, we decouple the primary contributions of this work into: (1) developing a practical framework to extract ABAC attributes from natural language artifacts, and (2) generating a set of realistic synthetic natural language access control policies (NLACPs) to evaluate the proposed framework. The experimental results are promising with regard to the potential automation of the task of interest. Using a convolutional neural network (CNN), we achieved - in average - an F1-score of 0.96 when extracting the attributes of subjects, and 0.91 when extracting the objects' attributes from natural language access control policies.