Doubly fed induction generators (DFIGs) are widely used in wind power generation. For controlling DFIGs to maintain network frequency within a safety range, the proportional-derivative (PD) type virtual inertia controllers (VIC) are used in the active power control of DFIGs. However, as is well known, wind power generation conditions change directly with wind conditions in nature. Such changes create great challenge for the VIC design and actually force the control designs to go beyond the traditional problem formulation of using explicit objective functions associated with specific optimality. Controller parameter tuning thus necessarily becomes a part of the controller design. In this paper, we propose an approximate dynamic programming (ADP) structure for online tuning of the PD type virtual inertia controller parameters. The proposed ADP structure naturally takes into account the PD control into design objective and provides the PD controller with online parameter tuning capability through learning. Design and implementation details of the proposed methodology, including neural network weight initialization, design of the reinforcement signal, data preprocessing, and a bound on the online tuned parameters are discussed in this paper. Simulation studies carried out on the Power System Computer Aided Design/ Electro Magnetic Transient in DC System (PSCAD/EMTDC) software are used to demonstrate the effectiveness and efficiency of the proposed ADP-based online VIC parameter tuning methodology.