The grand transition of electric grids from conventional fossil fuel resources to intermittent bulk renewable resources and distributed energy resources (DERs) has initiated a paradigm shift in power system operation. Distributed energy resources (i.e. rooftop solar photovoltaic, battery storage, electric vehicles, and demand response), communication infrastructures, and smart measurement devices provide the opportunity for electric utility customers to play an active role in power system operation and even benefit financially from this opportunity. However, new operational challenges have been introduced due to the intrinsic characteristics of DERs such as intermittency of renewable resources, distributed nature of these resources, variety of DERs technologies and human-in-the-loop effect. This paper mainly focuses on demand response (DR), which is a major type of DERs and is highly influenced by human-in-the-loop effect. A data-driven based analysis is implemented to analyze and reveal the human-in-the-loop effect. The results confirm the critical impact of demographic characteristics of customers on their interaction with smart grid and their quality of service (QoS). The proposed framework is also applicable to other types of DERs.