In disasters such as the earthquake in Haiti and the tsunami in Japan, people used social media to ask for help or report injuries. The popularity, efficiency, and ease of use of social media has led to its pervasive use during the disaster. This creates a pool of timely reports about the disaster, injuries, and help requests. This offers an alternative opportunity for first responders and disaster relief organizations to collect information about the disaster, victims, and their needs. It also presents a challenge for these organizations to aggregate and process the requests from different social media. Given the sheer volume of requests, it is necessary to filter reports and select those of high priority for decision making. Little is known about how the two phases should be smoothly integrated. In this paper we report the use of social media during a simulated crisis and crisis response process, the ASU Crisis Response Game. Its main objective is to creat a training capability to understand how to use social media in crisis. We report lessons learned from this exercise that may benefit first responders and NGOs who use social media to manage relief efforts during the disaster.