MT-diet demo: Demonstration of automated smartphone based diet assessment system

Background: According to several recent research results [1]-[4], obesity can increase the risk of many diseases such as diabetes, chronic kidney disease, metabolic disease, cardiovascular disease, etc. To prevent and treat the obesity efficiently and effectively, diet monitoring is an important factor. Purpose: Manual self-monitoring techniques for diet suffer from drawbacks such as low adherence, underreporting, and recall error [5]-[7]. Camera based applications that automatically extract type and quantity of food from an image of the food plate can potentially improve adherence and accuracy. However, state-of-the-art systems [8] have fairly low accuracy for identifying cooked food (only 63%) and are not fully automatic. To overcome these drawbacks such as low adherence, underreporting, recall error, low accuracy, and semi-automatedness, we introduce MT-Diet, a fully automated diet assessment system. It can identify cooked food with an accuracy of 88.93%. This is a significant improvement (over 20%) from the current state-of-the art system. Method: MT-Diet is a smartphone-based system that interfaces a thermal sensor with a smartphone. Using this system a user can take both thermal and visual images of her food plate with just one click. We used a database of 80 frozen meals which contain several different types of foods so that the actual total number of our food database 244 and the database has 33 different types of foods. By using the database, we demonstrate two core components: a) food segmentation, separating food items from the plate and recognizing multiple food items as a single food item, and b) food identification, determining the type of foods. Result: MT-Diet food segmentation methodology is fully automatic and requires no user input as opposed to recent works, the accuracy of separating food parts from the plate was 97.5%. The accuracy of food identification using Support Vector Machine with Radial Basis Function kernel based on color, texture, and histogram of oriented gradients features is 88.5%. Conclusion: We suggest a new and novel approach for diet assessment, MT-Diet. Our approach can potentially be an inexpensive, real time for the feedback on calorie intake, easy-to-use, privacy preservation, personalization based on eating habits of individuals, and fully automated diet monitoring system. The tool can also be used to conduct clinical studies to develop models of meal patterns that can be incorporated to design better artificial pancreas.