Modeling Human Behavior Using Smartphone and Smart Watch Data

Abstract

In this talk I will present some of our recent research on modeling Stress, Sleep and Transit using data from Smartphones and Smart Watches. There is a tremendous amount of high level inference that we can do by fully enabling the sensors on a smart phone, and we are just beginning to tap into this rich source of information. The new popularity of heart rate enabled smart watches could even further extend this trove. Although there is much that can be learned from call logs and location data alone, in our experiments we monitor data from a rich variety of sensor channels by implementing a wake lock on the phone CPU and recording dozens of channels. We additionally record features of audio from which we can determine when the user is speaking and from which we can additionally recognize salient audio events such as coughing, snoring, traffic and the presence of other people, along with qualities of other nearby people such as gender and age. Using high level inference we can determine the approximate locations of a person’s home and work and the mode of transport by which they commute each day. We can also start to model whether or not certain characteristics of a person’s day are “normal” for that person or “unusual” and correlate these changes with self-reported feelings of stress and other affective states. With a model of a person’s normal activities, people and their care givers could begin to track longitudinal changes which might reflect disease progression or the efficacy of a course of treatment.