While we’ve seen new robotic devices that can do everything from driving a car to thwarting underwater terrorists, a team of robotics researchers from Cornell and Stanford just want to make sure you never forget to put away the milk. At the International Conference on Robotics and Automation yesterday, the team presented Watch-Bot, a sort of robo-sentry that keeps an eye on the most mundane tasks in your home or office and politely shames you with a reminder if you forget to do them.
Using its Xbox Kinect 3D sensor, a camera, a laptop and a laser pointer, Watch-Bot observed a week’s worth of human activity in a kitchen and an office. During that time, it collected 458 videos — about half of which included someone human deliberately “forgetting” to do something. The team then made Watch-Bot analyze the videos and use its unsupervised learning algorithm to determine which human actions were intentional and which ones — like leaving the milk out on the counter — were accidental. Using probabilistic learning models, Watch-Bot was able to independently figure out which actions the humans were forgetting.
When Watch-Bot does notice the clumsy human in the room forgot to do something, it quietly highlights that item with the laser pointer until it is put away or dealt with. Among the tasks that Watch-Bot was able to identify were such pet peeves as: not putting a book away after reading it, not turning a computer monitor off after using it, forgetting to put the milk away and leaving food in the microwave. Although it is only able to correctly identify those brainfarts about 60 percent of the time, according to the researchers, “most study participants thought that the robot was helpful,” and not actually just a passive aggressive roommate.
One interesting thing to remember here: Watch-Bot also has no context for the objects in the room. It only acts based on probabilistic patterns it has observed and therefore has no idea what the milk even is, much less that it has an expiration date.