Kitchen Manager: Your shopping and Food Inventory Made Easy Lina Alaoui (lina.alaoui@gmail.com ) James Deen (gtg118g@mail.gatech.edu) Stephanie Ahn (sahn36@gatech.edu) Fei Ye (yefei@gatech.edu) ABSTRACT The Kitchen Manager is a smart food inventory management system for tracking food. This inventory system tracks food coming into the household and when it is discarded. This is done through two methods: 1. Items are added to the home s inventory by linking right up to the store. So a user buys something and it is automatically added to the users inventory either by integrating into credit cards or loyalty cards. 2. Items are deleted from the home s inventory by a smart trashcan. The trash has camera located around the rim which when combined with image recognition allows the system to recognize when items are discarded. Tracked discarded items are combined with the user s habits which can be used to automatically build a grocery shopping list. This shopping list can either be printed out or ordered (via a screen in the fridge door) for pick up right from the grocery store. Buying items will of course add them back to the inventory list. So by taking the focus away from one appliance like other inventions have done our idea focuses on the whole system (The home inventory, the store, and the trash can). Author Keywords Interface, User Experience, Ubiquitous Computing INTRODUCTION Managing the inventory of a house hold is not necessarily a difficult task it however can be quite tedious. Keeping a list is a popular method, whether digital or physical, however this method requires constant upkeep. Most people can relate to getting back from the grocery store and forgetting an item just because they forgot to put it on their list. Also figuring out what one has can sometimes require digging to the back of a spice cabinet or refrigerator. The kitchen manager aims to automate not only the process of list making but of getting the groceries too. The kitchen manager automatically builds an inventory of what is in the home based off credit card or frequent buyer cards. This automatic list building gets rid of the need for the user to manually enter items. So instead of bending over and digging through your fridge to check what you have in your house hold all you have to do is do a digital search. This can be done through an interface on fridge or through a mobile app. Items are deleted off the list when they are thrown in the trash. Cameras on the trash can capture images of items being thrown away and then image recognition software can identity the item. Using image, as opposed to bar code scanning, eliminates the need for the user to put the item in a specific orientation before discarding it. The kitchen manager also make shopping easier by providing the feature to send your grocery list to the store and let them do the shopping. Systems like this are already in place, for example Publix, so integration would not be difficult. SUPPORTING TECHNOLOGY In this project, the fridge screen is a surface computing platform that responds to natural hand gestures. We will use Microsoft Surface technology to develop it. Here is a brief introduction of Microsoft Surface technology. Microsoft Surface is a surface computing platform that responds to natural hand gestures and real world objects [1]. It has a 360-degree user interface, a 30 in (76 cm) reflective surface with a XGA DLP projector underneath the surface which projects an image onto its underside, while five cameras in the machine's housing record reflections of infrared light from objects and human fingertips on the surface. The surface is capable of object recognition, object/finger orientation recognition and tracking, and is multi-touch and is multi-user. Users can interact with the machine by touching or dragging their fingertips and objects such as paintbrushes across the screen, or by placing and moving placed objects. This paradigm of interaction with computers is known as a natural user interface (NUI). Surface has been optimized to respond to 52 touches at a time. During a demonstration with a reporter, Mark Bolger, the Surface Computing group's marketing director, "dipped" his finger in an on-screen paint palette, then dragged it across the screen to draw a smiley face. Then he used all 10 fingers at once to give the face a full head of hair.
Partner companies use the Surface in their hotels, restaurants, and retail stores. The Surface is used to choose meals at restaurants, plan vacations and spots to visit from the hotel room. Starwood Hotels plan to allow users to drop a credit card on the table to pay for music, books, and other amenities offered at the resort. The anchor analyzes polling and election results, views trends and demographic information and explores county maps to determine voting patterns and predict outcomes, all with the flick of his finger. In some hotels and casinos, users can do a range of things, such as watch videos, view maps, order drinks, play games, and chat and flirt with people between Surface tables. Microsoft notes four main components being important in Surface's interface: direct interaction, multi-touch contact, a multi-user experience, and object recognition [2]. Direct interaction refers to the user's ability to simply reach out and touch the interface of an application in order to interact with it, without the need for a mouse or keyboard. Multitouch contact refers to the ability to have multiple contact points with an interface, unlike with a mouse, where there is only one cursor. Multi-user is a benefit of multi-touch several people can orient themselves on different sides of the surface to interact with an application simultaneously. Object recognition refers to the device's ability to recognize the presence and orientation of tagged objects placed on top of it. Fridge Inventory The Fridge inventory has four views to view the inventory. The Picture view graphically displays the items in the fridge. Each item has a corresponding icon. The A Z view organizes the inventory items alphabetically. The Category view organizes the items in logical groups such as dairy, beverage, meat, vegetables, etc Finally, the By Date view shows the items based on the date they were purchased and added to the inventory. FRIDGE INTERFACE The fridge interface is designed for people to easily manage the fridge inventory and shopping lists. The interface starts with three main components: a fridge inventory, a shopping list, and an order pick up.
Shopping List The Shopping list has two views, A Z view and Category view. When items are removed from the fridge and thrown into a trash can, a scanner or an image recognition system on the trash can detects which items are removing and adds the items into a shopping list. By receiving information from stores, the shopping list has price information. Also, the interface can easily access to discount information and apply the discount instantly to the sale items on the shopping list. Healthier Suggestion When users finalize the shopping list, the fridge interface provides healthier suggestions based on the shopping list items. The healthier suggestion provides an alternate item with nutrition information. Users can replace the item as suggested or ignore the message.
Order pick up Users can send the shopping list to the store and set up date and time to pick up the orders. Users can review their sent shopping list and cancel it if necessary. Update Fridge Inventory When the purchase is made, the payment information is sent to the interface, and the inventory is updated with new items. The interface indicates new items by highlighting or having new next to the items.
Manual Deletion Managing fridge inventory is mostly automated, but it can be done manually upon users need. The interface allows users to manually delete items from the inventory. Users can easily delete the items and cancel the deletion they made. category or alphabetically. Similarly to the fridge interface, it allows the user to select a pick up time when sending an order to the store. MOBILE APPLICATION INTERFACE The mobile application is mainly used to view a shopping list and send it to the store for scheduled pick up. The screenshots below show the mobile interface used to browse the shopping list items. Items are ordered by FUTURE WORK When we started this project, we considered several options for the actual tracking of food items in the fridge or more general in the kitchen. While we decided to focus on designing an interface for the scope of this project, we had evaluated technology options for inputting items into the inventory and removing them: Items input: integration from store frequent shopper card database, scanning RFID tags, QR codes or barcodes on food items. Items output: image recognition of food item or scanning RFID tags, QR codes or barcodes on food items. Suggested next steps for this project would be to chose on of these technologies and develop a prototype for tracking the food items in a database. On a commercial version of Kitchen Manager, we suggest stripping down some of the functionality out of the fridge interface. Out of the box, the product would be configured to offer only a view into the food inventory from the fridge interface. The mobile application and possibly the product s website would offer the shopping list functionality and ability to send the order to the store for pick up. Users should have the option to turn on full functionality on the fridge interface but we believe that they would tend to do the more time consuming tasks (e.g., edit the shopping list and send it to the store) on their mobile device. Limiting the fridge functionality would also allow us to further simplify the fridge interface. On the current design, food inventory shows whether an item is in the fridge or not but does not provide any indication of quantity. We suggest adding a quantity indicator in the next version. Several extensions to this system could be developed. We will discuss here two possibilities that would enable customized food tracking and improve quality of patient care. The first extension is to have a set of food containers
of different sizes that could be tracked by Kitchen Manager via RFID tags. These containers could be used to store home cooked food in the freezer or fridge as well as leftover food. Individuals often forget how long it has been since they have some meat in the freezer or since they brought in some takeout food. These items would allow a complete integrated food inventory tracking. The second extension is to allow physicians to access their patient s food inventory. Some patients have dietary restrictions and while doctors give their patients guidelines, knowing exactly what they are eating and how fast they are consuming it would allow them to better understand their patients eating habits. Physicians could potentially push healthy suggestions to their patients thru Kitchen Manager or even send them reminders. PRELIMINARY DEMO http://www.youtube.com/watch?v=es5ovluv_cy
REFERENCES 1. Wikipedia: Microsoft surface http://en.wikipedia.org/wiki/microsoft_surface 2. Microsoft surface http://www.microsoft.com/surface/ 3. [1] Suhuai Luo, Hongfeng Xia, Yuan Gao, Jesse S. Jin, and Rukshan Athauda. 2008. Smart Fridges with Multimedia Capability for Better Nutrition and Health. In Proceedings of the 2008 International Symposium on Ubiquitous Multimedia Computing (UMC '08). IEEE Computer Society, Washington, DC, USA, 39-44. DOI=10.1109/UMC.2008.17 4. Laurel Swan and Alex S. Taylor. 2005. Notes on fridge surfaces. In CHI '05 extended abstracts on Human factors in computing systems (CHI '05). ACM, New York, NY, USA, 1813-1816. DOI=10.1145/1056808.1057029 http://doi.acm.org/10.1145/1056808.1057029 5. 'Smart' Fridge Will Even Order Groceries - Machine Design 69(15) 44 Aug 7, 1997 6. Designing Technology for Domestic Spaces - A kitchem Manifesto - The Journal of Food and Culture http://alumni.media.mit.edu/~jofish/writing/gastronomica 24.pdf 7. Ki-sung Hong, Hyoung Joong Kim, Chulung Lee, "Automated Grocery Ordering Systems for Smart Home," fgcn, vol. 2, pp.87-92, Future Generation Communication and Networking (FGCN 2007) - Volume 1, 2007. DOI 8. Judy E. Scott and Carlton H. Scott. 2008. Online Grocery Order Fulfillment Tradeoffs. In Proceedings of the Proceedings of the 41st Annual Hawaii International Conference on System Sciences (HICSS '08). IEEE Computer Society, Washington, DC, USA, 90-. DOI=10.1109/HICSS.2008.335 http://dx.doi.org/10.1109/hicss.2008.335