Key Pain Points Addressed

Transcription

1 Xerox Image Search 6 th International Photo Metadata Conference, London, May 17, 2012 Mathieu Chuat Director Licensing & Business Development Manager Xerox Corporation

2 Key Pain Points Addressed Explosion in the number of images Finding a needle in a haystack Some images never retrieved Image search often relies on image file names and inexistent/unreliable tags Inexistent tags: photographers dislike having to type in keywords into the metadata of their image files Unreliable tags: anonymous taggers, sometimes crowds, in multiple languages Often search is intuitive and not based on textual description

3 Solution to Pain Points Technology Automatic extraction, identification and summarization of the visual content of images: digital image signatures Application Fast and intelligent search/navigation among very vast volumes of images Based on proximity/distance with digital signatures of other images (or distributions of images) Based on automatic tagging Combination of similar search & tag-based search Business benefits Improved image copyright commerce Higher search accuracy More effective & efficient digital asset management

4 Underlying Technology Compute digital signature of an image Calculate distance between two image digital signatures Calculate distance between image and category digital signatures?

5 Core Image Services Image Categorization Find Similar Image? class 1 ~? class N Image Matching Find duplicate images Filter out images Cluster Images

6 Visual Categorization at Xerox Automatically classifies images using visual content Current version: 700+ categories, interactive response time World-class results at ImageNet Large Scale Visual Recognition Challenge contest #2 in 2010, winner in 2011 Research work began in 2002 Technology breakthroughs the result of combining expertise in image processing, computer vision, and machine learning Strong IP Position 10 patents issued and 20 pending patent applications

7 Applications Stock photography Social networking Online advertising Digital asset management Visual mobile search Visual content filtering

8 Stock Photography Applications Help organize picture collections by category More reliable than user generated tags Rank the best representatives of a category Search a picture collection for identical or similar images Help buyers find the best picture in your catalog Rid picture collection of duplicate images Find out where images are used on the Internet Detect and prevent copyright infringements Other works on image aesthetics Color palettes retrieval Automatic image reflow for photo album creation

9 Demos I. Image recognizer II. Image search III. Categorizer training IV. Facebook social networking V. Android smartphone

10 2 complementary ways to image search Image search by similarity Image search by category

11 Visual search by category Auto-completion of category names Selection of a category out of 706 (e.g., gorilla) Computed signatures of ~6M photos from

12 Sceneries (67) Objects (278) Plants (111) Animals (136) Activities (69) 706 Food (46) Drawing (1) Flowers Fruits Mushrooms Nuts Trees Vegetables Dishes Drinks Vehicles Balls Clothing Monuments Instruments Tools Fish Birds Mammals Reptiles Butterflies Insects Arachnids Team Sports Solo Sports Mechanical Sports Outdoor Leasure Indoor Leasure Natural Sceneries Urban Sceneries Indoor Sceneries

13 Addressable markets Stock photo Web image search Online advertising E-commerce Digital Asset Management Verticals law enforcement intelligence IP medical brand management security Customization of categories Scalability of the business through training web service

14 Xerox training Customer training Expansion of the category system Customer owns the trained model Xerox may acquire the customer-trained model to expand the universal category system Customer acquires the trained model Xerox expands the universal category system Customer specific category Generic/universal category

15 Commercialization - For evaluation: - Similar search in test collections - Xerox categorizers - To train & test new categorizers - For commercial ramp-up For high commercial Service Levels

16 Contact point THANK YOU!

17

18 BACK-UP SLIDES

19

20 Image recognizer

21 Categorizer Training Web Service

22 Technology Differentiators High performance generic feature extraction Resulting in image digital signature performing very well for both categorization and image retrieval Processing speed Interactive response time Scaleability Easy to add incremental categories Robustness Robust to variations in view, imaging, lighting, occlusion Optimization capabilities Depending on target computing and memory constraints

23 Social Networking Applications Detect and block publication or use of copyright material Pictures and video Enable users to visually search content Show me similar pictures Help enforce content policies Detect inappropriate content Entertaining social applications What do your pictures say about you?

24 Online Advertising Applications Visual intelligence for media Align ads with content Display ads that complement Web content Show car ads when viewing car pictures Safeguard advertisers brands

25 Digital Asset Management Visually search large picture and video collections Discover where digital assets are used on the Internet Prevent misuse Index videos by identifying content that appears in the video frames Search videos for specific content Video summarization Extract a storyboard Query by picture or video Find similar pictures and videos

26 Visual Mobile Search Applications Query by picture Look up information, history, coupons and special deals, hours of operation, price Picture as transaction gateway Make reservations, buy tickets, order services, make inquiry

27 Visual Content Filtering Applications Monitor image and video attachments Enforce business security, IP, and content policies

28