Deep Learning Meets Heterogeneous Computing. Dr. Ren Wu Distinguished Scientist, IDL, Baidu

Transcription

1 Deep Learning Meets Heterogeneous Computing Dr. Ren Wu Distinguished Scientist, IDL, Baidu

2 Baidu Everyday 5b+ queries 500m+ users 100m+ mobile users 100m+ photos

3 Big Data Storage Processing Webpages Index Update Log >2000PB PB/day 100b-1000b 100b-1000b 1b-10b/day 100TB~1PB/day

4 Infrastructure

5 Infrastructure ARM Servers Higher density GPU 服 务 器 Much better performance Data center containers Faster deployment Self-design switches Much lower cost

6 Big Very large scale data mining analytics visulization, etc Data warehouse World class in size A.I. Brain Deep learning World s first R. I. Software foundation Elastic cloud 100+PB data processing Servers and Data centers Best in Asia Self designed Huge # of servers

7 Nine Technology Challenges On Aug 13, 2012, CEO Robin Li gave a keynote speech at ACM KDD, and proposed nine major technological challenges to the academic research community. The top three are: 1. OCR in natural images 2. Speech recognition and understanding 3. Content-based image retrieval (visual search)

8 Deep Learning Since /24/14 8

9 Deep Learning vs. Human Brain object models Deep Architecture in the Brain Area V4 Higher level visual abstractions object parts (combination of edges) edges Area V2 Area V1 Retina Primitive shape detectors Edge detectors pixels pixels Slide credit: Andrew Ng

10 Top breakthrough technology 2013 MIT Technology Review, April 23 rd, 2013

11 Baidu IDL n Announced its first research arm in Jan n Institute of Deep Learning (IDL) n The focus is Artificial Intelligence n Two locations: Beijing and Silicon Valley

12 Progress of Deep Learning at Baidu Big improvement on speech & image recognition (2013) Speech: error rate reduced by 25% OCR: error rate reduced by 30% Face: LFW benchmark, 94% correct DNN CTR for search ads was launched on May 20 th 2013, serving billions of search queries everyday substantial improvement

13 Baidu Visual Search

14 Visual Search: Faces

15 Visual Search Example

16 Visually similar images The competition Baidu

17 Another Example Image uploaded Baidu The competition

18 Visually Similar Images - Comparison Image uploaded Baidu The Google competition 搜 索 结 果

19 CBIR The Competition

20 Image Recognition - Flowers

21 百 度 魔 图 :PK 大 咖 Peak uploading rate at 100 million images per day! IOS APP #1 for 3 weeks

22 Deep Learning Voice, Text DNN for Speech 10k hours of voice data 10b training samples Months on a GPU cluster Image User

23 Typical scale of training data Datasets Image recognition: 100 millions OCR: 100 millions Training time: Weeks to Months on GPU clusters Speech: 10 billions CTR: 100 billions Big data + Deep learning + HPC = Success Projected training data to grow 10x each year

24 Mobile Applications of DNN

25 手 机 百 度 随 时 知 道

26 DNN Anywhere, Anytime DNN-based image recognition on mobile device No connectivity needed Real time, directly works on video stream Everything is done within the device What you point is what you get OpenCL based, highly optimized Large deep neural network models Thousands of objects, flowers, dogs, and bags etc Unleashed the full potential of the device hardware World s first in-place mobile DNN app? And the best!

27 DNN Anywhere, Anytime

28 百 度 酷 耳

29 DNNs Everywhere Supercomputers Datacenters ( cloud ) Wearable devices IoTs Tablets, smartphones

30 DNNs Everywhere Supercomputers Datacenters Tablets, smartphones Wearable devices IoTs 1000s GPUs 100k-1m servers 700m (in China) Billions? Supercomputer used for training Trained DNNs then deployed to data centers (cloud), smartphones, and even wearables and IoTs

31 Heterogeneous Computing Supercomputers Data centers (cloud) Smart phones Wearable devices! Big data + Deep learning + HPC HC = Success

32 OpenCL-based Open ECO-SYSTEM Diverse industry participation, from cell phones to supercomputers o Processor vendors, system OEMs, middleware vendors, application developers. OpenCL is the industry standard embraced by many companies. * Courtesy of Simon McIntosh- Smith and Tom Deakin

33 Summary Big data + Deep learning + High performance computing = Intelligence Big data + Deep learning + Heterogeneous computing = Success

34 Baidu USA And we are hiring Heterogeneous Computing experts Parallel algorithm and performance experts CUDA/OpenCL Experts FPGA experts Andrios/IOS experts Data scientist Infrastructure Engineer

35 Thank you! Dr. Ren 韧 在 百 度