Developing apps for Android Auto

Developing apps for Android Auto talk by Christian Dziuba & Sina Grunau 1

About us Sina Grunau, Christian Dziuba Android apps (1.5-6.0) Automotive context: research + media apps c4c Engineering GmbH Automotive software app team 2

Topics 1. Introduction 2. How does Android Auto work? 3. APIs 4. Development Workflow 5. Summary 3

1 Introduction 4

Today family car car sharing 5

25 % of car accidents in the U.S happen because of smartphone usage while driving source: http://ghsa.org/html/media/pressreleases/2011/20110707_sfdist.html 6

Goal: Safe interactions (Navigation, phone, music) while driving Minimal driver distraction 7

In-Vehicle-Infotainment-Systems Car Vehicle Data Head Unit Car control Navigation Calls Smartphone Apps Calls Music Music 8

Smartphone - Car - Integration Car Vehicle Data Head Unit Navigation Calls Smartphone Apps Navigation Calls Music Music 9

Smartphone - Car - Integration: Comparison Technology Platform Development Car support Android Auto Android app extension /service limited, but promising Apple Carplay ios extension (?) / contact Apple Mirrorlink Android apps / certification limited, but promising limited Bosch MySpin ios/android not yet public very limited 10

2 Android Auto 11

Current state of Android Auto June 2014 / first announcement at Google I/O 2014 March 2015 / first aftermarket head-units (e.g. Pioneer, Kenwood) March 2015 / public release of companion app in Play Store Summer 2015 / first car manufacturers integrate Android Auto 2016 / 20+ car manufacturers announced to support Android Auto 12

Car Manufacturers supporting Android Auto Source: https://www.android.com/auto 13

How to use Android Auto 14

How to use Android Auto 15

Features: General 16

Features: Navigation 17

Features: Phone 18

Features: Music 19

Features: Exit Screen + Developer Mode 20

Currently compatible Android Auto apps Source: https://play.google.com/store/apps/collection/promotion_3001303_android_auto_all 21

3 Android Auto APIs 1. Messaging 2. Audio 22

Android Auto Apps audio app or messenger Android Auto app head unit service / interface get data forward user commands display logic mobile phone 23

Add Android Auto support to your app AndroidManifest.xml <application> <meta-data android:name="com.google.android.gms.car.application" android:resource="@xml/automotive_app_desc" /> </application> res/xml/automotive_app_desc.xml <automotiveapp> <uses name="notification"/> </automotiveapp> OR <uses name="media"/> 24

Messaging User Workflow 25

Messaging API received message is shown in overview screen no message content shown! message is read aloud, when button is pressed reply by voice 26

Notifications Notification CarExtender UnreadConversation MessageRead BroadcastReceiver RemoteInput Read PendingIntent Reply PendingIntent MessageReply BroadcastReceiver 27

Notifications Notification CarExtender UnreadConversation MessageRead BroadcastReceiver RemoteInput Read PendingIntent Reply PendingIntent MessageReply BroadcastReceiver 28

Create Message Reply Pending Intent Intent replyintent = new Intent().addFlags(Intent.FLAG_INCLUDE_STOPPED_PACKAGES).setAction("com.mychat.app.MY_ACTION_MESSAGE_REPLY").putExtra("conversation_id", conversation_id); PendingIntent replypendingintent = PendingIntent.getBroadcast( getapplicationcontext(), conversation_id, replyintent, PendingIntent.FLAG_UPDATE_CURRENT); 29

Notifications Notification CarExtender UnreadConversation MessageRead BroadcastReceiver RemoteInput Read PendingIntent Reply PendingIntent MessageReply BroadcastReceiver 30

UnreadConversation.Builder // build a RemoteInput for receiving the spoken voice input RemoteInput remoteinput = new RemoteInput.Builder("reply_key").setLabel("please say your message").build(); // group your messages from a particular sender UnreadConversation.Builder unreadconvbuilder = new UnreadConversation.Builder("conversation participant name").addmessage("the message text goes here").setlatesttimestamp(system.currenttimemillis()).setreadpendingintent(readpendingintent).setreplyaction(replypendingintent, remoteinput); 31

Notifications Notification CarExtender UnreadConversation MessageRead BroadcastReceiver RemoteInput Read PendingIntent Reply PendingIntent MessageReply BroadcastReceiver 32

Create a notification NotificationCompat.Builder builder = new NotificationCompat.Builder(getApplicationContext()).setSmallIcon(R.drawable.notification_icon).setLargeIcon(BitmapFactory.decodeResource( getapplicationcontext().getresources(), R.drawable.contact)).setContentText("this is the message text").setcontenttitle("participant name").setcontentintent(readpendingintent).extend(new CarExtender().setUnreadConversation(unreadConvBuilder.build())); contains the message(s) notificationmanager.notify(conversation_id, builder.build()); 33

Notifications Notification CarExtender UnreadConversation MessageRead BroadcastReceiver RemoteInput Read PendingIntent Reply PendingIntent MessageReply Broadcast Receiver 34

Register BroadcastReceiver <application>... <receiver android:name=".messagereplyreceiver"> <intent-filter> <action android:name="com.mychat.app.my_action_message_reply"/> </intent-filter> </receiver> </application> 35

BroadcastReceiver Example public class MessageReplyReceiver extends BroadcastReceiver { @Override public void onreceive(context context, Intent intent) { int conversation_id = intent.getintextra("conversation_id", -1); Bundle remoteinput = RemoteInput.getResultsFromIntent(intent); } CharSequence reply = remoteinput.getcharsequence("reply_key"); //ToDo send reply to other user } 36

Media API extend your audio app to allow users to: browse play audio content in the car 37

Define your app icon res/drawable/ic_notification.png <application> <meta-data android:name="com.google.android.gms.car.notification.smallicon" </application> android:resource="@drawable/ic_notification" /> 38

Audio Playback API MediaBrowserService MediaSession / MediaSession.Callback Android Auto Android Wear UI App 39

Overview MyAudioApp ongetroot() Android Auto MediaBrowserService ROOT_ID, sessiontoken Media Browser onloadchildren(id) List<MediaItem> MediaSession onplay(), onpause(),....setmetadata().setplaybackstate() Media Controller 40

MediaBrowserService Announcement of your media browser Service <application> <service android:name=".mymediabrowserservice" android:exported="true"> <intent-filter> <action android:name= "android.media.browse.mediabrowserservice"/> </intent-filter> </service> </application> 41

Overview MyAudioApp ongetroot() Android Auto MediaBrowserService ROOT_ID, sessiontoken Media Browser MediaSession Media Controller 42

MediaBrowserService - ongetroot() Authentification & binding to your media browser service @Override public BrowserRoot ongetroot(string clientpackagename, int clientuid, Bundle roothints) { // Check if this client is allowed to use service } return new BrowserRoot("my_Root_Id", Bundle roothints); Read more: https://developer.android.com/samples/mediabrowserservice/src/com.example.android.mediabrowserservice/packagevalida tor.html 43

Overview MyAudioApp MediaBrowserService Android Auto Media Browser onloadchildren(id) List<MediaItem> MediaSession Media Controller 44

MediaBrowserService - onloadchildren() @Override public void onloadchildren (String parentmediaid, Result<List<MediaItem>> result) { } List<MediaBrowser.MediaItem> mediaitems = new ArrayList<MediaBrowser.MediaItem>(); // create your mediaitems here... result.sendresult(mediaitems); Don t block the UI thread! use result.detach() to detach in from the current thread! call result.sendresults(t result) when ready 45

MediaDescription MediaDescription.Builder builder = new MediaDescription.Builder(); builder.setmediaid(id); builder.seticonbitmap(icon); builder.settitle("first Line"); builder.setsubtitle("second Line");... MediaBrowser.MediaItem myitem = new MediaBrowser.MediaItem(builder.build(), MediaBrowser.MediaItem.FLAG_PLAYABLE OR MediaBrowser.MediaItem.FLAG_BROWSABLE); 46

Overview MyAudioApp MediaBrowserService Android Auto Media Browser MediaSession onplay().setmetadata().setplaybackstate() Media Controller 47

MediaMetadata MediaMetadata.Builder builder = new MediaMetadata.Builder(); builder.putstring (MediaMetadata.METADATA_KEY_TITLE, title); builder.putstring (MediaMetadata.METADATA_KEY_ARTIST, artist); builder.putbitmap (imagebitmap);... mediasession.setmetadata(builder.build()); https://www.google.com/design/spec-auto/audio-apps/audio-app-anatomy.html 48

MediaSession - PlaybackState PlaybackState playbackstate = new PlaybackState.Builder() Display state.setactions(playbackstate.action_pause PlaybackState.ACTION_SKIP_TO_NEXT PlaybackState.ACTION_SKIP_TO_PREVIOUS).setState(PlaybackState.STATE_PLAYING, position, 1.0f, SystemClock.elapsedRealtime()).addCustomAction(ACTION_SHUFFLE, "Shuffle", R.drawable.ic_shuffle).build(); mediasession.setplaybackstate(playbackstate) 49

MediaSession - Callback @Override public void onplay() { }... player control @Override song selection public void onplayfrommediaid (String mediaid, Bundle extras) {} @Override public void onplayfromsearch voice input (String query, Bundle extras) {} mediasession.setcallback(new MyMediaSessionCallback()); mediasession.setactive(true); 50

Voice Search Examples Play music from Madonna Play Stairway To Heaven from Led Zeppelin Google Knowledge Graph Artist focused search Title focused search 51

4 Development workflow 52

Android Auto Apps Restrictions Goal: Minimize driver distraction predictable interaction Design Guidelines enforce design & interaction patterns Functionality Guidelines enforces correct configuration and expected behaviour 53

Design Guidelines no animations or videos no visual ads, game features support voice input in media apps Read more: https://www.google.com/design/spec-auto/ 54

Design Guidelines Day Night 55

Functionality Guidelines app starts and loads content within 10 secs keep state between auto & smartphone usage app works as described in Google Play... Read more: http://developer.android.com/distribute/essentials/quality/auto.html 56

Functionality Guidelines Task Limit A single action must be completed within 6 steps. (US) 4 steps (Japan) max hierarchy of 2 levels max 5 items on one page 57

Functionality Guidelines (messaging) App can successfully receive incoming messages Messages properly grouped and ordered User is able to successfully reply to a message. Only short-form messaging app design patterns (e.g. no emails) Only peer-to-peer messaging services (no notification services, e.g. weather, stocks, and sport scores apps) Read more: https://developer.android.com/distribute/essentials/quality/auto.html 58

Testing on simulators Messaging Simulator (deprecated) Media Browser Simulator (deprecated) New: Android Auto Desktop Head Unit (DHU) Simulators: <path-to-sdk>/extras/google/simulators Head Unit: <path-to-sdk>/extras/google/auto 59

Android Auto Desktop Head Unit * adb forward tcp:5277 tcp:5277 60

Example DHU Commands Microphone (voice commands) e.g. mic play <absolute_path>/pause.wav ( Pause Music ) Day & Night mode e.g. day / night / daynight Controller and touch events ( hardware buttons) dpad up / dpad down tap x y 61

Publish / Test on Hardware Workflow: 1. upload apk to play store 2. get positive review 3. download app from store Direct 4. use on real device Rejected Approved Review 62

Example App Review Fast feedback Email from Android Auto Team - within 2 hours REASON(S) FOR EXCLUSION: - Your app continuous to play music during the use of the microphone, interfering with voice commands. - Your app does not perform all functions properly or as expected from a user perspective. 63

5 Summary 64

Learn more? Our favorite resources 65

Course: Android Auto Development on Udacity https://www.udacity.com/course/android-auto-development--ud875c 66

Universal Music Player https://github.com/googlesamples/android-universalmusicplayer 67

babbq Texas 2015 talk by +Ian Lake https://www.youtube.com/watch?v=xqwe30czffg 68

Thanks! Contact us: c4c Engineering GmbH Hildesheimer Str. 27 38114 Braunschweig Germany christian.dziuba@c4cengineering.de sina.grunau@c4cengineering.de www.c4cengineering.de 69