Lecture 09 Data Storage

Lecture Overview Lecture 09 Data Storage HIT3328 / HIT8328 Software Development for Mobile Devices Dr. Rajesh Vasa, 2011 Twitter: @rvasa Blog: http://rvasa.blogspot.com 1 2 Mobile devices have a specialization Mobile phones are a fashion statement Web is good at semantic and social contexts Context:: Semantic // Spacial // Social // Temporal Mobile devices are good for spacial and temporal contexts Fashion Culture Time-Bound User View: Ugly apps. ruin the look of a pretty (expensive) gadget 3 4 Concept Overload on the Web? Heuristics to Check for Usability Review, Like, Rate Cart, Wish list, 1-Click, Prime Paper, Kindle, Multiple Prices Usability/Design Check List Sketch Paper Prototype Used Book - New Book Frequently Bought Read Inside Reflect on Answers Interactive Prototype Early Release Read Reviews More Buying Options What others bought This what a first time online book buyer will experience if they go to Amazon Make Changes 6 Beta Release Final Release 5

Key Aspects of the Check list Logical UI Physical UI Graphical UI Data Model Wording (Labels) Soft Keyboard Functional Organisation Navigation/ Exploration Ergonomics Icons and Images Physical and Temporal Context Font Type/Size Style/Colours Appropriate use of components Tasks are verified within the context of a User Model Tasks are generally derived from scenarios (they can even be a 1:1 mapping) Lecture Roadmap - Where are we? 7 8 Android offers the following options for data storage: Shared Preferences (Key-Value pairs) Internal Storage (Store on Flash memory) External Storage (Store on SD Card) SQLite Databases (RDBMS) Network connection to store on server Storage (Android Design Choices) Static data should be put into Resources ( res folder) Place any text file or custom data files in the raw folder (as a resource) User modifiable data can be stored to Internal (or) External Storage Every Application has the option to store private data 9 10 Internal Storage (via Raw Folder) Store any raw data in this folder Working with Raw Resource Files InputStream inputstream = getresources().openrawresource(r.raw.dfile); BufferedReader br = new BufferedReader(inputStream); This data has to be provided at development time You can work with the BufferedReader (or) the InputStream directly as required for your application 11 12

Resources are Read Only in Android Resources Stored in Compressed Form Applications are deployed as APK files (compressed archives) Resources are also compressed as part of the package creation process Data stored in the res folder is read-only Database file that needs to be updated cannot be stored in res If you have to access any resources, they need to be uncompressed first 13 14 Resource Compression Issues On the surface the compress decision by Android designers seems reasonable given small persistent storage size. But, decompression is a slow process Additionally, prior to Android 2.3.: If the uncompressed size of a file was greater than 1 Mb -- it did not decompress Log Cat message when things fail in Android 2.2 due to large size!"#$%&'(()*+,-./01!'*'1)23))4(1$56789:";;<!=>=<8=?1 +@AA-B,A1C(1@DEF.,A/ So, where can an application write to? Android offers options for data storage: Shared Preferences (Key-Value pairs) Internal Storage (Store on Flash memory) External Storage (Store on SD Card) SQLite Databases (RDBMS) Network connection to store on server Writing to File System has Security Implications (a number of choices were made during the design of Android) 15 16 Lecture Roadmap - Where are we? Writing to File System (Typical Model) Application-A can create and update a file E.g.!"#$%&%#'()*+,-./%0)$%&%1&2& Application-B can read/update this file stored in C:\data (if Application-A is closed) Default option on desktop O/S: Applications have full access to the file system (typically) Security is achieved via access permissions bound to user 17 18

Traditional Security Model (File Access) Assumptions, User is smart & computer savvy (hmm..) User is (can be) responsible for access User has knowledge of file system (hmm..^2?) permissions These assumptions were reasonable in the early days of computing But in mobile... Security on Mobile Devices File system is completely hidden from the user -- it is a (smart) phone to most people. Phone == Consumer device (like a TV) Data security is achieved via Sand-Boxing User has UID (User ID) In most modern O/S: File Access Permissions are restricted based on a user id 19 20 Security -- Android Devices User Android Application has also has 21 UID (User ID) determines File System Access Permissions The UID is generated at install time based on the signature and package name in the APK Security Architecture Android runs on top of Linux Linux offers access restrictions by user id Linux also treat external devices as files (in general) Applications (by default) do not have permission to, Read/Write user data [e.g. contacts] / Intent Filters Read/Write another apps. data [File permissions] Access the network (which is mapped as a file) Accessing/Talking to sensors etc. (also mapped as files) 22 Security in Practice (on Android) Application-A cannot access data privately stored by Application-B Each application runs in its own Sandbox The user can get to data by Remote shell access (shell access phone) By default, the phone provider has root access to the device You can by-pass this by installing a custom kernel (or) by modifying the kernel Once you have root access -- you can view data Viewing the File System Macintosh-4:platform-tools rvasa$./adb -d shell dbdata config cache efs sdcard acct mnt d etc data init.smdkc110.rc... Shell access to phone (locked Galaxy S) Folder where Applications store data (typically) 23 24

File System is Locked (by default) Many areas of file system are restricted by default (i.e. phone is locked) $ cd data opendir failed, Permission denied $ Note: You have full access to entire file system on emulator Android/Apple/Windows Mobile have this restriction Emulator does not have any restrictions # cd data # ls property misc local app-private backup data anr dontpanic app dalvik-cache system lost+found # No permission problem here 345$4%** 34*6$ 7$%&%7%** -68/$,.8$1-9/&8'+:;1%*< -68/$,.8$1%--8=/;:;1%*< -68/10+5:>1%*< (.61%/$,.8$:;1%*< -68/1+2%'*0+-:>1%*< 34 25 26 Viewing Data on Emulator # cd data # ls property misc local app-private backup data anr dontpanic app dalvik-cache system lost+found 345$4$%&% 5.'1%/$,.8$1?985<-+%,5@A.2 5.'1%/$,.8$1*,.B8$+,-1$.6/0.%$- 5.'1%/$,.8$1*,.&8*- 5.'1%/$,.8$1-*%,+)*%,&- -68/$,.8$1-9/&8'+ 345$4-68/$,.8$1-9/&8'+ 08A C80+- /data/data 345$4C80+- 9-+,)=+.0.5%&8./- This is the folder where apps. store data 27 Some Areas are Public (like SD Card) $ cd sdcard SD Card on a locked Galaxy S Samsung sd Android DCIM layar media AussieWeatherRadar DiskCacheIndex41214.tmp Sounds download ScreenCapture swiftkey Music $ 28 Some applications use SD Card for data storage $ cd AussieWeatherRadar IDR022background_img IDR022topo_img IDR022location_img IDR022range_img IDR024background_img IDR024topo_img IDR024location_img IDR024range_img We can pull public files from phone $./adb -d pull /sdcard/aussieweatherradar./awr pull: building file list... pull: /sdcard/aussieweatherradar/idr023range_img ->./AWR/IDR023range_img pull: /sdcard/aussieweatherradar/idr023topo_img ->./AWR/IDR023topo_img pull: /sdcard/aussieweatherradar/idr021range_img ->./AWR/IDR021range_img...... IDR023topo_img The ADB command is helpful for debugging & if you are curious to see how an application works Lecture Roadmap - Where are we? You can also push files to phone 29 30

Saving Data into a File (locally) Two options typical available Internal Storage (built-in flash memory) 345$4$%&% 5.'1%/$,.8$1-*%,+)*%,&- -68/$,.8$1-9/&8'+ 345$4-68/$,.8$1-9/&8'+ 08A /data/data/files C80+- 345$4C80+- 9-+,)=+.0.5%&8./- External Storage (SD Card) $ cd AussieWeatherRadar IDR022background_img IDR022topo_img IDR022location_img IDR022range_img IDR024background_img IDR024topo_img IDR024location_img IDR024range_img Saving Data into a File Internal Storage (built-in flash memory) /** Sample code to help get started with saving location */ private void savegeolocation() throws Exception { String FILENAME = "user_geolocations"; String string = "Melbourne, -40.23, 143.23"; FileOutputStream fos = openfileoutput(filename, Context.MODE_PRIVATE); } fos.write(string.getbytes()); fos.close(); Stored in /data/data/files 345$4-68/$,.8$1-9/&8'+ 08A C80+- 345$4C80+- 9-+,)=+.0.5%&8./- 31 32 Saving Data into a File Internal Storage (built-in flash memory) FileOutputStream fos = openfileoutput(filename, Context.MODE_PRIVATE); Modes available: Private, Append, World_Readable, World_Writable Lecture Roadmap - Where are we? 33 34 Assumptions (before we jump into sqlite) You are able to read/write SQL statements You know the basic concepts that underpin a relational database (Tables, Primary Key, Indexes, Data types etc.) You know the difference between a database (a file) and a DBMS (e.g. Oracle/sqlite/SQL Server etc.) You have a general understanding of how to programmatically access a database (via the DBMS) Android (and ios) provide sqlite Android and ios both provide a light weight (in-process) database engine called sqlite in their phones What is SQLite? http://en.wikipedia.org/wiki/sqlite Interesting (sensible) design choices: Locks entire file when updating Weakly typed (does not guarantee domain type integrity) 35 36

Another explanation of sqlite SQLite is a library (like java.io.*) you use to write/read a file -- your database It is a library, so it is part of your application You do not need to ship/bundle the library in your apps. because sqlite is part of the Android (and ios) libraries uses read write ClassX java.io.* locations.txt uses read write ClassZ sqlite locations.db sqlite is an embedded/in-process DBMS ClassZ and sqlite are in same process uses ClassZ sqlite locations.db similar to custom File I/O ClassX and library are in same process uses read write read write ClassX java.io.* locations.txt 37 38 Getting Started - First Steps Typical tasks when working with databases: Open or Create the database (a file) Create/Read/Update/Delete tables in DB If query returns more than one row -- iterate over this set of rows (using a Cursor) Display/Use data in columns (of each row) Database Table(s) Row(s) Column(s) Open/Create Database - Concerns API offers a helper class for opening/creating a database In practice, we also encounter the following situations and have to design for: Upgrading the DB from old to new version Downgrading the DB (sadly.. also needed) Handing errors during the opening/upgrade Upgrade (or) Initialisation that takes a long time 39 40 Open/Create DB - Android API Android provides DEF8&+G*+/H+0*+, in API Simplifies DB access via conventions (for instance it stores db files is a standard location) DEF8&+G*+/H+0*+, designed to simplify: Opening/Creating/Upgrading databases Handling errors Managing long running tasks by using asynchronous background processes Key DB Classes/Interfaces (in Android) SQLiteOpenHelper SQLiteDatabase SQLiteStatement Open/Create DB Interact with DB Encapsulates SQL Statement 41 42

What happens to a query? SQL Query on execution returns a Cursor Cursor offers random Read/Write access to the result set of a query Cursor (Interface) What happens to a query? When a Query is executed on SQLiteDatabase, it returns a SQLiteCursor Cursor (Interface) implements SQLiteCursor Warning: Cursor implementations are not required to be synchronized Warning: SQLiteCursor implementations is not synchronized 43 44 Result Set & Cursors in DB Access When you execute a select query on the DB, It will return a set of matching rows -- the cursor allows access to these rows The result set is a copy of the information in the DB -- hence the cursor will not reflect any changes made after query (or) insert Implications: Close the cursor as soon as possible to avoid keeping large volumes of data in memory Repopulate any views if data changes Lets get our feet wet 45 46 Database Access - Code Walkthrough BirthDayDB Project Demo of the functionality Code walk through in the lecture DB Access can be complex Consider a table with 30,000 rows Query all will return a cursor for this entire set of matching rows Android O/S has a primary UI thread If actions on this primary UI thread take too long (more than a few milliseconds) -- the OS will shut down the thread What will happen to long queries? How can we handle this situation? 47 48

Lecture Roadmap - Where are we? Android SDK has a Monkey Android SDK provides a dumb stress testing tool called Monkey It will (pseudo) randomly press buttons on the phone -- generate events and does a whole lot of silly things Ideally, your app. should survive this beating $ adb shell monkey -p swindroid.suntime -v 500 49 50