Using Python for Cross-platform Mobile Application Development

Transcription

1 Using Python for Cross-platform Mobile Application Development Draft version c Alexandru E. Şuşu alex.susu@gmail.com Abstract In the last year, we are witnessing how the sales of mobile devices - smartphones and tablets, are surpassing the ones of PCs. All big players in mobile OS and hardware manufacturers propose their own different application programming frameworks, making the development efforts for all platforms considerable. In order to offer a software product for most of the mobile market, companies are normally required to have trained software engineers for the various OSes and this reflects dramatically in the project implementation cost. A few programming frameworks allowing a more platform independent approach for mobile devices are proposed by companies or open-source communities. In this paper we use a cross-platform methodology in which we employ mainly Python to develop our mobile application. Our end result, the icam service, which offers inexpensive Camera surveillance with smartphones, is deployed on most phone platforms. This service is running for more than two years on Symbian, Android, Windows Mobile and ios phones, providing video information from remote sites. To the best of our knowledge, we are the first to have taken such approach. Since we created the methodology while developing the application itself, the implementation costs were considerable, but, once this generic framework becomes more robust and better documented, the savings in development effort should become very attractive. Our icam application requires more work to implement all features, mostly for ios and Windows, but with this occasion we explored many facets of cross-platform development and document our experiences. Keywords: cross-platform mobile development, Python, video surveillance system I. INTRODUCTION With mobile computing becoming mainstream, we face a very heterogeneous scene of OSes and platform SDKs proposed by the various players: Google, Apple, Microsoft, Nokia and RIM. It is late spring for mobile technology and nature is still experimenting with respect to OS and even hardware standards. In the last five years we witnessed the creation of many crossplatform frameworks for smartphones, which allow cutting down considerably development costs. The main contribution of this paper is the cross-platform development methodology employed for a smartphone application and media service we developed, icam, available at icam is an Internet enabled video surveillance solution using a server, normally Google YouTube and Picasa, which receives media content from mobile phones equipped with one or two cameras. Our method relies on the existing Python implementations for smartphone OSes - Py4A for Android, PyS60 for Symbian, PyObjC for ios and PythonCE for WindowsCE. We develop the application logic platform-independent and have different calls to the specific IO and multimedia functionality. To the best of our knowledge, we are the first to propose such a mobile application development method. The solution can be improved, hopefully with help from the community: we can design a better universal API wrapping for the existing Python platforms, making cross platform development easier. The moment our method is robust and very well documented, we estimate the entire effort taken by a developer to port the complex application to another platform to be reduced to a few days. II. CROSS-PLATFORM APIS FOR SMARTPHONES A great reference enumerating most of the existing frameworks is [AGL10]: as of 2010, more than a dousin solutions were offered by various usually small companies. Among the many mobile programming frameworks available (cross platform solutions using standard programming languages - Qt, Java MicroEdition, cross platform solutions relying on web frameworks - PhoneGap, Titanium Mobile [AGL10]; Java based - Android SDK; C++ based - Symbian, Android NDK), Python gains adepts on all platforms because of it is simple to program in, it is open source, with a liberal license, it has good performance and a wide industrial support/adoption [FM09]. The Qt framework [FMT10] is available officially on Symbian, MeeGo, Windows, MacOS, Linux, and through community effort on Android (the project android-lighthouse, available at and ios (project UIKit lighthouse - and the less active project Qt-iPhone from In fact, we encourage as a solution for industry-quality, native smartphone aplication development the use of PySide on top of the Qt runtimes for all platforms. For this to become an incontestable reality, the major competitors in the field of mobile computing software are required to cooperate to promote the solution: Google, Apple, Nokia and Microsoft. We must not overlook that all these players already incorporate Python their development tools strategies. Of course, developers must consider seriously some of the existing company developed frameworks, such as PhoneGap, when planning the implementation of their smartphone software. But, we consider there will always be a concern on the licensing for commercial application development, which is not the case when using Python, eventually with LGPL Qt. III. PYTHON IMPLEMENTATIONS FOR SMARTPHONES The biggest advantage of using Python for the development of smartphone applications is that Python is popular, very concise and the APIs offered are more abstract than the native ones, making programming simpler, yet without surprise, sacrificing flexibility and performance. This offers a very straigt-forward way for Rapid Application Development. In fact, doing a new iteration of the application can be simply done by just copying the.py script on the target platform. Compiling even big Python scripts on old smartphones, such as the Nokia 6680 with an ARM processor at 220Mhz, takes up to 5 seconds, which can be reasonable for most cases. Clearly, if running the same script many times on the phone, to avoid compiling it at each execution, you can obtain the bytecode on the development PC and copy on the target the.pyc file.

2 Due to its popularity, we can easily use in our smartphone applications many interesting generic Python modules: Google s GData Python client library, atom and so on. The standard way to use them is to include the corresponding bytecode files,.pyc, (or even.py) in a ZIP archive and then give the following command before importing the desired modules: sys.path.append(os.path.join(sys.path[0], lib std.zip )). This was tested with success on most smartphone platforms. Running the application in simulator is feasible, but given the fast prototyping obtained directly on the real platform, we consider it can be discarded. Python is in use on other embedded platforms: projects such as Python-on-a-Chip or p14p ( which allows running a significant subset of the Python language on microcontrollers without an OS, for example, the popular open architecture Arduino, which uses the ATmega micro-controller. Another project, nxt-python ( is a Python interface for the Lego Mindstorm NXT robots. In the following paragraphs we enumerate the various Python implementations in decreasing order of the respective community activity, at the time of writing. Note that all these projects are opensource and have in the case of Py4A, PyObjC and PyS60, the least, few company members directing the development, but the community effort is very important in evolving the project. But before doing this, we want to clarify that programming frameworks for smartphones contain two major components: the standard computing platform that normally runs on PCs (think of Python with the standard computation modules) and the special APIs for UI, IO (platform DBs; multimedia - camera, microphone, sound, GPS, accelerometers, Bluetooth, WiFi, NFC, etc), which we call mobility APIs, terminology inspired from Qt. A. Py4A with SL4A Android comes with a generic scripting solution, namely SL4A (Scripting Layer for Android), allowing the development of mobile applications. SL4A, originally named Android Scripting Environment, is available to the public from June Good references are [Fer11] and Chapter 5 in [JG11]. SL4A has a client-server architecture, in the sense it uses RPC, which allows using many scripting languages as clients [Fer11], such as Py4A, the Android native port of Python version 2.6, Perl, JRuby, Lua and so on. It is possible that SL4A and especially Py4A gains more visibility also due to the ongoing law suit between Google and Oracle debating the Java license issues in the Android SDK. Also, we must not forget that the creator of Python, Guido van Rossum, is affiliated with Google, which stresses the importance Google gives to this language. Our icam application, developed usign Py4A and SL4A, can be downloaded from Google Play as an.apk. When running it, it will automatically download the SL4A and Py4A runtimes if they are not already installed on the phone. B. PyS60 PyS60, Python for Symbian S60, is the first port to a smartphone platform - an easy argumentation given the fact Symbian is historically the first important player in the mobile OS market. The PyS60 project started in the Nokia research labs in 2003 and was made public in October 2005 [Cen12]. A distinctive feature of PyS60 w.r.t. the other Python mobile APIs we present in this paper is that it uses the mobile APIs through.pyd modules written directly in Symbian C++, which brings more efficiency. Nokia allows the distribution of applications written in PyS60 in the Nokia OVI store. A port for Symbian UIQ phones exists as well, codenamed PyUIQ, but its development, branched from the PyS60 project, was not continued and does not offer support for UI features. PyS60 has about five years of community support (2006-present); during this period quite a few modules using the mobile APIs were written. Despite the fact the Symbian platform is currently losing market, Nokia producing in 2012 only one more Symbian Belle phone ( symbian/), PyS60 is, in our opinion, the most mature Python smartphone platform at the time of writing. PyS60 has a few commercial-quality applications written in Python, such as, for example, IYouIt [BKL + 08]. PyS60 is well documented in reference manuals and two books. The first, [ST07], is a thorough introduction to the APIs of PyS60 1.4, providing also example of a few very interesting projects. The second, [NG10], takes us a few steps further by presenting the latest API for PyS60 version 2.0, released in It presents in detail its APIs, in some cases with uses of external components such as the Qik service. Note that PySide, Python for Qt, offered by Nokia, under the more convenient LGPL license (opposed to the similar project, PyQt [Sum07], offered by Riverbank Computing under GPL license) for Linux, Maemo, MeeGo, Microsoft Windows and Mac OS X, has an infant sub-project that targets Python on Symbian. However, PyS60 has a big advantage over an eventually running PySide for Symbian: the memory footprint is much smaller. In fact, Qt is recommended for newer Symbian S60 phones - from S60 v5.0 onwards. So for phones up to S60 2nd edition the only Python choice is PyS60 version 1.4.5, which uses Python 2.2. For S60 3rd edition or newer phones the best choice is PyS employing Python 2.5. C. PyObjC Developing mobile applications under Python for ios requires getting familiar with the ObjectiveC language, the UIKit framework (the MacOS equivalent is AppKit) from the Cocoa Touch API, the mobile extension of Cocoa for MacOS. PyObjC for MacOS is a mature open-source project, started in 2000, available at It currently includes Python 2.5, with ObjectiveC bindings to access the Cocoa (Touch) APIs. One can find examples with MacOS UI applications written in PyObjC, using the AppKit framework for Cocoa. The standard PyObjC required a few changes to be ported on ios, which were performed by Jay Freeman (alias Saurik) - see As discussed, the UI and mobile APIs of the ios are available in the UIKit framework [App12] and are accessible through the ObjectiveC bindings of PyObjC. PyObjC for ios is not a very active project and has a small community, but we hope the situation changes. Nevertheless, an interesting pro argument is that the standard Python environment (without the UIKit PyObjC bindings) is robust. We put effort to port the icam application to the ios platform. Indeed, there are almost no examples using UIKit with PyObjC and little documentation. We provide some information on how to install PyObjC and develop for ios. PyObjC for ios can be installed through the Cydia Packager. The installation creates an icon HelloPython, which basically calls an example script located at /Applications/HelloPython.app/HelloPython.py. We can change the contents of this HelloPython.py script and add more modules if wanted in the /Applications/HelloPython.app folder - one can use ispirit ( to transfer files to the ios device. An important incentive to use PyObjC for ios is that you do not require to have a MacOS system and register for the ios Developer Program, which costs 99 USD per year (see to obtain a certificate in order to easily install the application built with XCode. Indeed, after jailbreaking your ios device, a relatively simple procedure, you can immediately deploy and run your Python scripts on it. D. PythonCE Windows Mobile, Microsoft s smartphone platform until 2010, uses the Windows Embedded CE (from now on we will refer to it as

3 WinCE) kernel up to version 5.0. Windows Phone 7 (WP7) released in 2010, relies on WinCE 6.0 [Pro10], [Sin10]. The PythonCE open-source project ( is a standard Python 2.5 environment ported to WinCE. It does not have any bindings to the mobility APIs. We access some of the standard OS functions (e.g., create a new process) of WinCE via the ctypes module. The effort of porting and testing our Python icam application from the PyS60 version to PythonCE was around 5 days-man. The original PyS60 code itself was able to run immediately on the PythonCE VM, but we had to adapt the OS specific function calls and the multimedia calls. This is, in our opinion, a good example of cross-platform mobile development. Note that the multimedia APIs for WinCE are, in our opinion, more complex than the average: they require the use of DirectX and some encoders are not available for free. E. PyMaemo For the last three Python frameworks we describe, we did not have (mostly time) resources to port icam to. However, we are happy if you join us in our development efforts. Although icam is not currently supported on Nokia s Maemo/MeeGo platforms, the reason being we did not have extensive access to such devices, PyMaemo implements well the mobile extensions and this should allow us to easily port our application to PyMaemo. As discussed above, Nokia also offers PySide for MeeGo/Maemo, providing us another runtime to target our Python application to. OVI store for PyMaemo?!!!! F. Windows Phone 7 Very recently, in Mar 2012, the open-source project IronPython ( announced that the latest release targets WP7 and even Android, through the Mono for Android VM, and they plan to target ios. While the partnership between Nokia and Microsoft made many Nokia phone application developers wait for a port of Qt on WP 7, this is currently not supported. G. JythonME, Jython for J2ME J2ME is the native programming solution for Blackberry smartphones and feature phones. The latter, although are not really smartphones, are, in our opinion, mentionable for programming using Python - most of the feature phones (from Sony Ericsson, Samsung, Nokia) have potent CPUs and multimedia capabilities. JythonME is an inactive project. H. More Information about the Python Frameworks We now present some useful information about the mobile Python implementations discussed above: Python runtime versions used, type of licenses they use, download links, mailing lists and, in order to quantify how active the respective developer communities are we compute the average number of messages per day for the developer mailing lists in the last weeks. Android - Py4A with SL4A Python 2.6 Apache 2.0 license download from mailing lists SL4A msg/day Py4A msg/day Symbian PyS60 S60 1st & 2nd ed - PyS Python 2.2 S60 3rd ed or newer - PyS Python 2.5 Apache 2.0 license download from mailing list msg/day last complete build in 2010 PyUIQ Python (based on PyS ) download from inactive/discontinued ios - PyObjC Python 2.5 download through Cydia mailing lists - 0 msg/day in 2012 for both MacOS - ios - (last binaries built in 2010!!!!) other: Windows Mobile - PythonCE Python mailing list - 0 msg per day in last binaries built in Oct WP 7 - IronPython Python 2.7 active, released in Mar 2012 mailing list - 0 msg per day in PyMaemo JythonME IV. DESCRIPTION OF ICAM As discussed, the icam smartphone video surveillance application can upload media to a server, normally Google YouTube and Picasa. By running our icam phone application, we transform the mobile phone in an IP (Internet) surveillance camera. Such system can be used either for indoors, outdoors spaces or car video monitoring. The server can detect out of the ordinary elements in the recorded videos and alarm the user, who can take action, if required. The user can view the video broadcasted either on a mobile phone, tablet or PC. The mobile application can be downloaded from and the source code can be found at We present the icam service, from the perspective of Symbian S60 phones. We first developed the application for Nokia phones and have good expertise for this platform. When icam is started for the first time it will ask for an Internet access point. The application can work offline, but we recommend to use it with an Internet connection, through 3G/EDGE/GPRS or WiFi. We recommend you to use the Google services which provide free storage. The videos and photos icam uploads to YouTube and Picasa can be tagged as private, so nobody else could access them unless you specifically want to. Note that you need a Google account with YouTube and Picasa enabled. You need to specify the username and password for the Google account, in order to upload media to YouTube and Picasa. Normally, the application is configured to record 30 seconds every two minutes. You can change these settings from the menu of the application. Once the configuration of the icam application is done, hit Start to begin capturing. When the video is being recorded the application shows a counter incremented every second. The moment the recording is finished, the video is uploaded to YouTube, operation that will take a few seconds; the video can be stored in the icam Media subfolder, as well, if specified. Another operation mode is the Photo mode in which every photo is taken at a certain interval of time, pauseinterval, and then stored and/or sent to Picasa. Another operation mode is the Burst mode in which the viewfinder is always on and the phone takes photos as fast as it can. In this mode we implemented a motion detection algorithm. While the viewfinder frames are unchanged we can simply be in standby mode, when the algorithm returns a positive, that photo frame(s) can be transmitted. The videos can then be viewed on YouTube, under the My Videos section, in your account, in any browser supporting YouTube, an example being given in Fig 1. We can view the uploaded videos with another application we created, icamviewer, as depicted in Fig 2. This is a complex and

4 Fig. 1. The videos uploaded can be found in the My Videos section of YouTube, at videos. Fig. 3. The Remote Control Panel web page. to check less often for new commands to avoid having less Internet traffic, resulting in a smaller energy consumption. Fig. 2. The icamviewer application. VI. CONCLUSION The icam application is the first, to the best of our knowledge, cross-platform smartphone application using Python, running on Symbian, Android, Windows Mobile and ios phones. Such generic techniques for mobile development are highly desirable, cutting down considerably project costs. Although our solution cannot entirely compete with commercial solutions, such as PhoneGap, we have the advantage of using only free licensed software and frameworks. It is important to point out that there are many Python mobile frameworks backed up by the various mobile OS players, which can converge to making Python a serious, reliable, industry-proof solution for cross-platform application development. REFERENCES highly extensible PySide application, which displays simultaneously in a split screen format videos from the various devices selected and performs thorough video and audio analysis to detect if there are events of interest to report. Although currently not available for public, we also have the alternative of viewing the most recent video uploaded to your account by logging in at the This webpage is mostly suitable for mobile platforms that cannot run the icamviewer application themselves. V. ICAM REMOTE CONTROL PANEL To change the settings of the icam phone application you can directly manipulate the UI on the phone, or you can send commands to icam via the web, a useful alternative especially if you cannot access the phone. To send a command from the icam Remote Control Panel, depicted in Fig 2, you choose the desired configuration and press Submit. Please keep in mind that the icam phone client downloads these commands roughly every pauseinterval seconds, which can incur a considerable delay in reaction if the value is big. We designed icam [AGL10] Sarah Allen, Vidal Graupera, and Lee Lundrigan. Pro Smartphone Cross-Platform Development: iphone, Blackberry, Windows Mobile and Android Development and Distribution. Apress, Berkely, CA, USA, 1st edition, [App12] Apple. UIKit Framework Reference, reference/uikit Framework/, [BKL + 08] Sebastian Boehm, Johan Koolwaaij, Marko Luther, Bertrand Souville, Matthias Wagner, and Martin Wibbels. Introducing IYouIt. In Proceedings of the 7th International Conference on The Semantic Web, ISWC 08, pages , Berlin, Heidelberg, Springer-Verlag. [Cen12] Nokia Research Center [Fer11] Paul Ferrill. Pro Android Python with SL4A. Apress, Berkely, CA, USA, 1st edition, [FM09] Michael J. Foord and Christian Muirhead. IronPython in Action. Manning Pubs Co Series. Manning, [FMT10] Frank H P Fitzek, Tommi Mikkonen, and Tony Torp. Qt for Symbian. John Wiley and Sons Ltd., Chichester, [JG11] Lucas Jordan and Pieter Greyling. Practical Android Projects. Apress, Berkely, CA, USA, 1st edition, [NG10] P. Nathani and B. Galiceanu. Python on Symbian: Mobile App Development Made Easy. Createspace, 2010.

5 [Pro10] Emil Protalinski. Leaked Windows Phone 7 docs describe updates, customization, [Sin10] Tarandeep Singh. Windows Phone 7 Architecture, [ST07] Jrgen Scheible and Ville Tuulos. Mobile Python: Rapid prototyping of applications on the mobile platform. Wiley Publishing, [Sum07] Mark Summerfield. Rapid GUI Programming with Python and QT: The Definitive Guide to PyQt Programming. Prentice Hall Press, Upper Saddle River, NJ, USA, first edition, 2007.