Native code in Android Quad Detection

Transcription

1 Native code in Android Quad Detection 21 July 2013 This document provides information on how to build and run native (C/C++) code in Android platform using Android NDK and then use its scope to auto capture an image when a quadrilateral object (close to a rectangle) is detected inside the camera view of an Android phone. Author: Rahul Bhat

2 Table of Contents S.No Topic Page No 1 Introduction 3 2 Setting up the Environment 3 3 Calling the native code 6 4 Quad Detection 7 5 Auto Capture when a Quadrilateral is detected 8

3 1. Introduction This document provides information on how to build and run native (C/C++) code in Android platform using Android NDK and then use its scope to auto capture an image when a quadrilateral object (close to a rectangle) is detected inside the camera view of an Android phone 2. Setting up the Environment 1. Prerequisites: Eclipse with Android SDK and ADT plugin. 2. The NDK plugin currently works with CDT or CDT Install CDT from Eclipse update site 2. Install Android NDK Plugin from Eclipse update site 3. Set the path to Android NDK: Eclipse -> Window -> Preferences -> Android -> NDK -> set path to the NDK 3. For running native C/C++ code in Android platform, you need to convert your project to C/C++ project to add native nature to the project. Right click on your project and then click on New Other Under C/C++

4 4. Now right click on your project and then click Properties. You will see C/C++ Build item. Give the path of your ndk-build.cmd to Build Command edit box or you can set an environment variable to your ndkbuild.cmd file path. 5. Now click on Behavior tab and set the environment variables as shown. 6. Now to go to C/C++ General item and click on Include tab and set the path to Android NDK include folder for GNU C and/or GNU C++. Eg: C:\Users\rahul.bhat\Downloads\android-ndk-r8e-windows-x86\android-ndk-r8e\platforms\android- 9\arch-arm\usr\include

5 7. Write your C/C++ code and place the file inside jni folder alogn with Android.mk and Application.mk file. a. In Android.mk file, you can refer any library you want to include in your native code. Eg: include C:\Users\rahul.bhat\Downloads\OpenCV android-sdk\sdk\native\jni\OpenCV.mk b. Also you can assign the source file. LOCAL_SRC_FILES := ImageProcessing.cpp c. This is how a Application.mk file looks like:- APP_STL := gnustl_static APP_CPPFLAGS := -frtti -fexceptions APP_ABI := armeabi-v7a APP_MODULES := ImageProcessing

6 3. Calling the native code Calling the native C/C++ code from Java class. 1. Create a function with native keyword. // Native JNI // public native boolean ImageProcessing(int width, int height, byte[] framedata, int [] pixels); 2. Call the method from your java class. ImageProcessing(PreviewSizeWidth, PreviewSizeHeight, FrameData, pixels); 3. Your function in the C/C++ file should be in the following format. Java_package_callingClass_nativeMethodWithParameters E.g: com.ied is my package name so it would be written as com_ied. Java_com_ied_CameraPreview_ImageProcessing( JNIEnv* env, jobject this, jint width, jint height, jbytearray NV21FrameData, jintarray outpixels) { //your code in C/C++ return booleanvalue; } 4. Now whenever you try to run your code, it will compile/build your native code and create a library under obj and lib folder with name as libclassname.so. In this example, it is libimageprocessing.so. So you need to load this library through you java code. static { } System.loadLibrary("ImageProcessing");// Just the name of the C/C++ file 5. If your C/C++ code contains error, the build will fail. Resolve the errors and try again. After build is successful and a library file is created, your android application is ready to use the native code.

7 4. Quad Detection 1. First, I tried Image processing (detecting edges) using Java (Sobel Algorithm) but it turned out to be very slow. It took around 10-12s for detecting edges and hence, it was not feasible for real time quadrilateral detection. Original Image Final Image 2. Next step I took was to use C++ for image processing part of the application. 3. Thanks to OpenCV they provide API for Canny Edge Detection that is written in C++.

8 4. For using C++ along with Java, I had to use Android NDK and CDT plug-in. 5. CDT plug-in for eclipse helps you to write and compile the code in C/C Android NDK builds all the native code (C/C++) using ndk-build.cmd. All the native code will be built before the android application is built and launched. 7. After the native code is built, you will get a library file at the following location :-..\workspace\project_name\obj\local\armeabi-v7a\objs\ 8. In my current approach, I created a Camera View in Android which loads the above built native library (containing built C/C++ code) and using a thread constantly sends camera data to native code for processing and when a quadrilateral figure is detected, native code returns true as Boolean value and image is captured 5. Auto Capture when a Quadrilateral is detected 1. The data received by the camera view is passed to Open CV API for canny edge detection. 2. Next step to retrieve the contours from the data received after canny edge detection. 3. Going through each contour, skip small or non-convex objects and check if the contour has a size of If contour has size=4, then go through following steps: a. Number of vertices of polygonal curve b. Get the cosines of all corners c. Sort ascending the cosine values d. Get the lowest and the highest cosine e. Use the degrees obtained above to determine the shape of the contour f. If min Cosine value is greater than -0.1 and max Cosine value is <=0.3, we have a quadrilateral figure. g. Click the image instantly.

9