In Cycles, SYCL is currently used solely to target Intel GPUs, but one of SYCL’s goal is to deliver high performance across various GPU vendors.

Is this goal currently achieved with Cycles? That’s actually easy to try, let’s see!

Ever since we’ve introduced SYCL support in 2022, we already had put a lot in place to keep this multi-vendors support in, and we tried to avoid introducing Intel GPUs specific changes that would break it.

Such things could still ended up in the codebase, breaking or lowering runtime performance.

I’ve recently went on testing this support again, and cleaned up a few of them in the past weeks:

• Cycles: oneAPI: Use max_compute_units in get_num_multiprocessors
• Fix building for non-Intel SYCL targets without ocloc
• Cycles: oneAPI: Restrict use of experimental copy optimization to L0
• …

So, while it was all already possible with the same steps since Blender 3.3, you’ll run into less issues now with Blender 4.5 alpha 31633d14b53c and newer (from April 4).

Step 1: Get a SYCL compiler with support for AMD and Nvidia GPUs

Blender is using several important oneAPI extensions to SYCL, hence a version of oneAPI DPC++ compiler is required.

Blender is using and providing a self-compiled version of oneAPI DPC++ compiler 6.0.1, with only support for Intel GPUs.

You currently have 3 possibilities to get one with also AMD and Nvidia support:

1. Get a binary release of intel/llvm: Linux packages for nightlies do contain AMD and Nvidia GPUs support. These releases are compiled on Ubuntu 22.04 so are compatible with distributions using glibc 2.35 or higher.
2. Build your own version of intel/llvm: to get a build supporting AMD and Nvidia GPUs on Windows or any flavor of Linux. Compile time is rather long but the process is straightforward, documented here: https://github.com/intel/llvm/blob/acf6d3ba65aed812e3f27e487e5028c62ceac5e1/sycl/doc/GetStartedGuide.md
   1. HIP on Windows can work almost straight since https://github.com/intel/llvm/commit/999c682f583faf05ed5f6091b8c4946daed7d1bd. It has one remaining compilation issue with __vectorcall, worked on in https://github.com/intel/llvm/issues/7738
3. Get the proprietary version from intel.com, with the Codeplay plugins: https://codeplay.com/solutions/oneapi/plugins/

On my end, I went with option 2, on Windows, using Cuda 12.8, HIP SDK 6.2.4, VS2022 17.12.1, and nightly-2025-04-01@intel/llvm.

Step 2: Build Blender with specific CMake options

You should follow the standard’s Blender documentation: https://developer.blender.org/docs/handbook/building_blender/

Once done, you can come back and diverge from the default configuration, and edit your CMake configuration with:

• SYCL_ROOT_DIR=your path to oneAPI DPC++ compiler
• WITH_CYCLES_EMBREE=OFF # Embree is not supported on Nvidia and AMD backends and there is currently no logic to disable it per-target
• CYCLES_ONEAPI_SYCL_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda;spir64_gen
• CYCLES_ONEAPI_SYCL_OPTIONS_amdgcn-amd-amdhsa=--offload-arch=gfx1032
• CYCLES_ONEAPI_SYCL_OPTIONS_nvptx64-nvidia-cuda=--offload-arch=sm_86

Where gfx1032 and sm_86 correspond to the AMD and Nvidia devices you want to build for.

In this case, newer sm_9x and higher nvidia devices will still run after recompiling from the embed PTX, but there is no such mechanism for AMD, so make sure to pick the right device. (Yes, that’s quite a blocking issue if you want your build to be able to run on various AMD devices).

Step 3: Set CYCLES_ONEAPI_ALL_DEVICES environment variable and run

To avoid having enumeration issues and users complaining about failures on unsupported devices, only recent Intel GPUs are made accessible out of the box in Blender.
To remove this limitation and show more SYCL devices, set CYCLES_ONEAPI_ALL_DEVICES=1 environment variable before running Blender.

Step 4: Profit and Measure

All devices can be now selected in System->Preferences.

Rendering can even happen concurrently on these various devices but Cycles scales well across devices only when their performance is equal, so using such Blender build for this purpose is not necessarily a good idea.

What’s more interesting is that you can use native vendor tools for debugging and profiling, such as Nvidia Nsight Compute below, with proper source linking when compiling with -g:

And how is performance, at the moment?

It’s competitive! See below a comparison of the speed on a RTX3080:

I’ve deviated from the original build by also adding CMake option SYCL_CPP_FLAGS=-Xcudaptxas --maxrregcount=128, as it’s what ptxas chooses with native CUDA, and it gives a good speedup.

Such comparison between oneAPI and CUDA is fair and shows how good SYCL is already.

Is there a catch?

Blender Cycles is a bit specific as a good portion of the execution is spent on kernels doing intersection, that can be accelerated using Hardware Ray Tracing.

Hardware Ray Tracing is enabled on Nvidia GPUs when using the OptiX backend instead of the CUDA backend, you can see what it practically means by looking at aggregated kernel times below, generated using --verbose 4 blender argument and CYCLES_PER_KERNEL_PERFORMANCE environment variable.

Apart Intel’s Embree that of course does mix well with SYCL, current vendor specific SDKs (OptiX, HIP-RT), as well as cross-platform Vulkan Ray Tracing, cannot be efficiently mixed with anything else. What I mean with “efficiently” here is that, for good performance, we need hardware ray tracing calls to get called inline from the kernels doing intersections, and that’s not something that can be achieved with high-level interoperability.

Nonetheless, it’s pleasing to see SYCL performance for a complex real world code base like Cycles does deliver. Sure, it does so while we don’t take into consideration the intersection kernels relying on hardware acceleration, but this part is a quite specific case.

Troubleshooting typical issues

Compilation errors

clang-offload-wrapper: error: '--offload-compress' option is specified but zstd is not available. The device image will not be compressed.: Blender 4.4 and higher have enabled device images compression so it sets --offload-compress when compiling the kernels library. It’s however quite easy to get a compiler that lacks support for it and unfortunately, it errors out. I’ve opened a PR to turn the error into a warning: github.com/intel/llvm/pull/17914

Meanwhile, the solution is to build oneAPI DPC++ with LLVM_ENABLE_ZSTD=FORCE_ON instead of LLVM_ENABLE_ZSTD=ON to avoid silently running into this scenario when building the compiler, or to work around it by commenting out --offload-compress --offload-compression-level=19 from ./intern/cycles/kernel/CMakelists.txt.

Devices not enumerating

Devices failing to enumerate can be due to missing runtime dependencies, this can be traced with SYCL_UR_TRACE=1 environment variable, verified with ldd on Linux and https://github.com/lucasg/Dependencies on Windows.

On Linux, Blender calls patchelf --set-rpath $ORIGIN on its versions of all the shared libraries when building them, doing the same should fix most issues. Another temporary solution for testing is to add the compiler’s lib folder to LD_LIBRARY_PATH environment variable.

The new NDK support requires the use of Android Studio 1.3 RC1+/2.0+ and the android gradle-experimental plugin. This article is for those who are willing to give it a try. If you’re looking into the NDK support while using the gradle(-stable) plugin, you can check this older (but still up-to-date) article on the NDK and Android Studio.

last update – 2016/09/20: Android Studio 2.2 now supports external CMake and ndk-build tools very well. It’s in my view much better than the gradle-experimental plugin explained here. Please follow this guide to learn how to use it: https://developer.android.com/studio/projects/add-native-code.html

Migrating to the gradle-experimental plugin and the new com.android.model.*

The gradle-experimental 0.7.2 requires using gradle-2.10. Start by setting it from your project settings:

Or inside gradle/wrapper/gradle-wrapper.properties:

distributionUrl=https\://services.gradle.org/distributions/gradle-2.10-all.zip

Then, change the reference to the android gradle plugin to the new gradle-experimental plugin, from ./build.gradle:

// Top-level build file where you can add configuration options common to all sub-projects/modules.

buildscript {
    repositories {
        jcenter()
    }
    dependencies {
        classpath 'com.android.tools.build:gradle-experimental:0.7.2'
    }
}

The gradle-experimental plugin introduces a change in the DSL. The android plugins com.android.model.application and com.android.model.library are replacing the former com.android.application and com.android.library plugins.

You need to migrate your apps and libs build.gradle files to use these new plugins. Here is an example of the same configuration, with the old DSL (top) and the new (bottom):

apply plugin: 'com.android.application'

android {
    compileSdkVersion rootProject.ext.compileSdkVersion
    buildToolsVersion rootProject.ext.buildToolsVersion

    defaultConfig {
        applicationId "com.ph0b.example"
        minSdkVersion 15
        targetSdkVersion 23
        versionCode 4
        versionName "1.0.1"

        ndk {
            moduleName "mymodule"
            ldLibs "log"
            stl "gnustl_static"
            cFlags "-std=c++11 -fexceptions"
        }
    }

    signingConfigs {
        release {
            storeFile file(STORE_FILE)
            storePassword STORE_PASSWORD
            keyAlias KEY_ALIAS
            keyPassword KEY_PASSWORD
        }
    }

    buildTypes {
        release {
            minifyEnabled true
            shrinkResources true
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.txt'
            signingConfig signingConfigs.release
        }
        debug {
            jniDebuggable true
        }
    }
}

apply plugin: 'com.android.model.application'

model {
    android {
        compileSdkVersion rootProject.ext.compileSdkVersion
        buildToolsVersion rootProject.ext.buildToolsVersion
    
        defaultConfig {
            applicationId "com.ph0b.example"
            minSdkVersion.apiLevel 15
            targetSdkVersion.apiLevel 23
            versionCode 4
            versionName "1.0.1"
        }
    }

    android.ndk {
        moduleName "mymodule"
        ldLibs.addAll(['log'])
        cppFlags.add("-std=c++11")
        cppFlags.add("-fexceptions")
        platformVersion 15
        stl 'gnustl_shared'
    }

    android.signingConfigs {
        create("release") {
            keyAlias KEY_ALIAS
            keyPassword STORE_PASSWORD
            storeFile file(STORE_FILE)
            storePassword KEY_PASSWORD
        }
    }

    android.buildTypes {
        release {
            shrinkResources true
            useProguard true
            proguardFiles.add(file('proguard-rules.txt'))
            signingConfig = $("android.signingConfigs.release")
        }
    }
}

To summarize the changes required: all the android declarations are now going under model {}, the various assignments now have to use explicitly ‘=‘, collections must not be overwritten, use .removeAll(), .add(), .addAll() instead. Variants and other new configurations have to be declared using ‘create()‘. Properties like xxxSdkVersion have changed to xxxSdkVersion.apiLevel.

As it’s experimental, you can expect regular changes around the DSL across versions. For example, minifyEnabled has been changed to isMinifyEnabled, then to minifyEnabled again, and now there is also useProguard; jniDebuggable has been changed to isJniDebuggable and then to ndk.debuggable (and is now set by default for debug builds).

You’ll notice that with both DSLs, there is a configuration block for the NDK. This is the place where you’re supposed to set all the NDK configuration when using gradle, as by default Android.mk and Application.mk files will be simply ignored.

Enjoying the new C++/NDK support in Android Studio

To activate the C++/NDK support inside Android Studio, you only need to have a NDK module declared inside your application or library build.gradle:

model {
    android {
    //...
    ndk {
        moduleName "mymodule"
    }
  }
}

Once it’s done, you can go to your Java sources, create a method prefixed with the native keyword, and press ALT+Enter to generate its C or C++ implementation:

Yes, it’s that magical 🙂

The implementation will be added under ‘jni‘, inside an existing cpp file if there is one, or inside a new one.

In order to get started with NDK modules, you can have a look at all the samples that have been ported to use the new gradle-experimental plugin: https://github.com/googlesamples/android-ndk

Here is everything you can configure for a ndk module:

    android.ndk {
        moduleName = "mymodule"
        ldLibs.addAll(['log'])
        ldFlags.add("")
        toolchain = "clang"
        toolchainVersion = "3.9"
        abiFilters.add("x86")
        CFlags.add("")
        cppFlags.add("")
        debuggable = false
        renderscriptNdkMode = false
        stl = "system"
        platformVersion = 15
    }

Since 0.7.0, you can also add ABI-specific configurations:

    android.abis {
        create("x86") {
            cppFlags.add('-DENABLE_SSSE3')
            ldLibs.add('')
            ldFlags('')
        }
        create("armeabi-v7a") {
            cppFlags.addAll(["-mhard-float", "-D_NDK_MATH_NO_SOFTFP=1", "-mfloat-abi=hard"])
            ldLibs.add("m_hard")
            ldFlags.add("-Wl,--no-warn-mismatch")
        }
    }

Debugging a NDK project

In order to get the debug capabilities of AS, create and use a new Run/Debug configuration from the “Android Native” default. While it was possible to use GDB in Android Studio 1.3, now only the LLDB backend is available.

Use it with your debug variant, which will have ndk.debuggable flag set to true by default.

Going further with the NDK with Android Studio

Many advanced features, such as the ability to have dependencies between native libraries, reuse prebuilts, tune specific toolchain options and having dynamic version codes while still having a project in a good shape is a bit complex, as the gradle-experimental plugin is still undergoing a lot of improvements across versions.

Getting the APP_PLATFORM right

When you’re building a NDK module, the android platform you’re compiling it against is a quite important setting, as it basically determines the minimum platform your module will be guaranteed to run on.

With earlier versions than gradle-experimental:0.3.0-alpha4, the chosen platform was the one set as compileSdkVersion. Fortunately with subsequent releases, you can now set android.ndk.platformVersion independently, and you should make it the same as your minSdkVersion.

Using external libraries and separate modules

with sources

If you have access to your 3rd party libraries source code, you can embed it into your project and make it statically compile with your code.

There is an example of this with the native_app_glue library from the NDK, inside the native-activity sample. For example, you can copy the library sources inside a subfolder inside your jni folder and add a reference to its directory so the includes are properly resolved:

    android.ndk {
        //...
        cppFlags.add('-I' + file("src/main/jni/native_app_glue"))
    }

with sources in different modules

Now from 0.6.0-alpha7 version, you can finally have clean dependencies between native libraries, by setting the dependency on another module from your model:

    android.sources {
        main {
            jni {
                dependencies {
                    project ":yourlib" buildType "release" productFlavor "flavor1" linkage "shared"
                }
            }
        }
    }

In order to keep debugging working, you may have to edit your app-native run configuration, to add /build/intermediates/binaries/release/obj/[abi] to the symbol directories.

with native prebuilts

This technique works with static and shared prebuilts too! Inside your model, you’ll have to add a “lib repository”:

    repositories {
        libs(PrebuiltLibraries) {
            yourlib {
                headers.srcDir "src/main/jni/prebuilts/include"
                binaries.withType(SharedLibraryBinary) {
                    sharedLibraryFile = file("src/main/jni/prebuilts/${targetPlatform.getName()}/libyourlib.so")
                }
            }
        }
    }

And declare the dependency on this library:

    android.sources {
        main {
            jni {
                dependencies {
                    library "yourlib" linkage "shared" 
                }
            }
        }
    }

Shared linkage is the default, but of course you can use static prebuilts by using a static linkage, and declaring StaticLibraryBinary/staticLibraryFile variables.

When having dependencies on shared libraries, you need to make sure to integrate these libs inside your APK. It will be the case if they’re under jniLibs, otherwise you can add them manually:

    android.sources {
        main {
            jniLibs {
                source {
                    srcDir "src/main/jni/prebuilts"
                }
            }
        }
    }

Multiple APKs

When you publish multiple APKs (per architecture, per density, etc), you have to give them different version Version Codes. I couldn’t find a way to do it with the current DSL, neither when using splits or abiFilters and flavors. If you do see a way, plese contact me and I’ll add it there. For now, I would advise to go back using the stable gradle plugin.

The good news is you can mix use of the stable gradle plugin, and the new experimental one, even while keeping debug features working! Please follow this gist.

Using Android.mk/Application.mk

If the built-in gradle support isn’t suitable to your needs, you can get rid of it, while keeping the goodness of Android Studio C++ editing.

Declare a module that correctly represents your configuration, as this will help AS to correctly resolve all the symbols you’re using and keep the editing capabilities:

android.ndk { // keeping it to make AS correctly support C++ code editing
        moduleName "mymodule"
        ldLibs.add('log')
        cppFlags.add('-std=c++11')
        cppFlags.add('-fexceptions')
        cppFlags.add('-I' + file("src/main/jni/prebuilts/include"))
        stl = 'gnustl_shared'
    }

Then, set the jniLibs location to libs, the default directory in which ndk-build will put the generated libs, and deactivate built-in compilation tasks:

model {
    //...
    android.sources {
        main {
          jniLibs {
            source {
                srcDir 'src/main/libs'
            }
        }
    }
}

tasks.all {
    task ->
    if (task.name.startsWith('compile') && task.name.contains('MainC')) {
        task.enabled = false
    }
    if (task.name.startsWith('link')) {
        task.enabled = false
    }
}

Thanks to Alex Cohn and Vitaly Dyadyuk who shared this technique to disable compilation tasks.

This way, you can call ndk-build(.cmd) yourself from the root of your src/main directory. ndk-build will use your usual Android.mk/Application.mk files under the jni folder, your libs will be generated inside libs/<ABI> as usual and get included inside your APK.

You can also add the call to ndk-build in your gradle configuration so it’s done automatically:

import org.apache.tools.ant.taskdefs.condition.Os

model {
    //...
    android.sources {
        main {
          jniLibs {
            source {
                srcDir 'src/main/libs'
            }
        }
    }
}

// call regular ndk-build(.cmd) script from app directory
task ndkBuild(type: Exec) {
    if (Os.isFamily(Os.FAMILY_WINDOWS)) {
        commandLine 'ndk-build.cmd', '-C', file('src/main').absolutePath
    } else {
        commandLine 'ndk-build', '-C', file('src/main').absolutePath
    }
}

tasks.all {
    task ->
    if (task.name.startsWith('compile') && task.name.contains('MainC')) {
        task.enabled = false
    }
    if (task.name.startsWith('link')) {
        task.enabled = false
    }
    if (task.name.endsWith('SharedLibrary') ) {
        task.dependsOn ndkBuild
    }
}

Additional Resources

• official samples
• gradle-experimental user guide
• Important bugs that may hit you:
  • Android Studio cannot find jni.h, but compilation still works (ETA: Android Studio 2.2)
  • ndk.platformVersion < 21 prevents generating 64-bit libs (ETA: Android Studio 2.2)
  • can’t dynamically modify APKs versionCodes with gradle-experimental plugin (No ETA)
  • adding lint checks for incoherent inclusion of .so files (No ETA)

Seven distinct CPU-architectures are currently supported by the Android OS: ARMv5, ARMv7 (since 2010), x86 (2011), MIPS (2012), ARMv8, MIPS64 and x86_64 (2014). Each of them is associated with a respective ABI.

An Application Binary Interface is the definition of how binaries (esp. .so files) should be made in order to work on the platform, from the instruction set used and memory alignment, to the system libraries that are available. On Android there is one per CPU architecture: armeabi, armeabi-v7a, x86, mips, arm64-v8a, mips64, x86_64.

Why should you care about .so files in the first place?

If you’re using the NDK, you’re generating .so files, so you obviously already care. If you’re dealing only with Java code, you could think you don’t need to, as it’s cross-platform… but in fact, even in this case, quite often, you may rely without knowing it on libraries and engines that make your app embed .so files and become ABI-dependent/

Using the RenderScript support Library? OpenCV? Unity? android-gif-drawable? SQLCipher? You’ve got .so files inside your APK, and you need to care about these.

The ABIs supported by your Android application are determined by the presence of .so files inside your APK, under the “lib/<ABI>” folders where ABI can be one or more of armeabi, armeabi-v7a, arm64-v8a, mips, mips64, x86, x86_64 (the current seven ABIs).

Native Libs Monitor can help you understanding what .so files are embedded inside your APK and installed on your device, and which libraries and frameworks they’re coming from.

Most devices are compatible with more than one ABI. For example, ARM64 devices and x86 devices can run armeabi-v7a and armeabi binaries too! But it’s always better to provide libraries compiled for the preferred ABI of the device. This way, libraries can run without an emulation layer (in case of arm libs on x86 devices), and get more performance thanks to the recent architecture improvements (hardware fpu, more registers, better vectorization, etc).

You can get the list of supported ABIs, sorted by preference, from Build.SUPPORTED_ABIS. But you shouldn’t need to read it from your app, as Android’s package manager will automatically install the .so files compiled for your device preferred ABI that are present inside your APK, if they’re properly present under lib/<ABI>.

That’s where things may go wrong for your app!

There is a dead simple but not well-known important rule to follow when dealing with .so files.

You should provide libraries optimized for each ABIs if possible, but it’s all in or nothing: you shall not mix. You have to provide the full set of the libraries you’re using in each ABI folder.

When an application is installed on a device, only the .so files from one of its supported architectures are getting installed. On x86 devices, that can be the libs from the x86 folder if there is one, else from the armeabi-v7a, else armeabi folder (yes, x86 devices support these too).

But they may go wrong in many other ways too…

When you include a .so file, not only the architecture matters. Here is a list of really common mistakes I see from other developers, that are leading to “UnsatisfiedLinkError”, “dlopen: failed”, and other crashes or bad performance:

using a .so file that has been compiled against android-21+ platform version, to run it on android-15 device. Ouch.

When using the NDK, you may be tempted to “compile against the latest platform”, but it’s in fact quite wrong as ndk platforms aren’t backward compatible, they’re forward compatible! Use the same platform level than your app’s minSdkVersion.

That means that when you’re including a .so file that has already been compiled, you have to check the version it has been made to run on.

mixing .so files that are using different C++ runtimes

.so files can rely on various C++ runtimes, statically compiled or dynamically loaded. Mixing versions of C++ runtimes can lead to many weird crashes and should be avoided. As a rule of thumb, when having only one .so file, it’s ok to statically compile a C++ runtime, else you should have them all dynamically linking the same C++ runtime.

That means that when you’re including a .so file that has already been compiled and you already have some of them, you have to check the C++ runtime it’s using.

not providing all the libs for each of the supported platforms

This was already previously described, but you should really pay attention to this, as it may happen behind your back.

Example: You have an Android app with .so files for armeabi-v7a and x86 architectures. Then you add a dependency on another library that embeds .so files and supports more architectures, as android-gif-drawable: compile ‘pl.droidsonroids.gif:android-gif-drawable:1.1.+’.

Release your app, and then you’ll see it crashing on some devices, like the Galaxy S6, because only the .so files from the 64-bit directory got installed.

Solution: recompile your libs to support the missing ABIs, or set ndk.abiFilters to the one you support.

One more thing: If you’re a library provider, and you’re not supporting all the ABIs, you’re screwing up your customers, as they can’t support more ABIs than you do.

putting .so files in the wrong place

It’s easy to get confused on where .so files should be put or generated by default, here is a summary:

• jniLibs/ABI inside an Android Studio project (can be changed using jniLibs.srcDir property)
• libs/ABI inside an eclipse project (that’s where ndk-build generates libs by default too)
• jni/ABI inside an AAR (the libs will get automatically included in the APK that references the AAR).
• lib/ABI inside the final APK
• inside the app’s nativeLibraryPath on a <5.0 device / nativeLibraryRootDir/CPU_ARCH on a >=5.0 device (done by PackageManager during app installation)

providing only armeabi libs and ignoring all the other ABIs

armeabi is supported by all the x86/x86_64/armeabi-v7a/arm64-v8a devices, so that may look like a good trick to remove all the other ABIs to generate a light APK. But it’s not: there is an impact on performance and compatibility, and not only for the library itself.

x86 devices can run ARM libraries quite well, but it’s not 100% crash proof, particularly on the older devices. 64-bit devices (arm64-v8a, x86_64, mips64) can all run 32-bit libraries, but they do so in 32-bit mode, running a 32-bit version of ART and Android components, loosing the 64-bit performance improvements that have been made on the 64-bit version of ART, webview, media…

Reducing APK’s weight is in any case a false excuse, as you can upload distinct ABI-dependent APKs to the Play Store (for the same app entry, of course). You only have to switch to advanced mode and upload your APKs with different ABI support and versionCodes. Generating these APKs is as simple as adding this to your build.gradle:

android {
    ... 
    splits {
        abi {
            enable true
            reset()
            include 'x86', 'x86_64', 'armeabi-v7a', 'arm64-v8a' //select ABIs to build APKs for
            universalApk true //generate an additional APK that contains all the ABIs
        }
    }
 
    // map for the version code
    project.ext.versionCodes = ['armeabi': 1, 'armeabi-v7a': 2, 'arm64-v8a': 3, 'mips': 5, 'mips64': 6, 'x86': 8, 'x86_64': 9]
 
    android.applicationVariants.all { variant ->
        // assign different version code for each output
        variant.outputs.each { output ->
            output.versionCodeOverride =
                    project.ext.versionCodes.get(output.getFilter(com.android.build.OutputFile.ABI), 0) * 1000000 + android.defaultConfig.versionCode
        }
    }
 }

Using private system libraries

The libraries you can use are the ones which are part of the NDK ones. Any other lib isn’t guaranteed to work across devices and may trigger crashes starting with Android N: https://developer.android.com/preview/behavior-changes.html#ndk

TL;DR:

If you’ve never heard about Battery Historian, you can first have a look at the official page and some of Android Performance Patterns videos. Then, come back here if you’re using Windows, to get ready fixing your app 🙂

The first version of Battery Historian released last year was a simple cross-platform Python script. Two weeks ago, during Google I/O 15, a new version of Battery Historian (2.0!) has been quietly released.

This new release brings a lot of welcome improvements: more clarity in the reports, per-app summaries, a quicker execution, the presence of a Readme, etc.

While all these new features are awesome, Battery Historian got more complicated to install because of the addition of many dependencies and all its documentation and internal scripts are written for Unix compatible systems (Linux and Mac OSX). So out of the box, it’s simply not working on Windows.

Last week, I held a talk at Droidcon Berlin on power optimization for Android, and I’ve made a quick poll to the audience: 30-40% were actually developing on Windows (me included, 50%+ of the time).

So if you’re an Android developer who is developing on Windows, let me make you gain some time, here is my short guide on how to install Battery Historian 2.0 on Windows:

1. Installing the dependencies

Git

While you may already have some version of git installed on your system, it will be easier if you get the official client and make it available to Windows Command Prompt. In order to do so, download git’s latest version and trigger its installation.

When you get the chance to do it, do not forget to pick “Use Git from the Windows Command Prompt” :

Python

Battery Historian 1.0 requires Python 2.7, and Battery Historian 2.0 works with both Python 2.7 and 3.0 (with a small fix).
If you want to be able to generate and look at 1.0 reports from Battery Historian 2.0 (the interface gives you this possibility – but it’s not that interesting in my view), use Python 2.7, else you’re free to choose Python 3.0.
Download and trigger the installer of your choice from Python.org.

Go

Battery Historian 2.0 has been rewritten in Go, so you need first to download and install it.
Once this is done, create a go workspace (ie. a directory):

mkdir %USERPROFILE%\go-workspace

2. Installing Battery Historian 2.0

Open a Windows Command Prompt (cmd) and set useful Go specific environment variables, then go into your Go workspace:

set GOPATH=%USERPROFILE%\go-workspace
set GOBIN=%GOPATH%\bin
set PATH=%PATH%;%GOBIN%
cd %GOPATH%

Get the latest battery historian and install it:

go get -u github.com/google/battery-historian
cd %GOPATH%\src\github.com\google\battery-historian
go run setup.go

update from 3/25/16: Battery historian has been updated with a much improved support for Windows, so the following method isn’t required anymore:

Now get additional go dependencies (proto, protoc-gen-go) as well as Battery Historian, by executing:

go get -u github.com/golang/protobuf/proto
go get -u github.com/golang/protobuf/protoc-gen-go
go get -u github.com/google/battery-historian

Then you need to add closure library and compilers to battery historian. Go into battery-historian: %GOPATH%\src\github.com\google\battery-historian and create two directories: third_party and compiled.

Download the Closure compiler and unzip it inside third_party\closure-compiler

Clone the closure library to third_party\closure-library

git clone https://github.com/google/closure-library third_party/closure-library

If you’re using Python 3, change the ligne 123 of third_party/closure-library\closure\bin\build\source.py from this library: (that fix has been upstreamed)

-    fileobj = open(path)
+    fileobj = open(path, encoding="utf8")

While we’re editing sources, if you want the Battery Historian 1.0 tab to work from Battery Historian 2.0, change line 266 of battery-historian\analyzer\analyzer.go to add first the reference to your Python executable (python.exe is enough if the directory is part of your path):

-       cmd := exec.Command("./historian.py", "-c", "-m", "-r", reportName, filepath)
+       cmd := exec.Command("C:\\Python27\\python.exe", "./historian.py", "-c", "-m", "-r", reportName, filepath)

Finally, to finish the installation of Battery Historian 2.0, execute these two commands from the battery-historian directory:

third_party\closure-library\closure\bin\build\depswriter.py --root="third_party\closure-library\closure\goog" --root_with_prefix="js ../../../../js" > compiled\historian_deps-runfiles.js

java -jar third_party\closure-compiler\compiler.jar --closure_entry_point historian.Historian --js js\*.js --js third_party\closure-library\closure\goog\base.js --js third_party\closure-library\closure\goog\** --only_closure_dependencies --generate_exports --js_output_file compiled\historian-optimized.js --compilation_level SIMPLE_OPTIMIZATIONS

3. Running Battery Historian 2.0

Now that you’ve went through all the installation steps, you can finally launch Battery Historian:

set GOPATH=%USERPROFILE%\go-workspace
cd %GOPATH%\src\github.com\google\battery-historian
go run cmd\battery-historian\battery-historian.go

Tip: place these lines inside a .cmd file you’ll be able to double click on to launch Battery Historian.

Go to http://localhost:9999 and you should see your Battery Historian 2.0 local server up and running:

Now you can upload a bugreport to it, and enjoy this tool on Windows!

This device, made by Asus embeds 8GB memory, 1GB of ram, costs 99$ and features an Intel® SOC, the Atom Z3560 (codenamed Moorefield). That means it has got a 4-cores 64-bit Silvermont CPU clocked at 1.83 Ghz and a PowerVR Series 6 GPU (G6430) that supports OpenGL ES 3.1.

Let’s look at how you can port your app to this device and Android TV in general.

Android TV requirements for applications

Simply said, If you want your application to be available for Android TV devices and visible from their launcher, you need to provide an activity that will handle the action.Main intent for the LEANBACK_LAUNCHER category, to integrate TV-specific assets, and to specify that your application doesn’t require a touchscreen.

These are the very basic requirements for an application to be installed and used on Android TV devices like the Nexus Player. Let’s look at how this works in practice and then how to adapt your application so it provides a good user experience on Android TV and the Nexus Player:

Declaring an Activity for the Leanback Launcher Main intent

Like a classic Android Launcher, the Android TV Launcher will look for the activity of your application that you declared as a target of the android.intent.action.MAIN intent.

But in the case of the Android TV Launcher, the category of the intent isn’t android.intent.category.LAUNCHER but android.intent.category.LEANBACK_LAUNCHER.

Here is how you should declare the Main Activity of your application for Android TV:

<activity
            android:name=".TvMainActivity"
            android:banner="@drawable/ic_banner"
            android:theme="@style/TvAppTheme">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LEANBACK_LAUNCHER" />
            </intent-filter>
</activity>

Note: There are two categories of apps inside the Leanback Launcher -> Apps and Games. If you want to appear among the Games, you need to set the isGame property of your application to true:

<application
  ...
  android:isGame="true"
  ...>

On top of the specific intent filter, that’s also in this activity android:banner property that you specify your application’s banner that will be displayed on the Launcher. You can set this banner inside your <activity> or inside your <application>

The banner

The logo of your application that will be displayed on the launcher is the android:banner associated to your application or to the activity that accepts the Leanback Launcher intent. Here it’s the drawable we called ic_banner.

It has to be a 320x180px picture that you have to put inside your drawable-xhdpi resource folder. This pictures has to include the Name of your application, localized if needed, and have no alpha channel:

All this will allow your application to be seen and started from the Android TV launcher.

But For your application to be compatible with Android TV, you also need to support non-touchscreen Android devices and declare it:

Supporting non-touchscreen Android devices

First, you need to implement support for D-pad navigation. Standard Android UI elements all support it, but in practice some simple adjustments may be needed:

Adjusting Initial Focus

Adjust the element that will have the initial focus inside your view using requestFocus() from Java or <requestFocus /> from your xml layouts:

<View android:id="@+id/my_view" android:focusable="true" >
      <requestFocus />
</View>

The android:focusable property may need to be set to true if you want the focus request to work when your view is inflated.

Adjusting navigation flow

Sometimes you may need to adjust the navigation within your view. You can choose where the focus is going from one element to another using nextFocus properties:

<View 
    android:nextFocusDown="@id/bottom_view"
    android:nextFocusUp="@id/top_view"
    android:nextFocusLeft="@id/left_view"
    android:nextFocusRight="@id/right_view"
/>

Declaring non-touchscreen devices support

Once D-pad navigation is working for you app, you can declare it inside your AndroidManifest.xml. Simply declare that the “touchscreen” feature isn’t required:

<uses-feature
    android:name="android.hardware.touchscreen"
    android:required="false" />

Shaping a UI for Android TV

While the previous requirements may be enough for your app to be capable of being installed and appear on the Android TV launcher, there are more steps to follow to provide the right experience.

The theme of your activity has to be one with no ActionBar/Toolbar. (Even if the ActionBar can potentially work with D-pad, that’s a really bad experience to give to your users).

Two potential themes you can use are Theme.Leanback from the Leanback support library and android:Theme.NoTitleBar.

These themes don’t integrate overscan margins themselves, hence you may need to add some to your activities. Note that Leanback elements already include margins for proper overscan handling.

<LinearLayout
...
    android:layout_marginTop="27dp"
    android:layout_marginLeft="48dp"
    android:layout_marginRight="48dp"
    android:layout_marginBottom="27dp"
...>

Using the Leanback support library

The Leanback support library is available from API Level 17 and provides ready-to-use elements and a theme for your app.

If you want to continue supporting lower API levels from your application, you can keep all the references to the Leanback theme and elements inside the TV-specific part of your app and resources-v21, then use gradle’s manifest merger feature to avoid your minSdkVersion to be upgraded:

<manifest 
    xmlns:tools="http://schemas.android.com/tools"
    ...>

    <uses-sdk tools:overrideLibrary="android.support.v17.leanback" />
   ...

You can start looking at the sample to see how to use the Leanback support library. I also recommend reading this good introductory article from Sebastiano Gottardo (musixMatch).

Going further with Games

Android TV games can really be improved by adding a good multiplayer experience and Gamepad support.

From your manifest, you can specify that you support gamepads:

<uses-feature 
    android:name="android.hardware.gamepad"
    android:required="false" />

Set the feature as required only if gamepads are mandatory for your app, but remember that devices like the Nexus Player aren’t bundled with one. Additional Android devices can be used as controllers, but they only feature D-pad input as of now.

To support multiple user inputs, you can refer to this training.

Integrating with the Recommendations system

The Recommendations system occupies half of the space of the Launcher, to appear inside it, send a notification with “recommendation” set as category, as described in the official training.

Having a specific APK for Android TV

Through the features filtering functionality of the Play Store, you can restrict your APK for distribution to Android TV compatible devices, by setting the feature android.software.leanback as required:

<uses-feature android:name="android.software.leanback"
        android:required="true" />

When you require the leanback feature, your APK will be available only on Android-TV compatible devices.

You can choose to distribute this APK as a specific application on the Play Store, but you also have the choice to upload it as a TV-version of an already existing application, through the “Multiple APKs” feature of the Play Store.

You simply have to upload this TV-specific apk that requires the leanback feature and has a different versionCode than your classic application.

Optimizing your Application for the Nexus Player

While the Player is a fully 64-bit capable platform and the kernel is indeed 64-bit, all the userspace remains x86 32-bit as of now.

The device is able to run ARM-specific binaries perfectly fine, but providing x86 binaries is always better. You can read more about compatibility considerations for x86 devices from my previous article: How to improve x86 support of Android apps, libs, engines

Last things to check before uploading your app

A recapitulation of the guidelines for Android TV can be found on the Android developer website: Tv App Quality.

If you have android:screenOrientation=”portrait” added to any activity of your application, the android.hardware.screen.portrait requirement is implicitly set to true.

You need to explicitly set it to false inside your manifest if you want your application to be available on Android TV devices:

<uses-feature
    android:name="android.hardware.screen.portrait"
    android:required="false" />

The same is needed for:

• android.hardware.location.android.permission.ACCESS_FINE_LOCATION
• android.hardware.camera.autofocus and android.hardware.camera implied by android.permission.CAMERA
• android.hardware.microphone implied by android.permission.RECORD_AUDIO
• android.hardware.telephony implied by many telephony-specific permissions

Submitting your application

Once your application is ready and you’ve uploaded your APK to the Play Store developer console, you still need to add a screenshot as well as your banner:

| | | |—|—|

Then tick the proper checkbox to ask for distribution on Android TV:

It will be manually reviewed before being available on the Play Store browsed from Android TV devices.

It doesn’t take much to compile the proper x86 versions of JavaCV packages, but if you’re in a hurry, here they are:

• javacv-0.7-bin-android-x86.zip
• javacv-0.7-cppjars-android-x86.zip

Now let’s look at what was needed to build these. You need the usual build tools (gcc, make…) as well as maven.

You first need to recompile cppjars:

Recompiling cppjars (OpenCV and ffmpeg)

You can easily do so by reusing the build scripts provided in the cppjars release package:

wget https://javacv.googlecode.com/files/javacv-0.7-cppjars.zip
unzip javacv-0.7-cppjars.zip
cd javacv-cppjars

The build scripts for Android are: build_opencv-android-arm.sh and build_ffmpeg-android-arm.sh. We need to create build_opencv-android-x86.sh and build_ffmpeg-android-x86.sh equivalents of these.

Start by copying these scripts, then in the -x86 version, replace :

• arm-linux-androideabi- by i686-linux-android- (toolchain binaries prefix)
• arm-linux-androideabi- by x86- (toolchain)
• -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -mfpu=neon… by -mtune=atom -mssse3 -mfpmath=sse (compiler flags)
• arm-linux with i686-linux
• other -arm suffixes by -x86

Don’t do this blindly – it will work better if you understand what you’re doing 🙂

In the end here is the content I have for build_ffmpeg-android-x86.sh:

ANDROID_BIN=$ANDROID_NDK/toolchains/x86-4.8/prebuilt/linux-x86_64/bin
ANDROID_ROOT=$ANDROID_NDK/platforms/android-14/arch-x86
tar -xjvf ffmpeg-$FFMPEG_VERSION.tar.bz2
mv ffmpeg-$FFMPEG_VERSION ffmpeg-$FFMPEG_VERSION-android-x86
cd ffmpeg-$FFMPEG_VERSION-android-x86
tar -xjvf ../last_stable_x264.tar.bz2
X264=`echo x264-snapshot-*`
cd $X264
./configure --enable-static --enable-pic --disable-cli --disable-opencl --cross-prefix=$ANDROID_BIN/i686-linux-android- --sysroot=$ANDROID_ROOT --host=i686-linux --extra-cflags="-fpic -pipe -DANDROID -DNDEBUG -mtune=atom -mssse3 -ffast-math -mfpmath=sse -fomit-frame-pointer -fstrict-aliasing -funswitch-loops -finline-limit=300" --extra-ldflags="-lm -lz -Wl,--no-undefined -Wl,-z,noexecstack"
make -j8
cd ../
patch -p1 < ../ffmpeg-$FFMPEG_VERSION-android-x86.patch
./configure --prefix=$ANDROID_NDK/../ --enable-shared --enable-gpl --enable-version3 --enable-libx264 \
 --disable-static --disable-symver --disable-doc --disable-ffplay --disable-ffmpeg --disable-ffprobe --disable-ffserver --disable-encoders --disable-muxers --disable-devices --disable-demuxer=sbg --disable-demuxer=dts --disable-parser=dca --disable-decoder=dca --disable-decoder=svq3 --enable-network --enable-version3 --disable-amd3dnow --disable-amd3dnowext --disable-outdev=sdl\
 --extra-cflags="-I$X264" --extra-ldflags="-L$X264" --enable-cross-compile --cc=$ANDROID_BIN/i686-linux-android-gcc --sysroot=$ANDROID_ROOT --target-os=linux --arch=x86 --cpu=i686 \
--enable-asm --enable-yasm --enable-pic --extra-cflags="-DANDROID -DNDEBUG -fPIC -pipe -mtune=atom -mssse3 -ffast-math -mfpmath=sse" \
--extra-ldflags="-lm -lz -Wl,--no-undefined -Wl,-z,noexecstack" --disable-stripping --disable-symver --disable-programs

make -j8
LIBS="libavcodec/libavcodec.so libavdevice/libavdevice.so libavfilter/libavfilter.so libavformat/libavformat.so libavutil/libavutil.so libpostproc/libpostproc.so libswresample/libswresample.so libswscale/libswscale.so"
$ANDROID_NDK/toolchains/x86-4.8/prebuilt/linux-x86_64/bin/i686-linux-android-strip $LIBS
mkdir -p com/googlecode/javacv/cpp/android-x86/
cp $LIBS com/googlecode/javacv/cpp/android-x86/
jar cvf ../ffmpeg-$FFMPEG_VERSION-android-x86.jar com/
rm -Rf com/
cd ../

Here ffmpeg-$FFMPEG_VERSION-android-x86.patch is exactly the same as the arm one, you can cp it.

And for build_opencv-android-x86.sh:

tar -xzvf opencv-$OPENCV_VERSION.tar.gz
mkdir opencv-$OPENCV_VERSION/build_android-x86
cd opencv-$OPENCV_VERSION
cd build_android-x86
ANDROID_BIN=$ANDROID_NDK/toolchains/x86-4.6/prebuilt/linux-x86_64/bin/ \
ANDROID_CPP=$ANDROID_NDK/sources/cxx-stl/gnu-libstdc++/4.6/ \
ANDROID_ROOT=$ANDROID_NDK/platforms/android-9/arch-x86/ \
cmake -DCMAKE_TOOLCHAIN_FILE=platforms/android/android.toolchain.cmake -DANDROID_ABI=x86 -DOPENCV_EXTRA_C_FLAGS="-O3 -ffast-math -mtune=atom -mssse3 -mfpmath=sse" -DOPENCV_EXTRA_CXX_FLAGS="-O3 -ffast-math -mtune=atom -mssse3 -mfpmath=sse" -DCMAKE_INSTALL_PREFIX=$ANDROID_NDK/../ -DBUILD_SHARED_LIBS=ON -DBUILD_TESTS=OFF -DBUILD_PERF_TESTS=OFF -DBUILD_ANDROID_EXAMPLES=OFF -DBUILD_JASPER=ON -DBUILD_JPEG=ON -DBUILD_OPENEXR=ON -DBUILD_PNG=ON -DBUILD_TBB=ON -DBUILD_TIFF=ON -DBUILD_ZLIB=ON -DBUILD_opencv_java=OFF -DBUILD_opencv_python=OFF -DENABLE_PRECOMPILED_HEADERS=OFF -DWITH_1394=OFF -DWITH_FFMPEG=OFF -DWITH_GSTREAMER=OFF -DWITH_TBB=ON -DWITH_CUDA=OFF -DWITH_OPENCL=OFF ..
make -j8
LIBS="lib/x86/libopencv*.so lib/x86/libtbb.so"
$ANDROID_NDK/toolchains/x86-4.6/prebuilt/linux-x86_64/bin/i686-linux-android-strip $LIBS
mkdir -p com/googlecode/javacv/cpp/android-x86/
cp $LIBS com/googlecode/javacv/cpp/android-x86/
jar cvf ../../opencv-$OPENCV_VERSION-android-x86.jar com/
rm -Rf com/
cd ../../

Here I have deleted the call to the -arm patch that was generating another android.mk file to directly rely on platforms/android/android.toolchain.cmake that is provided by OpenCV and I’ve also added specific flags to get more optimization:

-DANDROID_ABI=x86 -DOPENCV_EXTRA_C_FLAGS="-O3 -ffast-math -mtune=atom -mssse3 -mfpmath=sse" -DOPENCV_EXTRA_CXX_FLAGS="-O3 -ffast-math -mtune=atom -mssse3 -mfpmath=sse"

Now if you call:

sh ./build_all.sh android-x86
cd ..

This will generate you ffmpeg-2.1.1-android-x86.jar and opencv-2.4.8-android-x86.jar.

Now you can finally build the JavaCV package:

Packaging x86 version of JavaCV

First retrieve and install JavaCPP:

git clone https://code.google.com/p/javacpp/
cd javacpp/
git checkout 0.6
mvn install
cd ..

Once its done, retrieve JavaCV source code:

git clone https://code.google.com/p/javacv/
cd javacv
git checkout 0.7

You can now package the android-x86 version of JavaCV by setting the android-x86 property to JavaCPP a:

mvn package -Pffmpeg -Djavacpp.options="-properties android-x86 -Dplatform.root=$ANDROID_NDK -Dcompiler.path=$ANDROID_NDK/toolchains/x86-4.6/prebuilt/linux-x86_64/bin/i686-linux-android-g++ -Dcompiler.includepath=$ANDROID_NDK/sources/cxx-stl/gnu-libstdc++/4.6/include\
:$ANDROID_NDK/sources/cxx-stl/gnu-libstdc++/4.6/libs/x86/include\
:../javacv-cppjars/opencv-2.4.8/build_android-x86/\
:../javacv-cppjars/opencv-2.4.8/modules/core/include\
:../javacv-cppjars/opencv-2.4.8/modules/androidcamera/include\
:../javacv-cppjars/opencv-2.4.8/modules/flann/include\
:../javacv-cppjars/opencv-2.4.8/modules/imgproc/include\
:../javacv-cppjars/opencv-2.4.8/modules/highgui/include\
:../javacv-cppjars/opencv-2.4.8/modules/features2d/include\
:../javacv-cppjars/opencv-2.4.8/modules/calib3d/include\
:../javacv-cppjars/opencv-2.4.8/modules/ml/include\
:../javacv-cppjars/opencv-2.4.8/modules/video/include\
:../javacv-cppjars/opencv-2.4.8/modules/legacy/include\
:../javacv-cppjars/opencv-2.4.8/modules/objdetect/include\
:../javacv-cppjars/opencv-2.4.8/modules/photo/include\
:../javacv-cppjars/opencv-2.4.8/modules/gpu/include\
:../javacv-cppjars/opencv-2.4.8/modules/nonfree/include\
:../javacv-cppjars/opencv-2.4.8/modules/contrib/include\
:../javacv-cppjars/opencv-2.4.8/modules/stitching/include\
:../javacv-cppjars/opencv-2.4.8/modules/ts/include\
:../javacv-cppjars/opencv-2.4.8/modules/videostab/include\
:../javacv-cppjars/ffmpeg-2.1.1-android-x86 \
-Dcompiler.linkpath=$ANDROID_NDK/sources/cxx-stl/gnu-libstdc++/4.6/libs/x86\
:../javacv-cppjars/opencv-2.4.8/build_android-x86/lib/x86\
:../javacv-cppjars/ffmpeg-2.1.1-android-x86/libswscale\
:../javacv-cppjars/ffmpeg-2.1.1-android-x86/libavcodec\
:../javacv-cppjars/ffmpeg-2.1.1-android-x86/libswresample\
:../javacv-cppjars/ffmpeg-2.1.1-android-x86/libpostproc\
:../javacv-cppjars/ffmpeg-2.1.1-android-x86/libavfilter\
:../javacv-cppjars/ffmpeg-2.1.1-android-x86/libavformat\
:../javacv-cppjars/ffmpeg-2.1.1-android-x86/libavutil\
:../javacv-cppjars/ffmpeg-2.1.1-android-x86/libavdevice"

You’ll get your packages in target/:

2.7M Jan 15 19:09 javacv-android-x86.jar
3.6M Jan 15 19:09 javacv-bin.zip
747K Jan 15 19:09 javacv-src.zip
734K Jan 15 19:09 javacv.jar

Adding this x86 version to your app

From the packages I gave at the beginning of this article or from what you’ve just built, you can now copy the generated .so files that are inside javacv-android-x86.jar, ffmpeg-2.1.1-android-x86.jar and opencv-2.4.8-android-x86.jar into the /lib/x86/ folder of your Android package, the same way as the arm versions inside /lib/armeabi-v7a/.

To get more information on how to deal with .so files and APKs, please refer to these two previous articles:

• http://ph0b.com/improving-x86-support-of-android-apps-libs-engines/
• http://ph0b.com/android-studio-gradle-and-ndk-integration/

While it’s fully compatible with Arduino Software Development Environment, it’s also capable of running a full Linux distribution.

Sept. 2014 update: GCC and other dev tools are now part of the official linux image you can download from Intel Developer Zone.

You can get such distribution from the official website. The “LINUX IMAGE FOR SD for Intel Galileo” is a poky distribution built using Yocto. It’s using uClibc and contains useful binaries like SSHd, bluez, nodejs and OpenCV while it can still run Arduino sketches !

It’s really simple to install it: you only need to unzip the .7z package at the root of a FAT32 formatted microSD.

I’ve came across some interesting nodejs modules, and some of them are relying on native extensions. Cross-compiling these seemed complicated so I’ve chosen another way to get these working on my Galileo: rebuilding the poky distribution to add dev tools.

And while it may sound complicated… it’s in fact dead simple thanks to the way Yocto is working.

First set up your build environment the same way you would build the same image than the one distributed on the official website, by downloading Galileo BSP and setting up Yocto:

wget http://downloadmirror.intel.com/23171/eng/Board_Support_Package_Sources_for_Intel_Quark_v0.7.5.7z
7z x Board_Support_Package_Sources_for_Intel_Quark_v0.7.5.7z
tar xzvf Board_Support_Package_Sources_for_Intel_Quark_v0.7.5/meta-clanton_v0.7.5.tar.gz
cd meta-clanton_v0.7.5
./setup.sh
source poky/oe-init-build-env yocto_build

Stop here to create your custom configuration:

cp ../meta-clanton-distro/recipes-core/images/image-full.bb ../meta-clanton-distro/recipes-core/images/image-custom.bb

Here I’ve chosen to triple the filesystem size to make it around 900mo:

IMAGE_ROOTFS_SIZE = "921600"

added -dev packages to the image by adding dev-pkgs feature:

IMAGE_FEATURES += "package-management dev-pkgs"

added the tools I wanted by adding this line:

IMAGE_INSTALL += "autoconf automake binutils binutils-symlinks cpp cpp-symlinks gcc gcc-symlinks g++ g++-symlinks gettext make libstdc++ libstdc++-dev file coreutils"

and also integrated the right firmware for my Wi-Fi/BT4.0 chipset (Centrino 6235) by adding this line:

IMAGE_INSTALL += "linux-firmware-iwlwifi-6000g2b-6"

Finally you can build your custom distribution:

bitbake image-custom

For me this process took around 20GB space and around an hour of automated download/build. Once its done you’ll find your generated files inside yocto_build/tmp/deploy/images. You can copy these to your microSD like so:

• latest bzImage-* as bzImage
• latest core-image-minimal-initramfs-* as core-image-minimal-initramfs-clanton.cpio.gz
• latest image-* as image-full-clanton.ext3
• boot folder as boot

Now you can easily get useful nodejs modules with native extensions – they will be compiled on the Galileo itself :

npm install socket.io

And if you don’t have the time and bandwidth to spare, you can directly download the custom image I’ve generated 🙂

update: There is now an official SD image with gcc support: https://software.intel.com/en-us/iotdevkit

If you want to get more information on how to build your own image or learn how to use another libc than uClibc, you should read this article: Intel Galileo – Building Linux Image

PS: I’ve encountered one small bug with the image I’ve generated, stdbuf couldn’t find ‘libstdbuf.so’. You can easily fix this by running ln -s /usr/lib/coreutils/libstdbuf.so /usr/lib/coreutils/coreutils/

Now this is really easy to integrate native libraries in your package and generate APKs for different architectures while correctly handling version codes (for more information on why this may be important, please refer to my first article).

update 2015/08/10: this article remains fully up-to-date if you’re using the recent Android Studio and the stable Android gradle plugin (set android.useDeprecatedNdk=true inside gradle.properties). Another article on gradle-experimental is in the work!

update 2014/11/1: changed output.abiFilter to output.getFilter(com.android.build.OutputFile.ABI) in order to work with gradle 0.14

update 2014/09/19: I’ve added information and modified my sample build.gradle to demonstrate the brand new APK Splits feature introduced by the latest android gradle plugin (0.13)

update 2: I’ve modified the sample build.gradle file to make it call ndk-build scripts by itself.

update 1: Here is a screencast on how to set up a project with NDK sources from Android Studio:

Integrating .so files into your APK

If you are using Android Studio and need to integrate native libraries in your app, you may have had to use some complex methods before, involving maven and .aar/.jar packages… the good news is you don’t need these anymore 🙂

You only need to put your .so libraries inside the jniLibs folder under sub-directories named against each supported ABI (x86, mips, armeabi-v7a, armeabi), and that’s it !

Once it’s done, all the .so files will be integrated into your apk when you build it:

If the jniLibs folder name doesn’t suit you (you may generate your .so files somewhere else), you can set a specific location in build.gradle:

android {
    ...
    sourceSets.main {
        jniLibs.srcDir 'src/main/libs'
    }
}

Building one APK per architecture, and doing it well!

You can use flavors to build one APK per architecture really easily, by using abiFilter property.

ndk.abiFilter(s) is by default set to all. This property has an impact on the integration of .so files as well as the calls to ndk-build (I’ll talk about it at the end of this article).

Let’s add some architecture flavors in build.gradle:

android{
  ...
  productFlavors {
        x86 {
            ndk {
                abiFilter "x86"
            }
        }
        mips {
            ndk {
                abiFilter "mips"
            }
        }
        armv7 {
            ndk {
                abiFilter "armeabi-v7a"
            }
        }
        arm {
            ndk {
                abiFilter "armeabi"
            }
        }
        fat
    }
}

And then, sync your project with gradle files:

You should now be able to enjoy these new flavors by selecting the build variants you want:

Each of these variants will give you an APK for the designated architecture:

app-x86-release-unsigned.apk:

But don’t stop reading here! These arch-dependent APKs are useful when developing, but if you want to upload several of these to the Google Play Store, you have to set a different versionCode for each. And thanks to the latest android build system, this is really easy:

Automatically setting different version codes for ABI dependent APKs

The property android.defaultConfig.versionCode holds the versionCode for your app. By default it’s set to -1 and if you don’t change it, the versionCode set in your AndroidManifest.xml file will be used instead.

Hence if you want to be able to dynamically modify your versionCode, you need to first specify it inside your build.gradle:

android {
    ...
    defaultConfig{
        versionName "1.1.0"
        versionCode 110
    }
}

But this is still possible to keep setting this variable only inside your AndroidManifest.xml if you retrieve it “manually” before modifying it:

import java.util.regex.Pattern

android {
    ...
    defaultConfig{
        versionCode getVersionCodeFromManifest()
    }
    ...
}

def getVersionCodeFromManifest() {
    def manifestFile = file(android.sourceSets.main.manifest.srcFile)
    def pattern = Pattern.compile("versionCode=\"(\\d+)\"")
    def matcher = pattern.matcher(manifestFile.getText())
    matcher.find()
    return Integer.parseInt(matcher.group(1))
}

Once it’s done, you can prefix the versionCode inside your different flavors:

android {
    ...
    productFlavors {
        x86 {
            versionCode Integer.parseInt("6" + defaultConfig.versionCode)
            ndk {
                abiFilter "x86"
            }
        }
        mips {
            versionCode Integer.parseInt("4" + defaultConfig.versionCode)
            ndk {
                abiFilter "mips"
            }
        }
        armv7 {
            versionCode Integer.parseInt("2" + defaultConfig.versionCode)
            ndk {
                abiFilter "armeabi-v7a"
            }
        }
        arm {
            versionCode Integer.parseInt("1" + defaultConfig.versionCode)
            ndk {
                abiFilter "armeabi"
            }
        }
        fat
    }
}

Here I’ve prefixed it with 6 for x86, 4 for mips, 2 for ARMv7 and 1 for ARMv5. If you’re asking yourself why!? please refer to this paragraph I wrote before on architecture dependent APKs on the Play Store.

Improving multiple APKs creation and versionCode handling with APK Splits

Since version 0.13 of the android plugin, instead of having a product flavors to get multiple APKs, you can use splits to have a single build (and variant) that will produce multiple APKs (and it’s much cleaner and faster).

    splits {
        abi {
            enable true // enable ABI split feature to create one APK per ABI
            universalApk true //generate an additional APK that targets all the ABIs
        }
    }
    // map for the version code
    project.ext.versionCodes = ['armeabi':1, 'armeabi-v7a':2, 'arm64-v8a':3, 'mips':5, 'mips64':6, 'x86':8, 'x86_64':9]

    android.applicationVariants.all { variant ->
        // assign different version code for each output
        variant.outputs.each { output ->
            output.versionCodeOverride =
                    project.ext.versionCodes.get(output.getFilter(com.android.build.OutputFile.ABI), 0) * 1000000 + android.defaultConfig.versionCode
        }
    }

Compiling your C/C++ source code from Android Studio

If you have a jni/ folder in your project sources, the build system will try to call ndk-build automatically.

As of 0.7.3, this integration is only working on Unix-compatible systems, cf bug 63896. On Windows you’ll want to disable it so you can call ndk-build.cmd yourself. You can do so by setting this in build.gradle: this has been fixed 🙂

The current implementation is ignoring your Android.mk makefiles and create a new one on the fly. While it’s really convenient for simple projects (you don’t need *.mk files anymore !), it may be more annoying for projects where you need all the features offered by Makefiles. You can then disable this properly in build.gradle:

android{
    ...
    sourceSets.main.jni.srcDirs = [] //disable automatic ndk-build call
}

If you want to use the on-the-fly generated Makefile, you can configure it first by setting the ndk.moduleName property, like so:

android {
 ...
 defaultConfig {
        ndk {
            moduleName "hello-jni"
        }
    }
}

And you’re still able to set these other ndk properties:

• **cFlags**
• ldLibs
• stl (ie: gnustl_shared, stlport_static…)
• abiFilters (ie: “x86”, “armeabi-v7a”)

You can also set android.buildTypes.debug.jniDebuggable to true so it will pass NDK_DEBUG=1 to ndk-build when generating a debug APK.

If you are using RenderScript from the NDK, you’ll need also to set the specific property defaultConfig.renderscriptNdkMode to true.

If you rely on auto-generate Makefiles, you can’t easily set different cFlags depending on the target architecture when you’re building multi-arch APKs. So if you want to entirely rely on gradle I recommend you to generate different libs per architecture by using flavors like I’ve described earlier in this post:

  ...
  productFlavors {
    x86 {
        versionCode Integer.parseInt("6" + defaultConfig.versionCode)
        ndk {
            cFlags cFlags + " -mtune=atom -mssse3 -mfpmath=sse"
            abiFilter "x86"
        }
    }
    ...

My sample .gradle file

Putting this altogether, Here is one build.gradle file I’m curently using. It’s using APK Splits to generate multiple APKs, it doesn’t use ndk-build integration to still rely on Android.mk and Application.mk files, and doesn’t require changing the usual location of sources and libs (sources in jni/, libs in libs/). It’s also automatically calling ndk-build script from the right directory:

import org.apache.tools.ant.taskdefs.condition.Os

apply plugin: 'com.android.application'

android {
    compileSdkVersion 21
    buildToolsVersion "21.1"

    defaultConfig{
        minSdkVersion 16
        targetSdkVersion 21
        versionCode 101
        versionName "1.0.1"
    }

    sourceSets.main {
        jniLibs.srcDir 'src/main/libs'
        jni.srcDirs = [] //disable automatic ndk-build call
    }

   project.ext.versionCodes = ['armeabi':1, 'armeabi-v7a':2, 'arm64-v8a':3, 'mips':5, 'mips64':6, 'x86':8, 'x86_64':9] //versionCode digit for each supported ABI, with 64bit>32bit and x86>armeabi-*

    android.applicationVariants.all { variant ->
        // assign different version code for each output
        variant.outputs.each { output ->
            output.versionCodeOverride =
 project.ext.versionCodes.get(output.getFilter(com.android.build.OutputFile.ABI), 0) * 1000000 + defaultConfig.versionCode
        }
    }

    // call regular ndk-build(.cmd) script from app directory
    task ndkBuild(type: Exec) {
        if (Os.isFamily(Os.FAMILY_WINDOWS)) {
            commandLine 'ndk-build.cmd', '-C', file('src/main').absolutePath
        } else {
            commandLine 'ndk-build', '-C', file('src/main').absolutePath
        }
    }

    tasks.withType(JavaCompile) {
        compileTask -> compileTask.dependsOn ndkBuild
    }
}

Troubleshooting

NDK not configured

If you get this kind of error:

Execution failed for task ':app:compileX86ReleaseNdk'.
> NDK not configured

This means the tools haven’t found the NDK directory. You have two ways to fix it: set the ANDROID_NDK_HOME variable environment to your NDK directory and delete local.properties, or set it manually inside local.properties:

ndk.dir=C\:\\Android\\ndk

No rule to make target

If you get this kind of error:

make.exe: *** No rule to make target ...\src\main\jni

This may come from a current NDK bug on Windows, when there is only one source file to compile. You only need to add one empty source to make it work again.

Other issues

You may also find some help on the official adt-dev google group: https://groups.google.com/forum/#!forum/adt-dev.

Getting more information on NDK integration

The best place to get more information is the official project page: http://tools.android.com/tech-docs/new-build-system.

You can look at the changelog and if you scroll all the way down you’ll also get access to sample projects dealing with NDK integration, inside the latest “gradle-samples-XXX.zip” archive.

This library is open-source (Apache License) and initially created by Qualcomm Innovation Center. The project has joined the AllSeen Alliance two days ago and now it’s hosted here: https://allseenalliance.org/source-code

Even if the precompiled x86 version isn’t distributed from the official website, compiling it from the sources is perfectly supported.

When you download the release package for Android, here is what you get:

cpp/    core AllJoyn functionality, implemented in C++
          - built from Git projects alljoyn_core and common
          - required for all AllJoyn applications
java/   optional Java language binding          (built from alljoyn_java)
          - required for Android apps
c/      optional ANSI C language binding        (built from alljoyn_c)
          - required by Unity binding
unity/  optional Unity language binding         (built from alljoyn_unity)
about/  implements AllJoyn About Feature. (built from about.git/(cpp and java))

If you use the Java binding alljoyn.jar, you’ll end up with liballjoyn_java.so in lib/armeabi/. The binaries (.so/.a files) from this package are ARMv5 only but compiling it from the sources is perfectly supported.

Here is the x86 release version I’ve compiled for you: alljoyn-3.4.5-android-sdk-rel-x86.zip. It follows the same architecture than the ARMv5 release.

If you’re just using the AllJoyn .jar on Android, you can directly get the liballjoyn_java.so and put it inside the lib/x86 directory of your Android application.

If you want to recompile it yourself or know more about the process, here is what I did:

Recompiling Alljoyn for x86-based Android platforms

First, get the sources:

git clone https://git.allseenalliance.org/gerrit/core/alljoyn
cd alljoyn

Then, retrieve libcrypto.so and libssl.so from a real x86 device or the x86 emulator and put them inside build/android/x86/release/dist/cpp/lib/ :

mkdir -p build/android/x86/release/dist/cpp/lib/
adb pull /system/lib/libcrypto.so build/android/x86/release/dist/cpp/lib/
adb pull /system/lib/libssl.so build/android/x86/release/dist/cpp/lib/

Now you can build the lib for x86 as explained by the project documentation – the tedious part being the need of having a copy of the AOSP sources:

export CLASSPATH=/usr/share/java/junit.jar
scons OS=android CPU=x86 ANDROID_NDK=/opt/android-ndk ANDROID_SDK=/opt/android-sdk ANDROID_SRC=/home/ph0b/android-build VARIANT=release BINDINGS=cpp,c,java,unity

Porting is a big word here ! As often, adding x86 support to libaacdecoder was a matter of fixing Makefiles.

libaacdecoder supports AAC as well as MP3 decoding through the use of OpenCore decoders. It’s based on opencore-aacdec which compiles fine for x86.

The release package, as of 0.6.1, includes only binaries for ARMv5 and ARMv7. If you try to recompile it by yourself from the sources, you may even think that other architectures aren’t supported… but this is wrong!

If you’re in a hurry, here is the compiled library package (version 0.6.1) with the x86 version of libaacdecoder.so added: aacdecoder-android-libs-0.6.1.zip

And now for the “technical details”, 2 small changes where involved:

1. As usual, add x86 to the list of targeted ABIs

Open ./decoder/jni/Application.mk and add x86 to the list of ABIs or simply put “all”:

APP_ABI := all

Now if you call ndk-build again, the toolchain will try to generate x86 versions of the library as well, but here it will fail with many errors like these:

...
     [exec] ./codecs_v2/audio/mp3/dec/src/asm/pvmp3_dct_16_gcc.s:225: Error: no such instruction: `smull r9,r7,r11,r7'
     [exec] ./codecs_v2/audio/mp3/dec/src/asm/pvmp3_dct_16_gcc.s:226: Error: too many memory references for `add'
     [exec] ./codecs_v2/audio/mp3/dec/src/asm/pvmp3_dct_16_gcc.s:227: Error: too many memory references for `sub'
...

because it’s trying to integrate ARM assembly code. Let’s fix this:

2. Fix the list of sources set inside opencore-mp3dec Makefile

Open ./decoder/jni/opencore-mp3dec/Android.mk.

You’ll see ARM asm sources included:

LOCAL_SRC_FILES := \
...
  src/asm/pvmp3_dct_16_gcc.s \
  src/asm/pvmp3_dct_9_gcc.s \
  src/asm/pvmp3_mdct_18_gcc.s \
  src/asm/pvmp3_polyphase_filter_window_gcc.s

Of course these can’t be compiled for x86… but inside src/ you already have C++ equivalents that can ! So you just need to conditionally include these:

ifeq ($(TARGET_ARCH),arm)
  LOCAL_SRC_FILES += \
    src/asm/pvmp3_dct_16_gcc.s \
    src/asm/pvmp3_dct_9_gcc.s \
    src/asm/pvmp3_mdct_18_gcc.s \
    src/asm/pvmp3_polyphase_filter_window_gcc.s
else
  LOCAL_SRC_FILES += \
    src/pvmp3_dct_16.cpp \
    src/pvmp3_dct_9.cpp \
    src/pvmp3_mdct_18.cpp \
    src/pvmp3_polyphase_filter_window.cpp
endif

While you’re here, you’ll also see an ARM macro always defined:

LOCAL_CFLAGS := -DPV_ARM_GCC_V4 $(PV_CFLAGS)

It will be better to conditionally set it:

LOCAL_CFLAGS :=  $(PV_CFLAGS)
ifeq ($(TARGET_ARCH),arm)
  LOCAL_CFLAGS += -DPV_ARM_GCC_V4 
endif

Once these changes are done you’ll be able to compile libaacdecoder for all the supported architectures.

Here is the latest release package, updated with all the libs: aacdecoder-android-libs-0.6.1.zip