An introduction to TensorFlow Lite development on Android

Android development is not limited to cute little apps that split the bill in restaurants (that seems to be everyone’s “genius app idea,” or is it just me?). Android is a powerful platform with backing from one of the biggest and most influential companies in the world. A company that is at the forefront of machine learning and considers itself “AI-first.”

Learning TensorFlow Lite for Android lets developers implement advanced machine learning into their creations. This greatly extends an app’s capabilities and introduces countless new potential use-cases. It also teaches invaluable skills that are only going to increase in demand over the coming years.

See also: Is your job safe? Jobs that AI will destroy in the next 10-20 years

This is the perfect introduction to machine learning, so let’s get started!

What is TensorFlow?

Let’s start with the basics: what is TensorFlow Lite? To answer that, we should first look at TensorFlow itself. TensorFlow is an “end-to-end” (meaning all-in-one), open-source platform for machine learning from the Google Brain Team. TensorFlow is an open-source software library that enables machine learning tasks.

A machine learning task is any problem that requires pattern recognition powered by algorithms and large amounts of data. This is AI, but not in the Hal from 2001: A Space Odyssey sense.

See also: Artificial intelligence vs machine learning: what’s the difference?

Use Cases

An example of a machine learning application is computer vision. It enables computers to recognize objects in a photograph or a live camera feed. To do this, the program must first be “trained” by being shown thousands of pictures of that object. The program never understands the object but learns to look for particular data patterns (changes in contrast, particular angles or curves) that are likely to match the object. Over time, the program becomes increasingly accurate at spotting that object.

As an Android developer, computer vision creates many possibilities: whether you want to use facial recognition as a security feature, create an AR program that can highlight elements in the environment, or build the next “Reface” app. This is before we consider the countless other uses for machine learning models: voice recognition, OCR, enemy AI, and much more.

Creating and implementing these types of models from scratch would be an extremely arduous task for a single developer, which is why it’s so useful to have access to ready-made libraries.

See also: What is Google Cloud?

TensorFlow is capable of running on a wide range of CPUs and GPUs but works particularly well with Google’s own Tensor Processing Units (TPUs). Developers can also leverage the power of the Google Cloud Platform by outsourcing machine learning operations to Google’s servers.

What is TensorFlow Lite?

TensorFlow Lite brings on-board (this means it runs on the mobile device itself) Tensor Flow to mobile devices. Announced in 2017, the TFLite software stack is designed specifically for mobile development. TensorFlow Lite “Micro”, on the other hand, is a version specifically for Microcontrollers, which recently merged with ARM’s uTensor.

Some developers might now be asking what the difference between ML Kit and TensorFlow Lite is. While there is definitely some overlap, TensorFlow Lite is more low level and open. More importantly: TensorFlow Lite runs off of the device itself, whereas ML Kit requires a Firebase registration and an active internet connection. Despite Google’s confusing nomenclature, note that ML Kit still uses TensorFlow “under the hood.” Firebase likewise is just another type of Google Cloud Platform project.

See also: Build a face-detecting app with machine learning and Firebase ML Kit

TensorFlow Lite is available on Android and iOS via a C++ API and a Java wrapper for Android developers. On devices that support it, the library can also take advantage of the Android Neural Networks API for hardware acceleration.

Which should you use for your projects? That depends very much on your objective. If you don’t mind relying on an external cloud service, ML Kit might make your life a little easier. If you want the code to run natively, or if you require a little more customization and flexibility, go for TensorFlow Lite.

How to use TensorFlow Lite

When solving a problem with machine learning, developers rely on “models.” ML models are files that contain statistical models. These files are trained to recognize specific patterns. Training essentially means feeding the model with data samples so that it can improve its success rate by refining the patterns it uses.

See also: ML Kit Image Labelling: Determine an image’s content with machine learning

So, a computer vision model might start off with a few basic assumptions about what an object looks like. As you show it more and more images, it will become increasingly precise while also broadening the scope of what it is looking for.

You will come across “pre-trained models” that have already been fed all of this data in order to refine their algorithms. This type of model is, therefore, “ready to go”. It can automatically perform a task such as identifying emotions based on facial expressions or moving a robot arm through space.

In TensorFlow Lite, these files are called “TensorFlow Lite Model Files” and have the extension “.tflite” or “.lite”. Labels files include the labels that the file is trained for (e.g.. “happy” or “sad” for facial recognition models.)

Training ML models

You may also encounter some other types of files that are used in the training process. GraphDef Files (.pb or .pbtxt) describe your graph and can be read by other processes. The TXT version is also designed to be human-readable. You can build these using TensorFlow too.

The Checkpoint File shows you the learning process by listing serialized variables – letting you see how the values change over time. The Frozen Graph Def then converts these values into constants and reads them from set checkpoints via the graph. The TFlite model is then built from the frozen graph using the TOCO (Tensor Flow Optimizing Converter Tool). This gives us a nice “pre-trained” file that we can then implement in our apps.

Discussing how to train and import models is beyond the scope of this post, though you can find a great tutorial here.

The good news is that the TensorFlow Task Library contains many powerful and simple libraries that rely on pre-trained models. These can handle all kinds of common tasks, such as responding to questions, recognizing faces, and more. This means those starting out don’t have to worry about Checkpoint Files or training!

Using TFLite Files

There are plenty of ways you can get hold of pre-trained TensorFlow Lite Model Files for your app. I recommend starting with the official TensorFlow site.

Follow this link, for example, and you will be able to download a starter model capable of basic image classification. The page also includes some details of how to use it via the TensorFlow Lite Task Library. You could alternatively use the TensorFlow Lite Support Library if you want to add your own inference pipeline (i.e. look for new stuff).

Once you have downloaded the file, you will place it into your assets directory. You must specify that the file should not be compressed. To do this, you add the following to your module build.gradle:

Setting up your Android Studio project

In order to utilize TensorFlow Lite in your app, you will need to add the following dependency to your build.gradle file:

Next, you need to import your interpreter. This is the code that will actually load the model and let you run it.

Inside your Java file, you will then create an instance of the Interpreter and use it to analyze the data you need. For example, you can input images and this will return results.

Results will be provided in the form of output probabilities. Models can never state with certainty what an object is. Thus, a picture of a cat might be 0.75 dog and 0.25 cat. Your code needs to

Alternatively, import the TensorFlow Support Library and convert the image into the tensor format.

These pre-trained models are capable of recognizing thousands of classes of images. However, there exist many different model “architectures” that alter the way the model defines the “layers” involved in the learning cycle, as well as the steps carried out to transform raw data into training data.

Popular model architectures include the likes of MobileNet and Inception. Your task is to choose the optimal solution for the job. For example, MobileNet is designed to favor lite and fast models over deep and complex ones. Complex models have higher accuracy but at the cost of size and speed.

Learning more

While this is a complex topic for beginners, I hope that this post has given you an idea of the basics, so that you can better understand future tutorials. The best way to learn any new skill is to choose a project and then learn the necessary steps to complete that task.

For a more in-depth understanding, we highly recommend Machine Learning With TensorFlow. This course includes 19 lessons that will show you how to implement common commercial solutions. Android Authority readers get a 91% discount right now, bringing the price down to $10 from $124.