Running machine learning models completely offline on Android gives you zero latency, zero API costs, and full user privacy. Google's LiteRT (com.google.ai.edge.litert, the evolution of TensorFlow Lite) makes it straightforward to run .tflite models directly on-device.
In this tutorial, we'll walk through how NutriLens — a Jetpack Compose + CameraX food & nutrition tracker — uses LiteRT to identify dishes from a live camera feed and estimate calories, macros, and micronutrients 100% offline.
What We Are Building
The NutriLens scanning pipeline works like this:
CameraX ImageAnalysis (RGBA_8888) │ ▼ Rotate upright & throttle (~850ms) │ ▼ Center-crop & scale Bitmap (192×192) │ ▼ LiteRT Inference (CompiledModel / Interpreter) │ ▼ Dequantize UINT8 Softmax → Top-3 Food Labels │ ▼ Temporal Smoothing + Nutrition Estimator → Compose UI Step 1: Add LiteRT and CameraX Dependencies
In gradle/libs.versions.toml:
[versions] camerax = "1.4.2" litert = "2.1.0" [libraries] androidx-camera-core = { group = "androidx.camera", name = "camera-core", version.ref = "camerax" } androidx-camera-camera2 = { group = "androidx.camera", name = "camera-camera2", version.ref = "camerax" } androidx-camera-lifecycle = { group = "androidx.camera", name = "camera-lifecycle", version.ref = "camerax" } androidx-camera-view = { group = "androidx.camera", name = "camera-view", version.ref = "camerax" } google-ai-edge-litert = { group = "com.google.ai.edge.litert", name = "litert", version.ref = "litert" } In app/build.gradle.kts, add the dependencies and disable compression for .tflite assets so Android can memory-map (mmap) the model file directly without decompressing it into RAM:
android { // ... androidResources { noCompress += "tflite" } } dependencies { implementation(libs.androidx.camera.core) implementation(libs.androidx.camera.camera2) implementation(libs.androidx.camera.lifecycle) implementation(libs.androidx.camera.view) implementation(libs.google.ai.edge.litert) } Tip: Never skip
noCompress += "tflite". If Gradle compresses your.tfliteasset inside the APK,assets.openFd(...)will fail and prevent zero-copy memory mapping.
Step 2: Bundle Your Offline .tflite Model and Labels
Place your model and label files inside app/src/main/assets/:
app/src/main/assets/ ├── vision_classifier_food.tflite # aiy/vision/classifier/food_V1 (2,024 classes) └── food_labels.txt # 2,024 class labels matching output indices Before writing Kotlin code, inspect your model's input and output tensor specs:
- Input:
[1, 192, 192, 3](UINT8RGB pixels0..255) - Output:
[1, 2024](UINT8quantized softmax probabilities)
Knowing whether your model uses UINT8 or FLOAT32 is critical — feeding FLOAT32 normalized data to a UINT8 model (or forgetting to dequantize UINT8 outputs) is one of the most common reasons an offline model outputs garbage.
Step 3: Initialize LiteRT (CompiledModel + Interpreter Fallback)
LiteRT 2.1.0 provides the modern CompiledModel + TensorBuffer API (com.google.ai.edge.litert.*) as well as a compatible Interpreter (org.tensorflow.lite.Interpreter bundled inside the litert artifact).
Older .tflite models (like aiy/vision/classifier/food_V1) sometimes lack SignatureDefs or use UINT8 quantization parameters (scale and zeroPoint). Initializing CompiledModel first and keeping Interpreter ready for tensor metadata & fallback gives you the best of both worlds:
class LiteRtFoodClassifier( private val context: Context ) { private var compiledModel: CompiledModel? = null private var inputBuffers: List<TensorBuffer>? = null private var outputBuffers: List<TensorBuffer>? = null private var interpreter: Interpreter? = null private var inputWidth = 192 private var inputHeight = 192 private var inputDataType = DataType.UINT8 private var outputClassCount = 2024 private var outputDataType = DataType.UINT8 private var outputScale = 1f / 255f private var outputZeroPoint = 0 private var labels: List<String> = emptyList() @Synchronized fun ensureInitialized(): Boolean { if ((compiledModel != null || interpreter != null) && labels.isNotEmpty()) { return true } labels = loadLabelsFromAssets() // 1. Modern LiteRT CompiledModel API val compiledReady = runCatching { val model = CompiledModel.create( context.assets, "vision_classifier_food.tflite", CompiledModel.Options(Accelerator.CPU) ) val inputs = model.createInputBuffers() val outputs = model.createOutputBuffers() if (inputs.isNotEmpty() && outputs.isNotEmpty()) { compiledModel = model inputBuffers = inputs outputBuffers = outputs true } else { model.close() false } }.getOrDefault(false) // 2. Inspect tensor shapes, DataTypes, and quantization params via Interpreter val interpreterReady = runCatching { val modelBuffer = loadMappedModelBuffer("vision_classifier_food.tflite") val tflite = Interpreter(modelBuffer, Interpreter.Options().apply { setNumThreads(4) }) val inTensor = tflite.getInputTensor(0) val inShape = inTensor.shape() // [1, 192, 192, 3] if (inShape.size >= 3) { inputHeight = inShape[1] inputWidth = inShape[2] } inputDataType = inTensor.dataType() val outTensor = tflite.getOutputTensor(0) outputClassCount = outTensor.shape().lastOrNull() ?: 2024 outputDataType = outTensor.dataType() val qParams = outTensor.quantizationParams() if (qParams.scale > 0f) { outputScale = qParams.scale outputZeroPoint = qParams.zeroPoint } interpreter = tflite true }.getOrDefault(false) return compiledReady || interpreterReady } } Memory-mapping the model from assets:
private fun loadMappedModelBuffer(assetName: String): ByteBuffer { return runCatching { context.assets.openFd(assetName).use { fd -> FileInputStream(fd.fileDescriptor).channel.use { channel -> channel.map( FileChannel.MapMode.READ_ONLY, fd.startOffset, fd.declaredLength ) } } }.getOrElse { val bytes = context.assets.open(assetName).use { it.readBytes() } ByteBuffer.allocateDirect(bytes.size).apply { order(ByteOrder.nativeOrder()) put(bytes) rewind() } } } Step 4: Preprocess Bitmaps & Run Inference
Before passing a camera Bitmap into LiteRT:
- Convert
Bitmap.Config.HARDWAREtoARGB_8888if needed (sincegetPixelscannot read hardware bitmaps). - Center-crop to a square and scale to
192×192. - Extract packed RGB bytes (
R,G,B) in row-major order.
private fun centerCropAndScale(source: Bitmap, targetWidth: Int, targetHeight: Int): Bitmap { val safeBitmap = if (source.config == Bitmap.Config.HARDWARE) { source.copy(Bitmap.Config.ARGB_8888, false) } else { source } val minDim = min(safeBitmap.width, safeBitmap.height) val cropX = (safeBitmap.width - minDim) / 2 val cropY = (safeBitmap.height - minDim) / 2 val cropped = Bitmap.createBitmap(safeBitmap, cropX, cropY, minDim, minDim) return if (cropped.width == targetWidth && cropped.height == targetHeight) { cropped } else { Bitmap.createScaledBitmap(cropped, targetWidth, targetHeight, true) } } When reading quantized UINT8 output bytes in Kotlin/Java, remember that Byte is signed (-128..127), so always mask with and 0xFF and apply the quantization formula:
val probability = ((rawByte.toInt() and 0xFF) - outputZeroPoint) * outputScale Running inference with CompiledModel + TensorBuffer:
val model = compiledModel val inputs = inputBuffers val outputs = outputBuffers if (model != null && inputs != null && outputs != null) { val probs = runCatching { val rgbBytes = bitmapToRgbByteArray(croppedBitmap, inputWidth, inputHeight) if (inputDataType == DataType.FLOAT32) { val floatIn = FloatArray(rgbBytes.size) { i -> ((rgbBytes[i].toInt() and 0xFF) / 127.5f) - 1.0f } inputs[0].writeFloat(floatIn) model.run(inputs, outputs) outputs[0].readFloat() } else { inputs[0].writeInt8(rgbBytes) model.run(inputs, outputs) if (outputDataType == DataType.FLOAT32) { outputs[0].readFloat() } else { val rawOut = outputs[0].readInt8() FloatArray(rawOut.size) { i -> (((rawOut[i].toInt() and 0xFF) - outputZeroPoint) * outputScale).coerceAtLeast(0f) } } } }.getOrNull() } Always release TensorBuffer, CompiledModel, and Interpreter resources when your ViewModel clears (onCleared()).
Step 5: Stream Live Frames from CameraX
In CameraPreviewView.kt, configure ImageAnalysis with OUTPUT_IMAGE_FORMAT_RGBA_8888 and STRATEGY_KEEP_ONLY_LATEST so slow frames never queue up. Don't forget to rotate the Bitmap using imageProxy.imageInfo.rotationDegrees — otherwise your model receives sideways images and accuracy drops sharply:
val imageAnalysis = ImageAnalysis.Builder() .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST) .setOutputImageFormat(ImageAnalysis.OUTPUT_IMAGE_FORMAT_RGBA_8888) .build() imageAnalysis.setAnalyzer(analysisExecutor) { imageProxy -> try { val rawBitmap = imageProxy.toBitmap() val rotation = imageProxy.imageInfo.rotationDegrees val uprightBitmap = if (rotation != 0) { val matrix = Matrix().apply { postRotate(rotation.toFloat()) } Bitmap.createBitmap( rawBitmap, 0, 0, rawBitmap.width, rawBitmap.height, matrix, true ) } else { rawBitmap } onFrameBitmap(uprightBitmap) } finally { imageProxy.close() } } Step 6: Stabilize Live Detections in the ViewModel
Running inference on every camera frame can cause flickering or stale UI states if you aren't careful. Here are three practical rules we applied in ScanFoodViewModel:
- Throttle & guard concurrency: Skip incoming frames if less than
850mshas passed or if anAtomicBoolean(isClassifyingFrame) is already active. Run inference onDispatchers.Default. - Use honest confidence thresholds: A 2,024-class softmax spreads probability across many labels, so real top-1 confidence on a good match is often
28%–65%, not90%+. Never artificially inflate confidence scores in code; instead calibrate your lock threshold (MIN_LIVE_CONFIDENCE_PERCENT = 28) to the model's real distribution. - Smooth transitions instead of hard-locking:
- First match (
>= 28%) updates the UI right away. - Switching to a different food requires 2 consecutive frames (
pendingCandidateStreak >= 2) so brief camera shake doesn't overwrite a good detection. - Pointing away from food (
< 18%for 3 consecutive frames) clears the previous item so the user can seamlessly scan a new plate.
- First match (
Step 7: Map Detected Labels to Offline Nutrition Data
Once LiteRT returns a food label (e.g., "Mango Sticky Rice", "Breakfast Burrito"), FoodNutritionEstimator maps the label offline to calories, protein/carbs/fat, serving weight, and micronutrients (fiber, sugar, sodium, potassium, calcium, iron, vitamin C, cholesterol). Users can also tap any macro card, adjust serving multipliers (0.5x, 1.0x, 1.5x, or custom grams), or switch between the top-3 predicted matches before logging to Room/SQLite.
Key Takeaways & Common Pitfalls
- Always inspect your
.tflitetensors: Check input size (192×192), channel order (RGB), and quantization (UINT8vsFLOAT32) before writing preprocessing code. - Avoid overriding fine-grained classifiers with coarse segmenters: If you combine a classifier with a semantic segmenter, let the whole-image classifier name cohesive dishes instead of forcing segment categories into hardcoded fallback names.
- Reuse direct
ByteBuffers for heavy outputs: Large segmentation tensors (like[1, 513, 513, 26] FLOAT32≈ 27 MB) should reuse a cached directByteBufferrather than allocating per frame. - Keep live scanning responsive: Use
STRATEGY_KEEP_ONLY_LATEST, throttle frames, and use short multi-frame streaks instead of permanently pausing the camera after the first detection.
With LiteRT + CameraX, you get a fast, privacy-first, zero-cost AI scanner that works anywhere — even in airplane mode.
Github: https://github.com/tusharahmed11/NutriLens