High-Performance Offline Edge-AI Biodiversity Atlas (Flora, Fungi & Fauna) powered by Rust NDK, Kotlin, Jetpack Compose, and Google LiteRT.
FoliaLens adopts a dual-engine hybrid architecture: high-level reactive UI and hardware capture in Kotlin on Android, coupled with an ultra-optimized native Rust core (libatlas_core.so) for zero-copy memory management, spatial indexing, decompression, and DSP algorithms.
flowchart TB
subgraph AndroidLayer["Android Application Layer (Kotlin & Jetpack Compose)"]
CameraX["CameraX Stream (YUV_420_888, 30 FPS)"]
AudioRecord["AudioRecord Stream (48 kHz PCM Float)"]
ComposeUI["Jetpack Compose Material 3 UI"]
DirectBufPool["DirectByteBuffer Pool (Zero-Copy Native Pointers)"]
CameraX --> DirectBufPool
AudioRecord --> DirectBufPool
end
subgraph ZeroCopyBridge["Zero-Copy JNI Bridge (*mut u8 / *mut f32)"]
JNI_Video["Direct JNI: Video YUV / RGB Pointers"]
JNI_Audio["Direct JNI: Audio PCM Stream Pointers"]
end
subgraph RustCore["Rust NDK Core (libatlas_core.so)"]
VisionMod["vision.rs: Center-Crop, BT.601 YUV-to-RGB, fast_image_resize SIMD, ImageNet Norm"]
AudioMod["audio.rs: Hann Window (n_fft=2048, hop=512), rustfft STFT, 96-bin Mel Filterbank"]
GeoMod["geo.rs: lat_lon_to_h3_res4 (h3o Res 4), 12-bit Seasonality Bitmask Verification"]
DbMod["db.rs: Read-only SQLite, FTS5 Search, Zstd Decompression, AVIF Media Blobs"]
BayesianFilter["Bayesian Postprocessing: Softmax Logits + H3 Spatial Penalty (x0.15)"]
end
subgraph LiteRTLayer["On-Device AI Engine (Google LiteRT)"]
VisionModel["Vision CNN / Transformer (INT8 / FP16, NPU / GPU Delegate)"]
AudioModel["Audio Bird / Bioacoustic CNN (FP16, GPU Delegate)"]
end
subgraph UniFFIBridge["Type-Safe UniFFI 0.28 Bridge (Auto-Generated)"]
UniFFI_API["AtlasEngine API (TaxonSummary, TaxonDetail, searchTaxa, getTaxonDetail)"]
end
%% Data flow connections
DirectBufPool --> JNI_Video --> VisionMod
DirectBufPool --> JNI_Audio --> AudioMod
VisionMod --> VisionModel
AudioMod --> AudioModel
VisionModel --> BayesianFilter
AudioModel --> BayesianFilter
GeoMod --> BayesianFilter
DbMod --> BayesianFilter
BayesianFilter --> UniFFI_API
DbMod --> UniFFI_API
UniFFI_API --> ComposeUI
- Zero-Copy Direct JNI: Raw high-frequency frame buffers (
ImageProxyplanes) and microphone audio chunks are mapped directly into native heap buffers (DirectByteBuffer). Rust reads native memory pointers (*mut u8,*const f32) without intermediate JVM allocations, preventing garbage collection stalls. - Type-Safe UniFFI 0.28: Complex domain structures (
TaxonSummary,TaxonDetail,Kingdom,SafetyStatus) and querying methods are generated via Mozilla UniFFI, guaranteeing safe memory ownership, exception propagation, and idiomatic Kotlin interfaces.
The entire real-time classification pipeline executes strictly under 30 ms, maintaining a smooth 30+ FPS viewfinder experience:
| Stage | Operation | Engine / Library | Latency | Memory Allocation |
|---|---|---|---|---|
| Stage 1 | Frame / Audio Capture into DirectByteBuffer |
Android CameraX / AudioRecord | < 0.2 ms |
0 B (Zero GC) |
| Stage 2 | DSP Preprocessing (BT.601, SIMD Resize, STFT) | Rust NDK (fast_image_resize, rustfft) |
~3.5 ms |
In-place pre-allocated |
| Stage 3 | Neural Inference (Vision INT8 / Audio FP16) | Google LiteRT (GPU / NPU Delegate) | 15.0 β 20.0 ms |
TensorBuffer reuse |
| Stage 4 | Spatial Prior Filtering, FTS5 Match & Zstd Decompression | Rust NDK (h3o, rusqlite, zstd) |
~1.0 ms |
Stack / Arena alloc |
| Total | Full End-to-End Real-Time Identification Loop | Hybrid Pipeline | ~20 β 25 ms | Zero GC pressure |
Note
Spatial Prior Penalty: If a taxon predicted by the neural network does not occur within the user's current Uber H3 hexagon (Resolution 4, ~11,000 kmΒ²) or is out of season according to the 12-bit monthly phenology mask, its probability is multiplied by a 0.15 penalty factor, effectively eliminating geographic hallucinations without discarding rare observations.
The offline encyclopedia is stored in an optimized, read-only SQLite database (atlas.db) designed for minimal storage footprint and sub-millisecond query performance:
-- Core taxonomic index (Clustered by taxon_id via WITHOUT ROWID)
CREATE TABLE taxa (
taxon_id INTEGER PRIMARY KEY,
scientific_name TEXT NOT NULL,
common_name_ru TEXT NOT NULL,
kingdom INTEGER NOT NULL,
safety_status INTEGER NOT NULL,
description_zstd BLOB NOT NULL
) WITHOUT ROWID;
-- SQLite FTS5 Full-Text Search with external content table
CREATE VIRTUAL TABLE taxa_fts USING fts5(
scientific_name,
common_name_ru,
synonyms,
content='taxa',
content_rowid='taxon_id',
tokenize='unicode61 remove_diacritics 2'
);
-- Spatiotemporal occurrence grid (Uber H3 Index Resolution 4)
CREATE TABLE geo_presence (
h3_index INTEGER NOT NULL,
taxon_id INTEGER NOT NULL,
months_bitmask INTEGER NOT NULL,
PRIMARY KEY (h3_index, taxon_id)
) WITHOUT ROWID;
-- Compressed offline iconography and species reference images
CREATE TABLE media_blobs (
media_id INTEGER PRIMARY KEY,
taxon_id INTEGER NOT NULL,
format TEXT NOT NULL,
data BLOB NOT NULL
) WITHOUT ROWID;-
WITHOUT ROWIDTables: Eliminates duplicate 64-bit integer rowid b-trees, reducing index storage overhead and caching whole rows directly within leaf index pages. -
Zstandard Dictionary Compression: Encyclopedic descriptions are compressed using
zstdlevel 3 with a domain-specific botanical training dictionary, shrinking text storage by >60% while decompressing in under0.5 ms. -
FTS5 Trigram & Prefix Matching: Supports instant search across scientific binomials and Cyrillic vernacular names with fault-tolerant prefix matching (
query*). -
Discrete Global Grid System (Uber H3): H3 cell identifiers are stored as compact 64-bit integers (
i64), enabling$O(1)$ spatial index lookups without bulky geospatial extensions like SpatiaLite.
bio-atlas/
βββ app/ # Android Application Module
β βββ build.gradle.kts # AGP 9 build configuration with cargo-ndk & uniffi
β βββ src/main/
β βββ java/com/example/folialens/
β β βββ MainActivity.kt # Compose UI (FTS5 search, inference sim, detail card)
β β βββ core/
β β βββ NativeBridge.kt # Zero-copy direct JNI method declarations
β β βββ uniffi/ # Generated UniFFI Kotlin bindings (atlas_core.kt)
β βββ jniLibs/ # Pre-compiled native libraries
β βββ arm64-v8a/libatlas_core.so
β βββ x86_64/libatlas_core.so
βββ rust-core/ # Native Rust Engine (atlas_core)
β βββ Cargo.toml # Dependencies: uniffi, rusqlite (fts5, bundled), zstd, h3o
β βββ uniffi.toml # UniFFI package bindings configuration
β βββ src/
β βββ lib.rs # UniFFI exports & AtlasEngine API facade
β βββ db.rs # SQLite + FTS5 + Zstd storage & seed dataset
β βββ geo.rs # Uber H3 (h3o) spatial indexing & seasonal bitmasks
β βββ vision.rs # SIMD image resizing & tensor pre-processing (TBD)
β βββ audio.rs # RustFFT audio spectrogram pipeline (TBD)
β βββ bin/
β βββ uniffi-bindgen.rs # CLI binding generator tool
βββ ARCHITECTURE.md # Comprehensive system architecture & specs
βββ LICENSE # MIT License
βββ README.md # Open-source technical documentation
- Android Studio (Ladybug Feature Drop or newer) with Android SDK 37 and NDK (r27+).
- Rust Toolchain (1.80+):
rustup default stable
- Android Rust Targets:
rustup target add aarch64-linux-android x86_64-linux-android
- Cargo NDK:
cargo install cargo-ndk
From the repository root directory:
# Assemble Debug APK (automatically builds Rust native libraries and UniFFI bindings)
./gradlew :app:assembleDebugOn Windows (PowerShell / Command Prompt):
.\gradlew.bat :app:assembleDebugThe resulting debug APK is created at:
app/build/outputs/apk/debug/app-debug.apk
# Deploy to a connected physical device (via USB) or Android Emulator
./gradlew :app:installDebugThis project is open-source under the terms of the MIT License.
Copyright (c) 2026 Yuri Egorov.