BoxyML/mobile-actions_q8_ekv1024.litertlm

LiteRT Model Card: Mobile Actions Q8 EKV1024

Model Overview

This repository contains a fine-tuned version of google/functiongemma-270m-it, optimized specifically for execution via the LiteRT framework on mobile and edge devices. It is specialized for mobile actions and function-calling capabilities.

Intended Use

Designed for low-latency, on-device mobile applications requiring function-calling, agent capabilities, and mobile-specific interaction models.

Format Details

• Framework/Format: LiteRT / LiteRT-LM format, tailored for efficient on-device sequence generation and deployment via LiteRT inference engines.

Quantisation Info

• Quantization Type: Q8 (8-bit integer quantization) to significantly reduce the model footprint and runtime memory consumption while maintaining high performance.

Embedding & Context Specifications

• Embedding KV Length: 1024 tokens, providing balanced context handling optimized for edge efficiency.

Usage Instructions

Load the .litertlm file into your mobile application utilizing the LiteRT-LM runtime or standard LiteRT interpretation layer equipped with custom operations support for sequence models.