A study compares functional token and schema-in-prompt methods for on‑device vehicle assistants, showing schema prompts generalize to unseen functions despite higher latency.
Researchers benchmarked 9,822 single‑turn requests across 79 Android Automotive functions using four small language models (270M‑1.7B parameters). They found that schema‑in‑prompt (SIP) generalizes to unseen functions, while functional tokens (FT) cannot, and that model size offers limited gains for seen functions.
Implement SIP to support unseen functions, accepting higher memory and latency costs.
Add schema definitions to prompts when extending the vehicle API without updating the model.
Tags
Sources
Also today