Voice Interaction changelog

3 releases since 26 Mar 2026 · latest 1.17.0

1.17.0Latest
9 new14 fixed
12 May47 views

Before you update

Migration steps

You must update onex-base for this release.

This update requires a minimum of onex-base v1.37.0 to work properly, and it resolves the issue where inventory was incorrectly flagged as full when it wasn't.

Config file

Check server/npc_config.lua

NPC configuration server logic updated for ElevenLabs voice caching and language routing. Copy your settings across instead of overwriting the file.

Config file

Check shared/config.lua

Main shared config updated with Config.VoipRelay toggle, language validation, and voice_llm_refine convar. Copy your settings across instead of overwriting the file.

VOIP Relay Control

A new config option lets you disable or enable the VOIP relay. If you're already supplying your own API key in the config, you can now disable the VOIP relay — this allows users to share the same key for both voice/VOIP and voice interaction.

Heads up: Disabling the VOIP relay can increase your API consumption by up to 80x. Every time someone speaks in the server, it counts as a request against your voice interaction API quota.

New9

ElevenLabs voice support

is now available for NPCs. Each NPC can be assigned a voice from an accent pool, and a new in-game UI editor lets you configure and preview ElevenLabs voices without editing files manually. Synthesized audio is cached and served via direct HTTP for fast playback.

Added a `voice_llm_refine` convar that enables LLM-assisted rewriting of speech-to-text transcripts before they are processed.

This helps clean up misheard words and improve accuracy for voice commands.

Dynamic LLM phrases

are now routed through the free Onex Cloud TTS service so NPCs can speak generated responses without requiring a paid TTS provider.

NPC Manager

now supports ElevenLabs voice configuration and caching, allowing persistent voice assignments across server restarts.

NPC Lift

(buses and taxis) now auto-accepts rides and auto-charges fares, removing the need for players to manually confirm lift interactions.

NPC Robbery

now blacklists animals and shop clerks from robbery targeting, preventing unintended interactions with those ped types.

Added a world-space debug overlay for the nearest NPC in the NPC debug panel, showing provider name and flagging voice mismatches to help server owners diagnose voice configuration issues.

Added a `Config.VoipRelay` toggle for servers supplying their own VOIP relay key (BYOK).

The relay configuration file is now exposed outside escrow so server owners can edit it directly. Debug logging for relay start, stop, and same-key detection has also been added.

Localization expanded

across the NPC Manager panel, VOIP same-key cancel hint, and STT recording-limit hint. Native translations have been added or updated for Arabic, German, Spanish, French, Hindi, Italian, Japanese, Korean, Dutch, Polish, Portuguese, Russian, Swedish, Thai, Turkish, Vietnamese, and Chinese.

Fixed14

  • Shop carry checks no longer incorrectly reject items as INVENTORY_FULL when the item is not recognized by ox_inventory. Unknown items are now treated as carriable so players are not falsely blocked from purchases.
  • Fixed a cold-join race condition with TTS where the provider state was not ready when a player first joined, causing TTS to fail silently on connection.
  • ElevenLabs synthesis now correctly sends the target language for translation when the server language is set to a non-English locale, ensuring subtitles and audio match the configured language.
  • Fixed the TTS audio cache becoming stale after updates. The cache has been renamed internally, so outdated cached audio is automatically discarded and regenerated.
  • Voice hints no longer attempt to set animals as voice targets, preventing errors and unintended NPC interactions when animals are nearby.
  • Ambient NPC crime reaction and hint translation now correctly skip animals during target scanning, preventing errors when animals are in range.
  • Fixed ambient NPC voice hints not translating correctly — hints are now translated via the native script pipeline as intended.
  • Fixed a missing backendUrl in the voice hints payload that could cause hint requests to fail when a custom backend is configured.
  • VOIP relay now correctly stops when a recording is cancelled, and the relay start is non-blocking so it no longer holds up other voice processing.
  • Fixed the NPC Manager displaying the wrong voice name (Edge TTS default) when an ElevenLabs voice was assigned — the correct ElevenLabs voice name is now shown.
  • Fixed the NPC debug panel hooks to use the VoiceHints system for target management, aligning debug behavior with the rest of the voice pipeline.
  • If an invalid or mistyped language code is set in config, the script now validates it and automatically falls back to English rather than breaking voice features.
  • Fixed the translation database upgrade path — the DB version has been bumped to 7 and variable restoration now handles a wider range of bracket formats, preventing data loss on upgrade.
  • Fixed the UI page loading from an incorrect path, which could prevent the voice interaction interface from displaying properly.

Files changed45

fxmanifest.lua
addons/ambient-npc/modules/4 files
  • crime_reaction.lua
  • feedback.lua
  • llm_conversation.lua
  • ped_manager.lua
addons/npc-lift/modules/2 files
  • ride_controller.lua
  • vehicle_manager.lua
addons/npc-robbery/modules/2 files
  • weapon_detector.lua
  • witness_handler.lua
addons/shops/1 file
  • server.lua
client/1 file
  • main.lua
client/api/3 files
  • npc.lua
  • speech.lua
  • tts_state.lua
client/modules/3 files
  • npc_debug.lua
  • npc_manager.lua
  • voice_hints.lua
data/1 file
  • npc_voices_elevenlabs.json
locale/18 files
  • en.json
  • ar.json
  • de.json
  • es.json
  • fr.json
  • hi.json
  • it.json
  • ja.json
  • ko.json
  • nl.json
  • pl.json
  • pt.json
  • ru.json
  • sv.json
  • th.json
  • tr.json
  • vi.json
  • zh.json
nui/whole folder
server/2 files
  • broker.lua
  • npc_config.luaHas your settings
shared/5 files
  • config.luaHas your settings
  • ambient-npc/default.lua
  • npc-lift/default.lua
  • npc-robbery/default.lua
  • shops/default.lua
utils/1 file
  • voip_relay.lua

1.10.0

7 Apr 202678 views
23 new18 fixed4 improved2 breaking51 files

Updated language list to 99+ languages.

Afrikaans, Amharic, Arabic, Assamese, Azerbaijani, Bashkir, Belarusian, Bengali, Tibetan, Breton, Bosnian, Catalan, Czech, Welsh, Danish, German, Greek, English, Spanish, Estonian, Basque, Persian, Finnish, Faroese, French, Galician, Gujarati, Hausa, Hawaiian, Hebrew, Hindi, Croatian, Hungarian, Armenian, Indonesian, Icelandic, Italian, Japanese, Javanese, Georgian, Kazakh, Khmer, Kannada, Korean, Latin, Luxembourgish, Lao, Lithuanian, Latvian, Malagasy, Maori, Macedonian, Malayalam, Mongolian, Marathi, Malay, Maltese, Burmese, Nepali, Dutch, Nynorsk, Norwegian, Occitan, Punjabi, Polish, Pashto, Portuguese, Romanian, Russian, Sanskrit, Sindhi, Sinhala, Slovak, Slovenian, Shona, Somali, Albanian, Serbian, Sundanese, Swedish, Swahili, Tamil, Telugu, Tajik, Thai, Turkmen, Tagalog, Turkish, Tatar, Ukrainian, Urdu, Uzbek, Vietnamese, Yiddish, Yoruba, Cantonese, Chinese, Zulu

Before you update

Breaking change

Config.NetworkAudio has been renamed to Config.NetworkSync — update your config if you had custom values

Breaking change

groq_api_key convar is deprecated — the Groq STT key is now automatically reused for LLM.

Remove the separate convar if set

Config file

Check shared/config.lua

Updated with new NPC voice modes, VOIP guard/relay, NetworkSync, BYOK providers, NumberToWords, VoiceSettings keybind, language auto-detection, and Translate table format Copy your settings across instead of overwriting the file.

Config file

Check shared/drug-sell/config.lua

Updated config Copy your settings across instead of overwriting the file.

New23

Added Azure Cognitive Services as a new BYOK text-to-speech provider

Added ElevenLabs as a BYOK TTS provider with 41 free-tier voices, gender-aware NPC voice fallback, and voice tier display

Added NPC voice configuration system with 3 modes: preconfigured, LLM-origin, and LLM-accent — includes 538+ unique NPC voice assignments

Redesigned voice settings panel

with collapsible sections, sticky test bar, ElevenLabs-style filter chips (search, accent, age, category, language), list layout, and instant voice previews

Added Azure and Google as BYOK speech-to-text providers with multi-provider fallback support

3D DUI voice hints now translate to the configured language automatically

Added VOIP Guard — voice interaction and VOIP push-to-talk can no longer share the same keybind, preventing unnecessary API calls.

Default voice interaction key moved from N to G. VOIP relay ensures nearby players can still hear you while using voice interaction, regardless of your keybind choice.

Added VOIP Relay — relays proximity voice audio during voice interactions so nearby players can still hear you

Ambient NPC crime reactions

nearby NPCs react to crimes (flee, call police, etc.)

Ambient NPCs can now follow the player or lead them to locations

Dynamic AI conversation with taxi/rideshare drivers

during rides (NPC Lift LLM Chat)

Added qs-banking integration for the banking addon

Added processing state spinner — visual feedback while audio is being processed after recording stops

NPCs being used by another player now show a lock icon with a shake animation

Added NPC ownership system to prevent audio conflicts when multiple players interact with the same NPC

Dual-bubble TTS

support — separate speech bubbles for player and NPC

Added unified currency configuration (shared/currency.lua) for all addons

Voice settings panel keybind is now configurable via Config.VoiceSettings

Added callback function support to the onDispatch config option

Added startNPCInteraction / stopNPCInteraction convenience exports for developers

NPC busy state

is now handled automatically with built-in queuing and retry

NPC config pending state now includes proactive voice warmup so TTS is ready faster

Keybind input now rejects invalid keys with a red shake animation and blinks while waiting for input

Improved4

  • Deprecated separate groq_api_key convar — Groq STT key is now reused for LLM
  • Applied IntegrationRegistry pattern to banking, hospital, and shops addons for cleaner integration management
  • Renamed NetworkAudio config section to NetworkSync
  • Translation system now auto-detects when translation is needed and wires up dual matching automatically

Fixed18

  • Fixed microphone selection — falls back to the first available device when a saved microphone ID is stale or disconnected
  • Fixed VOIP relay not stopping when the voice interaction key is released
  • Fixed processing animation not clearing immediately when a short recording is cancelled
  • Unified TTS animation timing with the progress bar and added an animation refresh loop to prevent stale states
  • Fixed default voice being cached before NPC config has finished loading
  • ElevenLabs and Azure TTS now correctly use English locale internally and let the backend handle script translation
  • TTS cancellation is now broadcast to nearby players so remote audio stops correctly
  • TTS progress ring now displays correctly when the NPC is in locked (busy) mode
  • Ambient NPCs now preserve their scenarios and animations during voice interactions instead of resetting to idle
  • Ambient NPCs no longer flee or act scared while actively in conversation with a player
  • Voice settings keybind now persists correctly and long key names no longer overflow the UI
  • Multiple voice UI polish fixes: voice card alignment, preview toggle behavior, and NPC manager access restriction removed
  • Fixed ox_inventory shop detection failing when shops.lua contains leading annotations
  • Fixed NPC communication issues — corrected RPC call structure and response handling
  • Voice settings keybind now syncs from config each time the panel opens
  • Improved startup connection banner with clearer status messages and retry feedback
  • TTS responseLanguage is now only sent when it differs from the primary language, reducing unnecessary API calls
  • Improved shop ped spawning with duplicate prevention, ground snapping, and coordinate override support

Files changed51

addons/banking/integrations/7 files
  • _base.lua
  • _template.lua
  • default.lua
  • esx-banking.lua
  • qb-banking.lua
  • qs-banking.lua
  • renewed-banking.lua
addons/banking/locale/1 file
  • en.lua
addons/hospital/2 files
  • client.lua
  • server.lua
addons/hospital/integrations/6 files
  • _base.lua
  • _template.lua
  • esx-ambulancejob.lua
  • qb-ambulancejob.lua
  • qb-medical.lua
  • standalone.lua
addons/hospital/locale/1 file
  • en.lua
addons/hospital/modules/1 file
  • feedback.lua
addons/shops/integrations/5 files
  • _base.lua
  • _template.lua
  • esx-shops.lua
  • ox-inventory.lua
  • qb-shops.lua
addons/shops/locale/9 files
  • ar.lua
  • de.lua
  • en.lua
  • es.lua
  • fr.lua
  • hi.lua
  • it.lua
  • pt-BR.lua
  • ru.lua
shared/2 files
  • config.luaHas your settings
  • currency.lua
shared/ambient-npc/4 files
  • default.lua
  • locations.lua
  • phrases.lua
  • responses.lua
shared/banking/2 files
  • default.lua
  • responses.lua
shared/drug-sell/2 files
  • config.luaHas your settings
  • haggling.lua
shared/hospital/2 files
  • default.lua
  • responses.lua
shared/npc-lift/2 files
  • default.lua
  • responses.lua
shared/npc-robbery/1 file
  • default.lua
shared/shops/3 files
  • default.lua
  • locations.lua
  • responses.lua
utils/1 file
  • commands.lua

1.2.0

26 Mar 202625 views
9 new7 fixed5 improved47 files

Before you update

Config file

Check server/npc_config.lua

Copy your settings across instead of overwriting the file.

New9

  • Shop NPC Interactions — Talk to shop NPCs using your voice to browse and buy items. Supports qb-shops, ox_inventory, and esx_shops out of the box
  • ESX Banking Integration — Added esx_banking support for the banking addon, so ESX servers can use voice banking
  • ESX Shops Integration — Added esx_shops support for the shops addon
  • Custom Integration Templates — Added _template.lua files for shops, banking, and hospital addons so server owners can easily add support for any custom resource
  • Voice Hint Customization — New setScale and setOffset APIs let addons control the size and position of 3D voice hint bubbles above NPCs
  • NPC-Lift: Free Exit — You can now leave a ride vehicle anytime. Removed the global cooldown that blocked requesting rides from other NPCs
  • NPC-Lift: 2D Hints in Vehicle — Voice hints switch from 3D bubble to a 2D screen overlay while inside the vehicle for better visibility
  • Dependency Check — The resource now requires onex-base v1.33.0+ and will show a clear error on startup if the dependency is missing or outdated
  • Shop Ped Spawning — For shop systems that don't spawn their own NPCs, the addon now spawns peds at configured locations automatically

Improved5

  • Shop Integration System — Rebuilt how shop systems are detected and loaded, making it easier to add new shop resources
  • Hospital & Banking Addons — Simplified to use the same integration pattern as shops for consistency
  • NPC-Lift Vehicle Manager — Simplified ride logic and payment handling
  • Shared Ped Utilities — Extracted common NPC utilities used across addons into a shared module
  • Dead Code Cleanup — Removed unused code across client, server, addons, and web UI

Fixed7

  • Fixed translation not using the configured target language — it was always defaulting to English
  • Fixed NPC driver driving off early when the player exits — handbrake is now enforced during exit
  • Fixed qb-banking ped using wrong behind-counter coordinates
  • Fixed voice hints sometimes appearing when too far from the NPC
  • Fixed ox_inventory shop locations not matching correctly when peds are nearby
  • Fixed ox_inventory shops.lua file not parsing correctly in some cases
  • Fixed duplicate ped spawning in the banking addon

Files changed47

addons/_shared/1 file
  • ped_utils.lua
addons/ambient-npc/1 file
  • client.lua
addons/banking/8 files
  • client.lua
  • modules/bank_manager.lua
  • integrations/_base.lua
  • integrations/_template.lua
  • integrations/default.lua
  • integrations/qb-banking.lua
  • integrations/renewed-banking.lua
  • integrations/esx-banking.lua
addons/hospital/9 files
  • client.lua
  • server.lua
  • modules/hospital_manager.lua
  • integrations/_base.lua
  • integrations/_template.lua
  • integrations/qb-medical.lua
  • integrations/qb-ambulancejob.lua
  • integrations/esx-ambulancejob.lua
  • integrations/standalone.lua
addons/npc-interact/1 file
  • client.lua
addons/npc-lift/4 files
  • client.lua
  • server.lua
  • modules/ride_controller.lua
  • modules/vehicle_manager.lua
addons/npc-robbery/2 files
  • client.lua
  • server.lua
addons/self-expression/1 file
  • client.lua
addons/shops/8 files
  • client.lua
  • server.lua
  • modules/shop_manager.lua
  • integrations/_base.lua
  • integrations/_template.lua
  • integrations/qb-shops.lua
  • integrations/ox-inventory.lua
  • integrations/esx-shops.lua
client/3 files
  • main.lua
  • modules/voice_hints.lua
  • modules/animation.lua
nui/**/whole folder
server/2 files
  • broker.lua
  • npc_config.luaHas your settings
shared/5 files
  • shops/default.lua
  • shops/locations.lua
  • banking/default.lua
  • hospital/default.lua
  • npc-lift/default.lua