SDKs

React Native

Voice conversations with agents in a React Native app on iOS, Android and the web — the client, the hooks, and push-to-talk, half-duplex and hands-free audio.

@lumavox/react-native is one install for a React Native app, on iOS, Android and the web: the client, the hooks, and the audio they lack — a player for the agent’s voice and a recorder for the person’s, for push-to-talk, half-duplex — the microphone left open for the whole conversation — and hands-free, talking over the agent to stop it (on a phone in early access). On a phone the audio runs on react-native-audio-api, installed beside it because React Native links only the native code your app lists itself; in a browser, on Web Audio.

npm install @lumavox/react-native react-native-audio-api

React Native 0.76 or later, on the New Architecture, with react-native-audio-api 0.13 (0.13.6 or later) — or, for hands-free on a phone, its early-access build (below).
iOS 17 and Android 12 (API 31) or later: supported and tested; older phones are not.
Expo needs a development build (npx expo run:ios, npx expo run:android), never Expo Go: the audio is native code.

Expo

List the plugin in app.json and build again:

{
  "expo": {
    "plugins": [["@lumavox/react-native", { "microphonePermission": "Hold to talk to your coach." }]]
  }
}

Never list react-native-audio-api’s own plugin as well. This one runs it with every option set: background audio off (it is on by default there), and Android’s RECORD_AUDIO and MODIFY_AUDIO_SETTINGS permissions listed (its own list lacks the first). That plugin runs once, so a second listing’s options would be dropped without a word; listed before this one, the build stops and says so.

Option
`microphonePermission`	what iOS shows when the app first asks for the microphone — your app’s words
`androidPermissions`	more Android permissions, added to the two above
`disableFFmpeg`	`true` leaves out FFmpeg, which decodes compressed audio files — a conversation’s raw audio never needs it, and the app is smaller
`disableStaticExternalLibs`	`react-native-audio-api`’s own switch, passed through

Bare React Native

After the install, cd ios && pod install. Then, by hand, what the plugin would do. iOS’s Info.plist says why the microphone is asked for — and has no audio background mode:

<key>NSMicrophoneUsageDescription</key>
<string>Hold to talk to your coach.</string>

Android’s AndroidManifest.xml asks for the microphone, and for the audio settings a call-like route needs:

<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />

Push-to-talk

Make the client once, put it in a provider, and give useAgent the audio. Pressing starts a recording — the agent’s voice stops, so it never reaches the microphone — and releasing sends it as the person’s turn:

import { AgentProvider, createAudio, createClient, useAgent } from "@lumavox/react-native";
import { useEffect, useMemo } from "react";
import { Pressable, Text } from "react-native";

const agents = createClient({
  baseUrl: "https://api.lumavox.ai",
  // Your server mints the key with your API key: POST /v1/ephemeral-keys.
  getToken: async () => (await fetch(keyServer, { method: "POST" })).text(),
});

export default function App() {
  return (
    <AgentProvider client={agents}>
      <Talk agentId="agt_..." />
    </AgentProvider>
  );
}

function Talk({ agentId }: { agentId: string }) {
  const audio = useMemo(() => createAudio(), []);
  useEffect(() => () => audio.dispose(), [audio]);
  const agent = useAgent({ agentId }, { audio });
  return (
    <Pressable
      onPressIn={() => {
        void audio.prime(); // in the press, before any await: the web needs it
        void agent.startRecording();
      }}
      onPressOut={() => void agent.stopRecording()}
    >
      <Text>{agent.recording ? "Listening…" : agent.speaking ? "Speaking…" : "Hold to talk"}</Text>
    </Pressable>
  );
}

The audio is yours: make it once per screen and dispose of it when the screen goes — useAgent never does. Everything else is the hooks’: turn, speaking, recording, cancel(), stopRecording({ discard: true }) for a press that slid off the button, stopRecording({ transcribeOnly: true }) for a dictation’s words with no turn.

Phones record at 16 kHz, mono, 100 ms a buffer; a phone that records at another rate is resampled to it. A browser records at 24 kHz.
The agent plays from the loudspeaker. Headphones and Bluetooth take over when connected. createAudio({ output: "earpiece" }), or audio.setOutput("earpiece"), plays it at the ear like a call — on iPhones; Android stays on the loudspeaker for now.
Bluetooth earbuds’ microphone is used on iPhones, like a call — and like a call, the agent sounds at call quality while they are on. Android records from the phone for now.
The microphone permission is asked on the first press. To ask earlier, at a moment your app explains, call requestMicrophone() — it answers whether it was given.

Interruptions

A phone call, Siri or another app’s audio pauses the conversation: the reply stops where it is and goes on when the audio comes back, its words still arriving meanwhile; a recording in progress is thrown away, since its speaker was cut off mid-sentence. Leaving the screen does the same: there is no background audio in this release, so the conversation pauses with the app and goes on when it is back.

const agent = useAgent({ agentId }, {
  audio,
  onAudioInterrupted: () => setNote("Paused for your call"),
  onAudioResumed: () => setNote(null),
});
// agent.interrupted: true while the phone has the audio

The audio comes back by itself when the system says it may. When it does not — another app went on playing — agent.interrupted stays true until the person presses to talk again, or you call audio.resume(), which rejects while the audio is still taken. On Android, a conversation on screen asks for the audio focus, so other apps’ music pauses.

The clock: a face, subtitles

The audio has a clock to draw against — a 3D character’s mouth, subtitles revealed chunk by chunk — on a phone and on the web alike. onScheduled tells every frame of the reply as it is queued, before it plays: its 16-bit samples (pcm, sample_rate), its chunk and turn, and startsAt and duration in seconds on the clock now() reads. Draw against heardTime(), what the person hears at this moment:

import { createAudio } from "@lumavox/react-native";
import { useEffect, useMemo } from "react";

function Character() {
  // 350 ms for your model to read each frame before it plays.
  const audio = useMemo(() => createAudio({ leadInMs: 350, outputLatencyMs: 0 }), []);
  useEffect(() => () => audio.dispose(), [audio]);
  useEffect(() => audio.onScheduled(({ pcm, sample_rate, startsAt }) => lipsync.add(pcm, sample_rate, startsAt)), [audio]);
  useEffect(() => audio.onStopped(() => lipsync.clear()), [audio]); // a cancel, a barge-in: never played
  // Each frame the face is drawn: lipsync.at(audio.heardTime())
  return null;
}

A reply’s first frame waits leadInMs — and again after any gap in the voice — so your model has that long to read it; frames after it follow back to back. It adds as much to the first word heard. None on a phone unless set (a buffer of silence played first), 60 ms on the web.
outputLatencyMs: what the phone’s sound takes to reach the ear, which heardTime() takes off the clock. react-native-audio-api 0.13 cannot say it, so it is 0 unless you set it — Bluetooth earbuds add about 150 to 250 ms. Its early-access build says the phone’s own, unless you set it; on the web the browser says its own.
onStopped fires when stop() drops audio still to play — a cancel, a mute, a barge-in, a new turn; onDrained then too, and when a reply plays out. onCapture((pcm, rate) => …) is every frame the microphone captures while it is open, for a level meter.
On a phone, a reply at another sample rate — the agent’s voice changed — makes a new player, whose clock starts again; what was queued is stopped, and onStopped says so. A browser keeps its clock.
Each on… answers its unsubscribe, and a handler that throws is rethrown on its own: it never stops the voice.

The web

The same code runs in a browser, on the client’s Web Audio: React Native’s bundler picks it for the web. audio.prime() must run in the press, before any await — browsers start sound only from a gesture, and a player started outside one is silent. There are no interruptions to tell and no earpiece there. A browser lets the app choose the microphone, its processing and the speaker — createAudio({ input, processing }), inputs(), setInput, setProcessing, and outputs() and setOutputDevice where canChooseOutput (Chrome and Edge): the client’s page has them. On a phone the system chooses: inputs() and outputs() answer [], canChooseOutput is false, and setOutput picks the loudspeaker or the earpiece. It needs Metro’s package exports, on by default from React Native 0.79 (Expo SDK 53); before, set resolver.unstable_enablePackageExports in metro.config.js. On iOS Safari, a page’s audio can stay quieter once the microphone has opened (WebKit’s bug 218012).

Hands-free

Two ways to talk without holding a button, both agent.listen() from the hooks: the microphone stays open for the whole conversation, and each thing the person says becomes a turn after 700 ms of silence.

	Half-duplex	Hands-free
Start it	`agent.listen({ halfDuplex: true })`	`agent.listen()`
Talk over the agent to stop it	no — nothing is heard while it speaks	yes
On a phone	every phone, either build of `react-native-audio-api`	the early-access build
The agent’s voice	as in push-to-talk	like a speakerphone call while it listens
In a browser	every browser	where the browser cancels echo: Chrome, Edge, Firefox

Half-duplex

Half-duplex runs on every phone: tap once and talk for the whole conversation. The microphone stays open, but nothing is heard while the agent speaks — so it never hears itself through the loudspeaker, and needs no echo cancellation. The agent’s voice cannot be talked over, so give the person a stop button (agent.cancel()).

const agent = useAgent({ agentId }, { audio });
// Tap once: the microphone stays open; nothing is heard while the agent speaks.
void agent.listen({ halfDuplex: true });
// agent.listening and agent.hearing say how it goes; agent.stopListening() ends it

The agent speaking is what the person hears, on the player’s clock — set outputLatencyMs with Bluetooth earbuds on, so it ends when they stop sounding.
The microphone waits 300 ms after the voice ends (afterVoiceMs), so the room’s echo of its last word does not open a recording. Words begun in it keep their start; a word over before it ends — a quick “yes” under 0.3 s — is not heard.
The phone shows the microphone in use for the whole conversation, as any app that keeps it open: the orange dot on an iPhone, the green one on Android.
The client’s page has the rest.

Hands-free on a phone (early access)

Talking over the agent to stop it needs the phone to cancel its echo: the agent’s voice, from the loudspeaker, must be taken out of what the microphone hears, or the agent answers itself. react-native-audio-api cancels echo from its 1.0, which is not released yet — so hands-free on a phone runs, in early access, on one build of it, named exactly:

npm install @lumavox/react-native react-native-audio-api@1.0.0-nightly-6fd4c68-20261102

That build, and no other. On 0.13, or a later test build of the library, listen() answers false and agent.error says why — on a later build, naming the one it runs on. Everything else — push-to-talk, half-duplex, the clock — runs on that build and on 0.13 alike, so you can move back with npm install react-native-audio-api@0.13.
Build the app again after changing it: npx expo prebuild --clean, then npx expo run:ios and npx expo run:android; bare, cd ios && pod install. The plugin above works with both builds — still never list the library’s own. This build’s own plugin writes its build options into your package.json, under "react-native-audio-api", as it runs.

Ask the audio first, then listen — hands-free where it may, half-duplex where it may not:

const agent = useAgent({ agentId }, { audio });

async function talk() {
  const may = await audio.handsFree();
  // On "no", may.reason says why, in words to show.
  await agent.listen(may.ok ? {} : { halfDuplex: true });
}

Talking over the agent lowers its voice at once; it stops when the person’s first words are heard — about two seconds in — and what they say becomes the next turn. Sound with no words — a cough, the agent’s own voice — never stops it: its voice comes back up.
While it listens, the phone’s sound is a call’s. On an iPhone, Apple’s voice processing takes the agent’s voice out of the microphone; the agent plays from the loudspeaker (the earpiece with output: "earpiece"), a little quieter and narrower, as a speakerphone does, and other apps’ sound is lowered. On Android, the phone’s call audio: its echo canceller, the loudspeaker, and the call’s volume, which the volume buttons then set.
Bluetooth earbuds take the agent’s voice and the person’s, by the earbuds’ microphone, on both.
stopListening() gives the phone its usual sound back. On an iPhone the voice has a short gap as listening starts and stops, while the phone’s audio restarts; on Android a reply playing finishes first.
Where a phone’s echo cancelling is poor, the agent’s own voice can still reach the microphone: it lowers its voice now and then for no one, and comes back up. Offer half-duplex as a setting.
The microphone shows in use for the whole conversation, as in half-duplex. Nothing is sent or billed between the person’s turns.

With headphones on, the agent’s voice never reaches the microphone, so hands-free runs on 0.13 too, at the usual sound:

const agent = useAgent({ agentId }, { audio });
// Headphones on: the agent's voice never reaches the microphone.
void agent.listen({ assumeNoEcho: true });
// agent.listening and agent.hearing say how it goes; agent.error, why it refused

Not in this release

Hands-free on a phone’s loudspeaker on react-native-audio-api 0.13: it needs the early-access build until the library’s 1.0 is released. Half-duplex and push-to-talk work on both.
Background audio — a conversation going on with the screen off, as a call does — will be your app’s choice, off by default. With it come the stores’ rules: App Review checks that background audio is used as meant and that recording shows; Google Play asks every app targeting Android 14 or later to declare its microphone foreground service, with a video of it.

Each audio frame reaches JavaScript as base64 inside React Native’s WebSocket, a cost no SDK can remove: keep the screen’s renders light while the agent speaks, or a long render is a gap in its voice.