NovaSynth by Noveum

A voice agent that works in a demo can still fall apart on someone's third call about a delayed refund. NovaSynth is built to produce that call on purpose. It generates synthetic callers with goals, moods and accents, such as an angry repeat caller, a rapid interrupter or a speaker who switches languages mid-sentence, and sends them at your agent over a real phone line, LiveKit audio or chat.

The scenarios go past polite requests. The site lists interruptions, background noise and adversarial intent, including prompt-injection probes, next to ordinary support calls. Each call leaves a recording, a transcript and evaluation results, so you can see where the agent lost the thread.

Single calls help you spot a problem, and batches turn that into a test suite. A batch runs every combination of personas and scenarios under one batch ID, so results are grouped and comparable, and the same batch can run again after a prompt or tool change to confirm the fix. Concurrency goes up to 2,000 lines at once for load testing, and the site asks you to warn the team before a large run so lines can be provisioned.

NovaSynth connects to SIP providers including Twilio, Plivo and Vobiz, to LiveKit and Pipecat, to LangChain and LangGraph, to the OpenAI and Anthropic SDKs, and to chat over HTTP. Custom agents can connect through an SDK or light hooks around their LLM calls.

NovaSynth comes with every Noveum plan, and plans are priced per organization rather than per seat. The free plan includes 2,500 credits a month with no card, and Pro costs $69 a month for 10,000 credits. Those credits are shared across voice and text testing, evaluation scoring and Noveum's analysis tool, so a big persona-by-scenario matrix draws on the same pool.

The rest of Noveum covers tracing, evaluation with a library of 112 published scorers, recommendations drawn from evaluated failures and a guardrail tool in beta. The privacy policy, from MagicAPI Inc, treats recordings and transcripts as customer content that it processes as a service provider.