Modular dialogue orchestration
Worked on typed scenario steps, branching, slot filling and pluggable extractors. YAML separated conversation logic from the code; regex, CRF and LLM-based extraction could be selected for different slots.
./Backend engineeringFrom a conversation
From a conversation
to a clear next step.
AI Voice Operations — an internal voice automation system built around one company’s operational workflows.
- My role
- Backend Developer
- Focus
- Dialogue & streaming audio
- Project type
- Internal company tool
Interface reconstructed for this portfolio under the presentation name “Relay”. All displayed contacts, transcripts and metrics are sample data; these screens are not captures of the original product.
The challenge
A voice agent needs more than a generated reply. It has to follow the purpose of a call, keep track of what has already been said, handle interruptions and leave a usable result for the operator.
This was a custom internal system for one company, not a SaaS product. I contributed to the backend and dialogue-processing components, tailored to the company’s call handling and request collection workflows. The work covered a modular scenario engine and, later, a prompt-driven voice flow integrated with telephony and call records.
My contribution
Context-aware voice responses
Contributed to the voice flow around LLM responses and streaming speech. The later implementation used gpt-oss-120b through Groq, with conversation history and handling for partially spoken, interrupted replies.
Audio preparation & interruptions
Worked with Node.js audio streams, noise suppression and speech detection for recognition. The pipeline combined ElevenLabs STT/TTS with playback cancellation and fading when the caller started speaking.
Detection, lifecycle & tests
Developed and refined stream-based hard-mute detection with typed events, configurable settings and cancellation. Tested audio fixtures, and explored synthetic-voice and dialogue-risk checks as separate experimental components.
How the architecture evolved
Scenario-led dialogue
YAML flows, intents, forms and slot mappings. Explicit handlers drove the conversation, while extractors supplied structured values.
Prompt-led voice flow
The voice backend used a central prompt and message history with gpt-oss-120b. The engineering focus shifted toward audio quality, response cancellation, call termination and useful call records.
The system around it
FreeSWITCH handled telephony, WebSockets carried audio, and PostgreSQL stored call status, outcomes and dialogue logs. The repository included an operator interface, a backend service and Docker configuration.