The project joined local inference with a real-time browser experience, keeping the conversation loop responsive while giving the avatar a visual presence instead of treating it as a text-only chatbot.
WorkProject Doru
Local-first real-time avatar
Project Doru
A local-first real-time AI avatar combining streaming speech, natural voice synthesis and an expressive Live2D interface.
Overview
Doru is an interactive systems project: speech arrives, language is generated, voice returns and the avatar responds through a deliberately expressive interface.
What I owned
Engineering challenges
Keeping streaming speech recognition, model response and voice synthesis inside one low-latency loop.
Synchronizing WebSocket events with an expressive Live2D frontend without making the interface feel mechanical.
Shipping local CPU inference and the supporting deployment pipeline as a usable product experiment.
Selected outcomes
A complete speech-to-avatar interaction loop running with local inference.
A visual frontend that responds through Live2D motion, not only chat bubbles.
A repeatable Docker and CI deployment path for the interactive system.
Built with
Sherpa-ONNXKokoroPixiJSLive2DWebSocketsDockerAzure