The Paradigm of Cognitive Twins
Most AI integrations are transactional: the user submits an isolated context, the model produces a stateless output, and the context is immediately cleared. To build a genuine cognitive twin—a digital mirror that retains context, learns across domains, and communicates fluidly in natural spoken audio—requires bypassing this stateless constraint.We achieved this by architecting the DEXter AI Voice & Chatbot Hub as an autonomous, high-performance edge microservice.
1. 🎛️ The Microservice Architecture
The system is built as a self-contained API route on our Next.js edge stack, handling:Client-Initiated E2EE Tunnels: All chatbot interactions are encrypted before transmission to prevent intermediate packet snooping.
Strict Rate Limiting: Employs an in-memory IP tracker allowing a maximum of 30 requests/min, protecting our underlying inference gateways.
Cross-Origin Embeds: Employs full CORS headers and preflight
OPTIONS routing, enabling third-party platforms to drop our script widget (dexter.js) without cross-origin blocks.2. 🧠 Persistent Memory Graphs: Mem0 Integration
Instead of overloading the prompt context with static data, we engineered a dynamic RAG retrieval loop powered by Mem0:Anonymized User IDs: On first load, the client generates a unique cryptographic UUID (
dexter_user_id) saved to localStorage.Intent Ingestion & PII Redaction: Before storing or querying context, the prompt is analyzed by a regex parser to redact credit card hashes, emails, and phone numbers.
Graph Database Memory: During each chat turn, the sanitized message is indexed into the Mem0 graph memory. When the user returns, the system automatically pulls the top-K relevant semantic nodes of historical context, prepending it to the Gemini system instructions.
3. 🎙️ Duplex Voice Pipeline: Real-Time Audio PCM
Duplex voice requires low latency to feel conversational. We avoided heavy WebRTC setups by building a direct, browser-native Audio Context stream:Buffer Ingestion: Capture microphone input in real-time at
16000Hz mono using a standard ScriptProcessorNode to ensure compatibility across Safari, Chrome, iOS, and Android.PCM to Base64 Conversion: Map the raw float32 array samples directly into 16-bit signed PCM integers, chunk them into base64 strings, and stream them via WebSockets to the Gemini Live gateway.
Duplex Interruption: The system constantly monitors outgoing client audio buffers. If incoming speech is detected while the model is playing synthesized text-to-speech, the browser context immediately clears the audio source nodes, allowing the user to naturally speak over the model's voice.
EOF