KAIST Unveils On-Device AI Accelerator for Real-Time Privacy-Preserving Personal LLMs
KAIST researchers have unveiled an on-device AI accelerator called “Soulmate,” designed to run personalized large language models directly on a user’s device. The system learns a user’s speaking style, preferences, and even emotional state in real time, without sending data to external servers.
The project is led by Prof. Yoo Hae-joon of KAIST’s AI Semiconductor Graduate School, with PhD student Hong Seong-yeon contributing to the work. KAIST announced the results on the 17th, describing Soulmate as a self-evolving, personalized LLM accelerator that operates entirely on-device.
Soulmate integrates memory of prior conversations and user feedback into its hardware, enabling what the team calls real-time personalization. It combines a search-augmented generation capability with a learning loop that updates its behavior directly inside the semiconductor, delivering responses in about 0.2 seconds while continuing to learn from interactions.
A key feature is its energy efficiency and privacy. By applying a mixed-rank architecture that optimizes processing, the researchers say the device can run on mobile hardware at roughly one-five-hundredth the power of typical smartphone processors. Since no personal data is transmitted off the device, the approach reduces the risk of data exposure.
The work was highlighted as a “Highlight Paper” at the International Solid-State Circuits Conference (ISSCC) held in San Francisco. The KAIST team plans to commercialize Soulmate through a university-affiliated startup founded by faculty members.
Beyond its Korea origins, the development speaks to broader trends in edge AI, where firms aim to move more intelligence onto devices rather than rely on cloud servers. For the United States, such on-device AI advances could influence consumer electronics, automotive systems, and wearables by offering lower latency, stronger privacy, and reduced reliance on network connectivity—factors that matter to digital markets, security considerations, and the resilience of supply chains for AI hardware.
The researchers say Soulmate represents a step toward AI that acts as a genuine, evolving companion by adapting to an individual user. The approach raises ongoing questions about data handling, user safety, and how best to balance personalization with privacy as such technologies move toward commercialization.