News for hackers. Stuff that matters.

Show HN: FaceTime-style calls with an AI Companion (Live2D and long-term memory)

https://thebeni.ai/

Hi HN, I built Beni (https://thebeni.ai ), a web app for real-time video calls with an AI companion.

The idea started as a pretty simple question: text chatbots are everywhere, but they rarely feel present. I wanted something closer to a call, where the character actually reacts in real time (voice, timing, expressions), not just “type, wait, reply”.

Beni is basically:

A Live2D avatar that animates during the call (expressions + motion driven by the conversation)

Real-time voice conversation (streaming response, not “wait 10 seconds then speak”)

Long-term memory so the character can keep context across sessions

The hardest part wasn’t generating text, it was making the whole loop feel synchronized: mic input, model response, TTS audio, and Live2D animation all need to line up or it feels broken immediately. I ended up spending more time on state management, latency and buffering than on prompts.

Some implementation details (happy to share more if anyone’s curious):

Browser-based real-time calling, with audio streaming and client-side playback control

Live2D rendering on the front end, with animation hooks tied to speech / state

A memory layer that stores lightweight user facts/preferences and conversation summaries to keep continuity

Current limitation: sign-in is required today (to persist memory and prevent abuse). I’m adding a guest mode soon for faster try-out and working on mobile view now.

What I’d love feedback on:

Does the “real-time call” loop feel responsive enough, or still too laggy?

Any ideas for better lip sync / expression timing on 2D/3D avatars in the browser?

Thanks, and I’ll be around in the comments.

( 26 comments | 3 points | on HN )
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by dfajgljsldkjag on Sunday January 25, 2026 @08:43PM (#46760786)
It creates a conflict to build a system that is both a private friend and a public performer. You cannot maximize intimacy and fame at the same time.
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by zemo on Sunday January 25, 2026 @09:15PM (#46761060)
You're describing Parasocial interaction: https://en.wikipedia.org/wiki/Parasocial_interaction

far from being impossible, it's the entire influencer economy. This form of social media has been extremely widespread for a decade or so running; it's probably the dominant form of social media.

Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by summerlee9611 on Monday January 26, 2026 @08:36PM (#46774383)
100% agree. Maximizing intimacy and scaling distribution pull in opposite directions. We’re experimenting with keeping the “character” consistent while letting personalization live in private memory and user-controlled settings. Still early, and this tension is real.
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by sghimire2022 on Sunday January 25, 2026 @08:50PM (#46760843)
This is cool.
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by summerlee9611 on Monday January 26, 2026 @08:36PM (#46774380)
Appreciate it. If you try it and anything feels off (latency, turn-taking, uncanny moments), I’d love concrete feedback. That’s what we’re grinding on right now.
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by october8140 on Sunday January 25, 2026 @08:55PM (#46760891)
This is disturbing.
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by xattt on Sunday January 25, 2026 @09:01PM (#46760932)
It will quickly distill down to clients using the service just for sex and sex-adjacent activities.

No kink-shaming, but this sort of thing enables self-destructive hard-to-return-from anti-social behaviour.

Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by summerlee9611 on Monday January 26, 2026 @08:34PM (#46774369)
Totally fair reaction. We’re building this with clear boundaries: we don’t position it as therapy replacement, we add safety rails, and gives user a choice what mode they want and guardrails differ based on this. Plus, age restriction is there as safety boundary
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by dummydummy1234 on Sunday January 25, 2026 @08:58PM (#46760914)
What are you using for tts/stt/models?
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by summerlee9611 on Monday January 26, 2026 @08:35PM (#46774377)
realtime api + elevenlabs but llms will be diversified based on persona moving forward. Using chatgpt/gemini as baseline model, we feel prompting has limitation
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by augusteo on Sunday January 25, 2026 @09:34PM (#46761193)
Building on zemo's point about parasocial relationships: traditional parasocial interaction involves a performer who doesn't know you exist. Here the AI does respond to you specifically, which changes the dynamic.

Is it still parasocial if the other party is responsive but not conscious? Or is this something new that we don't have good language for yet?

Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by Charon77 on Sunday January 25, 2026 @10:29PM (#46761540)
You need to first prove that AI is not conscious.

I find it hard to even convince others that I am a conscious person.

Maybe consciousness is just a matter of belief, if I see this AI and believe that it's a person, then I am talking to a conscious entity.

Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by neidnejnd on Sunday January 25, 2026 @11:05PM (#46761760)
Jesus Christ you AI fcks are out of your minds.

YOU prove it is what you say, you’re the one making claims like a mentally ill moron.

And it’s easy to see why people think you’re not conscious. Plain. As. Day.

Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by DauntingPear7 on Sunday January 25, 2026 @11:35PM (#46761911)
Yeah the “rocks are alive and conscious” crowd are certainly something
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by idiotsecant on Sunday January 25, 2026 @10:39PM (#46761603)
Give it access to a terminal and see what it does, unprompted. Does it explore? Does it develop interests? Does it change when exposed to new information?
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by krackers on Sunday January 25, 2026 @10:47PM (#46761659)
>Does it change when exposed to new information?

By this metric most humans are not conscious.

Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by summerlee9611 on Monday January 26, 2026 @08:32PM (#46774345)
We’re not giving it unconstrained tool access. In-product, actions are either not available or gated behind explicit user intent and strict allowlists. The interesting part for us is the real-time conversational loop and memory personalization, not autonomous exploration.
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by summerlee9611 on Monday January 26, 2026 @08:31PM (#46774343)
I’m not trying to make any claims about consciousness. For us, the practical question is: does the interaction feel supportive and useful, while staying transparent that it’s a model. The rest is philosophy, and I’m happy to read more perspectives.
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by idiotsecant on Sunday January 25, 2026 @10:41PM (#46761628)
I think maybe there needs to be a new word. It's still an asymmetric relationship. It's kind of a mix of DMing an influencer and chatting with the barista because you think she actually likes you. You're talking to a mirage.
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by summerlee9611 on Monday January 26, 2026 @08:31PM (#46774338)
I think “parasocial” still captures part of it (one-to-many distribution, performer vibe), but there’s also a true interactive dyad here. It’s closer to “synthetic social interaction” or “responsive parasocial.” I don’t have a perfect word yet, but the asymmetry and the responsiveness both matter.
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by nitroedge on Monday January 26, 2026 @01:46AM (#46762566)
For better lip sync you could try using rhubarb to extract from the mp3. What is your backend speech processor so you can get the real-time streaming response? Rhubarb would add a bit of latency for sure.
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by summerlee9611 on Monday January 26, 2026 @08:33PM (#46774352)
For real-time: we use WebRTC for streaming. Input is streaming STT, then a low-latency LLM, then TTS, then we drive Live2D parameters on the client. Lip sync: we currently do (simple phoneme / amplitude-based) and are testing viseme extraction. Rhubarb is on our list, but we’re cautious about added latency.
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by singular_atomic on Monday January 26, 2026 @03:36AM (#46763202)
Where’s the asteroid at
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by summerlee9611 on Monday January 26, 2026 @08:36PM (#46774379)
Same place as my latency budget: disappearing fast.
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by ryannampham on Monday January 26, 2026 @10:15PM (#46775072)
wow we got personal vtubers now!
Re:Show HN: FaceTime-style calls with an AI Companion (Live2... by summerlee9611 on Monday January 26, 2026 @11:57PM (#46775712)
yess you can have her 24/7!