The Human-Computer Interaction Facial Animation and Conversation Engine is a tool for developing new audio and text based interactions with new conversational AI.
This open source tool makes it easy for developers to create interactive experiences with digital personas. The website is a work in progress, but provides a few examples of what can be done with the tool. The backend can be configured for entirely offline use with free and open source models and software or can utilized online with paid cloud services. The HCI-FACE backend provides the key components for an interactive experience:
- Text to Speech
- Speech to Text
- Facial Animation (Emotions and Visemes)
- LLM Based Conversation
Together, these elements have been combined to create the following applets:
Have a conversation
Have a fully interactive conversation with an AI. Set the prompt and start a conversation!
Record your ideas
Have your browser write down your thoughts. You can use buttons to control a microphone or have it intelligently take notes on your next idea session.
Text-chat with AI
Have a text-based conversation with a Large Language Model. There are preset prompts or you can create your own!
Puppeteer the face
Visualize your robot face's behaviors and emotional expressions. You can also see and hear speech the face as it says what you want it to.
Work with the clock
Timer, Alarm Clock, Stopwatch, and Pomodoro Focus Timer all in one!
Setup/Test your mic
Visualize the sound waves, record with button controls or have the microphone pick out your voice as you talk automatically.
Visualize new expressions
Directly control the facial action units to customize the face's expression. See how it appears on a range of possible faces.
List to your text read aloud
Hear a voice read your text at your preferred speed.
Additional Pages:
Show just the face on this page!