TTS
If you want to have a speaking agent you first need the speech audio, if you do not already have *.wav files for the speech to use, you can generate the using Text-To-Speech (TTS), two options would be Google TTS or IBM Watson TTS
Both are free for a limited number of words, which however should be enough for most applications. However, they require you to register for free for the service. Python interfaces work well for both services:
These should be capable of creating *.wav files for you which we use for lip-syncing.
Lip Sync
This plugin provides different options for animating the face, see FaceAnimation of which Oculus Lip Sync will work best if you do not have any other tracking data and only the TTS *.wav files.
Each audio file needs to be preprocessed, e.g., with the OculusLipSyncWAVParser. This generates a visemes.txt file which is then used to animate the face. For more information about its usage, please check out the README.
After the preprocessing go back to your Unreal project. Choose the character, who is supposed to be conversational, and add a VHOculusLipSync component to it. Consider carefully if you want to add them to the respective character's blueprint or on the individual map instance.
If your character of choice happens to be a Character Creator 3 Model use the PluginContent/Characters/Henry/OculusLipSyncToCC3 pose asset and for a MetaHuman the PluginContent/Characters/MetaHuman/OculusLipSyncToMeta pose asset. The created *.txt file for the visemes has to be set as the animation file.
Information on how to start the lip sync (and the audio, see below) can be found here.
Audio
You are free to use any Unreal audio plugin and specify the audio file as well as an attachment.
However, for usage in the CAVE, we recommend using the VirtualAcoustics Plugin. You just have to add a VAFilenameSource component to your character and specify the audio file, relative to the VAServer's Data folder. For the source to move with the head you need to set the Position Setting to Attached to Bone and the Bone name for CC3 character/MetaHuman to head.