• General Usage
  • Installation
  • Citation
  • Settings
  • Usage
• Content
  • Dependencies
  • Toolkit Module
    • GamePlay Framework
    • Pawn
    • Replication
    • Utilities & Fixes
  • The RWTH VR Cluster Plugin and its Cluster Module
    • Configs & Setup
    • Cave Overlay
    • Cluster Console
    • Cluster Events
  • Editor Module
  • Example Maps
• Deployment/Building
• Development

General Usage

For development guidelines, please read the Development section carefully!

For more information, especially about CAVE usage, check out the general Unreal wiki

Installation

To install the plugin into your existing Unreal Engine project, either simply clone the Plugin into your Plugins folder, or add it as a submodule. This folder resides in your top-level Project directory. If you do not have it, just create it manually. More information on plugins can be found here in the Unreal Docs.

Make sure you have all respective plugins from the Dependency section enabled for your use-case. This should normally happen automatically. If you want to deploy to the cave, make sure you add the DTrackPlugin (Make sure to use our fork!) and the RWTH VR Cluster Plugin by cloning/adding a submodule similar to the Toolkit.

That's it already!

If you want to deploy your application on a standalone headset, follow the documentation for the Meta Quest

Citation

If you need to reference the toolkit in your scientific work, please use https://zenodo.org/records/10817754

Settings

Your project settings will vary depending on your use-case. For HMDs, different settings than for the Cave are recommended. As a standard set of settings, refer to the configuration files in the Project Template. As a summary, the following settings should be enabled:

• SM6 (Shader Model 6, in Platform Settings)
• VirtualTextures (if you want to use Nanite/Lumen)

Cave specific:

• r.DefaultBackBufferPixelFormat = 1 in your LinuxEngine.ini
• Make sure you have Instanced Stereo DISABLED if you deploy on the Cave, as this will lead to strange results!

Usage

The plugin comes with several Unreal Gameplay Framework extensions (GameMode, PlayerState, Pawn, ...). These extensions are explained in more detail in the Content section below. However, their initial usage is very simple:

Set the default GameMode of your project to the VRGameMode, which automatically sets all the other required defaults. Alternatively, you can set the default GameMode in the World Settings. More info on setting GameModes can be found in the Unreal docs.

Additionally, if you want to deploy your project on the Cave, you need to enable the RWTHVRCluster plugin. For Unreal 5.4+ projects, no additional configuration is necessary. For older project, see the respective section below. Make sure you have Instanced Stereo DISABLED if you deploy on the Cave, as this will lead to strange results!

For more information about how to deploy your project on the Cave, check the Unreal Wiki.

Content

Dependencies

For regular Desktop usage, the following Unreal-internal dependencies are required:

• LiveLink
• Enhanced Input
• Niagara

For VR usage, the respective VR plugins are required:

• OpenXR

For Cluster/Cave usage, additionally to the basic requirements:

• RWTH VR Cluster Plugin (Should enable all the others by itself)
• nDisplay (required for certain utility functions)
• LiveLinkOvernDisplay
• DTrackPlugin (Make sure to use our fork!)
• Switchboard (optional)

Toolkit Module

The main toolkit module RWTHVRToolkit contains the main functionality relevant to all platforms.

GamePlay Framework

The gameplay framework currently consists of the following extensions:

1. BP_RWTHVRGameModeBase
2. RWTHVRPlayerState
3. BP_RWTHVRPawn

1. BP_RWTHVRGameModeBase

The BP_RWTHVRGameModeBase blueprint is based on the RWTHVRGameModeBase C++ class, which in turn inherits from AGameModeBase.

It controls the flow of the application and handles new players and the corresponding default classes. It is important to notice that it only exists on the server, not on any clients! The BP_RWTHVRGameModeBase blueprint sets the following defaults:

• Default Pawn: BP_RWTHVRPawn
• Default Player State: RWTHVRPlayerState

Additionally, it overrides the InitNewPlayer function to check for specific join options in the Join-Session string that new clients send to the server on initial connection. This happens when nDisplay nodes join a server. We need to identify those here to spawn a dummy spectator pawn for all the secondary nodes, and only a regular default pawn for the primary nDisplay node.

In case a primary/secondary node is identified by the connection string, the Game Mode sets the corresponding player type on the new player's RWTHVRPlayerState.

In the PostLogin override, the player type is then queried and the correct pawn spawned and possessed.

2. RWTHVRPlayerState

The RWTHVRPlayerState is a simple extension of APlayerState. It extends the basic class by adding support for a replicated PlayerType variable, which stores the corresponding player's (hardware) type, and can be one of the following: nDisplayPrimary, nDisplaySecondary, Desktop, HMD.

For now, it is set to the first two in case the GameMode notices a corresponding connection string for the connecting client, or in the Pawn in case of standalone mode or the latter two types.

3. BP_RWTHVRPawn

As the Pawn is more involved, it gets its own Pawn section below.

Pawn

See Pawn section.

Replication

The Toolkit contains a fairly simple replication approach that can serve as a baseline for your project. Further resources on Unreal Multiplayer can be found in the Multiplayer Network Compendium, which can be used as a nice starting point. This section does not want to teach replication in general, but just highlights where it comes to play in the toolkit.

1. RWTHVRGameModeBase: The gamemode only exists on the server. It manages the new players and assigns the correct pawns as mentioned in the previous section. Be careful not to use it for client-side logic, as it simply won't exist there. However, it is very useful for functions you only want to be run on the server.
2. RWTHPlayerState: There exists one PlayerState for each player, and they are replicated to all other players. Therefore, this class is perfect for syncing information that is always relevant to all other players. As an example, the base class is extended by a replicated PlayerType variable.
3. ClientTransformReplication: In general, Actors/Components marked as "replicated", Unreal syncs basic things like their transform and attachment from the server to the clients. It does not sync transforms the other way round. If a player now modifies e.g. their pawn position, they only do this on their local machine. The UClientTransformReplication can be attached to your Actor to allow for Client-Side replication as well. It does this by simply sending an RPC to the server, asking it to modify the client position. This is then manually replicated by a UProperty marked as ReplicatedUsing to all the other clients. We're using a manual replication here for demo purposes and to save on bandwidth. Ideally, we would only sync Inputs instead of full transforms.
4. ReplicatedCameraComponent and ReplicatedMotionControllerComponent are simple extensions to their base-classes that pretty much do the same thing as described in 3. Due to being in VR, we need to have some kind of client-authority, which is emulated in this way.
5. RWTHVRPawn: The Pawn contains some additional functionality that searches and attaches the DCRA on the server to the Pawn corresponding to the PrimaryNode. This is then implicitly replicated to all clients, including the SecondaryNodes.

Utilities & Fixes

1. Utilities

The URWTHVRUtilities defines a BP Function Library that can also easily used via C++. It collects various static functions to determine the state/mode of the currently running applications regarding VR hardware devices, such as:

• IsRoomMounted/Cave/Desktop/HMDMode
• IsPrimary/SecondaryNode regarding nDisplay
• Various other nDisplay related utility functions

The UDemoConfig class, a derivative of UDeveloperSettings, serves as an example on how to create your own project settings including a DefaultDemo.ini file for your project that is still usable in packaged demos.

2. Fixes

There are currently two fixes in place.

• The first simply enables the activation of the console in shipping builds.
• The second fixes an issue in the FLiveLinkMotionController that prevents the LiveLink subjects to be accessed correctly. A PR is in place (here).

The RWTH VR Cluster Plugin and its Cluster Module

The cluster module RWTHVRCluster is part of the RWTH VR Cluster Plugin and extends the toolkits functionality by adding support for clustered rendering via nDisplay.

Configs & Setup

Version 5.4+

For Versions 5.4 and later, no additional setup is required. The plugin handles the setup and configuration of the DCRA automatically and no nDisplay related actor should be placed in the scene.

Upgrade from earlier versions to 5.4+

In newer plugin versions the BP_CaveSetup-blueprint has been removed. When you upgrade your plugins, you will find that your levels will no longer contain a CaveSetup-actor but an aixcave-actor instead. This aixcave-actor needs to be removed from your map for it to function correctly in the Cave. You will no longer need to add any actors or components to your maps manually, as they will be automatically added at runtime.

Version 5.3

The main actor that you need to add to your map if you want to deploy on the Cave is the BP_CaveSetup actor. It inherits from the C++ class CaveSetup documented below.

CaveSetup

The CaveSetup class has a few purposes:

1. It has the DCRA attached as a child actor. On non-cave systems, the DCRA will simply be destroyed.
2. The pawn searches for the CaveSetup actor and attaches it to itself, therefore attaching the DCRA to the pawn as well.
3. It is replicated, which means it replicates the attachment from 2. to all clients. This is required for using nDisplay multiplayer replication.
4. It applies the correct livelink preset on BeginPlay(), but only on the primary node. LiveLinkOvernDisplay handles the syncinc to the secondary nodes.
5. It spawns any actors that you add to the ActorsToSpawnOnCave array if running on the Cave. An example is the CaveOverlay, which gets spawned here.

Configs (DCRA)

Additionally, the module contains various nDisplay configuration files:

• aixcave (default aixcave config)
• aixcave_two_players (two player aixcave used for vikings demo)
• aixcave_two_players_tdw (two player with tdw)
• ExampleConfigs/NDC_CaveUnwrap (example unwrapped cave config for local tests on your own PC)

Cave Overlay

The Cave Overlay is used to display a warning tape and sign when the head or controller(s) come too close to the Cave walls. Additionally, it can be used to black out the door projectors when the door is opened. This isn't done automatically, but the door state can be toggled by pressing F10. ACAVEOverlayController is the actor that is being spawned by the CaveSetup actor and correspondingly attached to it and therefore the pawn. The OverlayController gets the Head, Left - and Right Hand components of the Pawn and enables/disables the corresponding warning signs.

It can be configured in the project settings.

Cluster Console

The FClusterConsole is a simple class that Registers a console command (ClusterExecute) which sends the appended string via ClusterEvents to the whole Cluster and executes it as a console command on all nodes.

E.g.: ClusterExecute stat fps to get a fps overlay on all nodes.

Cluster Events

For the usage of cluster events in C++, see Cluster events in C++

Editor Module

The editor module RWTHVRToolkitEditor in general contains features to enhace the editor experience, such as visualizers for certain components. Currently, no such features are implemented.

Example Maps

The ContentExamples-project provides example maps showcasing functionalities of the toolkit plugin, functionalities and pitfalls of the cluster-plugin and CAVE development, and Unreal Engines Replication System.

Deployment/Building

At accelerate your build process when creating executables and packaging your application, you can easily set up a shared derived data cache through Zen Storage Server. Instructions on how to do that can be found here: https://git-ce.rwth-aachen.de/vr-vis/unreal-wiki/-/wikis/Unreal/Derived-Data-Cache-(Shader-Cache)

Development

Please read the Development section carefully. As a summary:

• Use your own forks for developing features to not spam branches on this repository, keep it limited to release and dev branches.
• Use conventional commit syntax when commiting to improve readability for others: <type>(<scope>): <description>
• Keep commits encapsulated to one change/feature. Don't try to put all your changes into one commit, it makes the history and git blame absolutely unreadable, and git bisect doesn't work as well for debugging.
• Commit changes to shared blueprints early and put up PRs regarding them early. The longer they sit, the more nightmareish merging will become.
• Document your features both in code and in the Wiki.