THE NEW YORK TIMES

Open World Apollo Landing

Prototyping the “Story”

This Open World prototype, built around the Times’ Apollo landing assets, includes navigation controls and options available in Unreal Engine. The player moves around the space, interacts with and “collects” items in the 3D space, and views an inventory of found items with their descriptions.

Unreal Engine interactive start screen and inventory.

This prototype was designed from initial Figma mockups and wireframes that considered how current Times’ interactive storytelling models (such as the Scrollytelling format) could be adapted, expanded, and developed using open world gameplay feedback loops and mechanics in Unreal Engine. For this project, we considered how in-world arrows or 3D “+” markers might guide viewers/players as they navigate the 3D space. We also considered how the “narrative” told in this particular Times content lends itself to the non-linear experience dictated by the open world format.

Dynamic Audio

Exploring Audiovisual Dynamics

This Dynamic Audio prototype, built using a recording from one of the recent Modern Love podcast episodes, is conceptualized as an immersive/dynamic VR experience. There are two versions of this format prototyped for play. The first, subtitled “Dynamic Void,” positions the player/viewer at the center of a 3D void with 360-degree camera access. Periodically, as the player rotates their view around the space, dynamic illustrations appear at the point of the player’s gaze. The dynamic illustrations are currently represented by particle system placeholders, to show the movement quality of how dynamic illustrations should appear in a fully complete experience (without relying on the work of creating the dynamic illustrations for the prototype). In addition to the dynamic illustrations, 3D text quotations taken directly from the audio appear periodically in the 3D space, wherever the player is looking.

Gaze tracked VR simulation populates illustrations (represented by particle system here) and text from the audio recording played in 2D.

The second format, subtitled “Dynamic 3D,” positions the player along the “fourth wall” of a simple apartment scene. As the player rotates their view around the space, the same 3D text quotations taken directly from the audio appear wherever the player is looking.

3D Environment prototype for dynamic text to appear alongside 2D audio.

Dynamic Audio Figma Mockup using New York Times content

Story and Level Design Analysis

The following snapshot shows the breakdown of major Unreal Engine formats that were finalized in the research and design stages of this project. Each storytelling format is further categorized into experiences around particular types of interactions, features, and assets needed for prototyping and development.

The goal of this breakdown was to show how specific Unreal Engine capabilities (UMG, physics, photorealism, etc.) could be leveraged to create new Times experiences across Times’ content.

Those defined storytelling formats were then analyzed according to four categories of user experience to determine what kinds of Times content they would best suit. These categories included: Communication, Story Flow, Environment, and Reader Participation. The product of this outcome was the following four graphs, where each format was plotted accordingly.

Additional Unreal Engine Mockups & Prototypes

Beyond the prototypes developed in Unreal Engine, the project also yielded a series of additional mockup ideas for ways that The Times might use Unreal Engine and its virtual/interactive affordances for news media and journalistic storytelling.