From motion to magic: Real-time virtual-real stage effects via 3D motion capture

Xiongbin Lin, Xun Wang, Wenwu Yang

Article ID: 2336
Vol 4, Issue 2, 2023
DOI: https://doi.org/10.54517/m.v4i2.2336
Received: 16 October, 2023; Accepted: 6 November, 2023; Available online: 15 November, 2023; Issue release: 31 December, 2023


Download PDF

Abstract

Immersive cultural performances with virtual-real fusion effects are the future development trend in the exhibition and stage industry. However, current virtual-real stage performances heavily rely on traditional sequential design and arrangements. During the performance, actors must move to specific positions based on the musical beat and execute predetermined actions with a pre-designed amplitude and frequency to synchronize with the fixedly played stage visual effects; otherwise, major performance accidents such as plot inconsistencies or continuity errors may occur. To address the problem, this paper introduces a real-time generation system for stage visual effects based on multi-view multi-person 3D motion capture. The system utilizes multi-view 3D motion capture technique to achieve non-intrusive real-time interaction perception of target actors in the stage space. By perceiving the spatial position and performance actions of the target actors, corresponding stage visual effects are generated in real-time. This is followed by the seamless integration of sound effects and immersive high-definition display, ultimately realizing multidimensional real-time interaction between real actors and virtual visual effects in the stage space. We conducted an experimental virtual-real stage performance, lasting approximately two minutes, in a physical theater to validate the effectiveness of our proposed method. The experiment not only produced a unique innovative effect of blending stage and technology but also effectively enhanced the sense of presence and interactivity of the stage performance, providing actors with more freedom and control in their performances.


Keywords

immersive stage performance; interactive generation of stage effects; non-intrusive multi-person motion capture; action recognition


References

Refbacks

  • There are currently no refbacks.


Copyright (c) 2023 Xiongbin Lin, Xun Wang, Wenwu Yang

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.


This site is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).