Virtualized 3D Environment for Video Calls
I propose a new view mode for video calls to add to those listed in the video feature support page. But first let me elaborate a bit on the problem I want this new mode to solve:
The Problem (in current video call layouts/modes)
- Following the flow of conversation between more than 2 participants is not as intuitive in an online meeting as in person. It is a little harder to determine who is speaking (particularly if you're in an unfamiliar group) and matching faces with voices is not as smooth. Finding the video of a speaker requires either remembering their location on screen or searching the screen for them. With a large list of participants such a search may take longer than the person speaks, certainly longer than it typically takes during an in-person conversation.
- If a lot of people begin speaking at once it can get very hard to distinguish between individual voices. The sound just kind of muddles together into a mess.
These problems are not egregious and I'm definitely not saying that Discord handles it worse than others. Discord handles them as well as any of the other major video call services I've used (Teams, Hangouts, Skype, Zoom). That being said, I believe these problems greatly contribute to the reason why in-person meetings feel better than online meetings (particularly for games like D&D).
Causes of these Problems (My best judgement)
- Virtual calls do not use any spatial awareness cues. Everything is on a screen straight in front of you, the sound all comes from straight in front of you. This is fundamentally different from how conversation occurs in person, where directional audio cues lead you quickly to the speaker whether you remember where they are or not. Directionality also ties into memory and how voices/speakers are remembered.
- Audio direction provides cues that your brain can use to help distinguish between individual voices. When all voices come from straight ahead this is not possible
Current Near-Solutions
Some techniques have been tried to improve these problems (not necessarily in Discord):
- Automatically enlarging the loudest speaker
- Icons to highlight people that are speaking
- Different layouts (hotseat style layouts seem to help)
- Fancy Industrial Telepresence tools like large conference walls, conference robots, etc.
All of these fall short either in sufficiently solving the problems above or in being too expensive for widespread casual use.
3D games with integrated chat often utilize spatial audio in conjuction with the player's in-game characters, but lack of uniformity of features leads many gamers to use Discord for audio anyway. The use-case here is also slightly different from my proposed solution, as a gaming squad often values clear audio over spatial 3d sound features like attenuated volume due to distance.
And of course everything is different for VR chat, and there are probably lessons that can be learned from it (I've never experienced it so I can't comment), but VR equipment is too bulky and expensive for widespread use.
Proposed Solution
Add a new video mode (no need to remove the existing Grid or Focus mode) in which the video of the speakers is laid out in a virtual 3d environment as if everyone were gathered around a table. Use 3D spatial audio and allow the view to be panned and zoomed. On mobile this could be an ‘AR’ arrangement where they can move the phone around to change their view into the virtual environment. On PC I believe your target market is experienced with 3D software that they can reasonably be expected to navigate keyboard/mouse control of a 3D environment.
These techniques would be most powerful in a VR environment, but a full VR environment requires the previously mentioned equipment costs and may inhibit different kinds of gameplay. Despite this, I think the virtual 3D environment even outside of VR will add significantly to my ability to enjoy a large virtual video call.
-
So built-in phone speakers wouldn't have directional ability, but earbuds would. Interesting!
0
Bitte melden Sie sich an, um einen Kommentar zu hinterlassen.
Kommentare
1 Kommentar