Headsets in the thousand-dollar range are plenty good and still not selling. Take the hint. Push costs down. Cut out everything that is not strictly necessary. Less Switch, more Game Boy.
6DOF inside-out tracking is required, but you can get that from one camera and an orientation sensor. Is it easy? Nope. Is it tractable for any of the companies already making headsets? Yes, obviously. People want pick-up-and-go immersion. Lighthouses were infrastructure and Cardboard was not immersive. Proper tracking in 3D space has to Just Work.
Latency is intolerable. Visual quality, scene detail, shader complexity - these are nice back-of-the-box boasts. Instant response time is do-or-die. Some monocular 640x480 toy with rock-solid 1ms latency would feel more real than any ultrawide 4K pancake monstrosity that’s struggling to maintain 10ms.
Two innovations could make this painless.
One, complex lenses are a hack around flat lighting. Get rid of the LCD backlight and use one LED. This simplifies the ray diagram to be nearly trivial. Only the point light source needs to be far from the eye. The panel and its single lens can be right in your face. Or - each lens can be segmented. The pyramid shape of a distant point source gets smaller, and everything gets thinner. At some point the collection of tiny projectors looks like a lightfield, which is what we should pursue anyway.
Two, intermediate representation can guarantee high performance, even if the computer chokes. It is obviously trivial to throw a million colored dots at a screen. Dice up a finished frame into floating paint squares, and an absolute potato can still rotate, scale, and reproject that point-cloud, hundreds of times per second. But flat frames are meant for flat screens. Any movement at all reveals gaps behind everything. So: send point-cloud data, directly. Do “depth peeling.” Don’t do backface culling. Toss the headset a version of the scene that looks okay from anywhere inside a one-meter cube. If that takes longer for the computer to render and transmit… so what? The headset’s dinky chipset can show it more often than your godlike PC, because it’s just doing PS2-era rendering with microsecond-old head-tracking. The game could crash and you’d still be wandering through a frozen moment at 100, 200, 500 Hz.
I think the slightly more viable version of the rendering side is to send a combination of low res point cloud + low res large FOV frames for each eye with detailed (!) depth map + more detailed image streamed where the eye is focused (with movement prediction) + annotation on what features are static and which changed and where light sources are, allowing the headset to render the scene with low latency and continuously update the recieved frames based on movement with minimal noticable loss in detail, tracking stuff like shadows and making parallax with flawlessly even if the angle and position of the frame was a few degrees off.
Undoubtedly point-clouds can be beaten, and adding a single wide-FOV render is an efficient way to fill space “offscreen.” I’m just cautious about explaining this because it invites the most baffling rejections. At one point I tried explaining the separation of figuring out where stuff is, versus showing that location to you, using beads floating in a fluid simulation. Tracking the liquid and how things move within it is obviously full of computer-melting complexity. Rendering a dot, isn’t. And this brain case acted like I’d described simulating the entire ocean for free. As if the goal was plucking all future positions out of thin air, and not, y’know, remembering where it is, now.
The lowest-bullshit way is probably frustum slicing. Picture the camera surrounded by transparent spheres. Anything between two layers gets rendered onto the further one. This is more-or-less how “deep view video” works. (Worked?) Depth information can be used per-layer to create lumpen meshes or do parallax mapping. Whichever is cheaper at obscene framerates. Rendering with alpha is dirt cheap because it’s all sorted.
Point clouds (or even straight-up original geometry) might be better at nose-length distances. Separating moving parts is almost mandatory for anything attached to your hands. Using a wide-angle point render instead of doing a cube map is one of several hacks available since Fisheye Quake, and a great approach if you expect to replace things before the user can turn around.
But I do have to push back on active fake focus. Lightfields are better. Especially if we’re distilling the scene to be renderable in a hot millisecond, there’s no reason to motorize the optics and try guessing where your pupils are headed. Passive systems can provide genuine focal depth.
My suggestions are mostly about maintaining quality while limiting bandwidth requirements to the headset, wouldn’t a lightfield require a fair bit of bandwidth to keep updated?
(Another idea is to annotate moving objects with predicated trajectories)
Less than you might think, considering the small range perspectives involved. Rendering to a stack of layers or a grid of offsets technically counts. It is more information than simply transmitting a flat frame… but update rate isn’t do-or-die, if the headset itself handles perspective.
Optimizing for bandwidth would probably look more like depth-peeled layers with very approximate depth values. Maybe rendering objects independently to lumpy reliefs. The illusion only has to work for a fraction of a second, from about where you’re standing.
Alpha-blending is easy because, again, it is a set of sorted layers. The only real geometry is some crinkly concentric spheres. I wouldn’t necessarily hand-wave Silent Hill 2 levels of subtlety, with one static moment, but even uniform fog would be sliced-up along with everything else.
Reflections are handled as cutouts with stuff behind them. That part is a natural consequence of their focus on lightfield photography, but it could be faked somewhat directly by rendering. Or you could transmit environment maps and blend between those. Just remember the idea is to be orders of magnitude more efficient than rendering everything normally.
I thought the windows MR lineup filled that gap pretty well. Much cheaper than most of the other alternatives back then but it never really took off and MS has quietly dropped it.
Still $300 or $400 for a wonky platform. That’s priced better than I thought they were, but the minimum viable product is far below that, and we might need a minimal product, to improve adoption rates. The strictly necessary components could total tens of dollars… off the shelf.
I mean a wireless wearable computer would have network lag but the important lag – between the rendering GPU and the screen – would be nil.
I mean streaming the video data to the headset Miracast-style would be dumb, I agree, but afaik all the rendering hardware in the new Apple headset is inside the headset.
The new Steam Deck and its copycats have proven that a compact gaming-grade PC is doable. If not fully contained in the headset, then make a little fannypack with a cable up to vr headset, but you can walk around untethered otherwise.
He’s right, a product like that would have failed dramatically. At this point I just want them to release a dumb AF, streaming-only, inside-out tracking VR headset that connects to PCs. Forget trying to cram an expensive Qualcomm or AMD chip in there, it will never give you the ideal VR experience. Make something that’s $200 bucks, connects to any PC running SteamVR, and just does extremely well with streaming and low-latency. Both Airlink and VRDesktop have already shown that its possible to get extremely close to a cabled experience. All that’s left is some polish.
I just want to be able to buy something like an Oculus CV1 without Oculus software/proprietary hardware and a nicer screen. I’m still rocking the same unit I got several years ago and it’s still plenty fine for most things.
All of the fancy things like wireless and no-tower tracking are nice, but I imagine a lot of players are going to be seated and just want the immersion. Why not have a $300-400 offering that does this?
I could be wrong on this since I have no source but I always assumed that Oculus headsets were cheap because they’re a Meta product and you’re actually paying for it with your data/telemetry.
Like, the ungodly amount they spent on VR R&D is absolutely not being made up for by the few hundred dollar price tag on their headsets — I bet that barely covers the cost of materials. That must be for a reason.
I think having base stations not only increases price but also makes it unapproachable for a vast majority of people. Personally, I didn’t even consider the CV1 or the Index because I just didn’t have a room that could properly accommodate them. For the sitting use case, no-tower tracking is actually very suitable and probably works better.
even at ideal condition there is about 1~2ms latency(streaming 1080p game), while hitting 90hz requires 11ms frame time. so you are asking the game to at least perform at 111fps or above to function under said ideal condition. I think if some manufacturer can put together a chip set where they do the frame gen tech on the head set side, so the game just need to run at 60fps it would be a better option.(like PS VR ) Frame gen does require some other buffers to generate the in between frames, so that’s more info to stream over the bandwidth.
That’s basically what I got. Xreal Air (formerly Nreal until a C&D from Epic). 1080p per eye and something like 49PPD with a 45° FOV. Tracking is 6DOF and requires software on the host (only complaint) and connectivity is via a USB-C cable (uses DP alt mode).
It’s nearly as “dumb” as an HMD can get. From the teardowns that I’ve seen, it’s really just got an MCU, a GPIO expander, a 6DOF chip, and the displays + drivers. And I love that about it. No batteries or anything to worry about.
It is pretty narrow but also what makes it work, IMO. I don’t have them for immersion but for display replacement. The narrow field of view lets the 1080p display have nearly 0 screen door effect. Plus, the birdbath optics are really cheap compared to waveguides or fancy lenses in VR headsets.
This has been on my radar for a while to compliment my steam deck. But I believe it doesn’t do head tracking with the steam deck or does it? I just want a floating screen in front of me that stays still when moving my head around, otherwise I’m gonna hurl!
I think it’s great for my Deck but, that will indeed be a problem. The headset contains only the sensors and display systems but, none of the logic circuitry to “pin” displays. Including that would increase the price a good deal.
Understood thanks for the feedback. After posting you reignited my interest and I found out that they also have their product called beam which would do the trick to make a spatial display… if you’re willing to cough up another 120 for it!
Yeah… I’m not :P But, I am plotting a DIY solution. A solution that will probably cost more than $120 on components but, I think it will still be worth it.
So, I’ve got one for my steam deck and it’s less an issue than you might think, in my opinion.
When you’re focused on the screen, it doesn’t create too much incongruity when the background shifts, and it’s easy to just let you brain parse the screen as something that just floats in front of you.
It’s not immersive enough to get the inner ear involved and confused. It’s a lot closer to holding a phone sideways about six inches from your face and moving your head around.
The only time it felt weird was when I was using it in a well lit room, and I shifted my focus to something not on the screen, that was closer than the apparent distance to the floating display. It was weird feeling my vision try to reconcile that the nearer thing was moving behind the far thing.
Oh, you mean the problem they refused to acknowledge that was very common for a number of years? I wish I could get a refund or replacement on my 3 OFFICIAL NINTENDO controllers that suffer from drift. But alas, I don’t live where they do business “legally”
I hear you can get a replacement digital stick on amazon for the existing switch consoles. If the white ones that came with my oled switch ever get that drifting problem, I’m gonna buy one of those replacement stick components and send it and the drifting joycon to a tech repair shop I know of (that guy might have repaired like 200 of those already, pretty much everyone in my area these days owns a Switch). I don’t wanna risk shorting out a $70 controller doing it by myself
I think it’s saying a patent about hall effect sensors specifically relating to Nintendo and Switch style implementations. Obviously it’s not for the concept at large that’s been around for decades.
I loved my Dualsense too, and then the left stick started drifting so badly, it’s completely unusable now. It’s only about a year old, too. I blame Sekiro. Both my DS4s still work fine though, and they’ve seen much more use and abuse.
True, but you still do a lot of moving around with the left stick. And when you’re stressed out about imminent death at any moment, that can be hard on the sticks.
I loved Sekiro! My first time through the game, I probably died on that first miniboss a hundred times. On NG+, I got to and killed Lady Butterfly without dying once. What an amazing game. I should probably go back and finish up NG+ once Elden Ring lets go of me.
Ps5 controller was just as bad… I’m on my 3rd now and most of my friends are on their 2nd. I also had 1 switch controller go bad as well, but I also don’t play switch as much. This entire generation had the best controllers but also the worst problems I have ever had. Prior to these 2 systems I have never had a controller break before and I’m going back to original NES days.
I was talking about the PS5 controller. My DS4s (the PS4 controller) are holding up much better. At least the internals. The rubber on the sticks wore off, and I had to replace the tops. That was much easier than the 14 contacts-per-stick I have to de-and-re-solder on the Dualsense (PS5 controller) when I work up the courage to try that.
I loved my Dualsense too, and then the left stick started drifting so badly, it’s completely unusable now. It’s only about a year old, too
I really think that something changed with a major potentiometer manufacturer in the past few years. I don't recall stick drift on a PS2 controller that I used for many years, but I've seen it on a number of controllers from different vendors recently.
Only thing I can think of other than recent hardware problems is that maybe the controller hardware imposed a certain amount of deadzone at one point in time and stopped doing so in newer gamepads, and that masked the drift.
I really think that something changed with a major potentiometer manufacturer in the past few years.
I’ve heard a lot of hearsay that that is the case. Tech savvy people have taken apart some sticks and say that analog stick quality has taken a nosedive in recent years. Maybe it is just the effect of this sort of thing being discussed on the Internet more often, but I don’t doubt the veracity. I’ve had a few older controllers that I retired because of external wear whose internals were totally fine. Seems like controllers like Dualsense and particularly Switch Joycons are just poorly made.
None of my PS4, 5 or Switch controllers have had any drift. I even used the Joycons in Ringfit for ages, and I was sure that spending months being strapped to my leg would bugger it up.
I’m not sure if I’m the luckiest person on Earth, I just don’t use them enough, or others are doing something I’m not (smoking or vaping are possibilities here, along with greasy food fingers).
Yeah?!?! You think so huh? I’m holding my breath on this one and keeping my interests in other game pads like ASUS ROG one, or the one that Logitech is making
I fixed my drift with a small piece of cardboard. I figure Nintendo could have eliminated some drift by increasing the material thickness in the cad file they use. They just choose not to.
I lost four sets of Joycon to drift. I even sent two in to be “repaired”. Talking to support was worthless - I’m convinced that the people I spoke to had never seen or used a Switch before. I don’t think they did anything other than calibrate them and send them back. I ended up buying a 20 pair of knockoff Joycon that have worked perfectly ever since.
My old Amazon order says Vivefox, but they don’t seem to be for sale anymore. I’m pretty sure at the time there were a bunch of companies offering the same ones, like a Chinese dropship kind of thing.
repairing joycon drift is super easy. ifixit.com and the joystick modules are 5 bucks on amazon. People need to lose this fear over opening and repairing their own electronics.
Cause i’m never gonna let them fix it themselves after they lost one of my ($40) joycons then threw their hands up and said it was my fault. Plus it takes them 3 weeks to do it anyway which is a long time without the games console I paid for
I installed these on my wife’s joycons because she mashes the sticks like Gail the Snail. So far no issues with drift! The kit comes with all the tools you need to do the swap so it’s very straightforward.
I still wonder what was so special about my N64 joysticks that I never experienced drifting. They’d recalibrate every time you turned the console on (or held some key combination) and after that were golden.
Yeah, except they were also so horribly designed that normal use literally grinds away the plastic at the base of the stick until it starts flopping around like a wet noodle.
The N64 used optical sensors in its joysticks. If you take apart the N64 joystick you’ll see the joystick is attached to some disks with slits in them. The N64 had an optical sensor that would count how many slits passed by.
dexerto.com
Gorące