Earlier this week, Bloomberg Japan's report on a rumored Nintendo Switch "Pro" version exploded with a heavy-duty allegation: all those rumors about a "4K" Switch might indeed be true after all. The latest report on Tuesday teased a vague bump in specs like clock speed and memory, which could make the Switch run better... but jumping all the way to 4K resolution would need a massive bump from the 2016 system's current specs.
What made the report so interesting was that it had a technical answer to that seemingly impossible rendering challenge. Nvidia, Nintendo's exclusive SoC provider for existing Switch models, will remain on board for this refreshed model, Bloomberg said, and that contribution will include the tantalizing, Nvidia-exclusive "upscaling" technology known as Deep Learning Super Sampling (DLSS).
Since that report went live, I've done some thinking, and I can't shake a certain feeling. Nvidia has a much bigger plan for the future of average users' computing than they've publicly let on.
Making smoother gaming—arguably easier than Tom Cruise’s face
Here's an example of the latest build of Minecraft with ray tracing and DLSS upscaling to 4K enabled. This test ran at roughly 32 fps; with the RTX 3060, you can enjoy similarly clean DLSS upscales to 1440p with ray tracing on and enjoy performance closer to 60 fps.
DLSS upscaling from roughly 906p to 1440p in Watch Dogs: Legion's "quality" upscaling mode.
Watch Dogs: Legion at native 1440p with temporal anti-aliasing (TAA) applied.
DLSS upscaling from roughly 906p to 1440p in Watch Dogs Legion's "quality" upscaling mode.
Watch Dogs Legion at native 1440p with temporal anti-aliasing (TAA) applied.
DLSS upscaling from roughly 906p to 1440p in Control's "quality" upscaling mode.
Control at native 1440p with temporal anti-aliasing (TAA) applied.
DLSS, for the uninitiated, is a long-in-development system that relies on thousands of hours of existing video footage, largely taken from 3D games, that's been run through a machine-learning model. When you hear about "AI-trained" systems that, say, play video games or generate wacky names, they're usually working in the same way in order to reliably detect successful patterns. They study and duplicate existing data (dictionaries, Twitter conversations, Tom Cruise's face, videos of StarCraft competitions), then they go through double-, triple-, and sextuple-checking runs to confirm how well a computer can organically recreate that data before loosing their systems into real-world tests.
But those tests are usually run with a supercomputer managing those troves of data. This is where DLSS pivots sharply. As a consumer-grade offering, DLSS requires Nvidia's "RTX" graphics cards that (theoretically) cost as little as $329—and nothing else. No extra CPU, RAM, or other peripherals required. The idea being, RTX cards come with a slab of "tensor cores" baked onto the silicon, and these are dedicated to the mathematical grunt work of tasks like image interpretation and translation.
This works in part because DLSS is focused on a relatively nonvolatile prediction scenario: what a lower-resolution image would look like if it had more pixel depth. The shape of diagonal lines, the leaves on swaying tree branches, and even the letters and words on a street sign are all arguably easier to predict—at least when fueled by a lower-pixel base—than a StarCraft player's reaction to a Zerg rush. While DLSS's initial launch left a few things to be desired, the system has matured to a point where moving 3D images with DLSS processing generally looks better than their higher-resolution, temporal anti-aliasing (TAA) counterparts.
Side-by-side comparisons of lower-res DLSS and higher-res TAA show strengths and weaknesses, but as of 2021, it's become a wash. That wash has substantially higher frame rates and lower processing requirements for the DLSS side, since the traditional GPU grunt work is not only pixel-bound but also texture-bound. DLSS's savvy on upscaling doesn't just reduce needs for clock speed and processing cores; it also translates lower-resolution textures with more fidelity, so you can arguably hop, skip, and jump past high VRAM requirements.
Calling BS on another BS Satellaview
Nintendo Switch doesn't have exclusive license on lower amounts of VRAM, lower clock speeds, and rendering scenarios with limited die sizes and thermal envelopes to work with. Where else might those all be useful? Hmm.
The Nintendo Switch is a genius platform as a starting point, especially since we've said the same thing time and time again: "%GAMENAME% is amazing to play in Switch's portable mode, where it gets close to 720p resolution, but blowing the same action up on your 1080p TV looks like trash." You, the savvy gamer, shouldn't seriously consider using your Switch for TV play, in spite of stats showing tons of people craving a good TV-gaming experience (and an ever-expanding market in which more people than ever are rushing to buy Xboxes, PlayStations, and Switches for their TVs).
We’re 22 years past Nintendo’s last stab at that concept.
A DLSS-equipped "Switch Pro" (not a final name), as described by Bloomberg, would reach resolutions of up to 4K when docked and connected to a television. Assuming that DLSS in this scenario requires dedicated tensor cores, we're currently left wondering where Nvidia will slap said cores. Are we looking at a "split-motherboard" scenario, as we've seen on both Xbox Series X and PlayStation 5, where the TV-plugged dock takes on certain heat-generating elements like a block of tensor cores? Or will the Switch Pro's base hardware include tensor cores on its own SoC?
My completely speculative guess is the latter for a couple of reasons. First, the rumor of higher CPU clocks and more RAM on the base unit suggests that Nintendo plans to roll out an entirely new system, as opposed to add-on hardware that would boost the base. Nintendo has experience shipping add-on hardware for its home consoles in Japan—like the Famicom Disk System (which added more power and extra sound channels), the BS Satellaview (which added RAM), and the 64DD (which added a co-processor).
But we're 22 years past Nintendo's last stab at that concept, and after the mass confusion caused by the Wii U, I don't see modern-day Nintendo wading into the consumer confusion of questions like, "can I put an older Switch into a Switch Pro dock for more power?"
PSA on ISA
Second, and more importantly, Nvidia has a vested interest in proving that DLSS can ship and function on a portable device. We don't know whether Nintendo will limit DLSS as a "docked only" mode on Switch Pro, and Bloomberg's sources point to its portable panel sticking to 720p resolution, albeit at a larger size (7" compared to 6.2") and higher build quality (OLED over LCD).
Even if this is the case, Switch's highest-end games currently struggle to render at a native 720p resolution in portable mode, and they certainly drain its battery life. DLSS has yet to prove as convincing in RTX GPU tests on PC when they upscale from sub-720p resolutions, but Nvidia has been bullish about its upscaling ambitions. That's especially true for Nvidia with the RTX 3090's DLSS mode taking 1440p signals and ramping them all the way up to 8K resolution (as in, a 9x leap).
If Switch Pro can stably upscale a 540p signal (a common pixel count for the current system's portable-mode games) to 720p via DLSS, the results may be a win-win: smoother, sharper images and a reduced burden for Switch Pro's battery. Toggling those tensor cores for DLSS isn't entirely free in terms of power and system bandwidth, but it's definitely cheaper than raw rendering.
A new Nintendo Switch, on the other hand, is only the beginning of Nvidia's potential ambitions.
First up, you may recall that Nvidia is barreling toward a megaton acquisition of Arm, with those negotiations beginning in September (and since then facing serious scrutiny). Currently, Arm's value comes from its IP, as opposed to owning a chip-production line; chip makers like Qualcomm, Samsung, and (formerly) Apple take Arm's reference material, which is efficient and powerful enough to be worth their while, and go from there.
As my colleague Jim Salter wrote in February:
Owning Arm outright would allow Nvidia much greater leeway to innovate upon the design. We've spoken to several vendors who described the sort of innovation that the RISC-V architecture allows as effectively impossible with Arm; the vendors have said "they just won't let you" make changes, like adjustments to the instruction set.
If Nvidia owns Arm, it will no longer face such restrictions. This could make it easier for the GPU manufacturer to innovate on the CPU side and free it from the necessary alliance it has had with Intel in the manufacture of world-leading supercomputers... which likely looks increasingly important, since Intel is encroaching deeper and deeper into Nvidia's territory with its Xe GPUs and its oneAPI initiative, which aims to decouple GPU-based machine from the underlying GPU hardware.
So far, so good—and in fact, Nvidia describes its acquisition in just such terms, referring to it as "creating [the] world's premier computing company for the age of AI." But it also potentially offers Nvidia leverage over the entire mobile computing industry, which is currently wedded to the Arm instruction set architecture (ISA).
You guys all have phones, right?
Nvidia's interest in Arm arguably revolves more around ditching Intel as a necessary partner in certain industries. But taking a mobile-processing lead in smoother, more battery-efficient graphics wouldn't be a bad side dish.
Controlling Arm would give Nvidia a radical amount of wiggle room to institute paradigm-changing systems for things like graphics rendering in a baseline smartphone spec. And that's interesting when you look at how many people in the world, particularly in emerging markets, favor smartphones and portable devices for 3D-intensive software like video games, as opposed to consoles or home computers. (Currently, many of the world's top gaming moneymakers are all on mobile storefronts.)
What if an Nvidia sticker on a smartphone means longer Fortnite sessions on the go?
Maybe Nvidia simply takes Arm's ISA to its own factories and gets to work mass-producing a line of Nvidia-branded smartphones, all with DLSS built in on the ground floor. Or maybe Nvidia makes DLSS available as an instruction set to Arm's existing partners. Either way, that's a path to ensuring that Nvidia's proprietary DLSS systems land on more smartphones, which could thus guarantee better performance and better battery life. Unreal Engine 4, which runs a ton of high-earning software on mobile storefronts, has recently added baseline support for DLSS; if an Nvidia sticker on a smartphone means longer Fortnite sessions on the go before having to recharge batteries, that's no small sales pitch. (And it'd certainly be a jolt to rival engines like Unity, which have yet to adopt DLSS as widely.)
That's not Nvidia's only reason to add robust pixel upscaling to mobile hardware. What if someone wants to play higher-end games, maybe designed with ray tracing and other GPU-busting effects, on a game-streaming service like Google Stadia or Microsoft's Project xCloud... but their Internet access is spotty? What if a cloud service could pump out resolutions in the 480p-560p range and get natively upscaled somewhere closer to 1080p?
And doesn't Nvidia have its own burgeoning game-streaming service, GeForce Now, which has aggressively advertised support for more regions across the world than its rivals?A 2018 gamble, finally bearing mobile fruit
DLSS was one of Nvidia's big bets after its mid-2010s GPUs began soundly crushing AMD in every metric imaginable. In 2018, I wrote that Nvidia's RTX 2000 series took advantage of this lead by devoting less attention to sheer rasterization jumps. By focusing on entirely new processing cores in its GPUs, Nvidia seemed to declare that AMD wouldn't catch up in other metrics, anyway, so why not do something funky with that lead?
Sure enough, AMD needed two generations to nail its sales pitch of sheer power, and depending on the software in question, AMD's Big Navi products can claim big wins. But that took long enough for developers to see and embrace Nvidia's progress on its proprietary RTX systems, and so far, the refrain in my own coverage has been: when DLSS is in the equation, Nvidia wins each showdown. Last year, AMD announced something called "FidelityFX Super Resolution," but we're still waiting to hear exactly how it works, what hardware it requires, and whether it will scale to, say, portable hardware.
Should DLSS indeed launch as an upscaling method for Switch Pro, there's zero reason to believe Nvidia will stop there. Its aspirations as a player in the mobile space have been laid bare—and this Switch Pro leak could be a serious headache as Nvidia strives to complete the Arm acquisition, which already faces heavy scrutiny. Should Nvidia seal up that deal, however, prepare for much more mobile DLSS news to come.
https://ift.tt/3cpaWDN
Technology
Bagikan Berita Ini
0 Response to "How a “Switch Pro” leak may point to Nvidia’s megaton mobile-gaming plans - Ars Technica"
Post a Comment