Reflection probes

[Bree Giffen]

So I'm reading the blog posts and I see that one of the improvements will be reflection probes. Does this mean we will have working mirrors in SL? The vanity here is amazing and I think this will be a big hit. How does reflection probing work exactly? 

[Fluffy Sharkfin]

2 hours ago, Bree Giffen said:

How does reflection probing work exactly?

Essentially you place probes around your build (the viewer also places some default ones) and the probe automatically generates an environment map based on all the stuff within its boundaries (similar to how the 360 snapshot feature works but with a limited draw distance).  Anything reflective within the area of a probe will use the environment map generated by that probe for its reflections.

[Qie Niangao]

3 hours ago, Bree Giffen said:

So I'm reading the blog posts and I see that one of the improvements will be reflection probes. Does this mean we will have working mirrors in SL? The vanity here is amazing and I think this will be a big hit. How does reflection probing work exactly? 

Apparently reflection probes do not enable planar mirrors. This is at an implementation level I've never studied, but I remember reading somewhere that the folks developing reflection probes weren't really expecting somebody in management to promise mirrors, too, so they're trying to figure out how to make it happen Real Soon Now—but not on the imminent reflection probe timeline. Here's what Inara Pey reported from a recent Content Creation User Group:

Quote

Mirrors (in some capacity) are described as the “very next thing” the graphics team will commence work on after the PBR / reflection probes work gets to RC status (as Vir Linden pointed out, Mirrors currently require more reflection on how they are to be implemented!).

 

[Conifer Dada]

While we don't need mirrors in SL for practical reasons, such as checking our appearance, working flat mirrors would definitely add to the realism of the world.

[Fluffy Sharkfin]

Naturally mirrors are the first thing that springs to most peoples mind when they hear the term "reflection probes" but aside from the more obvious and visually striking uses like highly reflective surfaces reflection probes should also go a long way to increasing the overall visual quality of Second Life.  Like materials they're another step away from the decades old practice of baking all your lighting and reflections, etc. into a single colour texture and slapping it on some polygons which, along with forward rendering and bad lighting, is one of the main reasons SL often looks so dated.

Hopefully once they're introduced we'll finally be able to bid farewell to one of the few remaining excuses people still use for painting static reflections into their base textures because "shiny" won't equal "ugly" in SL anymore.

[Ceka Cianci]

7 hours ago, Bree Giffen said:

So I'm reading the blog posts and I see that one of the improvements will be reflection probes. Does this mean we will have working mirrors in SL? The vanity here is amazing and I think this will be a big hit. How does reflection probing work exactly? 

This was the last thing I heard on mirrors, which was 2 months ago.. If the video starts at the beginning , go to 19:35

 

Edited January 6, 2023 by Ceka Cianci

[Lewis Luminos]

I wonder what impact this will have on framerate and lag? I know my framerate takes a big hit when I raise the reflections on Linden water.

[wesleytron]

18 minutes ago, Lewis Luminos said:

I wonder what impact this will have on framerate and lag?

Probably 'some'.

[Conifer Dada]

4 hours ago, Fluffy Sharkfin said:

Naturally mirrors are the first thing that springs to most peoples mind when they hear the term "reflection probes" but aside from the more obvious and visually striking uses like highly reflective surfaces reflection probes should also go a long way to increasing the overall visual quality of Second Life.  Like materials they're another step away from the decades old practice of baking all your lighting and reflections, etc. into a single colour texture and slapping it on some polygons which, along with forward rendering and bad lighting, is one of the main reasons SL often looks so dated.

Hopefully once they're introduced we'll finally be able to bid farewell to one of the few remaining excuses people still use for painting static reflections into their base textures because "shiny" won't equal "ugly" in SL anymore.

I imagine textures will still benefit from some gentle painted-on highlighting and shading, especially where they are used for surfaces with lower levels of reflectivity.

[Fluffy Sharkfin]

56 minutes ago, Conifer Dada said:

I imagine textures will still benefit from some gentle painted-on highlighting and shading, especially where they are used for surfaces with lower levels of reflectivity.

Nope, not even the most miniscule smidgeon of a highlight or shadow!

If you're using PBR materials (which you absolutely should be by the time reflection probes are implemented) then your base colour/albedo map should contain no information other than the colour of the objects surface.  Your roughness, metalness and ambient occlusion maps alongside the reflection probes and dynamic in-world lighting should be left to take care of all of the rest since it can generate all those highlights, shadows and reflections dynamically based on light and camera angle rather than having them painted onto a static texture.

ETA: The above applies to more photorealistic objects, if you're aiming for a more stylized aesthetic then adding fake lighting is almost mandatory since SL doesn't have the custom shaders necessary to simulate the required "cartoony" effect.

Edited January 6, 2023 by Fluffy Sharkfin

[BillFletcher]

7 hours ago, Conifer Dada said:

While we don't need mirrors in SL for practical reasons, such as checking our appearance, working flat mirrors would definitely add to the realism of the world.

Of course if you were adjusting the position of a hat. So the head doesn't poke out anywhere. Being able to see the front and also the back in mirror, would be useful.

[EliseAnne85]

1 hour ago, Fluffy Sharkfin said:

then your base colour/albedo map should contain no information other than the colour of the objects surface.  Your roughness, metalness and ambient occlusion maps 

Fluffy, I have a question just to clarify something if you are around or perhaps someone else would know.  Wouldn't you mean the albedo should have no more information than color and pattern?  For example, let me take freckles on a skin and call the freckles a pattern.  If the freckles and the skin color don't go on the albedo, then where would the freckles for the skin go?  

[Fluffy Sharkfin]

28 minutes ago, EliseAnne85 said:

Fluffy, I have a question just to clarify something if you are around or perhaps someone else would know.  Wouldn't you mean the albedo should have no more information than color and pattern?  For example, let me take freckles on a skin and call the freckles a pattern.  If the freckles and the skin color don't go on the albedo, then where would the freckles for the skin go?  

Technically the "pattern" you're referring to is just a combination of colours so it's essentially the same thing.  For example a leopards skin texture consists of a pale cream/yellow colour on the belly ranging to a browny orange colour on the back with black spots of varying strength on different parts of the body and limbs, so all of those colours would appear on the albedo map.

[Qie Niangao]

Somebody will surely fret that their potato PC runs slower with Advanced Lighting enabled/mandatory in order for the PBR materials to appear. Won't creators bake lighting for as long as viewers exist that include a non-ALM pipeline?

(I suspect that viewers run faster with ALM now than they did without ALM a year ago, but comparing ALM and non-ALM on the same viewer generation will perpetuate anti-ALM bias.)

[EliseAnne85]

1 hour ago, Fluffy Sharkfin said:

Technically the "pattern" you're referring to is just a combination of colours so it's essentially the same thing.  For example a leopards skin texture consists of a pale cream/yellow colour on the belly ranging to a browny orange colour on the back with black spots of varying strength on different parts of the body and limbs, so all of those colours would appear on the albedo map.

I was wondering where the pattern of things would go but I think I gave a poor example because the skin is a different UV system and the freckles could be made a tattoo.  So, I'm still thinking pattern and trying to understand so now I'll use wood grain and bark as a pattern example. 

I'm thinking the wood grain or bark pattern would still need to be baked into the texture and that becomes the albedo, and then the roughness and metalness are trying to bump the grain in the wood or bark into looking more 3D.  IOW, I'm thinking the roughness and metalness are like bump maps but not understanding what these maps want to read if there is no pattern like wood or bark in the albedo itself?  So, this leaves me wondering if the wood grain/bark pattern is in the roughness map? But, let me keep it simple...on which map has the texture of things in our world like wood, bark, crushed velvet, corduroy, etc?  

[EliseAnne85]

41 minutes ago, Qie Niangao said:

Won't creators bake lighting for as long as viewers exist that include a non-ALM pipeline?

There already could be a billion dollar pipeline of non-ALM content that exists now, so I'm not sure what you are saying.  Do you think all the content that exists now for potatoes, as you call them, should be retired?  My computer may need some upgrades for ALM because ALM on part time has me cleaning my fans in my desktop once a month at least.  However, my computer is  in no way a potato (whatever that means) except for SL.  I could run this computer probably another 10 years or longer without SL because what else do I use it for - YouTube aside from reading news.  I don't need an upgrade for YouTube and I don't know how many gamers will like SL.  People switching to gaming computers needs a lot more than just SL.  LL could stream (cloud) SL for a monthly price, too.  

Edited January 6, 2023 by EliseAnne85

[Qie Niangao]

6 minutes ago, EliseAnne85 said:

There already could be a billion dollar pipeline of non-ALM content that exists now, so I'm not sure what you are saying. 

And that will all still work. But as I understand it, creators will soon have a choice:

• use PBR materials that depend on ALM and eschew baked lighting completely so they look cartoonish on non-ALM machines (but gorgeous otherwise, and generate better results with lower resolution textures), or
• continue to bake lighting for the non-ALM crowd, and use materials ineffectively if at all.

The more creators try to straddle that fence, the more their content will appear over-baked on viewers set to view PBR in all its glory. (This happens now, too, to a lesser degree, with ALM and non-PBR materials, especially with shadows enabled.)

The thing is, the rendering code is (supposed to be) so much faster now than a year ago that turning on ALM should only revert to that pre-improvement performance—unless other changes were made; it's very possible that folks raised their draw distance, for example, as the performance improvement made that less painful, so now turning on ALM really does slow things down.

[bigmoe Whitfield]

7 hours ago, Lewis Luminos said:

I wonder what impact this will have on framerate and lag? I know my framerate takes a big hit when I raise the reflections on Linden water.

Shadows has a big impact along with draw distance,  I leave them off,  I range from 50 to 90fps in most places,  in crowds I'm ranging about 25 fps.    if I turn those on...  4 fps.  my 980 hates it's life when I do that.

[Maitimo]

7 hours ago, Lewis Luminos said:

I wonder what impact this will have on framerate and lag? I know my framerate takes a big hit when I raise the reflections on Linden water.

15 minutes ago, bigmoe Whitfield said:

Shadows has a big impact along with draw distance,  I leave them off,  I range from 50 to 90fps in most places,  in crowds I'm ranging about 25 fps.    if I turn those on...  4 fps.  my 980 hates it's life when I do that.

I find shadows has a worse impact on my framerate than water reflection does, and presumably, since reflective surfaces are going to be smaller than the Linden Water ocean extended to your draw distance, I am assuming that the impact of this feature will be somewhat less than either. 

[EliseAnne85]

26 minutes ago, Qie Niangao said:

And that will all still work. But as I understand it, creators will soon have a choice:

• use PBR materials that depend on ALM and eschew baked lighting completely so they look cartoonish on non-ALM machines (but gorgeous otherwise, and generate better results with lower resolution textures), or
• continue to bake lighting for the non-ALM crowd, and use materials ineffectively if at all.

The more creators try to straddle that fence, the more their content will appear over-baked on viewers set to view PBR in all its glory. (This happens now, too, to a lesser degree, with ALM and non-PBR materials, especially with shadows enabled.)

The thing is, the rendering code is (supposed to be) so much faster now than a year ago that turning on ALM should only revert to that pre-improvement performance—unless other changes were made; it's very possible that folks raised their draw distance, for example, as the performance improvement made that less painful, so now turning on ALM really does slow things down.

All this sounds like is a big trade-off bit of chaos to me.  I'd rather LL look into offering cloud streaming PBR viewer so people can see if they like it at least before spending money on a gaming computer they may only use for SL because YouTube doesn't need the upgrades in order to work, that's for sure.  I don't game.  I only need upgrades for one thing - SL, otherwise, my computer could last 10 years or more.  

Edited January 6, 2023 by EliseAnne85

[Qie Niangao]

I kinda thought there was already a third-party server-side-rendered SL streaming service. That was an LL offering too, years ago (perhaps you recall) although it got little up-take because the service was very expensive back then and adequate network bandwidth was as great a market barrier as were graphics-capable machines. (At one point, I thought they might do this for the new mobile client but I gather that's not the plan now.)

Anyway, I agree there's a big trade-off ahead—at least for creators, unless the non-ALM pipeline is deprecated so they can adopt the "pure" PBR approach without excluding some of the market.

[arton Rotaru]

3 minutes ago, Qie Niangao said:

unless the non-ALM pipeline is deprecated

The forward renderer is no longer a thing in the PBR viewer. There is no choice to switch ALM (deferred renderer) off. What can be done is to turn reflections off, which will kill the reflection probes, and falls back to the current (old) environment map.

[arton Rotaru]

2 hours ago, EliseAnne85 said:

I was wondering where the pattern of things would go but I think I gave a poor example because the skin is a different UV system and the freckles could be made a tattoo.  So, I'm still thinking pattern and trying to understand so now I'll use wood grain and bark as a pattern example. 

I'm thinking the wood grain or bark pattern would still need to be baked into the texture and that becomes the albedo, and then the roughness and metalness are trying to bump the grain in the wood or bark into looking more 3D.  IOW, I'm thinking the roughness and metalness are like bump maps but not understanding what these maps want to read if there is no pattern like wood or bark in the albedo itself?  So, this leaves me wondering if the wood grain/bark pattern is in the roughness map? But, let me keep it simple...on which map has the texture of things in our world like wood, bark, crushed velvet, corduroy, etc?  

The bumpiness will be in the normal map, as always.
A PBR base color map won't have large scale ambient occlusion, or any kind of baked in specular reflections.
Micro scale AO is rather OK, though.

There will definitely be a period where we have to get used to, to the PBR look. At first I also felt the need to put "some" AO into the base color map, because it was the look I was familiar with, the SL look. Same with metals, I rather hesitated to set metals in the metallic map to pure white (metal) because of the darkening the PBR shader does on metals.

But the longer I worked with the PBR materials, the more and more I noticed how "wrong" these objects actually looked. So over time, I removed all of that AO from the base color, and let metals be metals by setting them to pure white in the metallic map. The look and feel is much more natural and lifelike once you get used to the look. But, it takes some time to get there.

Edited January 6, 2023 by arton Rotaru

[EliseAnne85]

44 minutes ago, arton Rotaru said:

The bumpiness will be in the normal map, as always.
A PBR base color map won't have large scale ambient occlusion, or any kind of baked in specular reflections.
Micro scale AO is rather OK, though.

There will definitely be a period where we have to get used to, to the PBR look. At first I also felt the need to put "some" AO into the base color map, because it was the look I was familiar with, the SL look. Same with metals, I rather hesitated to set metals in the metallic map to pure white (metal) because of the darkening the PBR shader does on metals.

But the longer I worked with the PBR materials, the more and more I noticed how "wrong" these objects actually looked. So over time, I removed all of that AO from the base color, and let metals be metals by setting them to pure white in the metallic map. The look and feel is much more natural and lifelike once you get used to the look. But, it takes some time to get there.

I'm trying to understand where the pattern of objects goes or will come from.  Patterns/textures of our world like the grain of wood, tree bark, corduroy fabric for some examples as Fluffy said the albedo is just color.  So, I'm assuming albedo is like a tinter or our inworld color palette because she said it's just color.  I'm just lost as to where texture/pattern of items would come from. So, I asked in my post above does the pattern come from the roughness map then?  

See photo I will place below because what I was asking for clarity on is the albedo actually color AND pattern.  See how the shirts of the PBR picture below have a pattern.  Fabrics have patterns, so to speak, and I was wondering how one does that in PBR.  Where does one create texture/pattern with PBR - which map area?  I was assuming with PBR we would create pattern/texture and color in the albedo.  So, I'm lost here.  

[arton Rotaru]

Such patterns will be in the base color (lets call it base color instead of albedo, because that's what the glTF spec is calling it) map as well. Ideally they will be in the normal map and, the ambient occlusion map (which is a standalone map in PBR) as well.