How realistic should avatars be?

[animats]

Here's an essay on that topic. Yes, those people are trying to sell their avatar system. But what's worth noting is that they went all the way to working photorealism and backed off. They discuss why.

This is relevant to Second  Life, where the avatars approach photorealistic, but don't quite get there. This puts them in the uncanny valley. With face tracking on the official SL roadmap for 2022, this becomes more relevant. If we have face tracking, what should avatars look like?

The uncanny valley.

The Polar Express movie (2004) was where Hollywood bottomed out on this scale. Facial motion capture was used in a mainstream movie for the first time. The result was considered creepy. The overall look was weird - too realistic to be a cartoon, not realistic enough to be live action.

The lesson was that once you get deep into the uncanny valley, you have to up your game in all areas to get out. The rendering has to get better. Especially for skin, where you need subsurface scattering, the effect that makes skin glow slightly. Motion has to look right.

Hollywood has managed, at great expense, to get out of the uncanny valley. Games are getting there. (See the Unreal Engine 5 Matrix demo.) SL is part-way into the valley. A question is whether facial motion capture moves it deeper down or starts to go up the other side to photorealism.

I'm not against motion tracking. I think it would force LL to up their game in other areas as well, though.

Comments?

 

[Fritigern Gothly]

I am quite pleased with the level of realism my avie has, particularly in the facial region. 
Would I go for more realism if I could? Maybe. 

[Rowan Amore]

I don't think face tracking is something SL needs nor something they should even spend time or money implementing.  Are they trying to attract companies again to host meetings here?  That's the only application where it might be useful.  The majority of people are never looking directly at an avatar who is speaking so I just don't see the.point.  Personally, when I want that much realism, I log out of SL.

[colleen Criss]

If I really wanted realisms I would not log in at all.

[Wulfie Reanimator]

I don't think the majority of SL is striving to have photo-realistic avatars, though obviously we've anyway reached the uncanny valley a long time ago. I don't think that's a super high bar to reach.

Face tracking is a big meh, it's not going to have much general use since avatar faces are so small on everyone's screens unless you're specifically focusing on the person using face tracking. But if we actually read the SL roadmap, it says:

Quote

Avatar “expressiveness” that brings camera-based gestures and movement to your avatar for a whole new level of interaction and connectedness

If this were to include full-body tracking capability (or at least upper body -- more than just facial expressions), that'd be neat.

If this were to pave the way for dynamic animations in general (with scripting support), that'd be even more neat. But now I'm dreaming 5+ years ahead.

Edited February 11, 2022 by Wulfie Reanimator

[ChinRey]

31 minutes ago, Wulfie Reanimator said:

I don't think the majority of SL is striving to have photo-realistic avatars

Yes, the SL trend seems to be more about Photoshop-realistic.

[Istelathis]

I don't experience the uncanny valley very often, so it is difficult to say.  I do like the exaggerated expressions often seen in cartoons and think they can look nice in games like World of Warcraft especially their cinematics.  With SL and more realistic avatars, it may be more difficult to see the expressions unless they are exaggerated and while that would be hilarious, it would not look as great for someone trying to get as realistic of a look as possible.  

I think face tracking is a neat option, it would give vloggers more charm to their videos, and in games that use it I thought it cool to see people's expressions.  I don't know how often I would use it myself, since my camera usually has a piece of tape covering it 🙃  I think I might try to find some slider to place over it on my laptop camera, because I do think waving would be pretty nice.  I think I would probably go for more cartoonish avatars when I did use it though.

Edited February 11, 2022 by Istelathis

[Coffee Pancake]

We're neck deep in the uncanny valley.

We have been for years and it's only getting worse.

We don't notice (for the most part) as the constant exposure adjusts how we react to uncanny features.

We do notice when we get a new mesh head, which is why it can take a few days to "get used to it".

New people coming to SL .. oh they notice and they leave and don't look back, even if they can't express specifically what it was that made them decide SL wasn't for them (often only requiring a single session!)

Thankfully video games have been doing an A+ job of breaking gamers brains too so they don't notice it here so much, they just notice all the other jank and leave because the platform is so out of date compared to the competition.

We're the uncanny butt of every metaverse joke going.

 

 

Avatar face tracking .. that's going to be a disaster no one will notice as we don't zoom in on people's faces anymore (because look at tracking means we get yelled at / hit on if we do).

If it stops Philip looking like this...

... probably worth it for that alone TBH.

[Quistess Alpha]

12 hours ago, Rowan Amore said:

Are they trying to attract companies again to host meetings here?  That's the only application where it might be useful.

If they make SL into a medium for v-tubers, that's another "Useful" applicaiton.

[Coffee Pancake]

19 hours ago, Quistess Alpha said:

If they make SL into a medium for v-tubers, that's another "Useful" applicaiton.

SL would be the worst possible option for that use case, even with face tracking.

[Quistess Alpha]

10 minutes ago, Coffee Pancake said:

SL would be the worst possible option for that use case, even with face tracking.

You're not wrong, but ~this is a rather uninformed opinion~ there are a few SL-video makers for whom not having their face look weird and unsynced as they record themselves would be a huge benefit to their content. That's like maybe 5 people though. Not a market I would have LL concentrate on if I were in charge, unless they can make it good enough where it's possible to run events like "hang out with XYZ 'famous' v-tuber". (unlikely)

[Sammy Huntsman]

If you want realism, then go about your real life. 

[Coffee Pancake]

2 hours ago, Quistess Alpha said:

You're not wrong, but ~this is a rather uninformed opinion~ there are a few SL-video makers for whom not having their face look weird and unsynced as they record themselves would be a huge benefit to their content. That's like maybe 5 people though. Not a market I would have LL concentrate on if I were in charge, unless they can make it good enough where it's possible to run events like "hang out with XYZ 'famous' v-tuber". (unlikely)

Oh I know, SL vloggers will be over the moon for it (assuming it's not janky AF) .. it's a "nice to have" but in the grand scheme of things, it's up there with 360 snapshots. 

[Sammy Huntsman]

10 minutes ago, Coffee Pancake said:

Oh I know, SL vloggers will be over the moon for it (assuming it's not janky AF) .. it's a "nice to have" but in the grand scheme of things, it's up there with 360 snapshots. 

Ngl, I don't want LL having access to my Web Cam. 

[Istelathis]

@Sammy HuntsmanI agree, I would not want that either.  I hope that all expressions, and movements that are translated are performed on our own computers rather than having a video feed uploaded to LL's servers.  Not only for our own privacy, but the bandwidth required, as well as any lag for people on slower connections.  I can imagine that leading to all sorts of entertaining, yet inappropriate reactions from people.

[Sammy Huntsman]

32 minutes ago, Istelathis said:

@Sammy HuntsmanI agree, I would not want that either.  I hope that all expressions, and movements that are translated are performed on our own computers rather than having a video feed uploaded to LL's servers.  Not only for our own privacy, but the bandwidth required, as well as any lag for people on slower connections.  I can imagine that leading to all sorts of entertaining, yet inappropriate reactions from people.

Knowing LL, if they did that. They would require people to give SL access to your webcam to play SL. If that were to happen. I would not come on ever again.

[Wulfie Reanimator]

2 hours ago, Sammy Huntsman said:

If you want realism, then go about your real life. 

2 hours ago, Sammy Huntsman said:

Ngl, I don't want LL having access to my Web Cam. 

1 hour ago, Sammy Huntsman said:

Knowing LL, if they did that. They would require people to give SL access to your webcam to play SL. If that were to happen. I would not come on ever again.

I wish I could live as blissfully as you.

 

LL already has "access" to your microphone. And your keyboard. And your computer.

[Bdobson]

I'm pretty content with how my avi looks.

[ChinRey]

On 2/11/2022 at 8:57 AM, animats said:

The one big flaw with this illustration and indeed with the whole uncanny valley concept as it's usually presented, is that it stipulates that uncanny valley is static. Nothing could be further from the truth.

It's different for different people for a start and even for the same person at different times.

But most of all, it's all about context. Uncanny valley was invented by scientists trying to create realistic humanoid robots for RL and it only really applies to that specific field. In a virtual reality it's not about how realistic or unrealistic your avatar looks; it's all about how it fits into the scene as a whole. Put a real life person into a cartoon environment and it's he or she that will trigger the uncanny feeling, not the scenario. So, to those who posted pictures of avatars here: sorry but without the surroundings those pics are simply not relevant.

This is of oucrse a serious, possibly unsolvable, problem for a virtual environment assembled together from umathced pieces with no coherent overall design and in Second Life we have the added complication that even the dominating trends and fashions are different for avatars than they are for scene objects.

[Katherine Heartsong]

To follow on to what @ChinReyhas just suggested, the concept and visual appearance that trigger's someone's uncanny valley response is different for everyone. While I love @animatsposting and knowledge, the suggestion that SL is in the uncanny valley as we generally assume it to mean is wrong, at least for me. I have yet to run into anyone's avatar that looks remotely human-like in game, and yes, I have all my settings cranked way up.

Everyone's dead, doll-like and static, and even the best AOs look so utterly silly and phony to my eyes. It doesn't help that the scale of almost everyone's avatar is off the charts height wise, and hardly anyone knows and uses the proportions of a realistic human being for an avatar, and that's why you all look sometimes pretty, but so obviously fake to me. 99% of the best female avatars look like nice mannequins, whose crotch is about 15–20% higher on their bodies that it is in real life. That alone breaks any sense of "realism" for me.

It doesn't help that flowing cloth is so obviously fake with no realistic physics.

We'll need way more realism for me to accept that SL offers anything like an uncanny valley look to avatars.

We're all pretty dolls, but not anything real. Yet.

Edited February 13, 2022 by Katherine Heartsong

[Persephone Emerald]

11 hours ago, ChinRey said:

The one big flaw with this illustration and indeed with the whole uncanny valley concept as it's usually presented, is that it stipulates that uncanny valley is static. Nothing could be further from the truth.

It's different for different people for a start and even for the same person at different times.

But most of all, it's all about context. Uncanny valley was invented by scientists trying to create realistic humanoid robots for RL and it only really applies to that specific field. In a virtual reality it's not about how realistic or unrealistic your avatar looks; it's all about how it fits into the scene as a whole. Put a real life person into a cartoon environment and it's he or she that will trigger the uncanny feeling, not the scenario. So, to those who posted pictures of avatars here: sorry but without the surroundings those pics are simply not relevant.

This is of course a serious, possibly unsolvable, problem for a virtual environment assembled together from unmatched pieces with no coherent overall design and in Second Life we have the added complication that even the dominating trends and fashions are different for avatars than they are for scene objects.

I think the uncanny valley problem is something that Philip Rosedale and Linden Lab are going to have to take seriously if they really want to animate avatars based on their RL expressions. I don't think it's necessary, but if they do it badly, it could actually end up turning people off from wanting to use SL. 

Also, the uncanny valley is not static, though I wish I had more data to show this. Over time, people get used to non-living objects that look human, such as dolls or robots. This article has some interesting examples of how the uncanny valley affects people. One example is of a robot the looks and moves sort of like a dog, which also elicited an uncanny valley effect from people. I couldn't help but keep thinking of the "Rat Thing" in Snow Crash .

Edited February 13, 2022 by Persephone Emerald

[Sammy Huntsman]

Avatars should look like computer generated people. They don't need to have hyper realistic emotions or skins. That is just creepy to me.

[Codex Alpha]

I don't think there are any rules, other than what the engine can handle, but I think it would have more to do with the AVERAGE detail of avatars on a platform.

A realistic avatar would look strange on Horizons or RecRoom, while others can be running around on Minecraft servers and perfectly acceptable and people still say "Oh that's a cool avatar".

Getting too realistic making anything can get boring, and there is a good argument (and probably sales data) to back it up that more stylized and less realistic forms of mesh creations are more popular. Also when we want 'realism' usually the texture sets have to be much larger and use more resources..

Depends on the end goal I guess. Here I am talking from the same default avatar I had when I joined hahaha.. I just don't care as my primary goal is to create here anyway, and not stand around and pose

I think in many ways SL avatars are quite balanced. Not quite so realistic (other than photography tricks), but acceptable enough to draw wows (to others some tongues hanging out their mouths) etc. so I would say mainly what you want to make anyway.

My issue in SL, OpenSim and related are the freakishly monster sized avatars people keep walking around in

[Kyrah Abattoir]

14 hours ago, entity0x said:

Getting too realistic making anything can get boring, and there is a good argument (and probably sales data) to back it up that more stylized and less realistic forms of mesh creations are more popular. Also when we want 'realism' usually the texture sets have to be much larger and use more resources.

Stylised also tend to hold the test of time a lot better.

[BigBankHankTV]

IMO as Realistic as we can lol, closed minds don't get feed. Can only be as realistic as the tech allows. The main issue is lighting in engines it what really brings out everything cause of shadows and good ambient occlusion, global illumination and such. You can make some really realistic characters these days even for real time its not the problem, what makes character real is animations, realistic proportions for shape, beside that facial imperfections and skin pores getting use a lot closer pass that uncanny valley point as from what I'm seeing lately. If you want to look at some case studies of seeing what is possible look at TexturingXYZ, skin pores and face scans of color maps from real people, once someone has enough data of those can really create some insane character avatar systems. A normal map or displacement map of skin pores really changes alot and lighting of course. Just wanted to show people whats possible.

 https://www.instagram.com/texturingxyz/

Heres a realtime character case study I found from Sefki Ibrahim, really cool stuff
https://texturing.xyz/pages/realtime-digital-double-with-character-creator-and-texturingxyz-sefki-ibrahim

Edited April 12, 2022 by BigBankHankTV