A.I. animated avatars

[amorhumanity]

Goal: Animate the face of an avatar to match real time generated audio answers being spoken.

Problem: Synchronizing the audio & facial expressions to continuously match one another

Would like to translate points of what would of been a talking face on another platform, then API in translated position points for the face rig to move.

Edited October 8, 2021 by amorhumanity

[Coffee Pancake]

There is no way to stream animations from the client, so you will have to create a ton of fractional anims to go with certain sounds.

There is no way to process audio in LSL, at all.

Viewer side code that then plays fractional anims on the client matched to heard audio, and hope it doesn't all screw up with round trip latency and load times.

It might look ok on the speakers screen, it will likely be a mess on others depending on network conditions.

[amorhumanity]

@Coffee Pancake hmmm... I will sketch it out & submit it to Firestorm for their review & see if it's possible to change that

[Coffee Pancake]

7 hours ago, amorhumanity said:

@Coffee Pancake hmmm... I will sketch it out & submit it to Firestorm for their review & see if it's possible to change that

keep in mind the viewer is dumb like a rock and entirely dependent on what the servers offer (and there is no facility to directly tell the server to move a specific bone and then relay that to nearby clients), there is also no direct viewer to viewer communication.

In viewer voice processing and triggering animations is the best you're going to get with the current system, anything more requires code on LL's side .. and that's unlikely to happen

 

[Dahlia Bloodrose]

How does the Lelutka Axis face posing HUD work?  It appears to use more bones than are available for animation.  It's strictly for static poses, but perhaps that technique can be used for comparatively slow micro-expressions.  Not what you really want, but still.

On a tangent, it would be really nice if dancing actually matched the music.  Not just the (almost) same BPM, but properly synchronized on 2 & 4 (at least for 4/4).

[Lucia Nightfire]

5 hours ago, Dahlia Bloodrose said:

How does the Lelutka Axis face posing HUD work?  It appears to use more bones than are available for animation.  It's strictly for static poses, but perhaps that technique can be used for comparatively slow micro-expressions.  Not what you really want, but still.

On a tangent, it would be really nice if dancing actually matched the music.  Not just the (almost) same BPM, but properly synchronized on 2 & 4 (at least for 4/4).

It just uses individual animations, one for each face bone, and plays them all together.

It's no different in operation than the AnyPose application that uses incremental animations per body bone.

[Coffee Pancake]

9 hours ago, Dahlia Bloodrose said:

How does the Lelutka Axis face posing HUD work?  It appears to use more bones than are available for animation.  It's strictly for static poses, but perhaps that technique can be used for comparatively slow micro-expressions.  Not what you really want, but still.

On a tangent, it would be really nice if dancing actually matched the music.  Not just the (almost) same BPM, but properly synchronized on 2 & 4 (at least for 4/4).

I deeply wish we had more control over animations from LSL. To be able to change things set in stone during upload would be awesome.

[amorhumanity]

Well I tried to make a compelling case for it.  I will make use of things as they are a hope one day we are able to see it happen.  Thank you for help clarify where things are at.

[Coffee Pancake]

6 hours ago, amorhumanity said:

Well I tried to make a compelling case for it.  I will make use of things as they are a hope one day we are able to see it happen.  Thank you for help clarify where things are at.

 

There was a long lost project from LL called puppeteering that was intended to send animations to the server and have them relayed to nearby clients, it got canned and forgotten.

[amorhumanity]

We can all avoid asking why they would not of spent the Sansar money on that, and move on with our lives.  Was it literally called Puppeteering? I would like to find info on it as reference for a follow up email.

 

Edited October 10, 2021 by amorhumanity

[Coffee Pancake]

1 hour ago, amorhumanity said:

We can all avoid asking why they would not of spent the Sansar money on that, and move on with our lives.  Was it literally called Puppeteering? I would like to find info on it as reference for a follow up email.

 

http://wiki.secondlife.com/wiki/Puppeteering

https://www.engadget.com/2008-03-28-cinemassively-avatar-puppeteering-possibly-coming-to-second-lif.html

http://npirl.blogspot.com/2008/03/avatar-puppeteering-for-second-life-in.html

[Quistess Alpha]

2008. . . why do all the cool abandoned projects seem super old?

[Coffee Pancake]

6 minutes ago, Quistess Alpha said:

2008. . . why do all the cool abandoned projects seem super old?

Because the people responsible were in MK exit layoffs I guess

https://www.gamedeveloper.com/pc/linden-lab-ceo-resigns-shortly-after-layoffs

[Mollymews]

a fyi

M Linden stepped down on the 24th of September 2010.  He was supposed to give a speech on the 23rd at SLB7 but cancelled and departed the company the next day

on the 21st two days prior, Philip Linden, as Board Chair, gave a speech to SLB7, the transcript is here:

http://wiki.secondlife.com/wiki/SL7B/Philip_speech

is kind of interesting this speech. Hearing it live at the time, it came across as a little bit harsh in parts, for a speech given by a Board Chair in a public forum. I have since read the transcript a number of times and the text maybe doesn't bear this out sans the context of the times. Approx. 100 staff (1/3rd of the workforce about) had been made redundant in the previous week

altho when the Board Chair publicly says, and you are the CEO:

Quote

But I think what we have to do — what I know is the kind of thinking that's informing our planning process going forward — is ask whether instead we can stop doing those many, many peripheral, highly usage-specific things to get people in here — and instead just take a step back, look at the basic problems that we are all faced by, and by fixing them, fill the moat. Tear down the walls. Stop trying to build over them. We have a product here that can deliver an unbelievable experience to everyone if we simply make the basic pieces of that experience work. Whether we're talking about how many people can stand together in a meeting like this, or how to put clothes on, or manage your inventory, or build basic objects inworld, or how voice works, how parcel media works, live music — all of these basic features are things that are amazing experiences when you can have them, but they're not easy enough yet. They're not — they just in many cases don't completely work, and we — it's so easy to get ahead of ourselves as a company and forget that. So going back to those basics and just trying to make this thing work for all of us is what you can expect to see from us next.

M Linden never showed up 2 days later to give his SLB7 address and had left the company the next day

 

Edited October 11, 2021 by Mollymews

[Coffee Pancake]

Quote

Whether we're talking about how many people can stand together in a meeting like this, or how to put clothes on, or manage your inventory, or build basic objects inworld, or how voice works, how parcel media works, live music — all of these basic features are things that are amazing experiences when you can have them, but they're not easy enough yet.

Funny .. all the things mentioned never got any love. Like, literally zero love.

[amorhumanity]

The 2 things that to this day boggle my mind is, #1. Why an easy to use, easy to install, standard language translation system was never put in so as to include all nations peoples as much as possible.  #2. Why the Marketplace wasn't made to be mind blowingly intuitive, fast & easier to use, like Amazon.   One would of thought, making it easy for people to talk & spend their money, would of been top tier priorities. It was almost as though the Lindens were always attracting so many people, since they were English speaking it was not a big deal for them, & that they spent so little of their own money on the MP, they never felt the struggle of the average user.  The inventory system... I don't think any of us have to elaborate on what is going on there lol

The mind reels

Anyway, well now I know, & am obviously going to have to work with another platform to synchronize what I want.  Which is okay... Unity, here I come baby !!! 

Edited October 11, 2021 by amorhumanity

[Quistess Alpha]

13 minutes ago, amorhumanity said:

The 2 things that to this day boggle my mind is, #1. Why an easy to use, easy to install, standard language translation system was never put in so as to include all nations peoples as much as possible.  #2. Why the Marketplace wasn't made to be mind blowingly intuitive, fast & easier to use, like Amazon.   One would of thought, making it easy for people to talk & spend their money, would of been top tier priorities.

Specialized things like web search/UI and automated translation require specialized talent ( I.E developers). On the marketplace I think they did as well as you could reasonably expect without totally outsourcing the search functionality to a 3rd party like google (I'm not saying that it can't be improved. . .). They already integrated 3rd party translation tools, problem is, the costs for that are pushed onto users rather than LL, as I understand it. That's probably as much on the 3rd parties as it is on LL.

[Jenaia Morane]

The MK era was a dark one for SL, IMHO
I met him at a conference, eager to talk about  the many and exciting possibilities and he wasn't interested at all.
SL is STILL groundbreaking - still has so much potential to be at the forefront of and shape the latest Metaverse craze if its owners would only invest.
It's so frustrating.