Workflow Demo: Cascadeur 3D + Rokoko

[Aglaia]

Hi!

I've tested a new workflow (at least new for me!) that i wanted to share because i think it is very promissing.

I made this dance using Rokoko mocap outfit + Cascadeur 3D. With the Rokoko mocap outfit, i just do the key poses. Then all the rest is made by Cascadeur 3D. If you don't know Cascadeur 3D yet, it's really worth to have a look at it. It is a kind of physics engine that handles gravity, that can make your character bounce, jump, have their knees bend before and after a jump, compensation movements, etc, it is really amazing. So in a few words : the key poses are made in mocap, and the transitions between them are made with Cascadeur 3D.

So this is my first animation using this workflow. There's still a big room for improvement, for sure, and i'm far to be fully satisfied: arms are not really animated yet, the head neither, not all transitions are smooth, but this is a base that i want to keep working on!

 

[Monstaar Mama]

I downloaded Cascadeur just to try to use it for SL when I get the time. Is it easy to bring the SL skeleton into Cascadeur? I appreciate you posting this here!

[Aglaia]

Yes, easy. Take a model that is skinned to the SL rig, export it as .fbx. Then import the fbx in Cascadeur (File > Import fbx/dae > Model).

If your model is a humanoid character, Cascadeur will open the "Quick Rig" tool. Here you just have to drag and drop the bone names of your model (mPelvis, mChest, etc) on the Cascadeur shape, and Cascadeur will do all the job. After that you get your model in the Cascadeur 3D view and can animate it.

When you're done with your animation in Cascadeur, export it as .fbx (File > Export fbx/dae > Scene). Since we cannot import fbx into SL, you have to convert it to either .bvh (using Blender) or .anim (using Avastar or Bento Buddy in Blender)

Edited February 24, 2023 by Aglaia

[Lucia Nightfire]

The dance looks like low keyframe count, though, IE, not that fluid.

How long, in seconds, is the loop and how many keyframes are in said loop.

Which Rokoko suit/software package did you use?

Was there anything limiting in the capture rate?

[Aglaia]

Yes you're right, i've had problems with FPS and i had to reduce it drastically to make the file smaller and i don't know yet how to reduce it properly. I had to reduce from 64 FPS to 20 FPS. This is something i have to work on.

- The animation is 32 seconds.

- The original animation, right after exported from Cascadeur, was 64 FPS. It was very smooth in Cascadeur, much more than on the video.

- It was one keyframe per frame. So 64 FPS meant 64 keyframes per second.

- 19 bones are involved in the anim (torso, arms and legs). So for each second, that was 64 keyframes for 19 bones... (i also keyframed the fingers but just one keyframe at the begining of the anim + when she rolls her ams so fingers are neglictable in this anim)

=> At the end, my animation was 2 MB. Which was way too big, since the limit for SL is 250 000 bytes.

So i opened it in Blender to reduce it drastically. I have resized it in the dopesheet and changed the FPS untill i got a file less than 250 000 bytes. At the end, my file was only 20 FPS, which is too low, and explains why it is not smooth.

I have to see how to do this in a more clean way. I am pretty sure that it is possible to delete keyframes in a clever way : i would be surprised that Blender would not be able to remove the "less important" keyframes, i mean the ones for which there are just very little variations along periods of time. I didn't take the time to find a better way, i was exhauted after these 2 weeks of work on this anim, but this is something that i want to work for the next dance anims i will do.

PS : for the mocap i use Rokoko Smartsuit Pro II with Rokoko Studio, but this has no importance here : i use Rokoko just for making the keyposes (1 key pose is 1 keyframe) then i import the key poses in Cascadeur where i do all the job.

Edited February 25, 2023 by Aglaia

[Aglaia]

I have removed the video to replace it by a new one that fixes the problem mentionned above. I wanted to just edit my first message to replace the video, but it seems that i cannot edit my message (i guess there's a limited time for editing a message?) so i will repost the video here. The animation is now 40 FPS instead of 20 FPS, so much smoother. It looks likes it stills shakes a bit sometimes on the video but this is due to the video, not to the animation itself.

 

 

By the way i made another one:

 

Edited March 3, 2023 by Aglaia

[BinBash]

5 hours ago, Aglaia said:

By the way i made another one

This is a lot of fun.  I think it's perfect the way it is, even if it looks a bit quirky which just adds more smiles.

[CandySwirl76]

Well done! Fun project.

[OptimoMaximo]

On 2/25/2023 at 12:32 AM, Aglaia said:

- 19 bones are involved in the anim (torso, arms and legs). So for each second, that was 64 keyframes for 19 bones... (i also keyframed the fingers but just one keyframe at the begining of the anim + when she rolls her ams so fingers are neglictable in this anim)

No, it's not neglectable at all. Adding hands means 30 more joints x number of frames, and that makes a huge difference.

See, it's not the number of key frames that get exported, the anim file gets ALL the frames your software is instructed to sample for export. There is no smooth interpolation, from a frame to another there is just a linear change.

Now, avastar has a sort of automated process where it compresses the animation as much as it can, that's why it isn't smooth as you'd like. The number of frames that get exported is less than what you set up at 20fps, and between a frame and another there is no smoothing, just rotation change at constant speed from angle A to angle B.

That's why higher fps means smoother, in SL. There simply are more frames to sample a smooth animation curve more accurately.

There's a hidden trick in Avastar to instruct it to not resample the animation, and that's to add a marker in the action editor called "fix" : the scripts will skip the keyframe in that location entirely. So if you would like to retain the entirety of the animation data, add a "fix" marker to every frame along the length of your animation (should do that with a script to avoid tearing your hair off). This will obviously result in a bigger file size.

To decrease output file size, you should consider splitting the hands animation from the rest of the body by performing 2 separate exports, muting the channels of those joints that you're nor exporting for that file. Then in world you should setup a device that starts the animations together, body and the associated hands animations.

[BinBash]

On 2/24/2023 at 7:32 PM, Aglaia said:

i would be surprised that Blender would not be able to remove the "less important" keyframes

There is a reduction algorithm in Blender but my experience with it doesn't seem to show that it's of much use, overall.

If you're using Bento Buddy your animation will not be altered, you get what you see from Blender to SL.  But, like OptimoMaximo said, you're eventually going to hit this 250kb mark but there is a fix.

Bento Buddy has an automatic animation splitter, which will export time segments based on your criteria, and creates a specific script just for these items that you can export along with the animations.  Once in-word you just toss the animations and script into an object and you're ready to go.

Good luck with your endeavors, you obviously have put a lot of effort into this and it's quite entertaining.

[Aglaia]

17 hours ago, OptimoMaximo said:

No, it's not neglectable at all. Adding hands means 30 more joints x number of frames, and that makes a huge difference.

Yes, this is what i figured out afterward. A friend of mine sent me a program that can export a json from a .anim file and i noticed that it keyframed all the fingers for each frame of the anim. However, using this program in the opposite direction (json to .anim), i was able to remove all these additional fingers keyframes and just kept the very first and very last, as it was in blender, and it worked well. This is what i will use for my next anims.

9 hours ago, BinBash said:

Bento Buddy has an automatic animation splitter, which will export time segments based on your criteria, and creates a specific script just for these items that you can export along with the animations.  Once in-word you just toss the animations and script into an object and you're ready to go.

Good luck with your endeavors, you obviously have put a lot of effort into this and it's quite entertaining.

This is nice. But i prefer some "stand alone" anims that don't require an additional anim and script, because if i share these dance animations, i would like that people can use them in their dance AO or dance machine, so i cannot force them to use a specific device that would play fingers anims in a separate anim with a script for launching them. But that is still a nice and useful feature of Bento Buddy 🙂

And yes i get a lot of fun doing this!

[Ricky Shaftoe]

On 2/24/2023 at 1:14 PM, Aglaia said:

Yes, easy. Take a model that is skinned to the SL rig, export it as .fbx. Then import the fbx in Cascadeur (File > Import fbx/dae > Model).

If your model is a humanoid character, Cascadeur will open the "Quick Rig" tool. Here you just have to drag and drop the bone names of your model (mPelvis, mChest, etc) on the Cascadeur shape, and Cascadeur will do all the job. After that you get your model in the Cascadeur 3D view and can animate it.

I love your work on this. But where would I find a model that is "skinned to the SL rig?" When last I animated, using Poser, back in the dark ages, LL provided a model for us to download from the SL website. 

[Aglaia]

11 hours ago, Ricky Shaftoe said:

I love your work on this. But where would I find a model that is "skinned to the SL rig?" When last I animated, using Poser, back in the dark ages, LL provided a model for us to download from the SL website. 

I took some female and male models on the web and skinned them to the SL rig. Sadly i cannot share them since they are not under a licence that would allow me to share them. For sure, i think that Linden should provide some good female and male skinned mesh models for builders.

Edited March 20, 2023 by Aglaia

[Ricky Shaftoe]

 

6 hours ago, Aglaia said:

I took some female and male models on the web and skinned them to the SL rig. Sadly i cannot share them since they are not under a licence that would allow me to share them. For sure, i think that Linden should provide this for builders.

Thanks for your reply. I can't even find the model I used to import into Poser to create animations there. Does LL no longer make that available? Put another way -- where can I get the SL rig? If I can find that, I can make my own 3D model in ZBrush and add the SL rig as its bones, can't I?

[Aglaia]

51 minutes ago, Ricky Shaftoe said:

 

Thanks for your reply. I can't even find the model I used to import into Poser to create animations there. Does LL no longer make that available? Put another way -- where can I get the SL rig? If I can find that, I can make my own 3D model in ZBrush and add the SL rig as its bones, can't I?

You can dowload it on the SL wiki: https://wiki.secondlife.com/wiki/Project_Bento_Resources_and_Information in the section "Current Content & Resources". But they warn that the female one is broken. (lol)

Better resource, the avatar workbench, that is provided for free by the Avastar team: https://www.avalab.org/avatar-workbench/

The avatar workbench even comes with basic meshes. But these meshes didn't fit my needs, it is way too low poly for making accurate and sensual animations in my opinion. But it is still super useful for the rig itself, so you can definitely use that.

Edited March 20, 2023 by Aglaia

[Ricky Shaftoe]

14 hours ago, Aglaia said:

Better resource, the avatar workbench, that is provided for free by the Avastar team: https://www.avalab.org/avatar-workbench/

The avatar workbench even comes with basic meshes. But these meshes didn't fit my needs, it is way too low poly for making accurate and sensual animations in my opinion. But it is still super useful for the rig itself, so you can definitely use that.

Thanks for this extremely helpful post! I was indeed able to get the Avatar Workbench avi into Cascadeur. I used the Quick Rig feature to map the model's bones into Cascadeur. (I seemed to have no choice but to do this?) That worked okay, but the rigged model is hard to manipulate. For one thing, some joints won't rotate, even though they should -- e.g., shoulders, knees. More worrisome, the rig doesn't impose many joint-movement constraints, so it's easy to have impossible elbow bends, at least for me. Should I try fixing this in Cascadeur? I would have thought the Blender rig already imposed joint constraints? Finally, the opening frame had to be a T pose when I worked in Poser; has that changed? The Avalab pose is an A pose. I'm not sure how to adjust it to become a T pose.

I actually prefer to use a low-poly model like this Avastar thing, so I'm happy with the model. It's the rig that's giving me fits. Maybe I'm missing something? 

[Ricky Shaftoe]

@Aglaia I've tried and failed to export the Avatar Workbench mesh and rig into Cascadeur. I get the mesh and bones into Cascadeur okay, but when I use the rig wizard, invariably it converts it into an A-pose. I can't seem to rotate the shoulder joints to turn it back into a T-pose, nor do I see "dials" or other numbers that I could input to fix this. If I try using the thing I"ve rigged, when I upload in SL, I get a distorted avatar.

I've asked for help at both the Avastar and Cascadeur discords; no luck. Any suggestions? I don't need anything fancy. I just want a simple mesh with a T-pose that is compatible with SL.