Testing Motion-Tracking with Second Life and possible applications for machinima!

[FinnfinnLost]

Howdy folks! Blender's motion tracker received a pretty big update recently so I wanted to play around with it again. Since motion tracking might offer a wide variety of possibilities for SL machinima (adding props and surroundings beyond SL's capabilities, limited changing to lighting or, given an ideal setup, even adding new features to characters), I wanted to check how well the tracker works with the graphics and camera movement.

https://streamable.com/grx015

The above doesn't look like much, seeing how I merely placed a cube in the environment, but it still serves to demonstrate what worked well and what didn't. First of all, a short explanations for those who wonder what the heck they are looking at without too much detail:

1. I recorded a video in SL, nothing fancy, just my avatar standing around and me zooming out.
2. I threw it at Blender and fired up the motion tracker, had it detect notable features and try to find out what the camera movements are (which is away from the character).
3. Blender applied its best "guess" to the camera.
4. I projected the video on a shape very roughly matching the SL surroundings.
5. Inserted a cube for testing.

Note how my avatar doesn't change his size compared to the rest of the scene, especially the cube and the plane which plays back the footage. Really, you could stick anything in there and it would stay roughly in place next to the avatar. That's because it's now Blender's camera moving away from the scene just like the SL camera did.

There's a few caveats though, especially visible on the cube sliding around a bit. Blender has difficulties detecting and tracking features due to the rather muddy textures. This makes camera solving somewhat imprecise, although this is largely owed to the lack of distinct features on the ground and the hedge. Probably works better on sharper textures or, better yet, a scene with points added specifically for Blender to track. Given these disadvantages, I still find the performance quite impressive.

So what do we do with this? A few ideas:

• Cut out a window in an interior scene and add a huge city with details that would not be possible with LI and space limitations!
• Land a spaceship next to a character for him or her to get on!
• Add complex animations of animals, monsters or machinery!
• Correct the camera motion in post-production (it's way more forgiving than you'd think)!
• Have elements added in Blender cast lights on SL scenery and vice versa!
• Add high-detail environmental effects like dust particles, smoke and fog!
• Render it all with raytracing and stress test your PC in the process!

Either way, this was hugely fun and, given the area I was chilling in, I got better results than I thought. I welcome feedback and, since this is not at all common knowledge or easy to understand, will try to answer any questions you might have. If you wonder if you can use it in a piece of footage you have for your hot new machinima, I'll be happy to play around with that as well.

Thanks for reading!

[Pryda Parx]

This looks really interesting and your example seems quite solid.

I have used the old Blender motion tracking with Second Life machinima a couple of times before, e.g. with the hovering video display at:

from my Future Shock series.

I will definitely be playing with this more, thanks for pointing it out Finn.

 

Edited March 19, 2021 by Pryda Parx

[FinnfinnLost]

4 minutes ago, Pryda Parx said:

This looks really interesting and your example seems quite solid.

I have used the old Blender motion tracking with Second Life machinima a couple of times before, e.g. with the hovering video display at:

from my Future Shock series.

I will definitely be playing with this more, thanks for pointing it out Finn.

 

Oh, nice. Clean solve right there. Any advice on placing tracking points? There's little difference in color and I'm wondering how you tracked it. Couldn't be automatic detection or could it?

[Pryda Parx]

This was Blender 2.79. I put 4 coloured balls in a square and set these as the tracking points in Blender. As I didn't have rez permissions on that land the corner balls were attachments on the Big Boss character set to avatar center who whilst animated was not moving. The display video was made in Blender too and tracked to the plane defined by tracking the 4 corners attachments in SL.

What you described seems to mean the attachment corners would not  be needed and the tracking could be derived from other content in the scene.

[Pryda Parx]

43 minutes ago, FinnfinnLost said:

Couldn't be automatic detection or could it?

It was mostly automated detection but I had to reset the tracking points a few times when they drifted

[FinnfinnLost]

18 minutes ago, Pryda Parx said:

This was Blender 2.79. I put 4 coloured balls in a square and set these as the tracking points in Blender. As I didn't have rez permissions on that land the corner balls were attachments on the Big Boss character set to avatar center who whilst animated was not moving. The display video was made in Blender too and tracked to the plane defined by tracking the 4 corners attachments in SL.

What you described seems to mean the attachment corners would not  be needed and the tracking could be derived from other content in the scene.

Cleverly done and mpressive! Thank you for the insight.

Yeah, what I attempted was basically the other way round. I didn't use tracking to translate an object attached to the tracks, in this case it was the camera motion being inferred from the translation (and scaling) of points in the scene. If it works, it offers greater flexibility, however, it requires more accurately trackable points and you need to block out the scenery to really make it worth the extra effort.

If you want to check it out more in-depth, Ian Hubert got a few videos on his Youtube channel that explain it quite well.