Unpredictable mesh import

[AcidJuice]

Hello again

 

Well, some more informations :

 

Concerning the low or high poly aspect : this 16k vertices models costs a bit more than 5 prims (lower lod set to zero). I dont know if it is a good indicator about what is good or not in matter of polycount ?

 

I have succeeded to load a good cat again (rigged this time **) !! And the amout of vertices and faces are still 18 468 and 33 088. So no difference with the bad import... but still a difference with blender informations...

 

@Kwakkelde :

You asked if there were sharp edges : no. All is defined as smooth in blender

Also you asked if there is multiple materials : now yes, but in the first imports no (no materials at all in the first imports)

 

@Gaia :

I am ok to give you the model but not to share it too all. I've tried to contact you inworld so that we could find a good way to give it to you, if you want.

I have not tried yet to triangulate the model. But by the past (on other models), i have noticed (or at least i made the supposition) that tri or quad models may influence the result. But i am not sure at all (because bad points are not allways the same from an import to another).

In this way, another question maybe the influence (or not) of mesh loops ? I dont know if sl takes care of it (for any optimisation or i dont know) or simply converts all in tris and no more...

 

For additionnal information, i know use the last 4.4 version of firestorm. I'll tell you if this seems to change anything. But previously i had the problem on both sl and firestorm viewers (and by the way both in aditi and agni).

 

Have a good day

 

** i have to say that i succeeded to do the rig because of the very good tutorial found on avastar blog here : http://blog.machinimatrix.org/2013/03/15/avastar-1-rc1-775/

 

 

[AcidJuice]

i can add few more informations :

 

The amount of vertices shown in the preview window is not allways the same (but the amount of tris seems to be stable).

 

 

There is also another bad thing. Do you notice the dark part on the cat chest ?

 

Also there are some bad imports that are not visible in the preview window : look at this screen capture below.. it has holes in the belly !! And yes, the mesh is verified about normals (and again this is exactly the same collada file...)

 

 

 

[Drongle McMahon]

Two things...

The internal mesh data format uses 16 bit values to define geometric positions. If you have triangles that are very small, the rounding of the coordinates to 16 bits may sometimes make two vertices the same, producing a redundant triangle with non area which will be culled from the triangle list. This could explain the small loss of triangles count that you describe.

I have described elsewhere a bug that limits the number of triangles for one material to 21844. The uploader secretly starts a new material when the count reaches this limit. I have only studies in detail the effects of this on texturing/colouring, but it is quite possible that it affects rigging as well. Which triangles go into which material depends on the order they are seen by the importer, I would expect that to depend only on the order in the collada file so that the effects should not vary unless the file is changed. However, there may well be some other things affectin that order. It might depend on the internal state of the collada library in the viewer. Anyway, I suggest you try reducing the triangle count below 21844 (a good thing anyway) and see if that solves the problem.

[AcidJuice]

Hello Drongle,

 

Concerning the vertices proximity, i thing i am not concerned here, because the mesh is really regular and clear... so i dont think this is the point here (for the less, for this model).

 

Concerning the limit of 21 844 vertices per material.

Knowing that, i have sorted the vertices per material [edit i have now 6 materials in the mesh] in blender and before exporting the model to collada.

After that, i made few import tests... and that seems to work !!!

 

But as all this does not occurs all the time, i am prudent and i will try some more tests to be sure...

 

... i'll tell you all in this thread...

 

Thanks Drongle

[Gaia Clary]

ok, so i took a look into the file and found 94 duplicate vertices in the mesh. After removing the duplicates, i end up with numbers similar to the SL Importer. This is from blender after removing duplicates:

 

• vertices: 16663
• faces: 16592
• tris: 33088

 

Furthermore: You use 6 materials on your cat and you have 45 islands in total (from a very quick look on your UV map). This explains well why the numbers of vertices reported by the SL Importer is higher than what you see in Blender.

Now after reading what Drongle wrote earlier, i remember that meshes with very small faces sometimes created MAV_BLOCK issues and other "missing requirements" issues.In all cases which i inspected i could get the meshes to import after reducing the face count and taking care to have not extremely small faces.

So what i would do in your case:

 

1. remove the doubles
2. reduce the mesh face count, especially the claws are way too detailed in my opinion.
3. Also the entire catbody looks like it can bemade with at least one level less of subdivision.

Maybe the spikes already go away when you clean up the mesh (from doubles). And i bet that once you decimated your face count and removed the small faces along the claws, your mesh will start behaving nicer during import as well.

[AcidJuice]

Thanks Gaia,

 

just few precisions :

 

The 94 duplicated vertices are recent (work in progress as i told you and this duplicated only concern the eyes newly added). And so i had this mesh import problem previously.

 

For the 'sl material split' i now understand the different amount of vertices comparing to blender. ok... but not necessarily that this amount vary when the import is bad ?

 

For the rest, surely i can reduce the face count in some way... but the point to me is to know precisely the limits : for instance if someone could say "eh ! in the documentation it is written that no mesh over xxxx vertices is allowed" or if the importer gives an error message concerning that... that could be really cool ! no ?

 

All this because in my first posts here, i gave another exemple of a 6 828 vertices mesh with the same problems... so where and why are the limits ?

 

Well... nevertheless, thanks again all for your responses and your attention

 

Lemon

[AcidJuice]

Well... i did over 30 import tests without reducing the vertice amount (and keeping the duplicated vertices that Gaia found).

 

As said previously the difference with these new tests is that i have sorted the vertices by material in blender.. this approach deduced from Drongle indications...

 

... and it seems to work well all the time !! Or... for once, i am very very lucky importing meshes !

 

As said, i have sorted meshes, not the faces or triangles.. i dont know how blender reacts internally in this sort...

 

But but but ^^

My other test model (the stick man) is only 13 600 triangles and 6 800 vertices... so under the given 21 844 ??

 

Anyway... i have sorted the stick man vertices too and tried the import about 20 times... and all is ok too !!!

 

So... to me, that could be the solution ... even if i dont understand completly all, to tell the truth...

 

Many thanks to all

 

[Rahkis Andel]

This seems to raise more questions than answers.

[AcidJuice]

I agree Rahkis !

 

Drongle said :

"

The internal mesh data format uses 16 bit values to define geometric positions. If you have triangles that are very small, the rounding of the coordinates to 16 bits may sometimes make two vertices the same, producing a redundant triangle with non area which will be culled from the triangle list. This could explain the small loss of triangles count that you describe.

"

Can that be solved using blender "remove double" function with a floor value for the "merge distance" ?

 

Drongle also said :

"

I have described elsewhere a bug that limits the number of triangles for one material to 21844. The uploader secretly starts a new material when the count reaches this limit. I have only studies in detail the effects of this on texturing/colouring, but it is quite possible that it affects rigging as well. Which triangles go into which material depends on the order they are seen by the importer, I would expect that to depend only on the order in the collada file so that the effects should not vary unless the file is changed. However, there may well be some other things affectin that order. It might depend on the internal state of the collada library in the viewer. Anyway, I suggest you try reducing the triangle count below 21844 (a good thing anyway) and see if that solves the problem.

"

Many things here...

- Why this limit ? A bug yes but where ?

- What is handled by the viewer side and what by the sl servers ? Remembering that an hypothesis was link to the bandwidth...

- Why does the problem exist, why "instability" from the same collada file ? And i precise again that the problem exists on non rigged meshes too... and with both sl and firestorm viewers...

- How this automatic material change affects the sl material count limit ? (if it does)

- ...

 

For the other point that is "i need to reduce my mesh"... well ok... i am not so good at all that but try to improve myself

But... when you see that this 16k vertices mesh "only costs" 5.7 equivalent prims... well...

 

Oh... for Gaia... a suggestion : including some kind of automatic vertices sorting in the "avastar collada" exporter ?

 

If any other ideas or things to experiment, just say and i'll try (if possible) to make some tests.

 

Cheers,

 

Lemon

[Drongle McMahon]

- Why this limit ? A bug yes but where ?

I would point you to the jira, but now you can't see that so...

It's in source code file llmodel.cpp, around the line "if (indices.size()%3 == 0 && verts.size() >= 65532)". the size being tested is the length of an array of all the indices from the triangle list into the vertex list. Every triangle has to have three vertex pointers. So this limit is always reached before the documented limit of 85536 vertices, even if there is no redundant use of vertex list entries. The code following that line starts a new material, which has its own new triangle and vertex lists. The odd thing is that it has the same material name, which may explain some strangeness in the effects. If you upload a mesh with more than 21844 (65532/3) triangles in one material, then check Select face and click on it and change the colour, you will se that it isn't  all one face! As for why, I don't know. I am not aware of anything that should limit the size of the triangle list, asI don't know where a pointer into it has to be only 16 bits.

- Why does the problem exist, why "instability" from the same collada file ? And i precise again that the problem exists on non rigged meshes too... and with both sl and firestorm viewers...

My guess would be that this depends on the internal state of the collada dom library, the part of the viewer that reads the collada and turnes it into data structures. It may be that this will present the triangles to the viewer code in a different order under different initial conditions.

- How this automatic material change affects the sl material count limit ? (if it does)

If you have just one material in the source, then when it gets beyond 8 x 21844 triangles, it looks like it goes on reading in triangles, but after it's finished some later code simply discards anything belonging to the 9th and subsequent materials. The result is that those triangles are missing from the upload. If you try to ulpoad more than 8 x 21844 = 174752 triangles, the upload counter will always say 174752, and there will be holes in the mesh where the triangles are missing. I didn't test how it works with more materials, but I suspect it would have holes with many fewer triangles because some of the materials would be used up  with less than 21844 each.

I should emphasize that I haven't retested this for a while. There could be a fix on the way or even already done. I can't tell because bugs now get spirited away into the internal jira which even the submitter can't see.

[Rahkis Andel]

---

AcidJuice wrote:

 

For the other point that is "i need to reduce my mesh"... well ok... i am not so good at all that but try to improve myself

But... when you see that this 16k vertices mesh "only costs" 5.7 equivalent prims... well...

---

You will probably find the response to the last point you made is overwhelmingly against looking at the LI as the singular point of reference to design towards. (LI is a pretty good measuring-stick, but it isn't perfect)

Considering what Second Life is -- a real time, persistent online simulation -- and the limitations such a thing carry with it for both the server and the client hardware, you should never even be allowed to approach that "ceiling", whether the ceiling is a bug or intentional.

The fact that an excessively dense mesh winds up with a small LI is irrelevant for practicality's sake. It is up to content creators to care that their content doesn't contribute negatively to the experience of everyone else.

In the ideal, controlled game environment, no character model (All clothes and hair included) needs be more dense than 10k triangles. That, of course takes advantage of the many tricks you can employ to cheat -- not adding any detail in the model that will not be visible to the player. In this "game", where the end user is rarely comfortable with the idea that there is no body under their clothing, obviously those tricks aren't possible. Plus, our content must be easily reusable, so we just plain lack the ability to cheat.

Does this mean we shouldn't care if our models are overly-complex? No, that means we need to care more!

That said, even now without normal maps, a naked, bald character composed of between 10k and 15k triangles will still look great by all reasonable standards! People just need to realize that they don't need their mesh to look perfectly smooth all around -- that all they really need to focus on is a mostly smooth silhouette. The shading groups and textures take care of the internal faces!

The problem? Making 10k to 15k triangles look and move properly requires a knowledge of topology when some people just want an easy button.

 

Anyway, you didn't ask for a rant and by all accounts you know this already, so I'll take my leave from the soap-box.

[AcidJuice]

Thanks a lot Drongle and Rahkis for your explanations and advices.

I'll try to rework on this mesh to lower it as possible !

 

cheers

 

Lemon

[AcidJuice]

Hello again all...

 

I am sorry to say that... in fact... the problem still occurs...

 

I retried today the upload of the same mesh... you remember the mesh that seemed to be good after vertices sorting... but today it is bad at import again

 

More than that... i reworked on a low(er) poly version of the cat. This time i have 7 068 triangles and 3 568 vertices.

 

Here is the cat in blender (colors are just due to a matcap display which does not affects the materials). There is no materials, no modifiers here... and no doubles...

 

So... here is a series of import in sl (firestorm / aditi) :

 

This time, no more bad vertices (it seems), but these dark parts (unpredictable again)...

 

Here is an example result of this kind of dark parts inworld (not the shadow under the belly, but the 'stars' in front of the arrow) :

 

 

Again... thanks for any suggestion or idea...

 

[Rahkis Andel]

I don't really know what to suggest offhand that hasn't already been suggested.

If you're willing to share the .blend I would be glad to see if I can see anything wrong with my own eyes. Hopefully it's just something silly and nothing hardware related.

[Rahkis Andel]

Well, I see nothing wrong with the model.

What version of blender are you running? I am going to export the cat and skeleton to a collada file and you can attempt to upload that to second life and see what happens. I'd do it myself, but I can't access SL until I get home tonight.

Edit: I noticed that the low poly cat isn't parented to a skeleton yet. Do you want to rig the low poly first and send that to me or do you want me to just export a static mesh?

[AcidJuice]

My blender version is 2.66.0 r54697

 

For the last exports (screen captures), I used the avastar collada exporter (but i had the same problems with the native exporter, for non rigged meshes).

 

For now I just want to understand what happens with the low model and try to find a 'secure way' to import meshes... so just a static mesh (my problems occur both with static and rigged meshes)...

 

I am ok to try an upload from your export (but not sure to have the possibility to do it very soon).

 

[Gaia Clary]

Hi;

I still can not reproduce your issues here. However i always use the default SL viewer for any of my testings. If you like, then please send me for inspection:

 

1. the .blend file
2. the .dae file that you created with avastar
3. what export settings you have used in the avastar export panel

I am certain that this is not a problem of the Collada exporters. However maybe we still can find at least the reason why the SL Importer does weird things.

cheers,
Gaia

[AcidJuice]

Rahkis, here is the result of 4 imports with your collada file :

 

 

What we can notice :

- The triangles and vertices count is not stable

- The two first imports seemed to be ok

- The two last imports are bad (the dark parts again near the arrow)

 

 

[Rahkis Andel]

If I had to make a hypothesis, it would be that it's failing at the import step. All the signs point to that. I can't test that hypothesis until I'm at home, though.

In the mean time, you could try re-installing the latest version of whatever viewer you're using.

As Gaia suggested, also try using the default LL viewer and see if you get better results.

[AcidJuice]

I have sent you the files, Gaia.

 

Concerning the viewers, I have tested with the LL viewer (recently reinstalled for the purpose of these tests) and this is the same.

For firestorm, i recently installed the 4.4 and have the same results as the previous version.

 

For me, these problems are not recent : i allways had this kind of problems... i opened this thread recently because i am a bit discouraged because of that and really hope to find a solution !

 

Concerning my configuration :

- Asus G75V (gamer laptop)

- i7 3610QM

- 12GB of memory

- hundred of free gigabits on disks...

- NVidia Geforce GTX 660M 2GB

- Windows 7 Family Edition 64

 

Hope that may help ? I dont know...

[AcidJuice]

Here are my export settings for the lowpoly cat (which is not rigged) :

 

 

I generally use the defaut values (except to say "selection only" using the blender exporter)

[AcidJuice]

Here are some more tests done with "Second Life 3.5.0 (273444) Apr  3 2013 12:48:32 (Second Life Release)"... so the official viewer this time.

I  used the collada file generated by Rahkis (so not generated by my version of blender).

 

Here are the results of the first two import attempts :

 

Dark parts on the neck :

 

Bad points on the tail or feet :

 

... and different amount of triangles between these two imports...

 

Well.. i dont understand... if i am the only (?) one to have this problem, maybe this is due to my environment...

But why with several viewers ?

And why with Rahkis collada file ?

 

Is there any shared system files (exe, dll, i dont know) between the several viewers ? Is that one bad componant somewhere on my computer ?

 

Really nobody else had this kind of problems ??

 

 

[EDIT : this cat is not rigged, has no material, and is around 7k triangles] 

[Rahkis Andel]

This whole mess is making my head hurt. I've never had this kind of issue myself.

Hopefully Gaia has better results.

If She also sees the same artifacts in her upload, anothing thing I would try is to start with a fresh .blend file and import in only the cat mesh and export from there. Perhaps something in your file itself is corrupt and messing with the upload -- that said, even if that worked, it doesn't explain why it would always be happening to you.

 

[AcidJuice]

Gaia talked about an example of bad behaviour due to her computer memory failure, in one previous post...

 

... ok, but why this only this function is impacted if i have a ram problem ? (and i found no notification about that on my system)

 

So Rahkis, if you never had this kind of issue, certainly my environment is the origine of the problem... but as i said, i reinstall recently both sl and firestorm viewers... and my computer is globally ok... or if it fails, it fails only in that particular import...

 

... i am ok to test to reimport the cat on another blend file, but as i told before i have regularly this problem on many meshes : and the solution was to try and retry imports so that at the end i obtain a good result (but i am sorry to say that i am fade up of doing that ^^)..

[Rahkis Andel]

At this point I'm just throwing anything out there that is "worth a shot". I'm stumped as for any obvious answers.

Hopefully something turns up that gives us more clues to go on.