Does anyone understand terrain height fields?

[Kayaker Magic]

I'm designing my new sim and trying to match sculpts up to the terrain. As usual, any task in SL starts with experiments to find out how things really work. The Wiki never seems to have all the information that I need.

I already knew that the values from the raw terrain map that you supply, the values that you get from llGround, and the terrain that the viewer displays do not match. I hypothesized that the viewer did some modification of the terrain values, perhaps to prevent “tortured”quadrangles. I tried a classic signal analysis test: I applied an impulse to the terrain system and examined the result. I set one pixel in the height map to 255 (times 0.180 resulting in a point about 46 meters tall) surrounded by pixels with a value of 120 (times 0.180 resulting in level ground about 1.5 meters above the water). I was kind of expecting that I would get four triangles pulled up around one point. To my horror, the result is a sync function! The surface of the ground rose up and down for a distance of around 6 meters in all directions around the peak! I include a picture of the result with a 10-meter rod for scale at the bottom of ths post. In a display this would be very sophisticated image processing. But I'm not sure if it is appropriate to use this on a height field. The height field is “displayed” at a much higher resolution than the original pixels. This means that the processing is visible as rings of distorted terrain rippling away from any sudden change. If I create a cliff with a sharp drop, there can be random bumps next to the edge that are sometimes 2 meters tall! I would rather that SL do no processing of my terrain map and just give me what I asked for.

 Perhaps I can perform the inverse filtering operation on my height maps. Put anti-bumps in the places where LL adds ringing bumps to my terrain. Does anyone know what this filtering operation is? The ringing is not symmetrical, but has a bias along the diagonal and dies off in a square pattern around the impulse.

 Another interesting research topic is the size of the height map. If a sim is 256 meters square then it should take a grid of 257x257 height values to describe all the corners of a 256x256 grid of rectangles. Sculpties have this problem, which is why you have to store a 32x32 quadrangle sculpt in a 64x64 pixel texture map. There is no room for 33x33 vectors to be stored in a 32x32 texture map, so they had to move up to the next size.

 So how did LL solve this height field problem? One way would be to define the values in the height field to be the center of each quadrangle, and interpolate the triangle corners between them. The impulse test above already answered this question. If the vertexes are cubic interpolated between height values, an impulse would produce a lower peak that was two meters wide, and it would ring positive and negative for 3.5 meters in all directions. But the impulse produces a sharp peak with ringing much farther than that. So where do the extra vertexes come from? A careful examination of my sim revealed that the vertexes along the north and east (top and right) edges of the height map were simply duplicated! This suggests that it is impossible to create terrain maps that cross multiple sims smoothly! (Unless your sims are boring and flat to start with). There should be a seam with a 1 meter wide stripe that has this duplicated value in it. In fact, there should be gaps in the seams, but since there is not, LL must have some special code to stitch the edges of sims together. Perhaps they simply steal the missing height values from the sim to the north and east.

 But I am trying to make sculpts that mate with the edge of the sim terrain and hang out farther. Not knowing the filtering function, I resigned myself to designing the sculpties to overlap the terrain a little and dive under the terrain. The duplicated height values on the north and east end are sticking out of the sculpties, I have to take that into account next.

 I went back and re-read the Wiki (http://wiki.secondlife.com/wiki/Tips_for_Creating_Heightfields_and_Details_on_Terrain_RAW_Files) for inspiration. It does have a good explanation for the multiplier scale factor. But it mentions nothing about filtering the values or dealing with outside edges. Is there anyplace else on the WEB that has more information?

 

 

[Helium Loon]

I can't say for certain, but that looks like 'ringing' caused by DCT compression.  By having a single pixel in the height map being so starkly offset from all those around it, it causes whatever 'interpolation' function being used to convert the map (which is 256x256 data points) to however many are being used to render the terrain, to exhibit such artifacts.

 

Try setting the closest pixels in the height map to a value closer in magnitude to the high pixel.....it may not reduce the aliasing completely, but will probably drop it's magnitude quite a bit.

 

Also, it is possible to utilize centroids by offset, to encode the heightfield, and avoid the need for 257 datapoints.  By taking into account the averaging, and weighting it, one can easily still have full height (or very similar) without causing all the adjacent points to be affected.  Or, more simply, they could simply interpolate the last set in each axis.  Or the first.  Or resample the 256x256 image to 257x257, which even with cubic interpoloation functions would rarely produce additonal adjacency changes except for large areas.

 

Without more info from LL, it's hard to say precisely what is going on.  I'm just guessing here, based on what I've seen.

 

[Ceera Murakami]

What I normally do when I make a custom raw file is use a height multiplier of 50% (Channel 2 at a greyscale 64 fill). That gives me a 0.5 M vertical resolution for each of the possible pixel locations in the heightfield, and a range of zero to 127.5 Meters for possible heights. Or else I will use a 25% modifier (Channel 2 at a greyscale of 32), so I have a vertical resolution of 0.25 Meters, and a range of zero to 64 Meters. The better your vertical resolution is, and the easier it is to calculate exact power of 2 values from the heightfield when applying that multiplier, the less distortion you'll get in the final result.

As far as I can tell, they treat all heightfield pixel values as the height for the center of the 1M square of terrain that they represent. Hence they don't need a 257th value, and don't need to negotiate values between sims if the edges disagree.

Since terrain in SL is defined by a grid that is spaced at 1 Meter intervals, it is impossible to have a true vertical face in terrain. If two adjacent areas are defined at different heights, the simulator interprets that as requiring a slope between the two levels, and will use an interpolated mid point as a halfway point between them. So when you use the in-world terraforming tools to raise a 4M x 4M square of terrain 20 M above an otherwise flat surface, you get a mid point that is 1 M further out in all directions (a 6 M square), and a base that is 2 M out in all directions (an 8 M square). Any slope will always have a mid-point, as far as I can see, regardless of how steep the difference is. So if you have half the sim at zero and half at 255, there will be a two meter wide transition between them for the cliff face.

But that still isn't the final result, since they round the edges, to smooth the surface. It's that edge-smoothing algorythm that kills a terraformer when you're trying to make a precise heightmap and import it. As far as I can tell, it introduces 'noise' into the values as part of making a smoothed transition. You can create a RAW file that is absolutely pristine, with pool table flat surfaces and crisp edges between levels, and the interpolation and smoothing that the simulators do will make the resulting surfaces near any transitions lumpy and uneven, occasionally with nasty spikes and holes. Having a height multiplier that is a random value, and not a simple fraction of the base values, also increases distortion. In my experience, a height multiplier that is evenly divisible by a simple fraction (one half, one fourth) seems to introduce less distortion.

No matter what, any terraforming I have done in SL has always required at least a little in-world tweaking, to eliminate the inacuracies caused by LL's poor interpretation of the RAW file data that was imported.

What they export is also inaccurate. I have had sims that I imported from a RAW file that had exactly 64 as the greyscale value for the multiplier layer, and when exporting that same sim back out, the height multiplier value is rarely that exact value any more. That, of course, also screws up what it says is the heightfield value. It's frustrating, but the LL coders never designed the terrain functions with precise control in mind.

[LindaB Helendale]

I did quite a bit research on terraforming when i made a tool to make the terrain from any height map with  LSL terraforming functions. Here's what I found out, but of course i can't say i know that

The ringing you see is noise the viewer adds to the rendered terrain. It's not  part of the real terrain (as you see with llGround() or by letting physical objects rest on the land). The noise is added near steep edges to 4-8 m from the edge and the amplitude depends on the heigth of the slope and especially on how steep crease there is between the plateau and the slope.

I assume it is there to make steep cliffs look more natural. 

About the intepolation, the ground is defined in grid points where the origo is at 0,0 and it is linearly interpolated  with each square meter split to two triangles diagonally in direction SW to NE. So grid points at  0 to 255 are in the sim and the last meter is interpolated from the coordinate 0 lines in the next sim.

You can find more in the manual of my terraformer, see Chapter 4   (https://d44ytnim3cfy5.cloudfront.net/assets/3384190/original/EdenScape%20User%20Manual.pdf?1302781935) but if it's not OK to link here stuff related to commercial products, please let me know and i remove the link.

 

[PeterCanessa Oh]

To the question you ask: no-one that I know.

My question: What has this got to do with scripting?  Shouldn't it be moved to Building & Texturing or to the Mainland forum?

[Pete Spires]

Maybe it should not be in scripting but here is some kind of answer. The terrain file is 256 x 256 which maps to the sim coordinates at 1 m intervals starting from (0,0) in the south west. If the sim is unconnected to others, the 256th element is copied from the 255th so the north and east edges are flat in one direction. But if the sim adjoins another, the 256th row is obviously the 0th row of the next sim.

If you want to make a sculpty that matches terrain, its X and Y size must be such that 1/255 (the sculpty measurement unit) is an exact fraction of 1 m. For example, if you use a 255 x 255 m megaprim, the sculpty units are 1 m. Another example, if you use a 63.75 x 63.75 m prim, then 4 units will equal 1 m.

As for the height, you will have to experiment because the terrain heights might be scaled to 1/8 m or 1/4 m steps, etc. The easiest way is perhaps to scale the terrain height range from 0 to 255 and then play with the sculpty height until it fits the terrain.