JSON Encoding of difficult Strings - uList2Json & uJsonSetValue

[LepreKhaun]

As many of you may be aware, LSL has the habit of "enhancing" Strings. This is regarded as a "feature" of the language and usually works out for the best, giving one the option of formatting chatted text by using "\t" and "\n". Unfortunately, one didn't have a way to opt out of this behavior. Put in computereze, LSL simply lacked "raw strings", http://en.wikipedia.org/wiki/String_literal#Raw_strings.

This has bedeviled those working with Json text, either for web communications or developing other uses for it, because some strings just wouldn't encode properly. That is to say, these are all perfectly valid Json strings that simply couldn't be directly formed with llList2Json() and llJsonSetValue():

• "\"Go!\" he yelled.\n"
• "She replied \"No!\""
• "Copyright symbol is \u00A9"
• "oops]"
• "Control characters are \t\n\r\f\b"

 

I've spent a few weeks studying the problem, most of it going about it the wrong way, but had an epiphany. A one line addition Maestro Linden added to https://wiki.secondlife.com/wiki/Json_usage_in_LSL on the 10th ("LSL strings which both begin and end with "\"" are interpreted literally as JSON strings, while those without are parsed when converted into JSON.") confirmed what I had begun to surmise- a Json String (being a LSL String that is further enclosed within double quotes) is a "raw string"!

 

Once I had that in hand, the following two functions practically wrote themselves. Hope some will find them useful.

[WARNING: Code optimized only for speed and memory. If you think this is unreadable, you'd go into convulsions to see some of the perl cgi programs I've written and maintain professionally.]

 

//////////////////////////////
// function string uList2Json (string type, list values)
// This function takes the exact same parameters as
// llList2Json() but correctly encodes all possible strings
// including those with escape characters within them.
//
// Initial strings must escape all instances of the
// desired escape character itself 
// (ie "\\t" => '\t', "\\\\" => '\\', "\\/" => '\/')
// as well as any double quotes ("\\\"" => '\"')
//
// Version 1.0 by LepreKhaun 9/19/2013
// May be freely used, modified and distributed with this header intact.
///////////////////////////////
string uList2Json (string type, list values) {

	integer iter = -1;
	integer listLength = llGetListLength(values);
	
	// Step through list, hitting every other item if JSON_OBJECT
	while ((iter = ++iter + (type == JSON_OBJECT)) < listLength)
		// necessary so we don't choke on next if test
		if (llGetListEntryType(values, iter) == TYPE_STRING)
			// make sure it is not a JSON_* Value or a Number
			if (llJsonValueType(llList2String(values, iter), []) == JSON_INVALID)
				values = llListReplaceList(values, ["\"" + llList2String(values, iter) + "\""], iter, iter);

	return llList2Json(type, values);
}	

//////////////////////////////
// function string uJsonSetValue ( string json, list specifiers, string value )
// This function takes the exact same parameters as
// llJsonSetValue() but correctly encodes all possible strings
// including those with escape characters within them.
//
// Initial strings must escape all instances of the
// desired escape character itself 
// (ie "\\t" => '\t', "\\\\" => '\\', "\\/" => '\/')
// as well as any double quotes ("\\\"" => '\"')
//
// NOTE: To encode a Float or Integer as a String
// within the Json text, enclose it with escaped quotes
// (ie "3" => 3 BUT "\"3\"" => "3")
//
// Version 1.0 by LepreKhaun 9/19/2013
// May be freely used, modified and distributed with this header intact.
///////////////////////////////
string uJsonSetValue(string json, list specifiers, string value) {
	// We don't want to change the string representation of 
	// an integer, a float or any Json Value Type
	if (llJsonValueType(value, []) == JSON_INVALID)
		value = "\"" + value + "\"";
	return llJsonSetValue(json, specifiers, value);
}


///////////
// Examples showing usage
///////////
 
default
{
	touch_end(integer i)
	{
		string jsonText;
		
		// To encode '{"A":"\"Go!\" he yelled.\nShe replied \"No!\"","Z":"\\escaped \\ slosh\\"}'
		jsonText = uList2Json (JSON_OBJECT, [
				"A", "\\\"Go!\\\" he yelled.\\nShe replied \\\"No!\\\"", 
				"Z", "\\\\escaped \\\\ slosh\\\\"
				]);
		llOwnerSay(jsonText);
		
		// To encode '{"Control Chars":"\b\r\f\n\t and Windows uses \r\n for EOL","©":"\u00A9"}'
		jsonText = uList2Json(JSON_OBJECT, [
				"Control Chars", "\\b\\r\\f\\n\\t and Windows uses \\r\\n for EOL", 
				"©", "\\u00A9"
				]);
		llOwnerSay(jsonText);
		
		// To encode '["WebSite","http:\/\/my.com\/ask.php?what%20is%20it","\t"]'
		jsonText = uList2Json(JSON_ARRAY, [
				"WebSite",
				"http:\\/\\/my.com\\/ask.php?what%20is%20it",
				"\\t"
				]);
		llOwnerSay(jsonText);

		// Make a Json object...
		string temp = uList2Json(JSON_OBJECT, [
				"A", 99,
				"Z", "88]",
				"C", JSON_TRUE
				]);
		// ... add it to end of the array ...
		jsonText = uJsonSetValue(jsonText, [JSON_APPEND], temp);
		// ... change our web address ...
		jsonText = uJsonSetValue(jsonText, [1], "http:\\/\\/www.google.com");
		// ... change that TAB in the third spot to PI
		jsonText = uJsonSetValue(jsonText, [2], (string)PI);
		// ... and add a new "Key":Value pair to our object
		jsonText = uJsonSetValue(jsonText, [3, "New"], ((string)PI + "\\n"));
		
		//  ["WebSite","http:\/\/www.google.com",3.141593,{"A":99,"C":true,"New":"3.141593\n","Z":"88]"}]
		llOwnerSay(jsonText);
			   
	}
}

 [Edited to remove an unused variable and obtuse reference to "JAPH".]

[LepreKhaun]

As many of you may be aware, LSL has the habit of "enhancing" Strings. This is regarded as a "feature" of the language and usually works out for the best, giving one the option of formatting chatted text by using "\t" and "\n". Unfortunately, one didn't have a way to opt out of this behavior. Put in computereze, LSL simply lacked "raw strings", http://en.wikipedia.org/wiki/String_literal#Raw_strings.

This has bedeviled those working with Json text, either for web communications or developing other uses for it, because some strings just wouldn't encode properly. That is to say, these are all perfectly valid Json strings that simply couldn't be directly formed with llList2Json() and llJsonSetValue():

• "\"Go!\" he yelled.\n"
• "She replied \"No!\""
• "Copyright symbol is \u00A9"
• "oops]"
• "Control characters are \t\n\r\f\b"

 

I've spent a few weeks studying the problem, most of it going about it the wrong way, but had an epiphany. A one line addition Maestro Linden added to https://wiki.secondlife.com/wiki/Json_usage_in_LSL on the 10th ("LSL strings which both begin and end with "\"" are interpreted literally as JSON strings, while those without are parsed when converted into JSON.") confirmed what I had begun to surmise- a Json String (being a LSL String that is further enclosed within double quotes) is a "raw string"!

 

Once I had that in hand, the following two functions practically wrote themselves. Hope some will find them useful.

[WARNING: Code optimized only for speed and memory. If you think this is unreadable, you'd go into convulsions to see some of the perl cgi programs I've written and maintain professionally.]

 

//////////////////////////////
// function string uList2Json (string type, list values)
// This function takes the exact same parameters as
// llList2Json() but correctly encodes all possible strings
// including those with escape characters within them.
//
// Initial strings must escape all instances of the
// desired escape character itself 
// (ie "\\t" => '\t', "\\\\" => '\\', "\\/" => '\/')
// as well as any double quotes ("\\\"" => '\"')
//
// Version 1.0 by LepreKhaun 9/19/2013
// May be freely used, modified and distributed with this header intact.
///////////////////////////////
string uList2Json (string type, list values) {

	integer iter = -1;
	integer listLength = llGetListLength(values);
	
	// Step through list, hitting every other item if JSON_OBJECT
	while ((iter = ++iter + (type == JSON_OBJECT)) < listLength)
		// necessary so we don't choke on next if test
		if (llGetListEntryType(values, iter) == TYPE_STRING)
			// make sure it is not a JSON_* Value or a Number
			if (llJsonValueType(llList2String(values, iter), []) == JSON_INVALID)
				values = llListReplaceList(values, ["\"" + llList2String(values, iter) + "\""], iter, iter);

	return llList2Json(type, values);
}	

//////////////////////////////
// function string uJsonSetValue ( string json, list specifiers, string value )
// This function takes the exact same parameters as
// llJsonSetValue() but correctly encodes all possible strings
// including those with escape characters within them.
//
// Initial strings must escape all instances of the
// desired escape character itself 
// (ie "\\t" => '\t', "\\\\" => '\\', "\\/" => '\/')
// as well as any double quotes ("\\\"" => '\"')
//
// NOTE: To encode a Float or Integer as a String
// within the Json text, enclose it with escaped quotes
// (ie "3" => 3 BUT "\"3\"" => "3")
//
// Version 1.0 by LepreKhaun 9/19/2013
// May be freely used, modified and distributed with this header intact.
///////////////////////////////
string uJsonSetValue(string json, list specifiers, string value) {
	// We don't want to change the string representation of 
	// an integer, a float or any Json Value Type
	if (llJsonValueType(value, []) == JSON_INVALID)
		value = "\"" + value + "\"";
	return llJsonSetValue(json, specifiers, value);
}


///////////
// Examples showing usage
///////////
 
default
{
	touch_end(integer i)
	{
		string jsonText;
		
		// To encode '{"A":"\"Go!\" he yelled.\nShe replied \"No!\"","Z":"\\escaped \\ slosh\\"}'
		jsonText = uList2Json (JSON_OBJECT, [
				"A", "\\\"Go!\\\" he yelled.\\nShe replied \\\"No!\\\"", 
				"Z", "\\\\escaped \\\\ slosh\\\\"
				]);
		llOwnerSay(jsonText);
		
		// To encode '{"Control Chars":"\b\r\f\n\t and Windows uses \r\n for EOL","©":"\u00A9"}'
		jsonText = uList2Json(JSON_OBJECT, [
				"Control Chars", "\\b\\r\\f\\n\\t and Windows uses \\r\\n for EOL", 
				"©", "\\u00A9"
				]);
		llOwnerSay(jsonText);
		
		// To encode '["WebSite","http:\/\/my.com\/ask.php?what%20is%20it","\t"]'
		jsonText = uList2Json(JSON_ARRAY, [
				"WebSite",
				"http:\\/\\/my.com\\/ask.php?what%20is%20it",
				"\\t"
				]);
		llOwnerSay(jsonText);

		// Make a Json object...
		string temp = uList2Json(JSON_OBJECT, [
				"A", 99,
				"Z", "88]",
				"C", JSON_TRUE
				]);
		// ... add it to end of the array ...
		jsonText = uJsonSetValue(jsonText, [JSON_APPEND], temp);
		// ... change our web address ...
		jsonText = uJsonSetValue(jsonText, [1], "http:\\/\\/www.google.com");
		// ... change that TAB in the third spot to PI
		jsonText = uJsonSetValue(jsonText, [2], (string)PI);
		// ... and add a new "Key":Value pair to our object
		jsonText = uJsonSetValue(jsonText, [3, "New"], ((string)PI + "\\n"));
		
		//  ["WebSite","http:\/\/www.google.com",3.141593,{"A":99,"C":true,"New":"3.141593\n","Z":"88]"}]
		llOwnerSay(jsonText);
			   
	}
}

 [Edited to remove an unused variable and obtuse reference to "JAPH".]