Jump to content

Key2UTF() and UTF2Key()


irihapeti
 Share

You are about to reply to a thread that has been inactive for 3464 days.

Please take a moment to consider if this thread is worth bumping.

Recommended Posts

this the latest iteration of encoding and decoding keys stored in UTF16 strings. Is called oranges encoder

it started as a convo on sluniverse. The convo moved to here LSL Library forum. Then to LSL Scripting forum. Thru the convo was 4 iterations of this based on feedback and eagle eyes and ideas of other scripters

so thanks to all them who contributed to it and also thanks to the earlier works by other scripters on the lsl wiki portal (: They all named in the script and bc their contributions are mostly CCby3 then if publish/post/distribute then you need maintain their credits ok. For mine credits then dont worry about it bc I just put anything I do in public domain bc I dont publish/write much public code anyways

+

just about CCby3 credits. Why scripters do this mostly is bc they do scripting/programming for a living in whole or part. Is the whole giving back thing also which people do as well, but credits helps them to get their name out there which helps them get paid work and feed themself and family. So just do it ok

+

ETA: changes to credit texts in the script to reflect contributiions better. See convo below

++

ETA again: Have modded to this version 1b. functions enU() and deU()

to make more clear which bits are being encoded to UTF. The mod to deU also shows better the symmetry between encoding and decoding.

the codes produced by version 1b are compatible with 1a

+++

ETA again. changes to test code examples and change the words "design contributions" to "design credits". See convo below

 

+

ok here is

 

// example key encoder decoder
// version: oranges 1b
// public domain May 2014 by elizabeth (irihapeti)
// a mod of works (CCby0 and CCby3) by: Adam Wozniak, Doran Zemlja, 
// Becky Pippen, Combined Library
// design credits: All named above plus Strife Onizuka, LepreKhaun
// with mentions for exacts in the codes
// also with new test code examples
//
// encodes a key (uuid) into a string of 9 UTF16 chars. 18 bytes
// str = Key2UTF(key) 
// key = UTF2Key(str) 
//
// UTF16 codes produced are copypasta into scripts and notecards in 
// SL viewers 
// 3.7.3 (287491) Mar 4 2014 05:01:31 to
// 3.7.7 (289405) Apr 21 2014 20:25:08
// on Windows 8.1. Default USA english
// dunno about earlier/later versions, LSL Editor or TPVs or other OS
// just need be careful when copypasta as
// - some UTF glyphs may not be visible but are in the string
// - some of them may show as the same glyph. but they are decodeable
// so best to Say wrapped in "" for copypasta purposes 
//
// warninkkks !!! is no error trapping !!! if you need that then mod 
// important  !!! codes produced by this version 1b are compatible 
// with version 1a. Neither version 1a or 1b are compatible with 
// earlier unversioned iterations of this encoder that were written by
// me, due to a encoding change of the end buffer character
//
// also is wrote this way to show the algos. can mod to inline for own 
// purposes as you like
 
// -- encoder --- 
integer enB;
string enU(integer n)
{   // credit: Strife
    return llBase64ToString(llIntegerToBase64(0xE0808000 | 
        ((n & 0xF000) << 12) | ((n & 0x0FC0) << 10) | ((n & 0x003F) << 8) ));
}
  
string enP(integer n)
{   
    enB = (enB << 2) | (n & 0x3);
    return enU(((n >> 2) & 0x7FFF) + 0x800) + enU(((n >> 17) & 0x7FFF) + 0x800); 
}
  
string Key2UTF(key k)
{   
    string s = (string)k;
    string r = 
       enP((integer)("0x" + llGetSubString(s, 28, 35))) +
       enP((integer)("0x" + llGetSubString(s, 19, 22) + llGetSubString(s, 24, 27))) +
       enP((integer)("0x" + llGetSubString(s,  9, 12) + llGetSubString(s, 14, 17))) +
       enP((integer)("0x" + llGetSubString(s,  0,  7)));
    return r + enU((enB & 0xFF) + 0x8800); // credit: LepreKhaun
}
// --- end encoder ---
  
// --- decoder ---
integer deB;
list deH = ["0","1","2","3","4","5","6","7","8","9","a","b","c","d","e","f"];
  
integer deU(string s)
{  
    integer n = llBase64ToInteger(llStringToBase64(s));
    return ((n >> 12) & 0xF000) | ((n >> 10) & 0x0FC0) | ((n >> 8) & 0x003F);       
} 
  
string deP(string s)
{
    integer n = ((deU(llGetSubString(s, 1, 1)) - 0x800) << 17) |
                ((deU(llGetSubString(s, 0, 0)) - 0x800) <<  2) | (deB & 0x3);
    deB = deB >> 2;
    return  
        llList2String(deH, (n >> 28) & 0xF) + llList2String(deH, (n >> 24) & 0xF) +
        llList2String(deH, (n >> 20) & 0xF) + llList2String(deH, (n >> 16) & 0xF) +
        llList2String(deH, (n >> 12) & 0xF) + llList2String(deH, (n >>  8) & 0xF) +
        llList2String(deH, (n >>  4) & 0xF) + llList2String(deH, n & 0xF);
}
     
key UTF2Key(string s)
{  
    deB = deU(llGetSubString(s, 8, 8)) - 0x8800;
    return (key)(
        deP(llGetSubString(s, 6, 7)) + "-" +
        llInsertString(deP(llGetSubString(s, 4, 5)), 4, "-") + "-" +
        llInsertString(deP(llGetSubString(s, 2, 3)), 4, "-") +
        deP(llGetSubString(s, 0, 1)));
}
// --- end decoder ---
 
// --- some test code ---

list keys = 
[
    "ffffffff-ffff-ffff-ffff-ffffffffffff",
    "77777777-7777-7777-8888-888888888888",
    "88888888-8888-8888-7777-777777777777",
    "88888888-8888-8888-8888-888888888888",
    "77777777-7777-7777-7777-777777777777",
    "01234567-8901-2345-6789-012345678901",
    "01234567-89ab-cdef-fedc-ba9876543210",
    "abcdeffe-dcba-abcd-effe-dcbaabcdeffe",
    "c9012a38-3f29-6ef3-fc21-4380aecd5927",
    "00000000-0000-0000-0000-000000000000",
    "77777777-7777-7777-7777-777777777777"   // dup test          
];

string utft = "蟿蟿蟿蟿蟿蟿蟿蟿裿⨢䱄⨢䱄旝䎻旝䎻裰旝䎻旝䎻⨢䱄⨢䱄蠏⨢䱄⨢䱄⨢䱄⨢䱄蠀旝䎻旝䎻旝䎻旝䎻裿橀⪳䡈㯄僑䲀奙࢑裝ᒄ䌪㚦蝮筻䳕奙࢑裰菿左㼮翿㋳癝菿左袚幉彦壠蘐掼➔劎沀蠳ࠀࠀࠀࠀࠀࠀࠀࠀ蠀";

string utfp = "蟿蟿蟿蟿蟿蟿蟿蟿裿|⨢䱄⨢䱄旝䎻旝䎻裰|旝䎻旝䎻⨢䱄⨢䱄蠏|⨢䱄⨢䱄⨢䱄⨢䱄蠀|旝䎻旝䎻旝䎻旝䎻裿|橀⪳䡈㯄僑䲀奙࢑裝|ᒄ䌪㚦蝮筻䳕奙࢑裰|菿左㼮翿㋳癝菿左袚|幉彦壠蘐掼➔劎沀蠳|ࠀࠀࠀࠀࠀࠀࠀࠀ蠀";

default
{
    touch_end(integer total_number)
    {
        key k;
        key d;
        list ukeys;
        string s;
        string table;
        integer i;
        integer x;
        integer len;
                
        // key encode/decode check. read from list keys
        llOwnerSay("begin... encode/decode check");
        len = llGetListLength(keys);
        for (i = 0; i < len; i++)
        {
            k = llList2Key(keys, i);
            s = Key2UTF(k);
            d = UTF2Key(s);
            llOwnerSay("key= " + (string)k + " : \"" + s + "\" : " + (string)d);
        } 
        llOwnerSay("...end");  
     
        // create a string table. adding only unique keys
        llOwnerSay("begin... add unique keys to string table");        
        table = "";
        len = llGetListLength(keys);
        for (i = 0; i < len; i++)
        {
            k = llList2Key(keys, i);
            s = Key2UTF(k);
            if (llSubStringIndex(table, s) >= 0)
               llOwnerSay("found= \"" + s + "\" : " + (string)k);
            else
               table += s;
            k = llGenerateKey();
            s = Key2UTF(k);
            if (llSubStringIndex(table, s) >= 0)
               llOwnerSay("found= \"" + s + "\" : " + (string)k);
            else
               table += s;   
        } 
        llOwnerSay("...end");  
           
        // return position of key in table
        llOwnerSay("begin... return position of key in table");
        len = llGetListLength(keys);
        for (i = 0; i < len; i++)
        {
            k = llList2Key(keys, i);
            s = Key2UTF(k);
            x = llSubStringIndex(table, s);
            llOwnerSay("pos= " + (string)x + " : " + (string)k);
        }      
        llOwnerSay("...end");  
     
        // extract key from table by position
        llOwnerSay("begin... extract key by position");
        len = llStringLength(table);
        for (x = 0; x < len; x += 9)
        {
            s = llGetSubString(table, x, x + 8);    
            k = UTF2Key(s);
            llOwnerSay("pos= " + (string)x + " : " + (string)k);
        }
        llOwnerSay("...end");  

        // return index of key in table where index in ordinal set [0,1,2,..n]
        llOwnerSay("begin... return index of key in table");
        len = llGetListLength(keys);
        for (i = 0; i < len; i++)
        {
            k = llList2Key(keys, i);
            s = Key2UTF(k);
            x = llSubStringIndex(table, s) / 9;
            llOwnerSay("idx= " + (string)x + " : " + (string)k);
        }
        llOwnerSay("...end");  
    
        // extract key from table by index
        llOwnerSay("begin... extract key by index");
        len = llStringLength(table) / 9;
        for (x = 0; x < len; x++)
        {
            s = llGetSubString(table, x * 9, (x * 9) + 8);    
            k = UTF2Key(s);
            llOwnerSay("idx= " + (string)x + " : " + (string)k);
        }
        llOwnerSay("...end");  
    
        // create table enclosed in "" from list 
llOwnerSay("begin... create table from list"); len = llGetListLength(keys) - 1; // skip dup test s = "\""; for (i = 0; i < len; i++) { k = llList2Key(keys, i); s += Key2UTF(k); } s += "\""; llOwnerSay(s); llOwnerSay("...end"); // decode to list a table (utft) copypasta into this script llOwnerSay("begin... decode table to list"); ukeys = []; len = llStringLength(utft); for (x = 0; x < len; x += 9) { s = llGetSubString(utft, x, x + 8); ukeys += UTF2Key(s); k = llList2Key(ukeys, -1); llOwnerSay("key= " + (string)k); } llOwnerSay("...end"); // create a pipeset from list at the cost of 1 extra char per key // for visual cue purposes if visual cue needed llOwnerSay("begin... create pipeset separated by |"); len = llGetListLength(keys) - 1; // skip dup test s = "\""; for (i = 0; i < (len - 1); i++) { k = llList2Key(keys, i); s += Key2UTF(k) + "|"; } k = llList2Key(keys, i); s += Key2UTF(k) + "\""; llOwnerSay(s); llOwnerSay("...end"); // decode to list a pipeset (utfp) copypasta into this script llOwnerSay("begin... decode pipeset separated by | to list"); ukeys = []; len = llStringLength(utfp); for (x = 0; x < len; x += 10) { s = llGetSubString(utfp, x, x + 8); ukeys += UTF2Key(s); k = llList2Key(ukeys, -1); llOwnerSay("key= " + (string)k); } llOwnerSay("...end"); } }

 

 

 

Link to comment
Share on other sites

i give you credit bc of the change to the end buffer character mapping. Is a really signicant change this. It allow multiple keys (self-terminating) to be stored in strings without cross-boundary conflict/collision

this change is a design change and a reallly good one

can see that I remapped to range 0x8800..0x88FF so that can be encoded/decoded using the same conversion code. But without your design change suggestion it wouldnt have happened in this codes

good design is what I think is all about

Adam and Zoran get credits bc of their design pioneering works in this field. Becky get a credit for advancing those designs and pioneering works herself. And bc CCby3 license, in the case of Combined Library. Assume Becky also CCby3  bc unspecified

I changed their designs to encode right-to-left instead of left-to-right to enable simpler decoding. Strife suggest to change the design to eliminate using int2hex in the encoding. You LepreKhaun suggest to change the design to enable self-terminating encoded keys

So like the others mentioned you deserve a credit for your contribution to the design. Is a design credit this. Is fully justified this and one I will always give you

I made a small change to the script credits text to reflect this a little bit better

Link to comment
Share on other sites

If you were using my suggestion (encoding the 9th character differently than the previous 8) correctly, you'd not have need for the pipe-delimited string. As it is, it appears my suggestion added nothing to the original algorithm as you wrote it in SLUniverse.

 

In other words, my suggestion has nothing to do with this code unless it is actually using a string for storage that doesn't depend on a pipe ("|") to separate the coded keys. Lacking that, it simply complicates the original algorithm unnecessarily. Your usage examples should reflect the change (ie 'string utfs = "蟿蟿蟿蟿蟿蟿蟿蟿裿|⨢䱄⨢䱄旝䎻旝䎻裰|旝䎻旝䎻⨢䱄⨢䱄蠏|⨢䱄⨢䱄⨢䱄⨢䱄蠀|旝䎻旝䎻旝䎻旝䎻裿|橀⪳䡈㯄僑䲀奙࢑裝|ᒄ䌪㚦蝮筻䳕奙࢑裰|菿左㼮翿㋳癝菿左袚|幉彦壠蘐掼➔劎沀蠳|ࠀࠀࠀࠀࠀࠀࠀࠀ蠀";' should read 'string utfs = "蟿蟿蟿蟿蟿蟿蟿蟿裿⨢䱄⨢䱄旝䎻旝䎻裰旝䎻旝䎻⨢䱄⨢䱄蠏⨢䱄⨢䱄⨢䱄⨢䱄蠀旝䎻旝䎻旝䎻旝䎻裿橀⪳䡈㯄僑䲀奙࢑裝ᒄ䌪㚦蝮筻䳕奙࢑裰菿左㼮翿㋳癝菿左袚幉彦壠蘐掼➔劎沀蠳ࠀࠀࠀࠀࠀࠀࠀࠀ蠀";') which eliminates breaking the string out into a list (just step through it 9 characters at a time).

Link to comment
Share on other sites


irihapeti wrote:

they examples of how it can be used

the string table example shows how to read and write native sets

the pipeset example shows how to format for that use case 

But the point is, the suggestion you're crediting me for makes a pipe delimited table completely unnecesary and doubly wasteful of memory ("doubly wasteful" in that 2 bytes are required for each pipe used and you are pulling out a list from the string table). Your third example of a "stringtable" is spot on but the first two, with the pipes, miss the mark because they imply that the pipes are somehow needed (which was the case in your original SLUniverse code).

 

Why incorporate a suggestion if you're not going to use it?

Link to comment
Share on other sites

changing the end character encoding doesnt mean that the pipeset is only now redundant. It was always redundant bc fixed length of 9 chars

for (i == 0; i < lenset; i += 9)
   k = UTF2Key(llGetSubString(set, i, i + 8));

is the same code that would be used for both old and new end encodings

+

it maybe that you had a impression that the first iteration was returning variable length codes. It wasnt, they were always 9 char lengths for all

+

Pipe sets was a use case raised in the first convo. Why the user chose to do that may simply have been so that when copypasta they got a visual cue that they did have the number of keys intended. At some loss of storage space

copypasta can have visual side effects depending on OS and fonts installed which can be visually confusing sometimes. Inserting an extra delimiter like | or ~ or , etc can be a aid to helping with this (at some loss of storage space)
 
i just coded this into the test to show how pipe/delimited sets may be encoded and decoded for that particular use case raised in the convo

+

is also true for old and new that with pipesets then this works also

for (i == 0; i < lenset; i += 10)
   k = UTF2Key(llGetSubString(set, i, i + 8));

+

if you wish to rewrite the test codes or even mod the whole thing as a help to intending users/readers then you are free to do this

Link to comment
Share on other sites


irihapeti wrote:

changing the end character encoding doesnt mean that the pipeset is only now redundant. It was always redundant bc fixed length of 9 chars

for (i == 0; i < lenset; i += 9)

   k = UTF2Key(llGetSubString(set, i, i + 8));

is the same code that would be used for both old and new end encodings

+

it maybe that you had a impression that the first iteration was returning variable length codes. It wasnt, they were always 9 char lengths for all

+

Pipe sets was a use case raised in the first convo. Why the user chose to do that may simply have been so that when copypasta they got a visual cue that they did have the number of keys intended. At some loss of storage space

copypasta can have visual side effects depending on OS and fonts installed which can be visually confusing sometimes. Inserting an extra delimiter like | or ~ or , etc can be a aid to helping with this (at some loss of storage space)

 

i just coded this into the test to show how pipe/delimited sets may be encoded and decoded for that particular use case raised in the convo

+

is also true for old and new that with pipesets then this works also

for (i == 0; i < lenset; i += 10)

   k = UTF2Key(llGetSubString(set, i, i + 8));

+

if you wish to rewrite the test codes or even mod the whole thing as a help to intending users/readers then you are free to do this

And now you've argued full circle back to my original contention that I have made no significant contribution to your code. So, crediting me alongside those whose work you did build upon is inappropriate.

 

The only required change to be made is removal of my name from it. Thank you.

Link to comment
Share on other sites

that a person may choose to implement their own use case with a extra visual cue delimiter for their own copypasta purposes dont negate the importance of the design change for those users who will implement string tables natively

i can no more not acknowledge your design contribution to the algo than I can not acknowledge people like Horst Feistel in other works I might publish. Would be unprofessional for me to not acknowledge. you, him or anyone else

 

Link to comment
Share on other sites

You're working on key compression using what is known as a "Base 32768 Scheme". Basically this means splitting the 128 bits of a UUID key into 8 groups of 16 bits each and then encoding 9 UTF16 characters within the Unicode BMP (Basic Multilingual Plane) from them.

This work apparently was started from a (somewhat off-topic) discussion in March of 2009
found here  starting with "Stephen Psaltery 2009-03-27 13:40:06-04" that includes Kelly Linden's response (which compressed a key to 22 characters) through Doran Zemlja's first compression to 9 UTF16 characters. Doran's implementation was later tweaked by their coding partner, Adam Wozniak, and Haravikk Mistral then included that version in a collection of compression schemes they had assembled (many of which were their own).

At the same time, Becky Pippen was also working on compression schemes that also (early on) used llUnescapeURL() to form UTC code points. Their work (here, here and primarily here) gives the best definition of the problem and (more importantly) gives the links to the reference material to properly understand both the problem as well as the offered solutions.

What is of note is that these early solutions all relied on llUnescapeURL() to obtain the final result and it was Strife Onizuka who made the radical suggestion in Jan of 2010 to use llBase64ToString(llIntegerToBase64()), which was based upon their work of the Combined Library, dating from 2008. This approach was proven to be superior in both time and memory usage (a rare combination, it's usually one or the other).

And that's about the sum of it- some great coders figured out this algorithm after untold hours of study and experimentations, published their results for the rest of us to use and Strife directed these early studies in the direction which you finally chose to modify.

You might notice the lack of my name in this list and for good reason: I not only had not yet joined Second Life at that point, I'll always be several ranks below the coding ability of any of those mentioned. In other words, I'm not only not in the same league as these coders, I wasn't even playing when they took to the field and worked all this out.

I have never knowingly nor willingly made any contribution towards any of your efforts that you could use my name without permission after the fact. Including my name with such as these coding greats is not flattering, nor is it appropriate. In fact, it is an embarrassment that I humbly beg you to forgo your stubborn, contrary arguments and simply remove the inappropriate credit from your code. PLEASE!

Link to comment
Share on other sites


LepreKhaun wrote:

You're working on key compression using what is known as a "Base 32768 Scheme". Basically this means splitting the 128 bits of a UUID key into 8 groups of 16 bits each and then encoding 9 UTF16 characters within the Unicode BMP (
) from them.

This work apparently was started from a (somewhat off-topic) discussion in March of 2009

found
 starting with "
Stephen Psaltery 2009-03-27 13:40:06-04
" that includes Kelly Linden's response (which compressed a key to 22 characters) through Doran Zemlja's first compression to 9 UTF16 characters. Doran's implementation was later tweaked by their coding partner, Adam Wozniak, and Haravikk Mistral then included that version in
they had assembled (many of which were their own).

At the same time, Becky Pippen was also working on compression schemes that also (early on) used
llUnescapeURL()
to form UTC code points. Their work (
,
and
) gives the best definition of the problem and (more importantly) gives the links to the reference material to properly understand both the problem as well as the offered solutions.

What is of note is that these early solutions all relied on
llUnescapeURL()
to obtain the final result and it was Strife Onizuka who made the
in Jan of 2010 to use
llBase64ToString(llIntegerToBase64())
, which was based upon their work of the
, dating from 2008. This approach was proven to be superior in both time and memory usage (a rare combination, it's usually one or the other).

And that's about the sum of it- some great coders figured out this algorithm after untold hours of study and experimentations, published their results for the rest of us to use and Strife directed these early studies in the direction which you finally chose to modify.

You might notice the lack of my name in this list and for good reason: I not only had not yet joined Second Life at that point, I'll always be several ranks below the coding ability of any of those mentioned. In other words, I'm not only not in the same league as these coders, I wasn't even playing when they took to the field and worked all this out.

I have never knowingly nor willingly made any contribution towards any of your efforts that you could use my name without permission after the fact. Including my name with such as these coding greats is not flattering, nor is it appropriate. In fact, it is an embarrassment that I humbly beg you to forgo your stubborn, contrary arguments and simply remove the inappropriate credit from your code.
PLEASE!

if I am being stubborn (and I am actual not) is bc of the credit/citation publishing rules

+

in all the previous LSL literature and works on encoding LSL keys to UTF-16 compatible chars is no mention anywhere to map the end buffer character to a different BMP section to enable llSubStringIndex to find UTF-16 compatible encoded keys in a string table without cross-boundary conflict/collision. Maybe someone somewhere else maybe did already think of it, but is not in the literature. You put it into the public domain literature [ETA] in a convo we were having about it elsewhere. So you get cited in this LSL published works for this invention

LepreKhaun you either invented this or you didnt. Is nothing in the prior literature or in what you have said to suggest that you didnt. So you get cited as the design inventor of the end buffer character mapping

if you (or anyone reading) can show somebody else who prior publish a LSL work with this end coding design invention or prior made such a suggestion in any LSL forums then I will change the citation/credit to them as is proper. I am not going to change the citation without this. bc is a code submission to the LL LSL library. Is not just some codes I pasted to some random forum chat

+

aww gosh imma not that great !!! is not grounds for a non-citation in a published work. bc credit/citation publishing rules. Nobody gets to choose which of these rules applies to them just bc of how they feel

if I did remove the citation then I would have to remove the script bc of the rules. I am not going to remove the script just bc awww! gosh

so either come up with another person as the design inventor to get yourself off the credit/citation list or your name stays as it must by the rules

Link to comment
Share on other sites


irihapeti wrote:

if I am being stubborn (and I am actual not) is bc of the credit/citation publishing rules

+

in all the previous LSL literature and works on encoding LSL keys to UTF-16 compatible chars is no mention anywhere to map the end buffer character to a different BMP section to enable llSubStringIndex to find UTF-16 compatible encoded keys in a string table without cross-boundary conflict/collision. Maybe someone somewhere else maybe did already think of it, but is not in the literature. You put it into the public
domain
 literature [ETA] in a convo we were having about it elsewhere. So you get cited in this LSL published works for this invention

LepreKhaun you either invented this or you didnt. Is nothing in the prior literature or in what you have said to suggest that you didnt. So you get cited as the design inventor of the end buffer character mapping

if you (or anyone reading) can show somebody else who prior publish a LSL work with this end coding design invention or prior made such a suggestion in any LSL forums then I will change the citation/credit to them as is proper. I am not going to change the citation without this. bc is a code submission to the LL LSL library. Is not just some codes I pasted to some random forum chat

+

aww gosh imma not that great !!! is not grounds for a non-citation in a published work. bc credit/citation publishing rules. Nobody gets to choose which of these rules applies to them just bc of how they feel

if I did remove the citation then I would have to remove the script bc of the rules. I am not going to remove the script just bc awww! gosh

so either come up with another person as the design inventor to get yourself off the credit/citation list or your name stays as it must by the rules

We've already shown that I "invented" nothing that you're making use of in your code. 

 

You seem to have taken the view that you get to decide what is the appropriate use of someone else's name, even when they beg you to reconsider. Well, that's your problem and all I can do is politely ask one last time to please remove the inappropriate use of my name in your code.

Link to comment
Share on other sites


irihapeti wrote:


LepreKhaun wrote:


 

We've already shown that I "invented" nothing that you're making use of in your code. 

 

the last example in the test codes clearly shows how your invention is used

Where you're relying on 

    return r + enU((enB & 0xFF) + 0x8800);

Which is YOUR OWN, original code.

 

 I had suggested an entirely different approach, for an entirely different program you'd written and my suggestion was merely what any intermediate programmer that was familiar with the subject and the literature of those that had worked within it would see as obvious. As Nexii Malthus did FIVE YEARS AGO.

 

"My invention", sheesh! Next you'll be crediting me with inventing the Marketplace.

 

I can't help you get your facts straight but that credit you give IS inappropriate, and I have no idea why you can't see it.

Link to comment
Share on other sites


LepreKhaun wrote:


irihapeti wrote:


LepreKhaun wrote:


 

We've already shown that I "invented" nothing that you're making use of in your code. 

 

the last example in the test codes clearly shows how your invention is used

Where you're relying on 
    return r + enU((enB & 0xFF) + 0x8800);

Which is YOUR OWN, original code.

 

, for an entirely different program you'd written and my suggestion was merely what any intermediate programmer that was familiar with the subject and the literature of those that had worked within it would see as obvious. As
did FIVE YEARS AGO.

 

"My invention", sheesh! Next you'll be crediting me with inventing the Marketplace.

 

I can't help you get your facts straight but that credit you give IS inappropriate, and I have no idea why you can't see it.

that I changed the map location dont negate the design. No more than me changing the codes I get of Strife from << & to & << does. [ETA} and removing Becky's code from deU and changing it to a >> & mod to show symmetry

seems to me that your main objection is to the test code examples. From which you seem to be extrapolating everything else. I made a mod to these to better show your design contribution for which you been properly credited

 

Link to comment
Share on other sites


irihapeti wrote:


LepreKhaun wrote:


irihapeti wrote:


LepreKhaun wrote:


 

We've already shown that I "invented" nothing that you're making use of in your code. 

 

the last example in the test codes clearly shows how your invention is used

Where you're relying on 
    return r + enU((enB & 0xFF) + 0x8800);

Which is YOUR OWN, original code.

 

, for an entirely different program you'd written and my suggestion was merely what any intermediate programmer that was familiar with the subject and the literature of those that had worked within it would see as obvious. As
did FIVE YEARS AGO.

 

"My invention", sheesh! Next you'll be crediting me with inventing the Marketplace.

 

I can't help you get your facts straight but that credit you give IS inappropriate, and I have no idea why you can't see it.

that I changed the map location dont negate the design. No more than me changing the codes I get of Strife from << & to & << does. [ETA} and removing Becky's code from deU and changing it to a >> & mod to show symmetry

seems to me that your main objection is to the test code examples. From which you seem to be extrapolating everything else. I made a mod to these to better show your design contribution for which you been properly credited

 

When the other has spoken plainly, "extrapolating" means you have chosen not to hear what is being said.

 

Too bad...

Link to comment
Share on other sites


LepreKhaun wrote:


irihapeti wrote:

jejejejje (:

can still remember that convo

+

when you going to start the ethics debate in GD ?? Kant and Rawls and Locke and all that

Thank you for clarifying your motivations.

one day like  probably never you going to get over yourself. in the meantime... oranges

Link to comment
Share on other sites


irihapeti wrote:


LepreKhaun wrote:


irihapeti wrote:

jejejejje (:

can still remember that convo

+

when you going to start the ethics debate in GD ?? Kant and Rawls and Locke and all that

Thank you for clarifying your motivations.

one day like  probably never you going to get over yourself. in the meantime... oranges

Oh, btw, I think it's great that you're modifying other people's code, we all had to start somewhere.

 

But there are some requirements when you're doing a derivative work of anything CC-by-# and you might want to learn them at some point. One of them is you're required to link to the license.

 

Here's a nice little tutorial to help you get started- http://creativecommons.org.au/materials/attribution.pdf

 

Link to comment
Share on other sites


irihapeti wrote:

this the latest iteration of encoding and decoding keys stored in UTF16 strings. Is called oranges encoder

it started as a convo on sluniverse. The convo moved to here LSL Library forum. Then to LSL Scripting forum. Thru the convo was 4 iterations of this based on feedback and eagle eyes and ideas of other scripters

so thanks to all them who contributed to it and also thanks to the earlier works by other scripters on the lsl wiki portal (: They all named in the script and bc their contributions are mostly CCby3 then if publish/post/distribute then you need maintain their credits ok. For mine credits then dont worry about it bc I just put anything I do in public domain bc I dont publish/write much public code anyways

+

just about CCby3 credits. Why scripters do this mostly is bc they do scripting/programming for a living in whole or part. Is the whole giving back thing also which people do as well, but credits helps them to get their name out there which helps them get paid work and feed themself and family. So just do it ok

+

ETA: changes to credit texts in the script to reflect contributiions better. See convo below

++

ETA again: Have modded to this version 1b. functions enU() and deU()

to make more clear which bits are being encoded to UTF. The mod to deU also shows better the symmetry between encoding and decoding.

the codes produced by version 1b are compatible with 1a

+++

ETA again. changes to test code examples and change the words "design contributions" to "design credits". See convo below

 

+

ok here is

 
// example key encoder decoder// version: oranges 1b// public domain May 2014 by elizabeth (irihapeti)// a mod of works (CCby0 and CCby3) by: Adam Wozniak, Doran Zemlja, // Becky Pippen, Combined Library// design credits: All named above plus Strife Onizuka, LepreKhaun// with mentions for exacts in the codes// also with new test code examples//// encodes a key (uuid) into a string of 9 UTF16 chars. 18 bytes// str = Key2UTF(key) // key = UTF2Key(str) //// UTF16 codes produced are copypasta into scripts and notecards in // SL viewers // 3.7.3 (287491) Mar 4 2014 05:01:31 to// 3.7.7 (289405) Apr 21 2014 20:25:08// on Windows 8.1. Default USA english// dunno about earlier/later versions, LSL Editor or TPVs or other OS// just need be careful when copypasta as// - some UTF glyphs may not be visible but are in the string// - some of them may show as the same glyph. but they are decodeable// so best to Say wrapped in "" for copypasta purposes //// warninkkks !!! is no error trapping !!! if you need that then mod // important  !!! codes produced by this version 1b are compatible // with version 1a. Neither version 1a or 1b are compatible with // earlier unversioned iterations of this encoder that were written by// me, due to a encoding change of the end buffer character//// also is wrote this way to show the algos. can mod to inline for own // purposes as you like // -- encoder --- integer enB;string enU(integer n){   // credit: Strife    return llBase64ToString(llIntegerToBase64(0xE0808000 |         ((n & 0xF000) << 12) | ((n & 0x0FC0) << 10) | ((n & 0x003F) << 8) ));}  string enP(integer n){       enB = (enB << 2) | (n & 0x3);    return enU(((n >> 2) & 0x7FFF) + 0x800) + enU(((n >> 17) & 0x7FFF) + 0x800); }  string Key2UTF(key k){       string s = (string)k;    string r =        enP((integer)("0x" + llGetSubString(s, 28, 35))) +       enP((integer)("0x" + llGetSubString(s, 19, 22) + llGetSubString(s, 24, 27))) +       enP((integer)("0x" + llGetSubString(s,  9, 12) + llGetSubString(s, 14, 17))) +       enP((integer)("0x" + llGetSubString(s,  0,  7)));    return r + enU((enB & 0xFF) + 0x8800); // credit: LepreKhaun}// --- end encoder ---  // --- decoder ---integer deB;list deH = ["0","1","2","3","4","5","6","7","8","9","a","b","c","d","e","f"];  integer deU(string s){      integer n = llBase64ToInteger(llStringToBase64(s));    return ((n >> 12) & 0xF000) | ((n >> 10) & 0x0FC0) | ((n >> 8) & 0x003F);       }   string deP(string s){    integer n = ((deU(llGetSubString(s, 1, 1)) - 0x800) << 17) |                ((deU(llGetSubString(s, 0, 0)) - 0x800) <<  2) | (deB & 0x3);    deB = deB >> 2;    return          llList2String(deH, (n >> 28) & 0xF) + llList2String(deH, (n >> 24) & 0xF) +        llList2String(deH, (n >> 20) & 0xF) + llList2String(deH, (n >> 16) & 0xF) +        llList2String(deH, (n >> 12) & 0xF) + llList2String(deH, (n >>  8) & 0xF) +        llList2String(deH, (n >>  4) & 0xF) + llList2String(deH, n & 0xF);}     key UTF2Key(string s){      deB = deU(llGetSubString(s, 8, 8)) - 0x8800;    return (key)(        deP(llGetSubString(s, 6, 7)) + "-" +        llInsertString(deP(llGetSubString(s, 4, 5)), 4, "-") + "-" +        llInsertString(deP(llGetSubString(s, 2, 3)), 4, "-") +        deP(llGetSubString(s, 0, 1)));}// --- end decoder --- // --- some test code ---list keys = [    "ffffffff-ffff-ffff-ffff-ffffffffffff",    "77777777-7777-7777-8888-888888888888",    "88888888-8888-8888-7777-777777777777",    "88888888-8888-8888-8888-888888888888",    "77777777-7777-7777-7777-777777777777",    "01234567-8901-2345-6789-012345678901",    "01234567-89ab-cdef-fedc-ba9876543210",    "abcdeffe-dcba-abcd-effe-dcbaabcdeffe",    "c9012a38-3f29-6ef3-fc21-4380aecd5927",    "00000000-0000-0000-0000-000000000000",    "77777777-7777-7777-7777-777777777777"   // dup test          ];string utft = "蟿蟿蟿蟿蟿蟿蟿蟿裿⨢䱄⨢䱄旝䎻旝䎻裰旝䎻旝䎻⨢䱄⨢䱄蠏⨢䱄⨢䱄⨢䱄⨢䱄蠀旝䎻旝䎻旝䎻旝䎻裿橀⪳䡈㯄僑䲀奙࢑裝ᒄ䌪㚦蝮筻䳕奙࢑裰菿左㼮翿㋳癝菿左袚幉彦壠蘐掼➔劎沀蠳ࠀࠀࠀࠀࠀࠀࠀࠀ蠀";string utfp = "蟿蟿蟿蟿蟿蟿蟿蟿裿|⨢䱄⨢䱄旝䎻旝䎻裰|旝䎻旝䎻⨢䱄⨢䱄蠏|⨢䱄⨢䱄⨢䱄⨢䱄蠀|旝䎻旝䎻旝䎻旝䎻裿|橀⪳䡈㯄僑䲀奙࢑裝|ᒄ䌪㚦蝮筻䳕奙࢑裰|菿左㼮翿㋳癝菿左袚|幉彦壠蘐掼➔劎沀蠳|ࠀࠀࠀࠀࠀࠀࠀࠀ蠀";default{    touch_end(integer total_number)    {        key k;        key d;        list ukeys;        string s;        string table;        integer i;        integer x;        integer len;                        // key encode/decode check. read from list keys        llOwnerSay("begin... encode/decode check");        len = llGetListLength(keys);        for (i = 0; i < len; i++)        {            k = llList2Key(keys, i);            s = Key2UTF(k);            d = UTF2Key(s);            llOwnerSay("key= " + (string)k + " : \"" + s + "\" : " + (string)d);        }         llOwnerSay("...end");               // create a string table. adding only unique keys        llOwnerSay("begin... add unique keys to string table");                table = "";        len = llGetListLength(keys);        for (i = 0; i < len; i++)        {            k = llList2Key(keys, i);            s = Key2UTF(k);            if (llSubStringIndex(table, s) >= 0)               llOwnerSay("found= \"" + s + "\" : " + (string)k);            else               table += s;            k = llGenerateKey();            s = Key2UTF(k);            if (llSubStringIndex(table, s) >= 0)               llOwnerSay("found= \"" + s + "\" : " + (string)k);            else               table += s;           }         llOwnerSay("...end");                     // return position of key in table        llOwnerSay("begin... return position of key in table");        len = llGetListLength(keys);        for (i = 0; i < len; i++)        {            k = llList2Key(keys, i);            s = Key2UTF(k);            x = llSubStringIndex(table, s);            llOwnerSay("pos= " + (string)x + " : " + (string)k);        }              llOwnerSay("...end");               // extract key from table by position        llOwnerSay("begin... extract key by position");        len = llStringLength(table);        for (x = 0; x < len; x += 9)        {            s = llGetSubString(table, x, x + 8);                k = UTF2Key(s);            llOwnerSay("pos= " + (string)x + " : " + (string)k);        }        llOwnerSay("...end");          // return index of key in table where index in ordinal set [0,1,2,..n]        llOwnerSay("begin... return index of key in table");        len = llGetListLength(keys);        for (i = 0; i < len; i++)        {            k = llList2Key(keys, i);            s = Key2UTF(k);            x = llSubStringIndex(table, s) / 9;            llOwnerSay("idx= " + (string)x + " : " + (string)k);        }        llOwnerSay("...end");              // extract key from table by index        llOwnerSay("begin... extract key by index");        len = llStringLength(table) / 9;        for (x = 0; x < len; x++)        {            s = llGetSubString(table, x * 9, (x * 9) + 8);                k = UTF2Key(s);            llOwnerSay("idx= " + (string)x + " : " + (string)k);        }        llOwnerSay("...end");              // create table enclosed in "" from list 

llOwnerSay("begin... create table from list"); len = llGetListLength(keys) - 1; // skip dup test s = "\""; for (i = 0; i < len; i++) { k = llList2Key(keys, i); s += Key2UTF(k); } s += "\""; llOwnerSay(s); llOwnerSay("...end"); // decode to list a table (utft) copypasta into this script llOwnerSay("begin... decode table to list"); ukeys = []; len = llStringLength(utft); for (x = 0; x < len; x += 9) { s = llGetSubString(utft, x, x + 8); ukeys += UTF2Key(s); k = llList2Key(ukeys, -1); llOwnerSay("key= " + (string)k); } llOwnerSay("...end"); // create a pipeset from list at the cost of 1 extra char per key // for visual cue purposes if visual cue needed llOwnerSay("begin... create pipeset separated by |"); len = llGetListLength(keys) - 1; // skip dup test s = "\""; for (i = 0; i < (len - 1); i++) { k = llList2Key(keys, i); s += Key2UTF(k) + "|"; } k = llList2Key(keys, i); s += Key2UTF(k) + "\""; llOwnerSay(s); llOwnerSay("...end"); // decode to list a pipeset (utfp) copypasta into this script llOwnerSay("begin... decode pipeset separated by | to list"); ukeys = []; len = llStringLength(utfp); for (x = 0; x < len; x += 10) { s = llGetSubString(utfp, x, x + 8); ukeys += UTF2Key(s); k = llList2Key(ukeys, -1); llOwnerSay("key= " + (string)k); } llOwnerSay("...end"); }}

 

Yeah, it's a shame Strife publishes their work under CC-by-3.0. I could never understand that, it certainly isn't recommended but it is their right. But the main problem is that you can't get automatic restoration of the license by rectifying your defect as one can under CC-by-4.0. Instead, you have to go explain to Strife what you'd done and then get their express permission before you can come under the CC umbrella. But I'm certain you won't have any problem with that, Strife's always willing to help.

 

And, when you do correct the defects in that licensing, I ask you to remove the "// credit: LepreKhaun" from the code body since it looks too much like I actually wrote the line of code, which is entirely your own work. The "design credits" in the header I can live with though, you can leave or omit them as you wish.

 

I'm asking this as a favor and because when I went to check into all the "publishing rules for crediting sources", I found all this, which pretty much contradicts your position.

http://copyrighttoolkit.com/moralrights.html

http://www.artslaw.com.au/info-sheets/info-sheet/moral-rights/

http://policy.mofcom.gov.cn/english/flaw!fetch.action?id=748b00c1-312f-489b-8d9e-cc535a50f422&pager.pageNo=73

http://niptech.wordpress.com/intellectual-property/intellectual-assets/creative-works/moral-rights/

http://www.smartcopying.edu.au/copyright-guidelines/copyright---a-general-overview/1-16-moral-rights

http://scholarship.law.georgetown.edu/cgi/viewcontent.cgi?article=1415&context=facpub

http://www.copyright.com.au/get-information/other-rights/moral-rights

http://www.seqlegal.com/blog/false-attribution-moral-right

http://www.artslaw.com.au/info-sheets/info-sheet/moral-rights-infringement-and-letter-of-demand/

http://books.google.com/books?id=smES_wO_NscC&pg=PA149&lpg=PA149&dq=false+attribution+of+authorship&source=bl&ots=YX6x-rsHbU&sig=UBbMX5gjNS6Kh0Uw8Gq148573ZQ&hl=en&sa=X&ei=Afh_U6qfCcGNyATZiYLgDA&ved=0CEYQ6AEwBDgU#v=onepage&q=false%20attribution%20of%20authorship&f=false

http://policy.mofcom.gov.cn/english/flaw!fetch.action?id=748b00c1-312f-489b-8d9e-cc535a50f422&pager.pageNo=73

http://www.newmediarights.org/guide/how_to/creative_commons/best_practices_creative_commons_attributions

Anyway, if you need help before your next submission to avoid these kinds of problems, just ask. Any number of us I'm certain would be glad to help, myself included.

Link to comment
Share on other sites

  • 4 months later...

I tested; you can store 20 bit per character by using only the 21-bit-code point range of utf8. sadly lsl is bound to use "utf8 code points" because thats how lsl converts unicode to base64. 21 bit per character wont work because you are skipping 2^16 to start with the 21-bit code point range. skipping 2^16 also skips all the invalid utf8 characters, including U+FFFE by simply adding +2^16 before you convert an integer to a character, and by substracting 2^16 after you converted the character back..

Your highest possible base is 2^21-2^16 (a higher base simply isnt worth the hassle of checking valid character ranges in more detail...), and since we only care for whole bits, thats base 2^20 to encode 20 bits per character. (and no higher base that doesnt reach base 2^12 will change that)

And no; there is no support for the 26-bit or 31-bit code point ranges for utf8 in sl whatsoever. someone once made an "RFC" to "cut off" these cote point ranges off utf8 (that barely anyone was using anyways) to ensure fast compatibility with utf16.

---

when i wrote my integer to utf8 encoder back in january 2008 (almost pre-mono era) ,  it was smart to only encode up to the 16-bit-code-point range, because lsl was still allowing utf8 characters to be written in object descripotions and names. but the 21-bit code point range was always written as two characters, and that made it a waste of space when you wanted to store many bits in as few characters as possible. bacj then you could store a key in 9 letters in an object description. this is no longer possible. You wanted to use the same function that stores in object description as you use to store in a string. so 15 bit per character was a reasonable limit.

now object descriptions only allow for 2^7 different characters of the small ascii range. And a 21-bit code point character is only 1 character for the mono compiler , stored as 2 byte character after a 16-byte string-header. Now you can happily store 20 bit per character of a string within lsl code or get data as compat via httprequest. Because ther is no more compatibility with storeing base 2^15 in a prims object description. base 2^7 is jsut too small to bother. use unicode to store in base 2^20,  all the way!

Link to comment
Share on other sites

You are about to reply to a thread that has been inactive for 3464 days.

Please take a moment to consider if this thread is worth bumping.

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now
 Share

×
×
  • Create New...