Web Prim Potentialities

[Macrocosm Draegonne]

Web on a prim is really cool, the localized data technique inspired me a lot, and especially after connecting jquery and other libraries externally, I am a web developer and dabble in a lot of code bases.

One idea about web-audio Ill share here with a few questions & thoughts if anyone wants to chime in....

Id love to build a sound track that is spatially experienced, depending on the area and context of the zone/activity, as well as micro-location within the zone, with differing and some crossover blending audio modulation and feedback.  This would be fairly minimal in most areas of course, using the howler.js cache plus procedurally generated sounds will mean very little to no bandwidth needed for most things.  Except delivering actual sound files (if used) of course, though... if you check out howler.js & other libraries you'll see a minimal set of sounds can be reused and tweaked in many ways, even synthesized in real time to produce any sound imaginable.

Can a webprims audio output be amplified and repeated/extended, spatially speaking?  Some things id want to triangulate an audio to amplify the sound in an area like a main track etc, not using prim copies as they could go off in timings.  Using a library like howler.js & others we can have excellent caching, looping, orchestrating, streaming, all manner of possibilities, all lightweight & fast webstandards based too.

Will making these web-audio web-prims hidden reduce viewer strain they may cause?  Along with hiding controls & interactivity, & setting to autoplay.   Web prims perform well on my system, but im not sure of others.  This use case really all I want is the audio, so that would be less strain potentially right?

What are the good the bad and the ugly about SL webprims that will effect such a creation and others?   Can LSL work those with good performance?  Is there something we could ask of LL to enhance this area for us? ?  It would be nice to have a few different types of web-prims, something like im discussing here could benefit from an Audio Only type with relevant functionalities.  

Generally speaking, this whole web-enabled area being expanded upon, with a standards based approach, could be a boon to in world potentials.  Imagine HUD's that are all html/css using no graphics files, but looking great, with visual cues & helpers, popups, mouseovers, animations, feedback.   Interactive signs/boards/books, that can be translated & edited, using fonts and little to no textures.      Connecting cloud based AI to SIM's and objects.  lol ?

------------------

Alright I have to stop or ill keep rambling, I will tinker with all this in time,  but I am curious what others think and have done in this area, there's not a lot googling since this area is more technical than I think most would want to bother with, but the potentials are too interesting not to at least check it out heheh.

I might post some web-audio examples I find below.

[Macrocosm Draegonne]

Here's a few examples

https://howlerjs.com/

https://tonejs.github.io/examples

https://www.borjamorales.com/3d-piano-player/

http://qiao.github.io/euphony/#158

https://www.npmjs.com/package/midi-sounds-react

https://webaudioapi.com/samples/

http://webaudiodemos.appspot.com/MIDIDrums/index.html

https://alemangui.github.io/pizzicato/

 

 

Edited October 7, 2018 by Macrocosm Draegonne

[Qie Niangao]

16 hours ago, Macrocosm Draegonne said:

Interactive signs/boards/books, that can be translated & edited, using fonts and little to no textures.

Here's something relevant: llGetAgentLanguage. Back before there was such a thing as Shared Media (aka Media on a Prim), I used that function to determine which translated wiki pages to show to SL6B visitors, using PARCEL_MEDIA_COMMAND_AGENT on parcel media surfaces of an exhibit.

But that was before RedZone reminded everybody that every web surface knows the IP of its viewer and by paying attention it's easy to determine who is alt of whom -- with drama-maximizing imprecision. Folks who remember that, or who are party to SL's collective memory of that, are skeptical of enabling media either Parcel or Prim.

Despite all that, for a more recent SL birthday, I planned to use MoaP to show about a dozen Place Pages with scenic 360 images and all that jazz. I quickly learned that the whole machine on which a viewer is running will come to its knees as it tries to show multiple streaming MoaP surfaces, each with its own heavyweight Windows process. I can't say whether this would happen with multiple MoaP audio sources as you're proposing, but if I were you I'd want to test it out at scale before getting too deep in the weeds.

Speaking of testing things out, I'd also want to check that the approach could achieve an acceptable degree of synchronization for whatever application is contemplated. Audio synchrony is always tricky; maybe there are ways to sync MoaP streams, but this isn't anything I know about.

[Macrocosm Draegonne]

34 minutes ago, Qie Niangao said:

Here's something relevant: llGetAgentLanguage. Back before there was such a thing as Shared Media (aka Media on a Prim), I used that function to determine which translated wiki pages to show to SL6B visitors, using PARCEL_MEDIA_COMMAND_AGENT on parcel media surfaces of an exhibit.

But that was before RedZone reminded everybody that every web surface knows the IP of its viewer and by paying attention it's easy to determine who is alt of whom -- with drama-maximizing imprecision. Folks who remember that, or who are party to SL's collective memory of that, are skeptical of enabling media either Parcel or Prim.

Despite all that, for a more recent SL birthday, I planned to use MoaP to show about a dozen Place Pages with scenic 360 images and all that jazz. I quickly learned that the whole machine on which a viewer is running will come to its knees as it tries to show multiple streaming MoaP surfaces, each with its own heavyweight Windows process. I can't say whether this would happen with multiple MoaP audio sources as you're proposing, but if I were you I'd want to test it out at scale before getting too deep in the weeds.

Speaking of testing things out, I'd also want to check that the approach could achieve an acceptable degree of synchronization for whatever application is contemplated. Audio synchrony is always tricky; maybe there are ways to sync MoaP streams, but this isn't anything I know about.

Yeah I was expecting to only tinker and test for the most part to see what's possible, definitely wont burn too much time, at least not until it stops being interesting and fun lol. 

Seeing IP address seems like something that should be fixable, though Im not sure why hiding it is important, or if someone who wanted to hide it didnt have other smarter measures to do so. lol

I have tested a little already, the audio is great with zero lag for me, and since SL does point sourced audio it mostly does what I want already, trick I want is to expand the sound and/or repeat it somehow without copies of the prim.   Then yes, as you say, to link them together when required, preferably with script, to keep timings & such, im fairly sure that might be doable from the javascript side though, it mostly depends on the chromium implementation I suppose.

If I keep the LSL to a minimum it should reduce overhead for doing an array of audio outputs hopefully. The LSL will be mostly for setting up the "speakers" and turning them off when nobody is around, or its switched off, the js and external libraries can do the really wild stuff, at breakneck speeds too.

My experience in testing the visual side of webprims has been that everything works great, the js is lighting fast, but it has some sort of cap on framerate that can make certain visual effects look choppy.

Edited October 7, 2018 by Macrocosm Draegonne

[Macrocosm Draegonne]

Ive read some Jira on the topic today, some of the issues people have had, and requests that have been made.  Its an interesting area that I hope gets a refresh someday too, there's a good amount of legwork already done in the requests and jira as to what would be desirable. Its already cool and can do a lot though.  I will keep my eye on this area and push all the limits in testing it out with audio and maybe some other experiments.

 

[ChinRey]

On 10/7/2018 at 5:57 AM, Macrocosm Draegonne said:

What are the good the bad and the ugly about SL webprims that will effect such a creation and others?

There is one, posibly two, problems with a setup like the one you describe.

The first one is that autoplay is not likely to work. Firestorm has it switched off by default and I would guess most people who use the official viewer have switched it off too. (For all I know, LL may even have gotten to their senses and followed Fs' example by now.)

I'm not sure if the other problem is relevant for what you want to do but playback can not be syncronized. A video or audio starts from the beginning when you click on the play button.

 

On 10/7/2018 at 5:57 AM, Macrocosm Draegonne said:

Will making these web-audio web-prims hidden reduce viewer strain they may cause?

Well, any visible object will add a tiny bit of load to the gpu of course.

 

On 10/7/2018 at 5:57 AM, Macrocosm Draegonne said:

Can LSL work those with good performance?

 

 

Media-on-a-prim is essentially just a system for the viewer's built-in web browser to display web content on the surface of a prim (or sculpt or mesh). There are some fairly simple lsl functions to control it but all the hard work is client side.

 

[Macrocosm Draegonne]

1 hour ago, ChinRey said:

There is one, posibly two, problems with a setup like the one you describe.

The first one is that autoplay is not likely to work. Firestorm has it switched off by default and I would guess most people who use the official viewer have switched it off too. (For all I know, LL may even have gotten to their senses and followed Fs' example by now.)

Media-on-a-prim is essentially just a system for the viewer's built-in web browser to display web content on the surface of a prim (or sculpt or mesh). There are some fairly simple lsl functions to control it but all the hard work is client side.

Im not too concerned about autoplay, if someones going to an audio experience, there would be a sign saying to turn it on at least for the experience.  As far as I know its only macs that have the bug/feature of not autostarting chromium, I have autoplay on, never bothered me in firestorm, I thought that was default setting?

I was reading into the cromium and CEF stuff, there is quite a lot that can be done, and more yet, depending on how SL implements it.  I bet syncing is possible, but it could be the way SL makes each surface separate, because its surface not audio or code only focused.   Its able to display a browser, but that is only one feature (probably the heaviest) of the codebase, it can also display OS UI, and other UI's and displays based on standards code, and then everything that code can do, it can spin up its own threads, and do all the things separate from the viewer and SL server, which is why its got so much future potentials.

Edited October 9, 2018 by Macrocosm Draegonne

[ChinRey]

1 hour ago, Macrocosm Draegonne said:

I have autoplay on, never bothered me in firestorm, I thought that was default setting?

It isn't and it's not a technical issue. If you have autoplay on, it means anybody in the sim and the next sim can bombard you with MoP sound. The only ways to stop it are to either to locate the media sources one by one with the not very helpful beacon feature or to switch autoplay off.

[Macrocosm Draegonne]

7 hours ago, ChinRey said:

It isn't and it's not a technical issue. If you have autoplay on, it means anybody in the sim and the next sim can bombard you with MoP sound. The only ways to stop it are to either to locate the media sources one by one with the not very helpful beacon feature or to switch autoplay off.

Ahh, well thats never happened to me that I can recall, but I bet it could be annoying lol.  Pretty much the only things I ever have to derender are some sculpties, overboard alpha blends, and overboard Particle effects lol.

Like I was saying though, I would use this in a special setup, people would be expecting it, with signs and notes and such.

[Klytyna]

4 hours ago, Macrocosm Draegonne said:

Like I was saying though, I would use this in a special setup, people would be expecting it, with signs and notes and such.

" Hi! Welcome to my sim...

I make poorly concieved IP-scanning laggy annoying arse-ware! 

Please enable all the things you ALWAYS DISABLE to experience my arse-ware fully! "

...

Yeah, signs. This reminds me of the time when some less-than-talented sim manageress, suggested that the greeter staff at a social hangout should be replaced with MoP signboards, and was really put out when we all pointed out that a) most of the visitors wouldn't be able to see the damn MoP, and b) most would run past the MoP before it had rezzed /loaded etc., anyway.

...

See it in perspective...

If YOU visited somebody else's sim, and were welcomed by a sign that said "I'm NOT a griefer... Please DISABLE your Anti-griefer settings NOW, because *I* am an ARTISTE... With that supercool E on the end...", how would you react?
 

[Macrocosm Draegonne]

3 hours ago, Klytyna said:

See it in perspective...

If YOU visited somebody else's sim, and were welcomed by a sign that said "I'm NOT a griefer... Please DISABLE your Anti-griefer settings NOW, because *I* am an ARTISTE... With that supercool E on the end...", how would you react?
 

There is no other way to do an audio thing im thinking of, LSL alone with the sound clips will not do it.  Parcel streaming could give different sound tracks to different areas alright, which is the most straight forward way to bring in sound, then mix in some smaller loops here and there around the sim.  Thats all fine and well and the normal way to do it with the current stuff.

I am not talking about that though, this is something different and if that scares folks from trying it then that actually pleases me greatly!   lol...  Imagine the fool who goes to an audio experience and yet will not enable audio. XD  I have no time to suffer such fools!

 

[Macrocosm Draegonne]

Besides, I never said I was building a SIM with this stuff, I am experimenting, and contemplating future potentials! ie. Stuff that is not actually possible, we can perhaps request from LL. 

[Qie Niangao]

11 minutes ago, Macrocosm Draegonne said:

There is no other way to do an audio thing im thinking of, LSL alone with the sound clips will not do it.  Parcel streaming could give different sound tracks to different areas alright, which is the most straight forward way to bring in sound, then mix in some smaller loops here and there around the sim.  Thats all fine and well and the normal way to do it with the current stuff.

Hmm. That suggests a way to meet some of the expectations without needing MoaP. It doesn't actually help with the privacy concerns, so folks with media disabled still wouldn't be able to get the full experience, but just for completeness: it should be possible to have "background" sound playing through parcel audio, and then push one sound stream of avatar-specific parcel media corresponding to where the avatar has wandered, what they're doing, etc.

FWIW, back when I was playing with this parcel media stuff, it seemed that PARCEL_MEDIA_COMMAND_TIME would roughly synchronize streams of certain formats on certain viewer OSs.

But again, it all assumes the audience has media enabled. As it happens, the simplest way to associate IP address with avatar identity is using that PARCEL_MEDIA_COMMAND_AGENT parameter to deliver avatar-specific streams, but it's no great leap of computing to get comparable accuracy with MoaP or even parcel audio streams. And yet you don't see folks getting exercised about the risk the DJ might figure out who's alt of whom. I guess the allure of endless 1980's pop music outweighs such precautions.

[Macrocosm Draegonne]

7 minutes ago, Qie Niangao said:

Hmm. That suggests a way to meet some of the expectations without needing MoaP. It doesn't actually help with the privacy concerns, so folks with media disabled still wouldn't be able to get the full experience, but just for completeness: it should be possible to have "background" sound playing through parcel audio, and then push one sound stream of avatar-specific parcel media corresponding to where the avatar has wandered, what they're doing, etc.

That's an interesting idea, I didnt know you could target an avatar with sound just for them, that could prove very useful someday!   

8 minutes ago, Qie Niangao said:

FWIW, back when I was playing with this parcel media stuff, it seemed that PARCEL_MEDIA_COMMAND_TIME would roughly synchronize streams of certain formats on certain viewer OSs.

Oh, so that shows some potential for syncing.  Ideally everything would either sync or be out of range so not to clash with sounds not desired in that area of the scene.   So if the BPM and modulations are to change in the next zone the traversal, spatially speaking, would create an audio fade effect.

Another way to do this would be with a HUD, so only one source of the audio, right where it needs to be, just a wildly different way to do a similar thing.

13 minutes ago, Qie Niangao said:

But again, it all assumes the audience has media enabled. As it happens, the simplest way to associate IP address with avatar identity is using that PARCEL_MEDIA_COMMAND_AGENT parameter to deliver avatar-specific streams, but it's no great leap of computing to get comparable accuracy with MoaP or even parcel audio streams. And yet you don't see folks getting exercised about the risk the DJ might figure out who's alt of whom. I guess the allure of endless 1980's pop music outweighs such precautions

Seems like LL could close up those minor "security" risks that scare folks needlessly.  I would imagine if someone is really paranoid they would take other precautions to hide their IP and such, and SL, or anything else, could not expose it right?  

Eventually hopefully LL could refresh MoaP, allowing other things than displaying a full web-browser.  Code only version is the biggie, audio only could be cool too, I know we can use data: to include a good amount of markup, and then connect scripts externally, but if there was more markup allowed, external connections would not be needed. 

There could be a lot of things done with that type alone, and it uses no networking at all, only processes code and outputs over to LSL to display and/or mesh/prims prepped to be manipulated by it (with their own set of rules and restrictions as to whats allowed/will-work). 

• Something like the Notecard system could get an additional type thats html enabled, with a WYSIWYG. 
• HTML HUDs. Has been done already, but MoaP has some caveats that have made it much less popular than it could be.
• Code generated visuals and audio feedback that can be displayed in various ways, not as a browser, but rather mesh/particles/textures/sounds
• AI and algorithmic things better suited to certain codebases optimized for such, that can be passed off to LSL after the heavy lifting.

I should stop, sometimes I can just ramble on forever if im in a flow state! lol  I will go back to my meshing now.   

 

[ChinRey]

12 hours ago, Macrocosm Draegonne said:

Seems like LL could close up those minor "security" risks that scare folks needlessly.  I would imagine if someone is really paranoid they would take other precautions to hide their IP and such, and SL, or anything else, could not expose it right?

Both MoaP and parcel media tap into internet resources outside LL's control and the owner of the website will always be able to get the avatar's IP address. I don't think there's anything that can be done about that.

Me, I love it when the DJ at one of my two favourite blues streams mentions they have a listener from such an exotic country as Norway. But other SL users may be quite a bit more cautious about revealing any RL information and for all I know, some may even have their good reasons.

Edited October 11, 2018 by ChinRey

[Wulfie Reanimator]

5 hours ago, Macrocosm Draegonne said:

Seems like LL could close up those minor "security" risks that scare folks needlessly.  I would imagine if someone is really paranoid they would take other precautions to hide their IP and such, and SL, or anything else, could not expose it right?  

This just isn't possible with the way MOP currently works. When you see a webpage on a prim, it's YOU who is directly requesting and connecting to that page.

For LL to make this system "secure," they'd have to become a proxy service and carry the burden of all of the network data going through MOP to any and all avatars. I doubt they have any interest in doing that.

Edited October 10, 2018 by Wulfie Reanimator

[Macrocosm Draegonne]

7 hours ago, Wulfie Reanimator said:

This just isn't possible with the way MOP currently works. When you see a webpage on a prim, it's YOU who is directly requesting and connecting to that page.

For LL to make this system "secure," they'd have to become a proxy service and carry the burden of all of the network data going through MOP to any and all avatars. I doubt they have any interest in doing that.

Yes my thoughts exactly, if someone wants to hide there IP while online, why is that LL's job?  To hide their identity will take a full system approach, not simply expecting various apps to protect them.   So the way to close up the falsely perceived security risk is to put the burden on the user where it belongs, I would imagine people who want to be hidden already know such things and do whatever it is they do. 

What I am speaking to are those who are not at all even remotely hidden or secure, and yet balk at MoaP as though its somehow going to put them in some sort of additional security risk, its simply using the internet, people do it all the time on various devices, and use whatever security measures they choose. And, that happens at a much higher level than individual apps, their connection needs to be secured. 

Besides, most people do not have a static IP, their ip address can change many times in a day, they change regularly.  People are just scared because all that furry fornication with their alts. ?

Edited October 11, 2018 by Macrocosm Draegonne

[Macrocosm Draegonne]

On 10/9/2018 at 6:17 PM, ChinRey said:

It isn't and it's not a technical issue. If you have autoplay on, it means anybody in the sim and the next sim can bombard you with MoP sound. The only ways to stop it are to either to locate the media sources one by one with the not very helpful beacon feature or to switch autoplay off.

But how? The sound only travels a very short distance before being totally inaudible. I have two 1024 parcels next to each other on the same SIM, if I play a MoP in one lot, it can barely be heard at all once walked over to the other side of that lot, and not at all in the next lot or anywhere else in the SIM.

Maybe LL has tweaked it some since you last tested it?

 

Edited October 11, 2018 by Macrocosm Draegonne

[ChinRey]

1 hour ago, Macrocosm Draegonne said:

But how? The sound only travels a very short distance before being totally inaudible.

...

Maybe LL has tweaked it some since you last tested it?

Interesting. They have made such secret fixes to other big blunders of the past recently so yes, it is possible they have changed something.

If I can allow myself a little bit of smugness, it usually takes about 2-3 years from when I file a JIRA and LL instantly rejects it to when they realize they have to fix the issue anyway. My JIRA about this problem is three years old so the timing is right.

The next big challenge then is to convince people to switch autoplay back on. That's going to be tough.

 

9 hours ago, Macrocosm Draegonne said:

So the way to close up the falsely perceived security risk is to put the burden on the user where it belongs, I would imagine people who want to be hidden already know such things and do whatever it is they do.

Don't count on that and besides, even if they do know how to take precautions, the easiest one is to switch MoaP autoplay and land media streaming off.

The security risk isn't completely bogus btw, there is at least one way IP addresses collected through media streaming are abused. There are some script systems on the market that pretend to protect land from griefers. These systems keep a database of IP addresses of known griefers and of people the scripters/sellers don't like collected via media streaming. If you try to visit a place that hs bought and installed such a system and your IP address happens to be in the database, you are instntly evicted and banned.

Edited October 11, 2018 by ChinRey

[Love Zhaoying]

9 hours ago, Macrocosm Draegonne said:

People are just scared because all that furry fornication with their alts. ?

..Mains..

[Macrocosm Draegonne]

1 hour ago, ChinRey said:

Interesting. They have made such secret fixes to other big blunders of the past recently so yes, it is possible they have changed something.

If I can allow myself a little bit of smugness, it usually takes about 2-3 years from when I file a JIRA and LL instantly rejects it to when they realize they have to fix the issue anyway. My JIRA about this problem is three years old so the timing is right.

The next big challenge then is to convince people to switch autoplay back on. That's going to be tough.

yea, I would imagine its a very demanding task to make adjustments, and sort thru all the different things that could be done at any given time.  At least it shows you're usually ahead of the curve on what to do next.

1 hour ago, ChinRey said:

Don't count on that and besides, even if they do know how to take precautions, the easiest one is to switch MoaP autoplay and land media streaming off.

The security risk isn't completely bogus btw, there is at least one way IP addresses collected through media streaming are abused. There are some script systems on the market that pretend to protect land from griefers. These systems keep a database of IP addresses of known griefers and of people the scripters/sellers don't like collected via media streaming. If you try to visit a place that hs bought and installed such a system and your IP address happens to be in the database, you are instntly evicted and banned.

Im surprised that would even sell, an ip address is not fixed unless someone pays extra, the overwhelming majority of people do not have a static ip, their ip changes all the time, their ISP is one of the few that can confirm who had what IP at what time.  Any MP product trying to block people by IP is really going to have like 99% miss for their intended targets lol.

[Wulfie Reanimator]

5 minutes ago, Macrocosm Draegonne said:

yea, I would imagine its a very demanding task to make adjustments, and sort thru all the different things that could be done at any given time.  At least it shows you're usually ahead of the curve on what to do next.

Im surprised that would even sell, an ip address is not fixed unless someone pays extra, the overwhelming majority of people do not have a static ip, their ip changes all the time, their ISP is one of the few that can confirm who had what IP at what time.  Any MP product trying to block people by IP is really going to have like 99% miss for their intended targets lol.

You'd be amazed of how many things you can sell to the technically illiterate.