Make your own "New/Updated Threads" page

[Ossian1488303077]

Since there isn't a good way to see which threads are new and updated,
I've made a Perl script (below).

This isn't perfect, but it's what I use to see what's happening here.

You should be able to call it with your account name and password, but the authentication seems
to have quit working.  So you can just call the script without... it isn't going to get the Off Topic posts, sorry.

The results aren't pretty, but I think they are better organized than the page provided,
and maybe someone could webify this and make it better.

 

# -----------------------------------------------------------------
# forumfix.pl [ ACCOUNT_NAME PASSWORD ]
#
# The account name and password are optional; in any case the
# authentication isn't working, so Off Topic posts are missed.
#
# This creates a file called UpdatedThreads.html that lists
# all posts from today and yesterday by thread.
#
# Feel free to use or modify.
# -----------------------------------------------------------------

use HTML::TreeBuilder;
use HTTP::Cookies;
use WWW::Mechanize;

# --- output filename ---

$threadFile = 'UpdatedThreads.html';

# --- are we logging in? ---

if ($#ARGV != 1) {

    $logging = 0;

} else {

    $logging = 1;

    $username = $ARGV[1];
    $password = $ARGV[0];
}

# --- URL ---

$nro = 1;
$url = 'http://community.secondlife.com/t5/forums/' .
       'recentpostspage/category-id/Forums/post-type/message/page';

# --- make your browser ---

my $mech = WWW::Mechanize->new();
$mech->cookie_jar(HTTP::Cookies->new());

$mech->get("$url/$nro");

# --- log in, if credentials were given ---

if ($logging) {

    $mech->follow_link(text => "Sign In");

    $mech->submit_form(
        form_number => 0,
        fields => { username => $username, password => $password,
          stay_logged_in => 'Yes',
          return_to => 'https://support.secondlife.com/openid_rp/hand-auth-id',
        },
        button => 'Submit'
    );
}

# --- take the page apart ---

$currDate = '';
$daysFound = 0;
$posted = '';
%Threads = ();

while (1) {

    $page = $mech->content();

    $root = HTML::TreeBuilder->new_from_content($page);

    @TR = $root->find('tr');

    $first = 1;

    # --- grab info from the page --
   
    foreach $tr (@TR) {

        # --- skip the header ---
        if ($first) { $first = 0; next; }

        @TD = $tr->find('td');

        # --- first get the key ---
        @anchors = $TD[4]->find('a');
        $key = $anchors[0]->attr('href');
        $key =~ s/^.*\///;

        # --- get post info and subforum ---
        @anchors = $TD[1]->find('a');
        $link = $anchors[0]->attr('href');
        $title = $anchors[0]->as_text();
        $posted = $title;
        $posted =~ s/^.* - \( //;
        $posted =~ s/ \).*$//;
        $title =~ s/ - \(.*$//;
        $forum = $anchors[1];
        $forum->attr('class', undef);
        $forum->attr('id', undef);
        $forum = $forum->as_HTML();
        chomp $forum;
        $forum = "<center>$forum</center><br />\n";
        $teaser = $TD[1]->find_by_attribute('class','message-subject-body wrapper-hide-overflow message-body justify');
        $teaser = $teaser->as_text();

        ##+ print "Link: $link\n";
        ##+ print "Title: $title\n";
        ##+ print "Teaser: $teaser\n";
        ##+ print "$forum\n";

        # --- get author ---
        $author = $TD[2]->find('a');
        $author->attr('class', undef);
        $author->attr('style', undef);
        $author->attr('target', undef);
        $author->attr('id', undef);
        $author = $author->as_HTML();
        chomp $author;
        $author =~ s/<span[^>]*>//;
        $author =~ s/<.span>//;

        ##+ print "$author\n";
        ##+ print "\n";

        # --- first time thread is mentioned, get the title ---

        if (!exists($Threads{$key})) {

            $Threads{$key} = $forum . "<b> $title &nbsp;{</b><i><a id=\"displayText$key\" href=\"javascript&colon;toggle('$key');\">show posts</a></i><b>}</b><br />\n" .
                "<ul id=\"toggleText$key\" style=\"display: none\">\n";

        }

        # --- add current post to thread ---

        $Threads{$key} .= "<li> <a href='$link'>$posted</a> by <i>$author</i><br />\n" .
            $teaser . "</li>\n";

       
    }

    # --- clean up ---

    $root->delete;

    ##+ print "-------------------------------------------\n";

    # --- should we stop? ---

    $posted =~ s/ .*$//;
    #++ print "$posted\n";
    if ($currDate ne $posted) {
        $currDate = $posted;
        $daysFound++;
        last if ($daysFound > 2);
    }

    # --- get next page ---
   
    $nro++;
    $mech->_reset_page();
    print ".";
    $mech->get("$url/$nro");
}
print "\n";

# --- prepare our output file ---

open OUTF, '>:utf8', $threadFile;

print OUTF <<EOT
<html><head><title>SL Forum New and Updated Threads</title>
<base href="http://community.secondlife.com/">
</head>
<body>
<script language="javascript">
function toggle(key) {
    var ele = document.getElementById("toggleText" + key);
    var text = document.getElementById("displayText" + key);
    if(ele.style.display == "block") {
            ele.style.display = "none";
        text.innerHTML = "show posts";
      }
    else {
        ele.style.display = "block";
        text.innerHTML = "hide posts";
    }
}
</script>

EOT
;

# --- use array to pull threads in order and print them ---

$lastForum = "";

foreach $thread (sort values %Threads) {

    # --- we only want to print the forum name once ---
   
    $forum = $thread;
    $forum =~ s/^<center><a[^>]*>//;
    $forum =~ s/<.*$//s;

    if ($forum ne $lastForum) {

        $lastForum = $forum;
        print OUTF "\n\n<hr>\n";

    } else {

        $thread =~ s/^.*<.center><br .>//s;
    }

    print OUTF "$thread</ul><br />\n\n";
}

print OUTF "</body></html>";

close OUTF;


[Peewee Musytari]

/me has no clue what all that means...It could be a recipe for chocolate cake for all I know hehe....But if that will work in the Answers forum then someone PLEASEEEEEEEE instal it !!!

[valerie Inshan]

---

Peewee Musytari wrote:

/me has no clue what all that means...It could be a recipe for chocolate cake for all I know hehe....But if that will work in the Answers forum then someone PLEASEEEEEEEE instal it !!!

---

LOL, I am not too sure I'd want a piece of a chocolate cake made out of this! /me understands NOTHING! (but votes for it if it works!!!)

[Darren Scorpio]

Thanks for this Ossian! I have been much too busy with RL stuff and lazy when I'm not to even think about doing this for a forum which is getting less and less use by the day I will look this over when I get home. Kuddos for you.

[Darren Scorpio]

---

Peewee Musytari wrote:

/me has no clue what all that means...It could be a recipe for chocolate cake for all I know hehe....But if that will work in the Answers forum then someone PLEASEEEEEEEE instal it !!!

---

LOL You have to use it on your own website, I doubt the Lindens would want to simplify the way this whole website works or they would have done it already. Seriously it is not hard to put a widget on the dashboard like we used to have, it is beyond me why there isn't one there already.

[Ossian1488303077]

I run it on my desktop computer. It creates a local file that I open in my browser.

When I want a refresh, I run the script again and refresh the page in the browser.

 

[Rhonda Huntress]

---

Peewee Musytari wrote:

/me has no clue what all that means...It could be a recipe for chocolate cake for all I know hehe....But if that will work in the Answers forum then someone PLEASEEEEEEEE instal it !!!

---

 Let me see if I can get something out of this ...

---

Ossian wrote:

use HTML::TreeBuilder;

use HTTP::Cookies;

use WWW::Mechanize;



---

First, we plant a tree.  This shows good eco-sense.  Save the tree, save the cheerleader.  Something like that.  It's technical.

Next, we get into the cookies.  You can't do anything right without cookies.  Unless you have bacon.  Bacon works too.  But not bacon cookies. That's a UNIX call.

And finally we join the World Wide Wrestling but we are Mechanized!  Bring on the gundams, baby! Nothing like a full metal full Nelson to get forum quirkiness in line.

[Peewee Musytari]

---

Ossian wrote:

I run it on my desktop computer. It creates a local file that I open in my browser.

When I want a refresh, I run the script again and refresh the page in the browser.

Also, it wouldn't be hard to modify this to sort threads in different ways,

for instance, to group threads into their respective subforums.

---

hehe you make it sound so simple ....Can you explain how to do it in words that a 5 year old would understand, preferably with pics and big red arrows LOL

Actually just the bit between it being here in a forum post and a clickable something on my puter.

[Ossian1488303077]

But this is what you need to run it on your own computer (instructions for Windows; Mac and Linux are similar):

1. Install Perl on your computer.

2. Make a new file on your computer called "forumfix.pl" and copy my script into it. Save it.

3. Open a command prompt and go to the directory where you saved forumfix.pl

4. Type:  perl forumfix.pl

5. Type: UpdatedThreads.html

 

Step four can take a minute to run.  Step five will open the new file in your browser.

It makes a long file with all the new threads and updates from today and yesterday.
The entries look like this:

Note that you can expand and collapse the list of posts.

If someone wanted to, they could install it on a webserver and run every couple of minutes.

(Sorry but I don't have a website to do it from.)

It could be adapted for Answers.

I don't know why authentication quit working (it was until yesterday), but maybe someone can figure
that out.

I think it's pretty simple and could easily be changed to group threads differently or display them differently.

It would also be easy to highlight Linden posts, but you can search for "Linden" to find them, or to find
posts from people you follow.

[Ossian1488303077]

I made some updates... I'm looking for a webserver to run this on,
and make pages that go back varying intervals (an hour, 6 hours, 24 hours).

I realize this is difficult for most people, but if anyone wants to take this script
and use it or change it, you're welcome to it.

[Luc Starsider]

I didn't look to close at your script, and probably wouldn't fully understand it, but it it very different than what you find under 'Recently created' on the forum front page? This, for instance, give you a list of all new posts created - replies and new threads - in any forum as well as in 'Answers'. The other 'posts' links give your updates for the different sections.

- Luc -

[Ossian1488303077]

In most web forums, you can look at a page that shows the threads that have been created or updated
since your last visit.  On SLU, for example, after six or eight hours, I can see in three short pages
all the threads that changed or appeared since my last visit.

Here, as far as I've seen, I'd have to look at ten or more pages to get the same information, and
see the same threads listed over and over, once for each post that was made.

My script makes a page that lets you scan down a list of thread titles, and if you want you
can see the posts that were made to any thread yesterday and today.  It's one page, and it takes me
a minute or two to see a whole day's activity and see whether there's anything of interest to me.

Also, your browser remembers the posts you've visited, so they are a different color.

I dunno. I use this. It's easier for me.

[Luc Starsider]

/me feels a bit slow...

I get what you mean now. Yes, that IS better. Much easier to get an overview of what's happened since the last visit.

Has it been suggested to LL? (It probably has, and I didn't understand it... )

- Luc -

[Ossian1488303077]

I think people have been asking for an easier way to see what's new... especially with Answers.
I haven't seen it mentioned in the CTUG meetings lately, though.

[Luc Starsider]

'Answers' has been brought up frequently in user group meetings, and is getting a lot of attention from LL because of it. Perhaps Lithium just don't provide the option for showing new and updated posts in the way you described.

- Luc -