SL Forums - Sloppy: LSL data store with Google App Engine

08-31-2008, 08:23 PM

Registered User

Join Date: Mar 2007

Posts: 1

Sloppy: LSL data store with Google App Engine

Hi, I've implemented a demonstration data store accessible to LSL scripts that works like Silo, but instead of being hosted by your ISP is instead hosted on the Google App Engine servers.

I would be particularly interested in hearing from anyone who uses Silo currently and would be willing to test their script against my code. I'm sorry, I won't be providing more support than is the README file for getting your own server up and running.

All the code for the server and client is freely available and released under a permissive BSD-style license.

Everything is here: http://fullycontrolled.com/code/

Here is the demo client you can drop into a script to access my test server.

EDIT: I've been told there was an issue with the script that was formatted here. I'm not sure what happened, but this version is now the correct one. Really, though, you'd be better off grabbing one from my svn repository with the spacing intact: http://fullycontrolled.com/public/index.cgi/public/sloppy/example.lsl?view=markup

Quote:

[php]
// [Snipped license for legibility in forums -- please see referenced LICENSE file for license information.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

// Configuration variables.

// NOTE: This server is only for use for this demonstration. Do not use this
// server for any production purposes. I reserve the right to sabotage
// your data in entertaining ways if you do. :)
string url = "http://sloppydemo.appspot.com/";

string demo_key = "fnord";
float request_timeout = 15.0; // in my experience these never get close to this number.
float idle_check = 60.0; // Check for a new message every idle_check seconds. This is pretty spammy.
integer listen_channel = 3;

// No real need to touch these.
integer HTTP_REQUEST_NONE = 0;
integer HTTP_REQUEST_GET = 1;
integer HTTP_REQUEST_PUT = 2;

integer http_last_request = HTTP_REQUEST_NONE;
key http_key = NULL_KEY;
string last_message = "";
integer touched = FALSE;

get_current_message() {
if (http_key == NULL_KEY) {
//llWhisper(0, "Requesting current message from demo server.");
http_key = llHTTPRequest( url + demo_key, [ ], "");
http_last_request = HTTP_REQUEST_GET;
llSetTimerEvent(request_timeout);
} else {
llWhisper(0, "There is already a request pending, sorry.");
}
}

update_text() {
llSetText(last_message, <1.0,0.8,0.8>, 1.0);
}

send_message(string name, string message) {
if (http_key == NULL_KEY) {
//llWhisper(0, "Sending message: " + message);
http_key = llHTTPRequest(url + demo_key, [ HTTP_METHOD, "PUT" ], name + ": " + message);
http_last_request = HTTP_REQUEST_PUT;
llSetTimerEvent(request_timeout);
} else {
llWhisper(0, "There is already a request pending, sorry.");
}
}

reset_state() {
http_last_request = HTTP_REQUEST_NONE;
http_key = NULL_KEY;
touched = FALSE;
}

say_hello() {
if (http_key == NULL_KEY) {
last_message = "";
llWhisper(0, "Demo client for Sloppy -- Touch to check for a new message at any time.");
llWhisper(0, "Any message set here will be seen by everyone who is running this script, anywhere in SL.");
get_current_message();
}
}

default
{
state_entry()
{
say_hello();
llListen(listen_channel, "", NULL_KEY, "");
}

on_rez(integer param) {
reset_state();
say_hello();

}

listen( integer channel, string name, key id, string message ) {
if (channel == listen_channel) {
send_message(name, message);
}
}

touch_start(integer total_number)
{
touched = TRUE;
get_current_message();
}

timer() {
if (http_key != NULL_KEY) {
// This is a request timeout.
llSetTimerEvent(idle_check);
reset_state();
llWhisper(0, "Timed out request.");
} else {
// This is an idle message update check.
llSetTimerEvent(0.0);
get_current_message();

}
}

http_response(key req, integer status, list meta, string content) {
if (req != http_key) {
//llWhisper(0, "Unexpected response...");
return;
}

if (http_last_request == HTTP_REQUEST_GET) {
llSetTimerEvent(idle_check);
if (status == 200) {
if (last_message != content) {
llWhisper(0, "New message: " + content);
llWhisper(0, "You can always set a new one by saying something on /" + (string) listen_channel);
last_message = content;
update_text();

} else if (touched) {
llWhisper(0, "Sorry, no new message.");
}

} else if (status == 404) {
llWhisper(0, "There is no key by the requested name.");
} else if (status == 403) {
llWhisper(0, "Forbidden from retrieving requested key.");
} else {
llWhisper(0, "Unexpected HTTP response to GET request: " + (string) status);
}
reset_state();
} else if (http_last_request == HTTP_REQUEST_PUT) {
llSetTimerEvent(idle_check);
reset_state();
get_current_message();

if (status == 200) {
llWhisper(0, "Message updated!");
} else if (status == 201) {
llWhisper(0, "Created new message!");
} else if (status == 403) {
llWhisper(0, "Forbidden from setting requested key.");
} else {
llWhisper(0, "Unexpected HTTP response to PUT request: " + (string) status);
}
} else {
reset_state();
llWhisper(0, "Unexpected HTTP request type.");
}

}
}
[/php]

Here is the full README:

Quote:

README for Sloppy v0.1 -- $Id: README 41 2008-08-31 20:07:18Z anari $

SUMMARY

Sloppy (Silo + .py) is software to host a web-accessible storage service
intended to be accessed from LSL scripts within Second Life. Other people
have solved this problem in other ways in the past; here is what might be
a little different about Sloppy:

- Designed as a drop-in replacement for Silo, an existing PHP-Based storage
solution for LSL scripts in Second Life written by Zero Linden.
- Written in Python with Google's App Engine SDK for free hosting on their
cloud computing service.

Sloppy was was implemented over the course of an evening or two primarily
as a testing ground and proof of concept for using Google App Engine servers
to provide persistent storage for LSL. For this reason, and since it is my
first Google App Engine application, the code is not as polished as it
could be.

If using Google App Engine this way proves valuable, I may look at
implementing a new system from the ground up with a better API. Feedback
is welcome on that front. Bug reports are also welcome.

DETAILS

There are some tradeoffs that come with Google App Engine and with using
a drop-in replacement for Silo.

Upsides:

+ If you already use Silo, this should hopefully "just work."
+ You don't need your own web hosting.
+ You don't need to trust some random SL user who is offering free web data
hosting.
+ Google is presumably less likely to have a serious, extended outage than
most smaller mom and pop web hosting service providers. Availability is
important; utilizing external data storage can introduce a single point
of failure for your scripts.
+ This code is released under a permissive BSD-style license (see the
included LICENSE file.)

Downsides:

- I don't particularly like the Silo protocol.
- You have to trust Google with your data.
- Getting something set up on App Engine is probably not for novices (though
Google provides plenty of documentation and some great examples to get
stated with.) I will not be providing support in getting your application
setup outside of this document.
- You may not be able to get a google app engine account. When I signed up,
they said there were no more beta accounts available, but a little later I
checked back and had access anyway.
- Google does not presently sell additional resources if you go over the
alotted resource quotas with your free account.

CONTENTS:

- README: this file.
- sloppy.py: this is the code that's implemented on Google's servers.
- app.yaml: configuration data for the application
- index.yaml: configures indexes for sloppys data store
- example.lsl: brain dead demonstration client.
- test.lsl: a few unit tests for the application.

QUICK SETUP

- Get a google app engine account.
- Create an application on google appe engine.
- Install the google app engine SDK on your test system.
- Extract the tarball this README was included into somewhere.
- Edit 'app.yaml'; update the name to the one you chose while creating
the application on Google's site.
- run appcfg.py against the directory you extracted the tarball into:
appcfg.py update sloppy/
- I'm not sure if your indexes are set up automatically or not. If they
are not, this should create the ones sloppy uses.
appcfg.py update_indexes sloppy/

LOCAL TEST SETUP

- Install the google app engine SDK on your test system.
- Extract the tarball onto your test system.
- run: dev_appengine.py sloppy/

By default, this will start a webserver running locally on
[url]http://localhost:8080/[/url] that you can use as a full sloppy
service.

CONTACT:

[email]anari.demme@gmail.com[/email]
Anari Demme in Second Life

ACKNOWLEDGEMENTS:

Schmobag Hogfather for the initial bright idea to use App Engine for this
purpose.

Last edited by Anari Demme : 08-31-2008 at 11:34 PM.