OpenSocial gives your application the power to store and retrieve data for each of its users. The functions to read and write this data are collectively known as OpenSocial's Persistence API. This persistent data is very powerful but can be complex when you take into account the different relationships between people writing data and others attempting to read it. For this reason, the way that the Persistence API works can be somewhat hard to conceptualize.

This article attempts to explain the process of reading and writing persistent data as well as illustrate the cases when the data will be available to different users. When you complete this article, you should have a good understanding of how the Persistence API works as well as an idea of how to structure your OpenSocial application to take advantage of some of the sophisticated uses of persistent data.

About this article

This article is written assuming that you have a beginner to intermediate knowledge of the OpenSocial API, and that you have written at least one OpenSocial application.

If you haven't completed an application or are looking for an introduction to OpenSocial, you may wish to take a look at these additional resources before continuing:

• OpenSocial Tutorial
• Developer's Guide
• /wiki/spaces/a/pages/527097

To help visually communicate the relationships between users, the Testington family will be used for examples:

Here is a picture depicting the Testington friend graph:

Persistence API basics

The Persistence API is one of the three main components of OpenSocial. It provides a storage mechanism to save and load data. This data is scoped per user per application, so it is a scalable way to store data for each of your application's users.

Data is stored as a series of key/value pairs, much like a dictionary or an associative array data structure. The keys and values stored must be strings, but OpenSocial provides some convenience methods to assist you in converting complex JavaScript objects into string representations for storing as persistent data.

Writing data

Most OpenSocial calls involve building a DataRequest object, sending it to the container, and processing the response. Reading and writing application data is included in this flow.

To write a value to a specific key, create a DataRequest and add the result of calling newUpdatePersonAppDataRequest to it:

var req = opensocial.newDataRequest();
req.add(
    req.newUpdatePersonAppDataRequest("VIEWER", "myKey", "myValue"),
    "set_data");
req.send(set_callback);

The previous piece of code attempts to save the string myValue under the key of myKey. It also specifies a callback function named set_callback that will get a DataResponse object indicating whether the update failed or succeeded. A function to handle this response could look something like this:

function set_callback(response) {
  if (response.get("set_data").hadError()) {
    /* The update failed ... insert code to handle the error */
  } else {
    /* The update was successful ... continue execution */
  }
};

You should be familiar with this style of coding from using the OpenSocial API. If you would like more information about making DataRequest requests, please check out the Requesting Data in OpenSocial article for more information.

Reading data

To retrieve the value of myKey you must create another DataRequest object like before, but this time include a newFetchPersonAppDataRequest call:

var idspec = opensocial.newIdSpec({ "userId" : "VIEWER", "groupId" : "SELF" });

var req = opensocial.newDataRequest();
req.add(
    req.newFetchPersonAppDataRequest(idspec, "myKey"),
    "get_data");
req.send(get_callback);

To specify who you want to retrieve data for, you need to create an IdSpec object and pass it as the first argument to the newFetchPersonAppDataRequest function. Following is a table that shows how to create IdSpec objects for different requests:

var idspec = opensocial.newIdSpec({ "userId" : "VIEWER", "groupId" : "SELF" });

var idspec = opensocial.newIdSpec({ "userId" : "OWNER", "groupId" : "SELF" });

var idspec = opensocial.newIdSpec({ "userId" : "VIEWER", "groupId" : "FRIENDS" });

var idspec = opensocial.newIdSpec({ "userId" : "OWNER", "groupId" : "FRIENDS" });

The meaning of each of these values will be covered in depth later in this article.

The callback specified by this function gets a DataResponse object back that contains the data that was stored, or an error message if something went wrong. A callback function implementation could look something like this:

function get_callback(response) {
  if (response.get("get_data").hadError()) {
    /* the fetch failed ... insert code to handle the error */
  } else {
    var data = response.get("get_data").getData();
    /* the fetch was successful ... "data" contains the app data */
  }
};

Assuming that the request succeeded, the data variable in the function above will be assigned a JavaScript object with the following layout:

{
  XXXXXXXXXXXXXXXXXXX : { myKey : "<value of myKey for user XXXXXX...>" }
}

where XXXXXXXXXXXXXXXXXXXXX is the ID number of the user who the data belongs to. If your request specifies many people to fetch data for, then the result will look like this:

{
  XXXXXXXXXXXXXXXXXXXX : { myKey : "<value of myKey for user XXXXXX...>" },
  YYYYYYYYYYYYYYYYYYYY : { myKey : "<value of myKey for user YYYYYY...>" },
  ZZZZZZZZZZZZZZZZZZZZ : { myKey : "<value of myKey for user ZZZZZZ...>" }
}

If you request multiple keys for each person, the keys will be scoped to each person's ID in the returned data object:

{
  XXXXXXXXXXXXXXXXXXXX : { myKey1 : "<value of myKey1 for user XXXXXX...>",
                           myKey2 : "<value of myKey2 for user XXXXXX...>" },
  YYYYYYYYYYYYYYYYYYYY : { myKey1 : "<value of myKey1 for user YYYYYY...>",
                           myKey2 : "<value of myKey2 for user YYYYYY...>" },
  ZZZZZZZZZZZZZZZZZZZZ : { myKey1 : "<value of myKey1 for user ZZZZZZ...>",
                           myKey2 : "<value of myKey2 for user ZZZZZZ...>" }
}

Fetching all keys

If you would like to fetch all data available for a request and not just a specific key, pass the value of "*" to your newFetchPersonAppDataRequest:

req.add(
    req.newFetchPersonAppDataRequest(idspec, "*"),
    "get_data");

Clearing data

There are situations where your application may need to delete a key. To erase a key and its value, use the newRemovePersonAppDataRequest method:

var req = opensocial.newDataRequest();
req.add(
{panel}
    req.newRemovePersonAppDataRequest("VIEWER", "myKey"),
    "clear_data");
{panel}
req.send(set_callback);

When this request is made, the key myKey and its stored value are both erased from Persistence data.

You can clear multiple keys at once by passing an array of keys to this method:

var req = opensocial.newDataRequest();
req.add(
    req.newRemovePersonAppDataRequest("VIEWER", ["key1", "key2", "key3"]),
    "clear_data");
req.send(set_callback);

When this request is made, the keys key1, key2, key3 and their stored values are erased from Persistence data.

You may want to erase all of a user's Persistence data if you are testing your application during development or if you use the storage as a temporary cache. If you wish to clear all data keys for your application, you can delete the "*" key.

Warning: The following call will erase all the data for your application, so be careful when using it:

var req = opensocial.newDataRequest();
req.add(
    req.newRemovePersonAppDataRequest("VIEWER", "*"),
    "clear_data");
req.send(set_callback);

Restrictions on the Persistence API

There are some important restrictions that you should be aware of when developing applications that save and load data.

Access control lists (ACLs)

It is up to each OpenSocial container to determine under which contexts persistent data can be written and retrieved. The policy that many live containers have implemented is that an application can only write to VIEWER data and only if the VIEWER has the application installed. This policy is fairly restrictive to prevent malicious users from writing data to arbitrary users, so it is expected to be the most commonly implemented Persistence data policy. This article was written under the assumption that data will only be writable to users with the application installed, and presents advice on how to structure applications around this limitation.

It is certainly possible that some containers may implement a more relaxed data policy that allows users to write data to other users' Persistence data. Additionally, some containers may choose to give their users the ability to set ACLs on their data. In this model, a user would be able to whitelist other user accounts to read from or write to their own persistent data.

When calls to set or get persistent data fail because of access control limitations, the corresponding DataRequest call will fail with the opensocial.ResponseItem.Error.UNAUTHORIZED error code.

Here's sample code that examines the result of an imaginary "set_data" call for an ACL error:

function response(data) {
  if (data.hadError() && data.get("set_data").hadError()) {
    if (data.get("set_data").getErrorCode() === opensocial.ResponseItem.Error.UNAUTHORIZED) {
      /** Looks like the set request was not authorized, put code to handle this case here **/
    }
  } 
  /** Continue normal execution here **/
};  

Character escaping

Since application data is visible to more than just the user who writes it, there is a danger that any given application data may contain content from a malicious user. For this reason, the OpenSocial specification stipulates that application data must be HTML escaped by the container before being returned to the application.

Escaping will prevent situations where application data is output without being filtered by the application first. Consider the following data string:

var data = "<img style="width: 1; height: 1;" src="adsfa" onerror="alert('hello')" />"

If the above string is put directly into the innerHTML property of a page element, a popup box containing hello will be displayed. While this sample is harmless, allowing JavaScript from other users to execute without being filtered is a security risk. Therefore, if that string is stored in application data, it will be returned as:

"<img style="width: 1; height: 1;" src="adsfa" onerror="alert('hello')" />"

which, if put into the innerHTML property of an element, will simply print the <img> tag and the alert() code, instead of executing the JavaScript directly.

If you need to encode additional user-supplied values, then you can use the gadgets.util.escapeString function to perform string escaping on arbitrary data. The escapeString function is idempotent, meaning repeated calls on a string will return the same output. Therefore:

gadgets.util.escapeString(data) === gadgets.util.escapeString(gadgets.util.escapeString(data))

If you need to undo this encoding operation for some reason, you may use the gadgets.util.unescapeString function to return the escaped string's original form. Be careful about displaying unescaped data, though, for the reason explained above.

Note that there are certain cases where the output of unescapeString is not guaranteed to get back the same string you passed to escapeString (for example, if the string you passed to escapeString already contained escaped entities).

gadgets.util.unescapeString("&") === "&"
gadgets.util.unescapeString(gadgets.util.escapeString("&")) === "&" //Not equal to "&"\!

OpenSocial 0.8 introduces a new ESCAPE_TYPE parameter that you may use to control the escaping of response data. To use this parameter, send it as part of the opt_params parameter map that you send to newFetchPersonAppDataRequest:

var idspec = opensocial.newIdSpec({ "userId" : "VIEWER", "groupId" : "SELF" });
var params = {};
params[opensocial.DataRequest.DataRequestFields.ESCAPE_TYPE] = opensocial.EscapeType.NONE;
var req = opensocial.newDataRequest();
req.add(req.newFetchPersonAppDataRequest(idspec, "myKey", params), "get_data");
req.send(get_callback);

String-only data

The data store only accepts strings for storage, which means that you must convert complex data structures to strings before saving them. Fortunately, you can use the function gadgets.json.stringify to convert JavaScript objects and arrays to a string value. The gadgets API also provides the inverse function gadgets.json.parse that will take a string generated by gadgets.json.stringify and convert it back into a JavaScript object.

Note that when you store stringified data, the data is escaped when you fetch it. You must therefore specify ESCAPE_TYPE = NONE in your request, or call gadgets.util.unescapeString on any JSON-encoded objects before running gadgets.json.parse on them. However, this has the side effect of unescaping all data in the returned object! Make sure that you escape any fields on this object manually before outputting their values directly to the gadget.

For an example, imagine that you need to write the following data structure to Persistence data under the key of "key":

 
var data = {
 html : "This is user supplied content"
} 

You could store this data using the following function:

 
function setRequest() {
 var data = {
   html : "This is user supplied content"
 };
 var req = opensocial.newDataRequest();
 req.add(
     req.newUpdatePersonAppDataRequest(
         "VIEWER", 
         "test", 
         gadgets.json.stringify(data)),
     "setData");
 req.send(setResponse);
}; 

Note that to store the data structure, this function calls gadgets.json.stringify on the JavaScript object before passing it to the update request.

To retrieve this data, the application could use the following request:

 
function getRequest() {
 var viewer_idspec = opensocial.newIdSpec({ 
     "userId" : "VIEWER", 
     "groupId" : "SELF" 
 });
 var req = opensocial.newDataRequest();
 req.add(req.newFetchPersonRequest("VIEWER"), "getViewer");
 req.add(req.newFetchPersonAppDataRequest(viewer_idspec,"test"), "getData");
 req.send(getResponse);
}; 

This example fetches the VIEWER because it needs the VIEWER's ID number to access the correct Persistence data key.

Now an application developer may be tempted to use the following code to display the value of the HTML parameter in the stored object:

 
function getResponse(data) {
 var viewer = data.get("getViewer").getData();
 var testData = data.get("getData").getData()[[]viewer.getId()][[]"test"];
 var unescapedTestData = gadgets.util.unescapeString(testData);
 var testObject = gadgets.json.parse(unescapedTestData);
 document.getElementById("outputDiv").innerHTML = testObject.html;
}; 

When the value of testObject.html is just text, this method works well, but what if the value is executable JavaScript, like this:

 
var data = {
 html : "<img style=\"width: 1; height: 1;\" src=\"adsfa\" onerror=\"alert('hello')\" />"
} 

Running the getResponse function results in an alert box popping up! To protect the VIEWER from having user-supplied JavaScript execute on their browser, you should escape this value before inserting it into the page:

 
function getResponse(data) {
 var viewer = data.get("getViewer").getData();
 var testData = data.get("getData").getData()[[]viewer.getId()][[]"test"];
 var unescapedTestData = gadgets.util.unescapeString(testData);
 var testObject = gadgets.json.parse(unescapedTestData);
 document.getElementById("outputDiv").innerHTML = gadgets.util.escapeString(testObject.html);
}; 

Which winds up printing <img style="width: 1; height: 1;" src="adsfa" onerror="alert('hello')" /> instead of executing the code contained in that string. Whenever you are printing user-supplied data, make sure you run it through gadgets.util.escapeString first.

Quotas

Social applications may potentially be used by millions of users, and unchecked use of application data stores may put a large storage burden on containers. It is expected that most containers will therefore implement a data storage quota which applications may not exceed. This quota will usually be measured as being a certain number of bytes per application per user.

When writing application data that would exceed the size of the current quota, the write request will fail and the container will return an error message indicating that the limit was exceeded.

Here is some example code that tests for the presence of an error and displays the error's message if it exists:

 
function response(data) {
 if (data.hadError() && data.get("setdata").hadError()) {
   if (data.get("setdata").getErrorCode() == opensocial.ResponseItem.Error.FORBIDDEN) {
     /** Looks like the request went over quota, put code to handle this case here **/
     alert(data.get("setdata").getErrorMessage());
   }
 } 
 /** Continue normal execution here **/
}; 

Naturally, you shouldn't just print the message in a real application. You should try and delete unneeded data--perhaps you've been using Persistence data as a cache that can be cleared, or maybe a user is just saving too much information and you can prompt them to delete something before saving additional data. A well-designed application should be able to deal with this error and continue functioning as normal.

Working with your data

If you want to save data for the current user of your application, you will need to use VIEWER data.

Who is this VIEWER person, anyway?

VIEWER represents one of the more misunderstood concepts of OpenSocial. VIEWER is the person currently logged in to the container and interacting with the application.

Consider the example of looking at your own profile:

If someone else looks at your profile, they are the VIEWER, even if they are not friends with you:

Anyone looking at an application is the VIEWER, even if they don't have the application installed. However, as covered in the ACL Section, most containers will probably not return any personal information if the VIEWER doesn't have the application installed, meaning that reading VIEWER data can be somewhat limited.

Who can see VIEWER data?

When you are testing your own application and interacting with it directly, you fulfill the role of VIEWER, so it may feel intuitive that VIEWER data should be available to the application. However, in a deployment setting, many users of your applications will not have installed your application or given it rights to access their profile data. Therefore, your application will usually get the least amount of information from VIEWER objects as compared to OWNER or OWNER_FRIENDS.

When writing data, this rule is reversed. As covered in the a:ACL Section, most containers will have policies indicating that VIEWER is the only user type that persistent data may be written to (assuming the current VIEWER has the application installed).

What is important to note is that data written to the VIEWER can show up in different ways. For example, VIEWER data can be accessed by retrieving:

• VIEWER data (this one is pretty obvious)
• OWNER data, if you are looking at your own profile
• OWNER_FRIENDS data, if someone is looking at one of your friends' profiles.

The following cases illustrate when data written to VIEWER is retrievable by requesting VIEWER data:

The data is available...

This data is not available through requesting VIEWER data for any users other than Alice.

What uses are there for VIEWER data?

Viewer data is the cornerstone of the Persistence API. Many containers enforce a policy that all data must be written through VIEWER, which means that users cannot change application data for anyone but themselves.

While you can read data written to VIEWER by getting OWNER or OWNER_FRIENDS data in different contexts, directly reading VIEWER data is really only useful for storing application preferences that need to travel with the user instead of the profile on which the application is installed.

Consider:

Super Friend Wall features sound effects, which can be distracting. The developers have implemented an option to disable sound effects. When a user changes this preference, the value is stored in VIEWER data.

Most of the time, you'll want a user to set VIEWER data when they are interacting with the application on their own profile. In this case, this data will be available as OWNER data for anyone accessing the application on that profile.

Other times you'll want a user to set VIEWER data when they are interacting with the application on a friend's profile. This has the effect of making this data available as OWNER_FRIENDS data for anyone accessing the application on the friend's profile.

Working with data on your profile

If you want to get data for the user whose profile your application is installed on, you will need to use OWNER data.

Who is this OWNER person, anyway?

OWNER represents the person whose profile the current application is installed on. In a production setting, OWNERs of your application will have approved your application to access their profiles during the install process, so your application will have access to OWNER and OWNER data much more reliably than VIEWER, OWNER_FRIENDS, or VIEWER_FRIENDS.

One thing to keep in mind is that frequently the OWNER will not be the VIEWER, and may not have any relationship with the VIEWER. To keep the OWNER's data secure, applications may not write to the OWNER data store on the VIEWER's behalf.

If I can't set OWNER data, who can?

Well, the VIEWER can when the VIEWER is the same person as the OWNER. To set OWNER data, an application simply writes to the VIEWER data store for a user who has the application installed. This data is available as OWNER data on that user's profile.

Who can see OWNER data?

As long as an application is installed on someone's profile, OWNER data will always be available, no matter who is looking at the profile. For example, if:

The data is available...

What uses are there for OWNER data?

By setting and getting OWNER data, an application can let each of its users customize the way the application works on their profile. For example:

Super Friend Wall allows the OWNER to leave a welcome message that is displayed whenever the application is displayed on that OWNER's profile. This message is saved in the OWNER's application data.

Working with data on your friends' profiles

If you want to get data for the friends of the user whose profile your application is installed on, you will need to use OWNER_FRIENDS data.

Who are these OWNER_FRIENDS people, anyway?

OWNER_FRIENDS represents the list of friends of the current profile's OWNER. As with OWNER data, you cannot write directly to OWNER_FRIENDS data--instead, you can write to each user's VIEWER data as they interact with the application.

Structuring apps to take advantage of OWNER_FRIENDS data

OWNER_FRIENDS data is the only way to use application data to let friends of the OWNER affect the state of an application on the OWNER's profile.

It may seem counter intuitive, but data must be saved to a user's VIEWER data in order for it to show up on one of his or her friends' profiles as OWNER_FRIENDS data. As you can see in the tables above, OWNER_FRIENDS does not change, no matter who is looking at the profile.

Imagine that an application wants to let an OWNER's friends comment on photos stored on the OWNER's profile. This application needs to retrieve the photo choice on the OWNER's application data, but needs to fetch the friends' comments in OWNER_FRIENDS data.

When the application renders, it should do the following:

• Request OWNER_FRIENDS data and display any comments it finds stored there.
• Determine if the VIEWER is the same person as the OWNER. In this case, the OWNER is looking at their own profile.
  • If this case is true, UI elements to pick the photo that is displayed should be shown to the user.
• Determine if the VIEWER is in the OWNER_FRIENDS list. In this case, a friend of the OWNER is looking at the OWNER's profiles.
• Determine whether the VIEWER has the application installed.
  • If the VIEWER has the app installed, UI elements to comment on the OWNER's choice of photo should be shown to the user. If the user decides to make a comment, this will be stored in VIEWER data and will be retrievable via OWNER_FRIENDS.
  • If the VIEWER doesn't have the app installed, UI elements to encourage the VIEWER to install the application are displayed: "Install this app to leave a comment on this photo!"

Scoping data per OWNER

As an application developer, your first impulse to store each friend's comments might be to define a comment data structure. Say that Alice wants to comment on Barry's photo. The application could use the following structure:

{
   timestamp : 1205204585029,
   comment : "Hey Barry, nice photo!"
}

and store an array of them in the application data store.

[
  { timestamp : 1205204585029, comment : "Hey Barry, nice photo\!" },
  { timestamp : 1205204813389, comment : "What kind of camera is that?" },
  ...
]

But what happens when Alice wants to leave a comment on both Barry and Digby's profiles? A simple list doesn't keep track of which profile the comment belongs to. Your application should then store a list of data structures scoped to each OWNER's ID that the data should be displayed on:

{
  "11111111111111111111" :
    [
      { timestamp : 1205204585029, comment : "Hey Barry, nice photo!" },
      { timestamp : 1205204813389, comment : "What kind of camera is that?" }
    ],
  "22222222222222222222" :
    [
      { timestamp : 1205205223492, comment : "Hi Digby!  Long time no see!" }
    ],
  ...
]

where 11111111111111111111 is Barry's ID number and 22222222222222222222 is Digby's ID. Now, when the application retrieves Alice's data, it can compare the current OWNER's ID number against Alice's data store to see if Alice has commented on the current profile's photo.

As an alternative, the application can scope the key it uses to store its data with the ID of the current viewer. For example, the same photo application could store

[
  { timestamp : 1205204585029, comment : "Hey Barry, nice photo!" },
  { timestamp : 1205204813389, comment : "What kind of camera is that?" }
]

using the key comments_11111111111111111111 and then store

[
  { timestamp : 1205205223492, comment : "Hi Digby!  Long time no see!" }
]

under the key comments_22222222222222222222. The application would need to get the owner's profile ID number to request the correct comments data key, but this would minimize the amount of unneeded data returned to the application.

Who can see OWNER_FRIENDS data?

OWNER_FRIENDS data is available in cases where any friends of the OWNER have application data stored on their profiles. This data will be available no matter which profile it was stored on.

Consider:

The data is available...

These examples are identical to the cases where OWNER data is available.

What uses are there for OWNER_FRIENDS data?

OWNER_FRIENDS data can be used for allowing friends of the OWNER to interact with the current application. For example:

The Super Friend Wall app allows members of OWNER_FRIENDS to leave comments about the welcome message on the OWNER's profile. These comments are accessed through OWNER_FRIENDS and displayed to the user.

Working with your friends' data

If you want to get data for the friends of the user who is currently using your application, you will need to use VIEWER_FRIENDS data.

Who are these VIEWER_FRIENDS people, anyway?

VIEWER_FRIENDS represents the list of friends of the current VIEWER:

Who can see VIEWER_FRIENDS data?

You cannot write directly to VIEWER_FRIENDS data but have to store data in each user's VIEWER data when they are viewing the application.

Under most container policies, VIEWER_FRIENDS Persistence data will only be returned:

1. If the VIEWER has the application installed.
2. For VIEWER_FRIENDS who have the application installed.

This makes VIEWER_FRIENDS data somewhat limited in availability, although there are still some cases where it may be useful. The following example illustrates when VIEWER_FRIENDS data will be available.

The data is available...

What uses are there for VIEWER_FRIENDS data?

VIEWER_FRIENDS data can be used for allowing the VIEWER to see detailed information about what his or her friends have been doing when using the application. Consider the following example:

The Super Friend Wall app offers a tab called "Your friends' recent comments". When a user views this tab, it displays a list of the ten most recent comments that their friends have made using Super Friend Wall.

By reading through this article, you should have gotten a thorough understanding of how OpenSocial application data is written and read, as well as some ideas of how to structure the data access model in your application.

For more information about the Persistence API and OpenSocial in general, you should consult some of these resources:

• OpenSocial homepage
• OpenSocial Developer Forum
• #opensocial on irc.freenode.net

For updates and news about OpenSocial, you can subscribe to the OpenSocial blog