Demo Video: EE_Demo.swf

Proposal Details

Scope of change: Medium
Reference to existing standard: Aligning with OEmbed, pending licensing issues
Champion Name: Andrew Davis(IBM), Ryan Baxter(IBM)
Specification editor names and declarations of time commitment: Andrew Davis(IBM), Ryan Baxter(IBM)
Ref Implementors commitment: 1 developers(Ryan) at IBM for 6-8 weeks, targeting shindig patch mid February

Background

What does "Embedded Experience" mean? It means to embedded the experience of another service into a container. One way to do this is to allow an OpenSocial gadget as a means of providing the embedded experience to the end user. This allows the user to interact with the content from the service as if they were using the service directly even though the content is embedded elsewhere. What differentiates an embedded experience gadget from other gadgets is the flexibility of the gadget.

Use Cases

• User A logs into their favorite social networking site and sends a message to User B. User B gets an email notification with an embedded experience. The embedded experience contains the content of the message and provides a way for User B to reply to the message right from their email without having to go to the social networking site to do so.
• User A logs into box.net and shares a file with User B. User B gets an email notification with an embedded experience of that file. The embedded experience in that email allows User B to preview the file right from his email and allows him to post a comment on the file.
• User A IM's User B via their favorite IM service. User A asks User B to check out a file which he has placed on box.net. He copies a snipped of code to IM an embedded experience of that file to User B. User B can then preview that file right from his IM client and let User A know what he thinks.
• User A is watching a video on YouTube and decides he wants to tweet about it. User A clicks the share button, and selects Twitter, and YouTube posts a tweet about the video to User A's Twitter account. User B is following User A on Twitter and sees User A's tweet about the video. The Tweet contains an embedded experience that allow User B to watch the video from Twitter. User B likes the video so much he decides he wants to subscribe to the video publisher's channel. User B does this from the embedded experience in the tweet.
• User A is watching a video on YouTube and decides he wants to share it on his favorite social networking site. User A clicks the share button, and selects the social networking site, and YouTube updates User A's status on that social networking site and includes an embedded experience for the YouTube video. User B is friends with User A on this social networking site and sees User A's status update in his activity stream. The status update contains an embedded experience that allow User B to watch the video inline. User B likes the video so much he decides he wants to subscribe to the video publisher's channel. User B does this from the embedded experience in the status update.

Today's Solutions

Several services have some type of solution for embedding content. Many social networking sites and email clients can recognize URLs in content and render those URLs. For example if I tweet about a YouTube video, Twitter can now recognize that the URL in the tweet is a URL to a YouTube video and provide a player for it. However the user cannot take any further action on the video, to do anything more than watch it they must go to YouTube.

Google's contextual gadgets take this approach a step further allowing gadgets to recognize any type of content in an email and then use that information in the gadget itself.

Microsoft has a feature which is part of Hotmail called active views.  Active Views are very similar to what we are trying to achieve with embedded experiences, however it is very specific to email and not based on open standards.  Active views also has some similarities to Google's contextual gadgets in that they are scanning emails looking for content, that they know how to handle.  For example a link to YouTube or Flickr. 

Proposal

All of the current solutions involve the service doing some kind of analysis of the content and seeing if there is anything that the service recognizes.  Instead of recognizing something in the content like a URL, a service can provide an embedded experience as a way to embed content.  By doing this the content host is TELLING the service how to render it's content instead of having the service figure out how to render the content. An embedded experience should contain two pieces of information. First, is a URL to the gadget that will actually render the object. Second, is some contextual information about the object. For example, there could be a single YouTube video player gadget definition which acts as the embedded experience that YouTube uses. When a YouTube video is embedded YouTube would provide the URL to that gadget definition as well as the contextual information. The gadget then uses the contextual information to render and play the correct YouTube video.  The contextual information would be requested by the gadget from the container.

By embedding the rendering of the object inside the content you eliminate the need to process the content for relavant information. The enhanced experience is no longer dependent of having some type or recognizer "installed". With embedded experiences the publisher of the content tells the render what they should use to render this type of content.

Data Model

In order for content to be embedded we need a way for providers to tell consumers how to embed their content. The ideal way to communicate the data of what is to be embedded is to use either Atom or JSON. These two data formats are easy to consume in several different contexts and are widely used throughout the community already.

No matter the syntax the data model will need to include a few key pieces of information.

*The data model must container at least a url or gadget.  If a gadget is provided than contextual data must also be provided.

ATOM

One place where embedded experiences may be used is inside Atom feeds. When an embedded experience is included within an Atom feed the embedded experience data model needs to be embedded within the Atom entry. We can embed the data model as part of the inline child element of link elements . This implies that the service also provide some type of REST API to access the data model. The href of the link element points to that REST service. If a user was to hit the URL in the href attribute of the link element it would be returned the embedded experiences data model. By inlining it we avoid the need to hit the REST service ourselves.
Also notice the different rel and type attributes of the link element. These would need to be registered with IANA.

URL Embedded Experience

....
<entry>
       <title>YouTube Player</title>
       <link href="http://www.youtube.com/watch?v=9gW2YVBrNVA"/>
       <!-- Link element with href pointing to embedded REST endpoint -->
       <link rel="render" type="application/embed+xml" href="http://www.youtube.com/embeddedexperiences?url=http://www.youtube.com/watch?v=9gW2YVBrNVA&amp;format=xml">
       	<inline>
         <embed>
           <url>http://www.youtube.com/watch?v=9gW2YVBrNVA</url>
         </embed>
        </inline>
       </link>
       <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>
       <updated>2003-12-13T18:30:02Z</updated>
       <summary>Some text.</summary>
    </entry>
.....

Gadget Embedded Experience

....
<entry>
       <title>YouTube Player</title>
       <link href="http://www.youtube.com/watch?v=9gW2YVBrNVA"/>
       <!-- Link element with href pointing to embedded REST endpoint -->
       <link rel="render" type="application/embed+xml" href="http://www.youtube.com/embeddedexperiences?url=http://www.youtube.com/watch?v=9gW2YVBrNVA&amp;format=xml">
       	<inline>
         <embed>
           <!-- Gadget Definition -->
           <gadget>http://localhost:8080/fiesta/embeddedexperiences/examples/YouTubePlayer.xml</gadget>
           <!-- Embedded Experiences Context -->
	   <context>9gW2YVBrNVA</context>
           <!-- Preview Image -->
           <previewImage>http://i2.ytimg.com/vi/9gW2YVBrNVA/hqdefault.jpg</previewImage>
         </embed>
        </inline>
       </link>
       <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>
       <updated>2003-12-13T18:30:02Z</updated>
       <summary>Some text.</summary>
    </entry>
.....

URL and Gadget Embedded Experience

A service may optionally include both a URL and gadget embedded experience and it will be up to the container to choose which embedded experience to render.

....
<entry>
       <title>YouTube Player</title>
       <link href="http://www.youtube.com/watch?v=9gW2YVBrNVA"/>
       <!-- Link element with href pointing to embedded REST endpoint -->
       <link rel="render" type="application/embed+xml" href="http://www.youtube.com/embeddedexperiences?url=http://www.youtube.com/watch?v=9gW2YVBrNVA&amp;format=xml">
       	<inline>
         <embed>
           <!-- URL Embedded Experience -->
           <url>http://www.youtube.com/watch?v=9gW2YVBrNVA</url>
           <!-- Gadget Definition -->
           <gadget>http://localhost:8080/fiesta/embeddedexperiences/examples/YouTubePlayer.xml</gadget>
           <!-- Embedded Experiences Context -->
	   <context>9gW2YVBrNVA</context>
           <!-- Preview Image -->
           <previewImage>http://i2.ytimg.com/vi/9gW2YVBrNVA/hqdefault.jpg</previewImage>
         </embed>
        </inline>
       </link>
       <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>
       <updated>2003-12-13T18:30:02Z</updated>
       <summary>Some text.</summary>
    </entry>
.....

ActivityStreams

Another data model where embedded experiences will prove useful is in ActivityStreams. OpenSocial should define an extension to the ActivityStreams spec in order to add the embedded experiences data model. All OpenSocial extensions will be contained within the 'opensocial' namespace. The embedded experiences data model will be placed within the embedded property.

URL Embedded Experience

 {
    "postedTime": "2011-02-10T15:04:55Z",
    "actor": {
      "url": "http://example.org/martin",
      "objectType" : "person",
      "id": "tag:example.org,2011:martin",
      "image": {
        "url": "http://example.org/martin/image",
        "width": 250,
        "height": 250
      },
      "displayName": "Martin Smith"
    }
    "verb": "post",
    "object" : {
      "url": "http://example.org/blog/2011/02/entry",
      "id": "tag:example.org,2011:abc123/xyz"
    },
    "target" : {
      "url": "http://example.org/blog/",
      "objectType": "blog",
      "id": "tag:example.org,2011:abc123",
      "displayName": "Martin's Blog"
    },
    "opensocial" : {
      "embed" : {
           "url" : "http://example.org/blog/"
      }
    }
 }

Gadget Embedded Experience

 {
    "postedTime": "2011-02-10T15:04:55Z",
    "actor": {
      "url": "http://example.org/martin",
      "objectType" : "person",
      "id": "tag:example.org,2011:martin",
      "image": {
        "url": "http://example.org/martin/image",
        "width": 250,
        "height": 250
      },
      "displayName": "Martin Smith"
    }
    "verb": "post",
    "object" : {
      "url": "http://example.org/blog/2011/02/entry",
      "id": "tag:example.org,2011:abc123/xyz"
    },
    "target" : {
      "url": "http://example.org/blog/",
      "objectType": "blog",
      "id": "tag:example.org,2011:abc123",
      "displayName": "Martin's Blog"
    },
    "opensocial" : {
      "embed" : {
           "gadget" : "http://example.org/blogs/gadget.xml",
           "context" : "abc123"
      }
    }
 }

URL and Gadget Embedded Experience

A service may optionally include both a URL and gadget embedded experience and it will be up to the container to choose which embedded experience to render.

 {
    "postedTime": "2011-02-10T15:04:55Z",
    "actor": {
      "url": "http://example.org/martin",
      "objectType" : "person",
      "id": "tag:example.org,2011:martin",
      "image": {
        "url": "http://example.org/martin/image",
        "width": 250,
        "height": 250
      },
      "displayName": "Martin Smith"
    }
    "verb": "post",
    "object" : {
      "url": "http://example.org/blog/2011/02/entry",
      "id": "tag:example.org,2011:abc123/xyz"
    },
    "target" : {
      "url": "http://example.org/blog/",
      "objectType": "blog",
      "id": "tag:example.org,2011:abc123",
      "displayName": "Martin's Blog"
    },
    "opensocial" : {
      "embed" : {
           "url" : "http://example.org/blog/2011/02/entry",
           "gadget" : "http://example.org/blogs/gadget.xml",
           "context" : "abc123"
      }
    }
 }

APIs

Gadget APIs

Embedded experiences will not add any new APIs to the spec.  Instead it will leverage a combination of existing features, RPC and data context, to accomplish everything it needs.  Essentially the container will communicate to the embedded experiences feature, via RPC, what the gadgets context is.  The embedded experiences feature will then store that context in that gadgets data context object with a reserved key.  The gadget can then use the existing data context apis to retrieve its context and act on it appropriately.  Below is an example gadget using the embedded experience feature.

<Module>
	<ModulePrefs title="Embedded Experiences Test" description="Embedded Experiences Test Gadget">
		<Require feature="embedded-experiences"></Require>
	</ModulePrefs>
	<Content type="html">
             <![CDATA[
                      <script type="text/javascript">
                           /*
                            *  Function which gets called whenever the embedded experience context of this gadget changes.
                            */
                           function myCallback(context){
     		                document.getElementById("contextData").innerHTML = "Context Information For This Gadget: " + context;
     	                   }

     	                   function initData() {
                                /*
                                 *  Register a listener to know that the value associated with the key org.opensocial.ee.context has changed.
                                 */
	                        opensocial.data.getDataContext().registerListener('org.opensocial.ee.context', function(key){
     			             myCallbackL(opensocial.data.getDataContext().getDataSet(key));
     		                });
     	                   }

     	                   gadgets.util.registerOnLoadHandler(initData);
                     </script>

                    <div id="contextData"></div>
            ]]>
	</Content>
</Module>

Container APIs

To make things easier on the container we will provide some APIs that build upon the common container to render all types of embedded experiences (URL and gadget).

shindig.container.Container.navigateEE

shindig.container.Container.navigateEE(eeElement, eeDataModel, eeRenderParams)

Parameters

Returns

Description

Renders the embedded experience on the page in the element provided.

shindig.container.Container.closeEE

shidig.container.Container.closeEE(site)

Parameters

Description

Closes the embedded experience contained within the site on the page.

Classes

To support the container in handling both embedded experience gadgets and URLs, I think we should introduce some functionality in the container to support URLs.  This functionality is modeled after what has been done with the common container code.  Specifically I think we would benefit from having the equivalent of a gadget holder and gadget site for URLs as well.  These classes should implement the same methods where possible so the container does not have to distinguish between what type of site or holder it is dealing with.  I have take a first stab at this, and here is what I have come up with.

shindig.container.EEUrlHolder

shindig.container.EEUrlSite

Views

Gadgets may also specify a view, named embedded, which will act as a gadgets embedded experience.  If a gadget does not specify the embedded view than the container should consider the default view the embedded view when rendering a gadget as an embedded experience.

<Module>
	<ModulePrefs title="Embedded Experiences Test" description="Embedded Experiences Test Gadget">
		<Require feature="embedded-experiences"></Require>
	</ModulePrefs>
        <Content type="html" view="home">
              .....
        </Content>
	<Content type="html" view="embedded">
             <![CDATA[
                      <script type="text/javascript">
                           /*
                            *  Function which gets called whenever the embedded experience context of this gadget changes.
                            */
                           function myCallback(context){
     		                document.getElementById("contextData").innerHTML = "Context Information For This Gadget: " + context;
     	                   }

     	                   function initData() {
                                /*
                                 *  Register a listener to know that the value associated with the key org.opensocial.ee.context has changed.
                                 */
	                        opensocial.data.getDataContext().registerListener('org.opensocial.ee.context', function(key){
     			             myCallbackL(opensocial.data.getDataContext().getDataSet(key));
     		                });
     	                   }

     	                   gadgets.util.registerOnLoadHandler(initData);
                     </script>

                    <div id="contextData"></div>
            ]]>
	</Content>
</Module>

Security

Embedded Experiences is focused on providing an integrated experience within an open social container that is providing collaboration functionality. In this model the third party services is providing
1) the content/message being rendered
2) specifying a gadget hosted on their server to render the content.

Because the content provided to open social container, is often via a feed, email or other transport mechanism is addressed to multiple individuals, the act of rendering a third party component, such as a gadget, specified by the third party service sent to a user, inherently opens up the container to two types of vulnerabilities that must be handled:
Threat 1) a classic phishing attack(think hidden embedded images being used to validate email addresses) experienced in email systems.
Threat 2) theft of information from the open social container by the third party service, such as a user's address book.

'''Phishing Attacks'''

To this end, for container to safely implement the embedded experiences API, the container implementation must first establish a trust relationship before it even invokes the Shindig renderer to download and render the specified gadget.

Two approaches are possible

1) choosing specific senders to trust, which in email systems is inherently unsafe due to email header and identity forgery. The same issues exist for feeds that can be federated or provided by unknown sources.

2) the container must have a configured set of trusted gadgets that may be rendered via the embedded experiences service. For example, if a user is sent an email tracking number for a fedex package and contains the EE xml to render it via a gadget provided by fedex, a container implementing EE should only render that fedex gadget if it is trusted. The mere act of Shindig downloading the specified gadget.xml is a vector for a phishing attack.

'''Restricting Access to APIs'''

If a container choose trust the gadget to render specific content, then we should render it via shindig and provide it access to the minimal set of optional feature apis. Our concern arrives from a gadget that has been granted access to use EE API, then subsequently uses other optional and recommended features provided by an open social container. For example, if our FedEx Widget above, has full access to osapi.people.* this supposed FedEx Widget could call osapi.people.get() and then steal the user's list of friends.

To guard against use Threat 2, it is necessary for Shindig to provider feature level security, by which a specific gadget is granted access to a limited set of feature(js apis). In most cases, a gadget rendered as part of embedded experiences would be trusted only to be rendered, and to access the EE API section, but would be explicitly prevented from accessing APIs, such as open social. Note securing these APIs involves securing both the JS apis, and more importantly the backend services that implement the JS APIs, as a gadget may always load its own JS library that attempts to access the same backend services.

Links:
[a:Shindig feature level security]

< Mark W 4/6>

It would be great if we could represent the entire embedded experience as a URL. Something like... 

http://xyzcompany.com/myapp.xml?view="embedded"&context="http://xyzcompany.com/UrlToResolveableContextInfo"

This way, we could also represent embedded experiences directly as a url. This would work very similar to the requestNaviagateTo where you bind to a url. We could specify the following view rules

1) The view the user specifies

2) "embedded" (a new view we are introduce)

3) home (if present)

Return an error if none of these are met. We could define other behavior if we need to.

</Mark W.>