Scope of change: Major
Champion Name: Eric Woods (IBM)
Spec Editors: Eric Woods
Ref implementation: Apache Shindig 3.0

Call for Meeting: December 13, 2011

We'll have an organizational meeting on December 13, 2:00pm EDT.

Calendar information is here...

USA Toll-Free: 888-426-6840
USA Caller Paid: 215-861-6239
For Other Countries: https://www.teleconference.att.com/servlet/glbAccess?process=1&accessCode=3295259&accessNumber=2158616239
Participant Code: 3295259

Introduction

A number of folks have been interested in aligning CMIS and OpenSocial. While there was an initial prototype of this done earlier this year, we need to take the next step and document the base set of use cases and high-level positioning.

When OpenSocial was initially created several years ago, a primary use case was sharing pictures and videos with your friends. This is reflected in the concept of albums and media items. However, as OpenSocial has become increasingly adopted by vendors such as Jive, IBM, SAP, Cisco, and others, the limited scope of albums and media items are not sufficient for enterprise scenarios. An effort was started at OASIS to standardized the data model and the APIs around content management.

In addition, many of these social platforms already have the concept of files, directories, and other constructs. Large enterprises often have a social platform AND a CMS system. The objective of this effort is NOT to make all social platforms CMIS systems, and all CMIS systems social platforms. Rather, it is to define a set of APIs that are consistent with the OpenSocial programming model that facilitate easy integration with CMIS based systems. This will allow both specifications and communities to focus on their specialties, OpenSocial on building and defining the social data model for the Web, and CMIS for building enterprise class content management APIs.

Many enterprise social platforms have some basic form of file management, and increasingly, OpenSocial is becoming part of the programming model that these platforms offer for integration at the application (gadget) layer, as well as the server to server integration layer (REST). Increasingly, customers of both CMIS systems, and social platforms are expecting these two systems to "just work together". By aligning with CMIS, the OpenSocial community will get the benefit of a more complete, feature rich data model that can easily be plugged in at the server layer with a CMIS system. The CMIS community benefits by having a rich social model that can be used to foster collaboration around the content that resides in their systems. The open ecosystem of vendors, application developers, and customers benefits by having clearly defined boundaries among standards, a consistent Web 2.0 social application model, and perhaps most importantly, an open standards based integration path for enterprise systems.

Use Cases

Document & Folder CRUD Operations

Explain basic crud use case in more detail here.

Create a Document

First, a document entry is posted to a CMIS folder (a collection). In response, an entry is returned containing metadata. This metadata contains a link to which document content may be PUT. Document content is PUT to this link to create the document content in the CMIS repository.

When the CMIS browser binding is approved and CMIS repositories support its capability, HTTP form posts may be used to upload document content. Until then, a mechanism will have to be established to retrieve the bits from the user's local machine and upload them to the PUT link.

Retrieve a Document

First, metadata for a document is retrieved from the repository. The metadata contains a link that points to the document content within the repository. Second, the link is followed to retrieve document content.

Update a Document

TODO

Delete a Document

TODO

Retrieve a Folder

TODO

Create a Folder

TODO

Delete a Folder

TODO

Navigation Services

A system must be established for navigating the document & folder tree. A folder's children ought to be exposed. Documents ought to identifiable by path.

We need a way to identify or discover the repository root.

TODO

Other Considerations

CMIS Document Sub-Types

CMIS supports the notion of document sub-types, which inherit certain properties and attributes from their parents. Sub-types may require certain properties/attributes to be defined in order to create documents of that type. We must, at minimum, allow gadget developers to retrieve the repository's sub-types and metadata to facilitate creation of those sub-types.

CMIS Queries

CMIS supports querying, such as full text searching within CMIS documents. For the first pass, OpenSocial will not support querying operations (subject to change).

CMIS Versioning

Multiple versions of a document may live in the CMIS repository. We need to handle versioning with respect to CRUD operations. For example, when a document is retrieved, only the latest version of the document is returned. When a document is updated, it may create a new version. These assumptions need to be established and documented.

Policies, Relationships, and Access Control Limits

Policies, relationships, and access control limits will not be supported or handled in OpenSocial. However, OpenSocial will express, verbatim, errors from incorrect permissions and other illegal CMIS operations.

Documents & Comments

A user creates a document in a CMIS repository using an OpenSocial gadget. Other users post comments which are associated with the document. Comments may be created, retrieved, updated, and deleted. Since CMIS does not support document comments, comments will live with the social container. We need to establish how OpenSocial comments (i.e. Activity Streams) will be associated with documents, likely using an IRI or GlobalId.

Technical Approach

An initial mapping of the data models between OpenSocial and CMIS has been done and is available as an attachment to this page.CMIS Mapping.xls

We will treat the delta in fields following the extension pattern.

Base OpenSocial Data Model

Document
*id
*contentStreamLength
*contentStreamMimeType
*contentStreamFileName
*contentStreamId
*name
*objectId
*createdBy
*creationDate
*lastModifiedBy
*lastModificationDate
*versionLabel
*description
*language
*location
*numComments
*numViews
*numVotes
*rating
*startTime
*taggedPeople
*tags
*thumbnailUrl
*title
*type
*url
*<OS owner>

Document Model

We'd like to have a simple subset of CMIS be the top level model and allow the full CMIS model to be accessible from a "cmis" namespaced extension.

TODO

Folder Model

TODO

OpenSocial JavaScript API for Documents & Folders

APIs should support retrieving @self's files or @friend's documents as well as by ID.

osapi.documents (TODO)

osapi.documents.get(...)

osapi.documents.create(...)

osapi.documents.update(...)

osapi.documents.delete(...)

osapi.folders (TODO)

osapi.folders.get(...)

osapi.folders.create(...)

osapi.folders.update(...)

osapi.folders.delete(...)

Deprecating Albums and Media Items

As a result of this effort, there will no longer be a need for the OpenSocial specification to carry forward the definition and APIs for Album and Media Item. As such, they will be deprecated when support for the new APIs & data model that is based on CMIS is introduced. Whenever possible, we'll outline the migration path from the old APIs to the new data model. In addition, there are social constructs that will be carried forward and remain the responsibility of the OpenSocial based container.

Meeting Minutes

12/06/2011

Proposing following structure for top level CMIS object. TBD if Document and Folder will subclass this.

class CMISObject {
  Map<String, String> attributes;
  Map<String, Map<String, Object>> properties;


  /** Accessors and mutators */
  Map<String, Map<String, Object>> getProperties()
  void setProperties(Map<String, Map<String, Object>> properties) { this.properties = properties; }
  Map<String, String> getAttributes() { return attributes; };
  void setAttributes(Map<String, String> attributes) { this.attributes = attributes; };
}

This top-level object is capable of representing all CMIS Object-Types (Document, Folder, Relationship, etc):

Example:

"object: {
  "attributes":{"fileable":true, ... },
  "properties":{
    "cmis:lastModificationDate":{
      "cardinality":"single",
      "type":"datetime",
      "displayName":"CMIS Last Modification Date Property",
      "value":1298396347156,
      "queryName":"cmis:lastModificationDate",
      "localName":"cmis:lastModificationDate",
      "id":"cmis:lastModificationDate"
    }, ... 
  }
}

But what do we want surface in the OpenSocial API? A few options:

Preserve native CMIS constructs, as above. For example, osapi.documents.get(); returns a JSON object like above. To access the object's ID, the gadget developer will have to say document.properties.objectId.value;
Wrap native CMIS constructs with simplified representation of a Document and Folder. For example, an OpenSocial document may have an 'id', 'displayName', 'created' and other standard fields. A call to opapi.documents.get(); returns a JSON object in this simplified form, allowing the gadget developer retrieve the document's ID with document.id;
A hybrid of the first two approaches; include CMIS native constructs (i.e. attributes & properties) AND provide a high level wrapper class for simplicity. Extensions may be used to implement this.

To decide which approach to take, we need to determine what the end goal is. If the end-goal is explicit alignment with CMIS, then approach #1 may suffice. If the goal is only to provide generic Document & Folder constructs to OpenSocial, which may later be mapped to a CMIS back-end, approach #2 may suffice. Approach #3 encompasses both.

For the first pass, I will be implementing only CMIS native constructs (approach #1). OpenSocial Document & Folder constructs (i.e. wrappers) may be implemented relatively easily.

CMIS Base Patch Available - 13 December 2011

Setup

Check out Apache Shindig.
Download and apply CMIS patch to Shindig.
Build Apache Shindig.
Deploy Shindig to your favorite application server.
Check out Apache Chemistry.
Build Apache Chemistry using Maven.
1. Via command line, navigate to the checked out Chemistry project.
2. Run: mvn clean install -Dmaven.test.skip=true
Deploy Apache Chemistry's in-memory repository to your favorite application server.
1. The WAR file is located under chemistry-opencmis-server > chemistry-opencmis-server-inmemory > target.
2. The patch is currently hardcoded to look for a CMIS repository at http://localhost:8081/repository, so deploy the WAR to port 8081 with context root "repository".
3. For more information, refer to Chemistry's documentation for building and deploying the in-memory repository.
Deploy Apache Chemistry's browser tool.
1. The WAR file is located under chemistry-opencmis-test > chemistry-opencmis-test-browser-app > target.
2. E.g. http://localhost:8081/repository/browser/
Restart servers.
Confirm environment setup.
1. Point your browser to the CMIS browser tool (e.g. http://localhost:8081/repository/browser/browse)
2. Enter http://localhost:8081/repository/atom into the browser tool.
3. Verify you can browse the CMIS repository using the tool.
4. Point your browser to the OpenSocial documents service (e.g. http://localhost:8080/social/rest/documents/john.doe/@self/1/)
5. Verify that john's documents are returned (John may not have any documents, but shouldn't thrown an error).

Documents API

POST http://localhost:8080/social/rest/documents/john.doe/@self/1?parentId=100

Posts a document for John Doe. The document's parent is identified by parentId [OSD:required].
Example minimal POST body:

{
   "properties":{
      "cmis:name":{
         "cardinality":"single",
         "displayName":"CMIS Name Property",
         "id":"cmis:name",
         "localName":"cmis:name",
         "queryName":"cmis:name",
         "type":"STRING",
         "value":"Erics Doc"
      },
      "cmis:objectTypeId":{
         "cardinality":"single",
         "displayName":"CMIS Object Type Id Property",
         "id":"cmis:objectTypeId",
         "localName":"cmis:objectTypeId",
         "queryName":"cmis:objectTypeId",
         "type":"ID",
         "value":"cmis:document"
      }
   }
}

PUT http://localhost:8080/social/rest/documents/john.doe/@self/1?documentId=136

Updates a document, identified by documentId. The PUT body is the same as the POST body above, but with updated fields.

GET http://localhost:8080/social/rest/documents/john.doe/@self/1

Retrieves all of John Doe's documents

GET http://localhost:8080/social/rest/documents/john.doe/@self/1?documentId=136

Retrieves John Doe's document with ID 136.

GET http://localhost:8080/social/rest/documents/john.doe/@self/1?documentId=136,137

Retrieves John Doe's documents with IDs 136 and 137.

DELETE http://localhost:8080/social/rest/documents/john.doe/@self/1?documentId=136,137

Deletes John Doe's documents with IDs 136 and 137.

Folders API

POST http://localhost:8080/social/rest/folders/john.doe/@self/1?parentId=100

Creates a folder whose parent is identified by parentId. If no parentId is given, the folder is created under the repository root folder.
Example minimal POST body:

{
  "properties": {
    "cmis:objectTypeId": {
      "id": "cmis:objectTypeId",
      "localName": "cmis:objectTypeId",
      "queryName": "cmis:objectTypeId",
      "value": "cmis:folder",
      "displayName": "CMIS Object Type Id Property",
      "type": "ID",
      "cardinality": "single"
    }, {
    "cmis:name": {
      "id": "cmis:name",
      "localName": "cmis:name",
      "queryName": "cmis:name",
      "value": "Erics Folder 2",
      "displayName": "CMIS Name Property",
      "type": "STRING",
      "cardinality": "single"
    }
  }

}

PUT http://localhost:8080/social/rest/folders/john.doe/@self/1?folderId=139

Updates the folder, identified by folderId. The PUT body is the same as the POST body (but with updated properties).

DELETE http://localhost:8080/social/rest/folders/john.doe/@self/1?folderId=139

Delets the folder, identified by folderId.

GET http://localhost:8080/social/rest/folders/john.doe/@self/1

Gets all of John Doe's folders.

GET http://localhost:8080/social/rest/folders/john.doe/@self/1?folderId=139

Gets John Doe's folder with ID=139.

GET http://localhost:8080/social/rest/folders/john.doe/@self/1?folderId=139,140

Gets John Doe's folders with IDs 139 and 140.

Notes

I used Apache Chemistry's InMemory Repository to build this patch. It's appropriate because it's lightweight and resets all data on each restart, like Shindig's JSON DB.
The CMIS repository is currently hardcoded to point to http://localhost:8081/repository/atom in JsonDbOpensocialService.java. To change the repository, change this hardcoding (near the bottom of the file).
The OpenSocial sample JSON DB is only persisting document & folder IDs; no document & folder metadata are being persisted in the JSON DB. This is to circumvent caching issues. If another user changes a document on the CMIS back-end, the OpenSocial metadata will become stale. By only persisting the document/folder ID, the latest document/folder metadata is retrieved from the server on each request.
Despite only folder/document IDs being persisted in the social container, these IDs may still become out-of-sync with the CMIS back-end. For example, a user creates several documents using the OpenSocial APIs. The OpenSocial container is then restarted, losing the user's records of those documents. The documents will still be persisted in the CMIS repository, but a GET via the OpenSocial APIs will return an error because the container does not have a record of the user owning the file. As a work-around, a PUT request may submitted via the OpenSocial API, inserting a record of the user owning the file with the container. From there, subsequent GETs, PUTs, and DELETEs will work as expected.
For convenience, several document and folder IDs should be pre-populated in Shindig's canonicaldb.json database so that GET calls return several objects by default. These IDs should align with Chemistry's in-memory repository default sample documents & folders.

References

11/30/2011 Update: We will be referencing the up-and-coming CMIS 1.1 draft spec for this work. This draft spec is here: CMIS 1.1 Draft Spec

Some notes from call on 15-Nov-11 -- Eric will clean up.

* Need to tie in discovery of both CMIS servers and the repositories w/in those server into OS discovery

* It's expected that the OS Container would be configured with a known set of servers, but the repositories should be exposed.

* Need to be able to handle multiple repositories -- e.g. like OAuth.

CMIS 0.5 JSON Binding (outdated) - This is a browser binding that, when invoked, will return JSON objects.

Next steps

David -- Will take a pass at updating the data model based on json binding

Mark -- Will try to put more details around the basic use cases

Ryan -- Verify & update links on this page

ToDo

Submit ideas for what the API would look like.

Need to get a test server up. Best to use chemistry. Maybe leverage EC2 instance as well.

OpenSocial Reference Material

CMIS Use Cases and Positioning

Call for Meeting: December 13, 2011

Introduction

Use Cases

Document & Folder CRUD Operations

Create a Document

Retrieve a Document

Update a Document

Delete a Document

Retrieve a Folder

Create a Folder

Delete a Folder

Navigation Services

Other Considerations

CMIS Document Sub-Types

CMIS Queries

CMIS Versioning

Policies, Relationships, and Access Control Limits

Documents & Comments

Technical Approach

Base OpenSocial Data Model

Document Model

Folder Model

OpenSocial JavaScript API for Documents & Folders

osapi.documents (TODO)

osapi.folders (TODO)

Deprecating Albums and Media Items

Meeting Minutes

12/06/2011

CMIS Base Patch Available - 13 December 2011

Setup

Documents API

Folders API

Notes

References