Web API maturity model

Web APIs (REST-based HTTP services with simple payloads, typically JSON) have proven to be a very simple and effective way for applications to consume capabilities provided by apps or expose some capabilities themselves. As we've experienced, there are various ways these APIs are defined from satisfying a large audience based on a single server (Twitter) to something more general that handles various kinds of resources and operations.

The reality of many of these Web APIs, their purpose and needs have evolved from their early inception and then evolving into their latest 3.1 version with additional bells and whistles, deprecating some baggage along the way.

So the title of this post is "Web API maturity model", so what is this and why is it needed?  Perhaps maturity model isn't the right title for it, let me describe the aspects of it and give some examples.  This looks a bit beyond Martin Fowler's article about this to go beyond his levels.

Basically it is a way to talk about how Web APIs are development and the various needs as the customer needs evolve and therefore the APIs evolve to meet those needs.

Maturity levels (loosely sketched out as):

1. REST+JSON (barely docs) - this is barebones, your app defines a simple Web API.  The goals at this level are not to make it rock solid and cover ever edge case but instead to focus on supporting an important scenario quickly, getting feedback on usage and evolve from there (if needed).  You rely on a sample, an incomplete wiki page or reading the source to figure out how best to use it.
2. REST+JSON (better docs, source) - a better web page dedicated to describing the API, samples and some source.  At this point you are promoting it a bit more as "the API" for your service.
3. REST+JSON (LD, docs, source) - needing a way to describe the payloads more consistently between various endpoints, could reuse existing vocabularies such as schema.org.  App developers may even want to share some of their vocabularies or capabilities
4. LDP (REST+JSON-LD&Turtle+containers) - needing a way to have a consistent pattern for interacting with "buckets" of like resources
5. Domain extensions (OSLC, etc) - needs around app development, systems engineering, ... which could create new vocabularies or capabilities themselves

In fact, these layers of maturity can be cleanly applied on one another without disrupting previous apps that consume the API.

Some may say "ya, dumbass this is just how we work", which they are right (at least not the dumbass part).  We often start small, find what works and evolve from it.  Though it is good sometimes to be clear and intentional in that is how you plan to operate as some look building integration APIs in a way that starts in the middle or end, instead of at level 1.  Though learning about various techniques and technologies in other levels helps guide developers to upgrade their mental toolbox of skills.

The levels I highlighted are just examples, incomplete and could even be viewed that there are different axis that might emerge.  Related to that is understanding in what scenarios stepping up the levels, or over to another axis, may make sense.  I plan to explore some more of these ideas, including how best to define, present, collaborate, evolve, enable all this. 

As always, interested to hear feedback (I prefer agreement but accept other forms as well)

Considering the right solution to your integration problem

So often we often get tied up in a specific subject or technology area it feels like we see this is the everything it is just to help people think different.
only game in town.  It reminds me of the Jim Carrey film "Yes man" (by the way, I relate all my work to Jim Carrey films -- please no "Dumb and Dumber "references ;-) where he takes part in a program to change his life by answering "Yes" to any question or opportunity.  He learns, it doesn't apply to all situations but just trying to force people to see a different perspective by opening up to a new way of approaching situations.

I relate my work in OSLC and Linked Data work in a similar way.  It is not the only solution but thinking this way, apart from some traditional ways of thinking about integrations, helps to find alternative solutions that could end up being more loosely-coupled, scale better and are more resilient to upgrades.  Other benefits are that it allows the data to stay where it can be properly protected via access-control and updated as needed.

Often people and companies are so passionate about a technology or solution, they often answer the customer's question before they ever fully hear what the problem is.  There are varying degrees of this, of course if your job is to sell a product then that is then it is hard to imagine that you'd recommend an alternative solution.  Though I think if you worked with the customer to determine it isn't the right fit or the trade-offs, they would be more willing to continue to attempt to business with you.

There are so many factors on deciding what is the right integration solution for your current integration problem.  It would be fantastic if I could clearly define a concise decision tree that fit on a cocktail napkin and handled 90% of the cases...unfortunately, it is not that easy (and a group of integration consultants might hunt me down).  I've worked with a number of customers to identify possible solutions to either simple 1-to-1 integration problems to defining a corporate integration architecture/approach and plan.

Here's some factors that I typically consider to drive a recommendation:

• problem statement & user stories, including any other constraints
• # of tools
• anticipated growth or consolidation
• integration technology already available in tools landscape
• ability to develop integrations
• timeframe and ownership

As I come up with more, I will add to the list but I'd be interested to hear what other considerations people have when tackling integration problems.  I'd like to elaborate more on each of these points and weight them compared to each other as well.

Creating Containers in LDP, it is just this easy

I've heard a number of times that it isn't very clear how one creates a Linked Data Platform (LDP) Container.

Let's start with an example of how to create a simple resource (say LDP RDF Source).  You need to know a URL that accepts POST of the media type you want.
Here's the POST request:

1:  POST /netWorth/nw1/liabilities/ HTTP/1.1  
2:  Host: example.org  
3:  Accept: text/turtle  
4:  Content-Type: text/turtle  
5:  Content-Length: 63  
6:    
7:  @prefix dcterms: <http://purl.org/dc/terms/>.
8:  @prefix o: <http://example.org/ontology>.
9:
10:  <> a <http://example.org/ontology#Liability>;
11:       dcterms:title "Home loan";
12:       o:value 200000.00 .
13:    # plus any other properties that the domain says liabilities have  

(Example "borrowed" from LDP spec example)

s Very simple, just POST some Turtle to a URL.

Now let's look at what it would look like to create a Container:

1:  POST /netWorth/nw1/liabilities/ HTTP/1.1  
2:  Host: example.org  
3:  Accept: text/turtle  
4:  Content-Type: text/turtle  
5:  Content-Length: 72  
6:  Link: <http://www.w3.org/ns/ldp#BasicContainer>; rel="type"  
7:  
8:  @prefix dcterms: <http://purl.org/dc/terms/>.
9:  @prefix o: <http://example.org/ontology>.  
10:
11:  <> a <http://www.w3.org/ns/ldp#BasicContainer> ;
12:       dcterms:title "Home loans" ;
13:       o:limit 500000.00 .   
14:    # plus any other properties  

That's it, just POSTing some content just like before. I added a Link header (line #6) on the POST request to be explicitly clear that I want this newly created resource to behave as an ldp:BasicContainer. Note that I wanted to be expressive so I included the type triple on line 11, though the spec doesn't require this as the Link header as the spec-defined way to clearly assign the desired behavior.

There are various other ways that ldp:Containers can come into existence, most of those reasons are dependent on the server's application-specific rules.  For example, an application might allow the creation of a resource of say "Customer", the application logic may determine to create a number of supporting Containers for the new Customer resource, such as which assets and liabilities they have.

From zero to LDP: a tale of our 5 day journey/sprint to create an LDP impl from scratch in Node.js

Goals:

Highest priority, produce a Node.js® based W3C Linked Data Platform (LDP) implementation.  I will discuss some of the goals later (and in subsequent posts).

Result:
Had a BasicContainer and RDFSource implementation at the end of the week, passing all the automated test cases from the test suite.  Guessing we spent < 40 hours total on the work.  40 sounds like a lot, though we had a bit of learning curve in areas.  And it is 'live' at: http://ldpjs.mybluemix.net/ (assuming our autodeploys continue to be green).

Some background on us that did the development, well just Sam Padgett and Steve Speicher.  To be fair, we know a fair amount about the LDP spec already and have already done a reference implementation in Java in Eclipse Lyo.  Neither of us have done anything in Node.js other than a “Hello Node” sample.  Sam on the other hand, is an experienced JavaScript developer and Steve has stumbled his way through a few applications.

Day 1, we started the effort on Monday July 21.  We had sparkle in our eyes and two thermos full of coffee.  Then we were in meetings for a while and caught up on LDP test suite items.  Well, we did do some initial project structure setup to share the code and auto deploys to Bluemix (more on that later), read a little bit to determine a general direction we were ready to head.  After day 1, we had a place to store code, a sample app checked in, using express.js and starting to play with rdfstore.js API.

Day 2, day started off with the promise of all day code slinging.  We only got about 1/2 a day in, due to other OSLC and Lyo normal operations (WG meetings, code reviews, …).  We may a good dent in a simple GET/PUT working with rdfstore.js.  Though we were struggling to make a few things to work with rdfstore.js and Steve’s newbie callback hell was not improving his drinking problems (he says he doesn’t have a problem, he has it figured out).

Day 3, again about 1/2 a day of hacking…some progress with some of the BasicContainer features and various formats supported.

Day 4, realization that we should reconsider using rdfstore.js to store and parse RDF data.  The needs we have with LDP are quite simple.  We looked at mongoDB model and what we were doing, and looked at a simple format for JSON we dealt with using N3 library.  It was fairly straightforward to do, greatly simplified our dependencies and removed a couple barriers we were hitting with rdfstore.js.  We ended taking the graph-centric approach, where each JSON document in mongo is an RDF graph.  This approach, and drawbacks, is outlined over in dotNetRdf's blog post.

Day 5, complete transition to mongoDB, handle all GET/HEAD/OPTIONS/POST/DELETE, got viz service working, all tests green (well Sam did all/most of the hard LDP work, Steve was the “idea man” and “bounce ideas off man”).

Days 6-10 we were able to add JSON-LD support and full support for all variations of ldp:DirectContainers.
What next?  We’d like to solidify the impl more, a little more testing, and do non-RDFSource.  We’ve talked about building a sample app as well, something like a poor-mans bug tracker, photo album or address book.  Oh yes, and making the source code available.  That wasn't an initial priority just we didn't know how much would be worth sharing, we'll be going through approval processes to make it available.

Be on the look out for an upcoming blog post that talks about our experiences with DevOps Services and Bluemix.

Supporting Accept-Post in JAX-RS applications

Recently in the W3C Linked Data Platform working group we did a survey of the various discovery (or also referred to as affordances) there are for various methods or actions a client may want to introspect or learn from error responses.  One specific scenario is the case of which resource formats (content-types) are accepted by a server when a client wants to send the representation of a resource using POST with the intent of giving birth to a new resource.  The current approaches rely on trial-and-error or some out-of-band knowledge the client application has.  The trial-and-error approach relies on the client sending content of the content-type it believes the server accepts, if the server does accept it and successfully processes the request, it will send back a friendly 201 (maybe 202 or other 200-range) status code.  If the server doesn't like the content-type the client is giving it, it can kindly reply with a 415 (Unsupported Media Type).  Well the client knows what doesn't work but has to guess what might.  Let me introduce you to Accept-Post which is being proposed as a way for a server to tell a client what content-types it prefers.  Accept-Post is somewhat like the Accept header but more closely matches the newer (and less supported) Accept-Patch header.

Ok, that is a enough about the motivation and usage.  I thought I'd share the few lines of Java code needed to support this in JAX-RS 2.0 based implementations.  Since I want the HTTP response header Accept-Post to be returned for a variety of use scenarios such as: when JAX-RS returns a 415, on OPTIONS and HEAD requests, and so on, I decided to always return the header.  To do this, I implemented the ContainerResponseFilter with a simple Class and filter() method as:

import javax.ws.rs.container.ContainerRequestContext;
import javax.ws.rs.container.ContainerResponseContext;
import javax.ws.rs.container.ContainerResponseFilter;
import javax.ws.rs.ext.Provider;

public class AcceptPostResponseFilter 
       implements ContainerResponseFilter {
   @Override
   public void filter(ContainerRequestContext requestContext,
                      ContainerResponseContext responseContext) 
                      throws IOException {
      responseContext.getHeaders().putSingle(
         "Accept-Post", 
         "text/turtle", "application/ld+json", "image/png");
   }
}

That is about it, except of course you needs to register this filter with your JAX-RS Application, such as:

import java.util.HashSet;
import java.util.Set;
import javax.ws.rs.core.Application;

public class MyApplication extends Application {
   @Override
   public Set<Class<?>> getClasses() {
      Set<Class<?>> classes = new HashSet<Class<?>>();
      classes.add(AcceptPostResponseFilter.class);
      return classes;
   }
}

I've made this change for the in-progress LDP reference implementation occurring at Eclipse Lyo.

Similar approaches to other web server implementations or configurations make implementing Accept-Post quite trivial as well. Feel free to provide feedback on the IETF working draft for Accept-Post as well.

OSLC's motivation on the design of the LDP Container

It is often that when some concepts in specifications exist, it is hard to tie together the learning from past experiences and what had been considered prior.  Sometimes preserving this and providing the background is too much filler for some already long documents, so I will use this opportunity to pull together a series of activities.  Some of these activities can be found in OSLC 1.0 specs and elsewhere.

All these approaches were motivated with a set of requirements which are quite similar:

• Common patterns for dealing with multi-valued properties
• These multi-valued properties often become large
• Easy way to get all the properties
• Easy way to add new property values

Below are a sample of approaches

OSLC CM 1.0 - @collref

In CM 1.0, when these patterns first evolved and we were working on applying REST we came up with a concept of @collref which was an XML attribute that held the value of the URL of the collection where you could POST to add new members and GET on to get a list of all members.  This approach was developed based on some legacy XML and pragmatic need to solve a narrow problem.  It would for a good number of cases but was a bit special and non-standard.

OSLC CM 2.0 - multi-value props

In CM 2.0 and Core 2.0, we took a more RDF-view of REST and simply treated these as multi-valued properties or in RDF terminology they were same-subject and same-predicate triples.  They were just like any other property on a HTTP (OSLC) resource, using GET to fetch the representation of the state of the resource, POST to create it and PUT to update.  This approach worked well in that it didn't require any new invention but was limited by what existed, it would be good if we could layer on semantics without losing the simple usage of RDF to define the resources.

OSLC CM 3.0 - multi-value props + LDP Container

In drafts of CM 3.0 and Core 3.0, we look to build off a concept called Linked Data Platform Container defined by Linked Data Platform 1.0.  This isn't far from what we've done in 2.0 except it is based on a W3C specification and has some additional semantics, namely layering on the ldp:Container semantics.

Other motivation

Another key scenario motivating similar requires is dealing with things such as query results or filtering long lists of resources and where to POST to create new ones.  In Core 1.0 and 2.0, we used concepts such as query capability and creation factory to meet these needs.  As we see with ldp:Container, this provides that same function while we can annotation it to provide some of the additional needs for knowing what kinds of resources a server will accept in a container it hosts and things like Web UI dialogs that can be used to select and create resources.

Applying LDP concepts to bug tracker

In this post I'll explore how some products that support bug tracking, such as Rational Team Concert, model their internal "Work Item" resource given W3C Linked Data Platform and OSLC concepts.  This will also highlight how various capabilities are needed to solve some key problems in a simple way.  I will be "paraphrasing" what the actual runtime resource will look like to help with human consumption of the examples (as the real-world ones can sometimes have additional concepts and capabilities that don't apply to the simple part of the model we are focused on here).

Let's take one of the most simple examples of what one might find, let's say a bug (aka "Defect") that has a few properties but no associated attachments or child bugs.

Representation of bug http://example.org/bugs/13:

@prefix dc:  <http://purl.org/dc/terms/> .
@prefix ocm: <http://open-services.net/ns/cm#> . 

<> a ocm:Defect ;
   dc:title "Bug in rendering large jpgs" ;
   dc:identifier "13" .

Now we'll explore how add a couple screen shots to the bug. Using this information I have with this resource, I'm not sure how I do that (assuming I am a Linked Data client). I could just attempt to PUT replace the contents and add a ocm:attachment statement referring to the screenshot. Depending on the server, it may reject it for a couple of reasons, such as: it doesn't known anything about ocm:attachment, or it has restrictions on the object (where the attachment is physically stored), or simply PUT updates not allowed for ocm:attachment.
To help with this problem, we can associate an ldp:Container with this bug resource to assist with this. So we expand our example, with out modifying any of the original model, just adding a few more statements.

@prefix dc:  <http://purl.org/dc/terms/> .
@prefix ldp: <http://w3.org/ns/ldp#> .
@prefix ocm: <http://open-services.net/ns/cm#> . 

<> a ocm:Defect ;
   dc:title "Bug in rendering large jpgs" ;
   dc:identifier "13" .

<attachments> a ldp:Container ;
   ldp:membershipPredicate ocm:attachment ;
   ldp:membershipSubject <>.

This tells my client now that we have an ldp:Container associated with the bug, since the ldp:membershipSubject connects this container to the bug. I can inspect also ldp:membershipPredicate to know for which same-subject and same-predicate pairing I can use this container to assist with managing and navigating them.
Now I have a URL http://examples.org/bugs/13/attachments where I can POST a screenshot to create an attachment and associate it with bug 13. Let's look at what the POST looks like:

Request:

POST http://example.com/bugs/13/attachments
Slug: screenshot1.png
Content-Type: image/png
Content-Length: 18124

[binary content]

Response:

HTTP/1.1 201 CREATED
Location: http://example.com/bugs/13/attachments/3
Content-Length: 0

Now that the attachment has been created, we can fetch bug 13 again to see what we have.

@prefix dc:  <http://purl.org/dc/terms/> .
@prefix ldp: <http://w3.org/ns/ldp#> .
@prefix ocm: <http://open-services.net/ns/cm#> . 

<> a ocm:Defect ;
   dc:title "Bug in rendering large jpgs" ;
   dc:identifier "13" ;
   ocm:attachment <attachments/3> .

<attachments> a ldp:Container ;
   ldp:membershipPredicate ocm:attachment ;
   ldp:membershipSubject <>.

We now see that there is an ocm:attachment statement associated with bug 13. This statement was added by the server when it processed the POST request to create a new resource (attachment) and then added it as a member of the attachments associated with the bug.
We can also see that this list can grow to be quite large. Experience has shown, that end users of bug trackers need to attach a number of documents, images, logs, etc. with bugs. This need also comes from a number of other capabilities such as having nested bugs or tasks. To illustrate, let's assume our bug tracking server has been upgrade or now exposes children resource within bug resources (or has had children added by other means). Let's take a look at bug 13 again:

@prefix dc:  <http://purl.org/dc/terms/> .
@prefix ldp: <http://w3.org/ns/ldp#> .
@prefix ocm: <http://open-services.net/ns/cm#> . 

<> a ocm:Defect ;
   dc:title "Bug in rendering large jpgs" ;
   dc:identifier "13" ;
   ocm:attachment <attachments/3>, <attachments/14> .
   ocm:child <../47>, <../2> .

<attachments> a ldp:Container ;
   dc:title "Attachment container for bug 13" ;
   ldp:membershipPredicate ocm:attachment ;
   ldp:membershipSubject <>.

<children> a ldp:Container ;
   dc:title "Children for bug 13" ;
   ldp:membershipPredicate ocm:child ;
   ldp:membershipSubject <>.

As you can see, the bug model stays very simple with statements about bug 13 being made directly about it using simple RDF concepts where the statements are of the form [bug13, predicate, object|literal]. We can repeat this pattern and use it in many other forms, such as a container of all the bugs the serve knows about, which I plan to illustrate in other posts.
Disclaimer: some of the predicates used in these examples show they are in the OSLC-CM namespace but have not been approved by the WG. This was done just for simple illustration.

Making progress towards an OSLC-CM 3.0

Been a while I have talked about a specific domain working group and what has been going on.  Though the work for a post-2.0 Change Management specification has been evolving for some time, it is coming along nicely.  I've been focusing much of my energy on getting the core-est part of Core (W3C Linked Data Platform and Core 3.0) rolling, while keeping an eye and hand involved with CM.

To summarize the OSLC-CM WG has looked at a number of prioritized scenarios:

• IDE Task federation and management
• Transitioning states
• Programmatically accessing and uploading attachments
• Querying across multiple states in a single request
• Additional semantics on regards to priority, severity and additional types of resources

We currently have draft specification text for most of the scenarios above.  We are planning to enter into convergence soon and start to do some implementation to validate the specifications.

Special thanks to Sam Padgett for leading the OSLC-CM efforts, planning meetings, keeping minutes, keeping track of issues and pushing forward with the draft specification.  Looking forward to getting some of these approaches implemented and specification wrapped up.

W3C Linked Data Platform WG 2nd Face to Face meeting March 13-15 Boston

I just returned from the 2nd face to face meeting for the W3C Linked Data Platform (LDP) working group located at MIT in Cambridge, MA. There were about 14 attendees in all, a good number but perhaps low for a WG that has 50 total registered. Being the 2nd face to face meeting of a relatively new working group, meant this was the first meeting where we could dig in our heals and start tackling issues. We reviewed our deliveries: a must deliverable of the spec and a number of supporting documents: use case and requirements, access control guidance, possible primer, deployment guide (need a better name) and test suite.

We prioritized the issues we wanted to discuss early on the first day to make sure we got all the "big hitter" issues out on the table. I added to this list some of the key OSLC items such as binary resources and metadata, patch and more. One thing we learned as we got into the next day, there still was some confusion on the model. I surfaced ISSUE-59 to hopefully help simplify the model.

Here are the detailed meeting minutes from: day 1, day 2 and day 3.

Some of the key issues discussed were:

ISSUE-15 Sharing binary resources and metadata
The WG came to the following resolution after much discussion about concerns of different behavior of POST'ing RDF formats which create LDP-Rs and which response location headers should used.

Resolution: Assuming the existing qualifications that POST is optional, and supporting "binary" media types is optional: The expected response to a post for create that creates a non-LDPR is a 201, with location header whose URI I identifies the resource whose representation matches the POST input, and the response SHOULD include a Link header rel="meta" href= another URI P identifying the resource whose state is the server-managed properties. The URIs I and P MAY be distinct, but this is not required. When the object of a membership triple (I) is DELETEd, the server MUST automatically deletes any related resource P that it created previously

ISSUE-17 Recommended PATCH document (and 27 & 45) 

There were a number of PATCH related issues, like 27 whether POST could be used to tunnel PATCH but WG decided just to use PATCH. Also there is 45, which suggested the need to use POST'ing of RDF formats to LDP-Rs to append/add to existing resources. The WG decided that this would be fairly different than what is done with POSTing to LDP-Cs which creates a new resource and adds membership triples, plus we have a solution to append by using PATCH-insert. For 17, the original proposal was to use the Talis changeset format but due to a variety of reason this was withdrawn (don't recall the problems with Talis changeset). Instead the WG decided to pursue its own simple PATCH document that leveraged existing quad formats such as TriG. The WG has an action to define the minimal and work with issues and proposals to expand on it.

Other issues we discussed were on our list to watch as of course the resolution could impact us but the issues listed were not identified by OSLC community members as a high priority.  Overall it was very encouraging to see the WG make progress on such key issues.  It is also a little disappointing we didn't make more progress in other areas.  It is most likely that we won't enter into "Last Call" until after our 3rd face to face which is being planned for mid-June.

W3C Linked Data Platform WG -- 1st Face to Face Meeting

I recently attended the 2012 W3C Technical Plenary / Advisory Committee (TPAC) in Lyon, France from October 29 to November 2.

RDF WG: Monday and Tuesday 

I attended the RDF F2F Working Group face to face meeting as an observer since I'm not an member of that WG though of course interested in a number of topics.  The RDF working group is nearing the end of its charter (Jan 2013) and making good progress on a number of its deliverables such as Turtle, JSON, TriG, n-triple, n-quad serialization formats.  There are a number of key issues that are of great interest such as ISSUE-77 marking rdf:Seq as archaic (this if particular interest to work at OSLC). There was a breakthrough in direction with named graphs by resolving to not define detailed semantics on identification of graphs, instead just allowing simply names/labels for graphs.

RDF WG Minutes: Day 1 and Day 2

TPAC Plenary Day: Wednesday

Jeff Jaffe CEO of W3C kicked off the day with a brief keynote and then logistics for breakout sessions.

I attended these sessions:

• Social Web: lots of interest in linked data and social web.
• Government Linked Data: discussion and update on work going on with existing WG, there is need for some guidelines and have need for LDP work
• Linked Data for Web Developers: a fair amount of discussion around JSON, including a "white-boarding" session by Tim Berners-Lee talking about some approaches using JSON.  There was also discussion of a need for some schema-like need for RDF data, which I explained we have similar need and have some experience with OSLC Resource Shapes.

Linked Data Platform (LDP): Thursday and Friday 

Of course this was the primary reason I was here.  Agenda covered a range of topics as it was our first face to face meeting. Discussed Use Case and Requirements, Primer, Test Suite, Implementations, next F2F and of course spec issues.  Tim Berners-Lee attended both days and was very active in discussing the issues, he had strong views on how some of LDP should be shaped.

Probably the biggest change is the resolution around ISSUE-25 where the ldp:Container follows strong composition model instead of weaker aggregation.  There was discussion for a common set of needs around paging and ordering of contents.

LDP WG Minutes: Day 1 and Day 2

Overall Impressions

It is clear there are many W3C members that are very passionate and believers of Linked Data, including Tim Berners-Lee.  Discussions with many shown they are interested in keeping LDP simple and willing to jump in and add support to their products early.

Trip Report from JavaOne 2012

Trip Report: JavaOne 2012 October 1-4, San Francisco 

I attended my first JavaOne conference, where I presented and attended a number of sessions and events.

Leveraging W3C Linked Data for Loosely Coupled Application Integrations

Tuesday Oct 2, 8:30

Track: Enterprise Service Architectures and the Cloud

There were about 90 people who preregistered for the session (see attachment) but seemed like there was a bit less than that at the actual session (50). Since they don’t serve the free coffee until after the 8:30 sessions, I felt like the attendees seemed very engaged by their responses and didn’t see anyone sleeping. The best was at the end where I had about 10 people approach me. Someone stated he was from a major news media company who asked how they could get involved in W3C Linked Data Platform (LDP) work and I highlighted some work at BBC that he might be interested in as well. Also a development manager from automative sector, asked how they could get involved as he was very interested in requirements traceability. Good news was that I could connect him with what other efforts in automative that were already occurring. There were a few others from healthcare and life sciences sector that said they were interested as well, handing out my card to a number of folks.

Linking the Java Application Lifecycle with Hudson

Wednesday Oct 3 4:30

Track: Development Tools

I was not on the original agenda to speak here but Mik Kersten CEO of Tasktop asked that I join him to explain Linked Data and OSLC, and give an overview of current activities. This session was also co-presented by Gregory Stachnick, Oracle Senior Director, Product Management. Greg is responsible for Oracle’s Eclipse-based tools strategy. Good overall flow considering some late changes, story around linking the lifecycle and need for standard integrations was good.

Some other activities:

Tuesday night Eclipse community meetup, good to talk with some Eclipse community members and talk about Lyo futures.

A number of sessions on JAX-RS, REST, WebSockets, Cloud, NoSQL, Hudson, CI, e4, ...

It was interesting to see the overall interest in REST, WebSockets, NoSQL and big data.

Overall very useful trip, look forward to trying to get back in 2013.

W3C Launches Linked Data Platform (LDP) Working Group

Following on the W3C Submission Linked Data Basic Profile 1.0 and the Linked Enterprise Data Workshop around needs for standardization around Linked Data, the W3C has created the Linked Data Platform Working Group to look into this problem and produce a specification (W3C Recommendation) to address the stated items in the charter.

This is great news and follows on what we have been leveraging and learning when developing OSLC specifications built off of Linked Data as an architectural basis.  Many of the things you see in the charter and use cases the working group will face, have been a result of work by the OSLC community and community members such as IBM Rational.  By taking this effort to W3C this enables a broader set of applications and data to interoperate.  How does it do this?  By prescribing a single, simple, consistent way for applications to support Linked Data this means that more we can expect to see more and more compliant servers that are exposing their data.  This has a great quality that it makes it easier for client implementers being able to wring a client once and use across multiple server vendors -- this is a standard reason that we have done standards and continue to do standards.  There is much value in this alone, no doubt.  We gain additional value by building applications that pull data from a variety of servers and sources, leveraging the relationships (links) between the data and the meaning behind these links using standard and common vocabularies.  Reading the data is all good, though we need to a way to create and update the data.  We additionally need to a have a simple way to have these clients be able to create and update this data, it improves the quality of the data from the source and all benefit from a social and distributed set of clients/applications that operate over this data.

As the charter points out "The combination of RDF and RESTful APIs is therefore natural".  Why is this natural?  Since we operate on resources on the Web: get them, create them, update them, remove them, we need a way to identify these things, which URIs provide a nice mechanism for that, and access them over a network, which HTTP URIs provide a nice mechanism for that.  Since these resources on the Web are really not on the Web, for example a URI that produces a representations for a toaster isn't the toaster but a Web resource representing some information about it: it is a useful to have a model (RDF) for representing the state of these things.  Even though these concepts seem simple, there are many variations that exist on interpretation of existing specifications, best practices when multiple options exist and so on.  The Linked Data Platform WG looks to address a number of questions such as:

What is the current set of RDF datatypes that must be supported?

Currently it is left open to implementations to select a set, hopefully from a common and compatible set.

Which RDF formats must be supported?

RDF is a "resource description framework" and not a format, it has multiple formats that support it: RDF/XML, Turtle, N-3, JSON-LD, etc.  It would be nice for clients to be able to rely on a minimal subset of representations.

Which vocabulary terms are recommended?

When pulling together resource representations from separate servers, you gain value in analyzing these two resources by seeing which links they have in common and other interesting relationship.  Though, if both the resource representations used common, well-understood and standardized vocabulary terms to describe their data and links, then applications could correlate meaning from these common term usages without the need for extensive mappings applied.

What URL should be used to create new resources?  What should I receive back from that URL?

A common best practice is to enable new resource creation by POSTing a representation on a given say container URL.  A subsequent GET of that container URL would find that new resource any any previous resources that were created and still exist in that container.  This same model could apply to deleting the previously created resource, by issuing a DELETE on the resource URL it would both delete the resource and remove it from the container.

How can a client ensure it can create and update resources that have constraints applied by the server?

There are a number of existing applications and cases for future applications, where constraints may need to be made on the validity of the resource representation prior to the server creating a resource or allowing a new updated revision to be created.  It would helpful to have a representation of these constraints that a client could easily fetch and apply to its representation prior to submitting it to a server.

These are just a sample of the issues that the charter highlights and the working group looks to tackle.  The member submission for a Linked Data Basic Profile address a subset of these issues and a future derivative or version will likely be introduced to handle these cases.  Though in the spirit of making progress and getting value out of something in the near future, it would seem logical that the working group would focus on a subset in its version 1.0 specification.

The working group charter outlines a timetable that is an initial guess on when a final "Recommendation" is available, which is looking like early 2014 with a first draft for review later in 2012.  I highly encourage  everyone to get involved: providing use cases and requirements, reviewing material, providing implementation feedback, writing specification...the more review and collaboration we have on the better changes for success we'll have.  This has proven true in OSLC and W3C.