Showing posts with label linked_data. Show all posts
Showing posts with label linked_data. Show all posts

Friday, November 7, 2014

Considering the right solution to your integration problem

So often we often get tied up in a specific subject or technology area it feels like we see this is the everything it is just to help people think different.
only game in town.  It reminds me of the Jim Carrey film "Yes man" (by the way, I relate all my work to Jim Carrey films -- please no "Dumb and Dumber "references ;-) where he takes part in a program to change his life by answering "Yes" to any question or opportunity.  He learns, it doesn't apply to all situations but just trying to force people to see a different perspective by opening up to a new way of approaching situations.

I relate my work in OSLC and Linked Data work in a similar way.  It is not the only solution but thinking this way, apart from some traditional ways of thinking about integrations, helps to find alternative solutions that could end up being more loosely-coupled, scale better and are more resilient to upgrades.  Other benefits are that it allows the data to stay where it can be properly protected via access-control and updated as needed.

Often people and companies are so passionate about a technology or solution, they often answer the customer's question before they ever fully hear what the problem is.  There are varying degrees of this, of course if your job is to sell a product then that is then it is hard to imagine that you'd recommend an alternative solution.  Though I think if you worked with the customer to determine it isn't the right fit or the trade-offs, they would be more willing to continue to attempt to business with you.

There are so many factors on deciding what is the right integration solution for your current integration problem.  It would be fantastic if I could clearly define a concise decision tree that fit on a cocktail napkin and handled 90% of the cases...unfortunately, it is not that easy (and a group of integration consultants might hunt me down).  I've worked with a number of customers to identify possible solutions to either simple 1-to-1 integration problems to defining a corporate integration architecture/approach and plan.

Here's some factors that I typically consider to drive a recommendation:

  • problem statement & user stories, including any other constraints
  • # of tools
  • anticipated growth or consolidation
  • integration technology already available in tools landscape
  • ability to develop integrations
  • timeframe and ownership
As I come up with more, I will add to the list but I'd be interested to hear what other considerations people have when tackling integration problems.  I'd like to elaborate more on each of these points and weight them compared to each other as well.


Thursday, April 11, 2013

Trip Report from EclipseCon 2013

I had a chance, along with Michael Fiedler,  to attend the US EclipseCon 2013 in Boston, MA in late March.   In addition to presenting a BOF and a session on OSLC, Linked Data and Lyo, we were able to attend several sessions, panel discussions and BOFs.   It was also a great chance to meet with members of the OSLC and Lyo communities to discuss the challenges of tool integration.

General Impressions from the Conference

It was cool to see and hear OSLC and Eclipse Lyo in a variety of sessions.  There were good active discussions with the Eclipse Mylyn team regarding their recent m4 proposal, including how OSLC and Lyo play a key part.  It was good to explore ways we could work together with Mylyn, Orion, Hudson and more.  We did do a lot of education and exploration with a number of attendees that had no or limited knowledge of what was going on within OSLC and Lyo.  Various panels, keynotes and sessions often called out the need for tool vendors to be able to collaborate on open interfaces....we asked them: "have you heard of OSLC?"  There were a number of people making the case for OSLC-based integrations very easy.
Here is a summary of the two sessions we presented at EclipseCon:

1) Lifecycle Tool Integrations: Linked Data, OSLC and Eclipse Lyo
Session type: Birds of a Feather

The participants ranged from experienced OSLC implementers interested in contributing to Eclipse Lyo to those new to OSLC looking for learning resources. 
The discussion covered the re-vamped tutorial on open-services.net, using the Lyo OSLC4J Bugzilla adapter as a learning resource and some general OSLC and linked data integration philosophies.  Topics included:
  • a description of an OSLC4J implementation in progress which exposes EMF Models as OSLC resources - being considered for contribution to Lyo
  • the experiences of a developer who has implemented several OSLC integrations to enable tools to participate in an ALM system
  • The OSLC CM 1.0 integrations developed for the Mantis bug tracker and FusionForge
  • Status of Eclipse Lyo - what is new in 1.1 and what is coming
  • One OSLC implementer strongly recommended developing integrations using 2.0 of OSLC.  He also pointed out authentication is a primary painpoint for the work he has done.


2) Leveraging W3C Linked Data, OSLC, and Open Source for Loosely Coupled Application Integrations
Session type: ALM Connect track

This session was a 35 minute tour through tool integration, problems with previous approaches and new approaches using linked data and OSLC.
This was followed by an overview of open source projects relevant to this space (Jena, Wink, Clerezza, Lyo etc) and a brief demo of some potential integrations between Eclipse Orion and a Change Management tool (Bugzilla) and an Automation tool (Lyo automation reference implementation).   
There was very little time left for questions at the end, but there were some good ones which pushed the session into overtime: 
  • Interest in the concept of delegated UIs and the responsibilities of the tool hosting them - described the OSLC concepts in some detail
  • Interest in the concept of UI previews and compact representations.   How these resources can be used to link to full representations was described.
  • Some clarifying questions were raised by a few folks about the details of the Bugzilla integration demonstrated.   Explained it is the live Eclipse Bugzilla instance, but that it could have been any OSLC CM provider.
Special thanks to Michael Fiedler for both authoring some of this content of this blog post, as well as building the demo, presenting and too many things to list here.

Wednesday, March 20, 2013

W3C Linked Data Platform WG 2nd Face to Face meeting March 13-15 Boston

I just returned from the 2nd face to face meeting for the W3C Linked Data Platform (LDP) working group located at MIT in Cambridge, MA. There were about 14 attendees in all, a good number but perhaps low for a WG that has 50 total registered. Being the 2nd face to face meeting of a relatively new working group, meant this was the first meeting where we could dig in our heals and start tackling issues. We reviewed our deliveries: a must deliverable of the spec and a number of supporting documents: use case and requirements, access control guidance, possible primer, deployment guide (need a better name) and test suite.

We prioritized the issues we wanted to discuss early on the first day to make sure we got all the "big hitter" issues out on the table. I added to this list some of the key OSLC items such as binary resources and metadata, patch and more. One thing we learned as we got into the next day, there still was some confusion on the model. I surfaced ISSUE-59 to hopefully help simplify the model.

Here are the detailed meeting minutes from: day 1, day 2 and day 3.

Some of the key issues discussed were:

ISSUE-15 Sharing binary resources and metadata
The WG came to the following resolution after much discussion about concerns of different behavior of POST'ing RDF formats which create LDP-Rs and which response location headers should used.

Resolution: Assuming the existing qualifications that POST is optional, and supporting "binary" media types is optional: The expected response to a post for create that creates a non-LDPR is a 201, with location header whose URI I identifies the resource whose representation matches the POST input, and the response SHOULD include a Link header rel="meta" href= another URI P identifying the resource whose state is the server-managed properties. The URIs I and P MAY be distinct, but this is not required. When the object of a membership triple (I) is DELETEd, the server MUST automatically deletes any related resource P that it created previously

ISSUE-17 Recommended PATCH document (and 27 & 45) 
There were a number of PATCH related issues, like 27 whether POST could be used to tunnel PATCH but WG decided just to use PATCH. Also there is 45, which suggested the need to use POST'ing of RDF formats to LDP-Rs to append/add to existing resources. The WG decided that this would be fairly different than what is done with POSTing to LDP-Cs which creates a new resource and adds membership triples, plus we have a solution to append by using PATCH-insert. For 17, the original proposal was to use the Talis changeset format but due to a variety of reason this was withdrawn (don't recall the problems with Talis changeset). Instead the WG decided to pursue its own simple PATCH document that leveraged existing quad formats such as TriG. The WG has an action to define the minimal and work with issues and proposals to expand on it.

Other issues we discussed were on our list to watch as of course the resolution could impact us but the issues listed were not identified by OSLC community members as a high priority.  Overall it was very encouraging to see the WG make progress on such key issues.  It is also a little disappointing we didn't make more progress in other areas.  It is most likely that we won't enter into "Last Call" until after our 3rd face to face which is being planned for mid-June.

Sunday, March 3, 2013

OSLC's relationship with Semantic Web technologies

The technical basis for accessing data through OSLC has its roots in Linked Data, which has its origins with Semantic Web technologies.  Some worry about the costs of supporting this for some simple integration scenarios, though OSLC only depends on a small amount of it.  For those that bought into the full stack of Semantic Web technologies for various domain solutions realize there is tremendous value in what it can provide but that it also comes at some cost.  The cost is in aspects of Semantic Web such as reasoners, inferencing, search engines, RDF and specialized repositories for dealing with these things.

OSLC takes a minimal incremental approach to depend on only what is needed to satisfy the integration scenarios.  So far that has led us to leverage a simple standard way of describing resources using RDF.  That is about where the Semantic Web technology dependency ends.  We leverage a few terms out of RDF Schema to help with defining our own vocabulary terms but do not go beyond that as it might imply that clients would need to process inference rules against the resource representations they receive to learn more.

Since a primary goal of OSLC is to not reinvent but leverage standards based approaches that meet our requirements, I can see cases where it might be good to adopt some more Semantic Web technologies.  Though to be clear, for tools to get value out of OSLC-based integrations, only some RDF syntax readers and writers are needed.  There is no need for tools to have be rehost or rewrite them onto a new technology base, they can simple adapt their solution with a simple fasade or update their existing REST APIs to provide this support.

Monday, November 5, 2012

W3C Linked Data Platform WG -- 1st Face to Face Meeting

I recently attended the 2012 W3C Technical Plenary / Advisory Committee (TPAC) in Lyon, France from October 29 to November 2.

RDF WG: Monday and Tuesday 
I attended the RDF F2F Working Group face to face meeting as an observer since I'm not an member of that WG though of course interested in a number of topics.  The RDF working group is nearing the end of its charter (Jan 2013) and making good progress on a number of its deliverables such as Turtle, JSON, TriG, n-triple, n-quad serialization formats.  There are a number of key issues that are of great interest such as ISSUE-77 marking rdf:Seq as archaic (this if particular interest to work at OSLC). There was a breakthrough in direction with named graphs by resolving to not define detailed semantics on identification of graphs, instead just allowing simply names/labels for graphs.
RDF WG Minutes: Day 1 and Day 2

TPAC Plenary Day: Wednesday
Jeff Jaffe CEO of W3C kicked off the day with a brief keynote and then logistics for breakout sessions.
I attended these sessions:
  • Social Web: lots of interest in linked data and social web.
  • Government Linked Data: discussion and update on work going on with existing WG, there is need for some guidelines and have need for LDP work
  • Linked Data for Web Developers: a fair amount of discussion around JSON, including a "white-boarding" session by Tim Berners-Lee talking about some approaches using JSON.  There was also discussion of a need for some schema-like need for RDF data, which I explained we have similar need and have some experience with OSLC Resource Shapes.
Linked Data Platform (LDP): Thursday and Friday 
Of course this was the primary reason I was here.  Agenda covered a range of topics as it was our first face to face meeting. Discussed Use Case and Requirements, Primer, Test Suite, Implementations, next F2F and of course spec issues.  Tim Berners-Lee attended both days and was very active in discussing the issues, he had strong views on how some of LDP should be shaped.
Probably the biggest change is the resolution around ISSUE-25 where the ldp:Container follows strong composition model instead of weaker aggregation.  There was discussion for a common set of needs around paging and ordering of contents.
LDP WG Minutes: Day 1 and Day 2

Overall Impressions
It is clear there are many W3C members that are very passionate and believers of Linked Data, including Tim Berners-Lee.  Discussions with many shown they are interested in keeping LDP simple and willing to jump in and add support to their products early.

Saturday, October 27, 2012

Trip Report from JavaOne 2012

Trip Report: JavaOne 2012 October 1-4, San Francisco 

I attended my first JavaOne conference, where I presented and attended a number of sessions and events.

Leveraging W3C Linked Data for Loosely Coupled Application Integrations

Tuesday Oct 2, 8:30
Track: Enterprise Service Architectures and the Cloud

There were about 90 people who preregistered for the session (see attachment) but seemed like there was a bit less than that at the actual session (50). Since they don’t serve the free coffee until after the 8:30 sessions, I felt like the attendees seemed very engaged by their responses and didn’t see anyone sleeping. The best was at the end where I had about 10 people approach me. Someone stated he was from a major news media company who asked how they could get involved in W3C Linked Data Platform (LDP) work and I highlighted some work at BBC that he might be interested in as well. Also a development manager from automative sector, asked how they could get involved as he was very interested in requirements traceability. Good news was that I could connect him with what other efforts in automative that were already occurring. There were a few others from healthcare and life sciences sector that said they were interested as well, handing out my card to a number of folks.

Linking the Java Application Lifecycle with Hudson

Wednesday Oct 3 4:30
Track: Development Tools

I was not on the original agenda to speak here but Mik Kersten CEO of Tasktop asked that I join him to explain Linked Data and OSLC, and give an overview of current activities. This session was also co-presented by Gregory Stachnick, Oracle Senior Director, Product Management. Greg is responsible for Oracle’s Eclipse-based tools strategy. Good overall flow considering some late changes, story around linking the lifecycle and need for standard integrations was good.

Some other activities:

Tuesday night Eclipse community meetup, good to talk with some Eclipse community members and talk about Lyo futures.
A number of sessions on JAX-RS, REST, WebSockets, Cloud, NoSQL, Hudson, CI, e4, ...
It was interesting to see the overall interest in REST, WebSockets, NoSQL and big data.

Overall very useful trip, look forward to trying to get back in 2013.

Friday, October 5, 2012

Quick Introduction to Linked Data and OSLC

I thought it would be useful to explain both Linked Data and OSLC using a simple buildup example.  Hoping to also show where Linked Data ends and OSLC picks up.  This was taken from a presentation that I did at Innovate 2012 and received better than bad feedback.

Monday, June 11, 2012

Innovate 2012: OSLC growing strong

This is my 4th IBM Rational conference where I've been a part of OSLC.  OSLC initiative was first announced at Rational Software Developer's Conference 2008 and after that I started working on OSLC specifications and implementations.  I co-presented then in 2009 with Mik Kersten of Tasktop Technologies, where we demonstrated OSLC in action.

Things have come a long way.  At Innovate 2011, there are a good number of OSLC sessions including a first even Jazz Interoperability Center powered by OSLC and other open specifications. I arrived early to help setup the renamed Integration Center which doubled in size from the previous year.  Thanks to the hard work of many, it was well organized and things went very smoothly.

I was able to meet with a number of customers who are very interested and excited about integrations using OSLC.  I was also able to attend a number of sessions where customers, partners and IBMers presented their findings, experiences and vision for integrations.  It was very rewarding to see so much OSLC content, there were about 40 sessions with OSLC content.

I ran a packed OSLC Workshop with Eclipse Lyo Project co-lead Michael Fiedler.  I get the feeling we'll be repurposing the workshop and running it many times over the year, perhaps even make it for a 3rd year in 2013?  I also co-presented with Angel Diaz, IBM VP Software Standards on "W3C Linked Data and OSLC".

It was also great to be part of the inaugural OSLC Steering Committee face to face meeting held in Orlando the day before Innovate kicked off.

There were many great conversations and events there in Orlando, the ability to put faces with names and have in person conversations is extremely valuable.  There is also a nice summary on OSLC website regarding OSLC activities at Innovate.

Thursday, May 10, 2012

W3C Launches Linked Data Platform (LDP) Working Group

Following on the W3C Submission Linked Data Basic Profile 1.0 and the Linked Enterprise Data Workshop around needs for standardization around Linked Data, the W3C has created the Linked Data Platform Working Group to look into this problem and produce a specification (W3C Recommendation) to address the stated items in the charter.

This is great news and follows on what we have been leveraging and learning when developing OSLC specifications built off of Linked Data as an architectural basis.  Many of the things you see in the charter and use cases the working group will face, have been a result of work by the OSLC community and community members such as IBM Rational.  By taking this effort to W3C this enables a broader set of applications and data to interoperate.  How does it do this?  By prescribing a single, simple, consistent way for applications to support Linked Data this means that more we can expect to see more and more compliant servers that are exposing their data.  This has a great quality that it makes it easier for client implementers being able to wring a client once and use across multiple server vendors -- this is a standard reason that we have done standards and continue to do standards.  There is much value in this alone, no doubt.  We gain additional value by building applications that pull data from a variety of servers and sources, leveraging the relationships (links) between the data and the meaning behind these links using standard and common vocabularies.  Reading the data is all good, though we need to a way to create and update the data.  We additionally need to a have a simple way to have these clients be able to create and update this data, it improves the quality of the data from the source and all benefit from a social and distributed set of clients/applications that operate over this data.

As the charter points out "The combination of RDF and RESTful APIs is therefore natural".  Why is this natural?  Since we operate on resources on the Web: get them, create them, update them, remove them, we need a way to identify these things, which URIs provide a nice mechanism for that, and access them over a network, which HTTP URIs provide a nice mechanism for that.  Since these resources on the Web are really not on the Web, for example a URI that produces a representations for a toaster isn't the toaster but a Web resource representing some information about it: it is a useful to have a model (RDF) for representing the state of these things.  Even though these concepts seem simple, there are many variations that exist on interpretation of existing specifications, best practices when multiple options exist and so on.  The Linked Data Platform WG looks to address a number of questions such as:

What is the current set of RDF datatypes that must be supported?
Currently it is left open to implementations to select a set, hopefully from a common and compatible set.
Which RDF formats must be supported?
RDF is a "resource description framework" and not a format, it has multiple formats that support it: RDF/XML, Turtle, N-3, JSON-LD, etc.  It would be nice for clients to be able to rely on a minimal subset of representations.
Which vocabulary terms are recommended?
When pulling together resource representations from separate servers, you gain value in analyzing these two resources by seeing which links they have in common and other interesting relationship.  Though, if both the resource representations used common, well-understood and standardized vocabulary terms to describe their data and links, then applications could correlate meaning from these common term usages without the need for extensive mappings applied.
What URL should be used to create new resources?  What should I receive back from that URL?
A common best practice is to enable new resource creation by POSTing a representation on a given say container URL.  A subsequent GET of that container URL would find that new resource any any previous resources that were created and still exist in that container.  This same model could apply to deleting the previously created resource, by issuing a DELETE on the resource URL it would both delete the resource and remove it from the container.
How can a client ensure it can create and update resources that have constraints applied by the server?
There are a number of existing applications and cases for future applications, where constraints may need to be made on the validity of the resource representation prior to the server creating a resource or allowing a new updated revision to be created.  It would helpful to have a representation of these constraints that a client could easily fetch and apply to its representation prior to submitting it to a server.
These are just a sample of the issues that the charter highlights and the working group looks to tackle.  The member submission for a Linked Data Basic Profile address a subset of these issues and a future derivative or version will likely be introduced to handle these cases.  Though in the spirit of making progress and getting value out of something in the near future, it would seem logical that the working group would focus on a subset in its version 1.0 specification.

The working group charter outlines a timetable that is an initial guess on when a final "Recommendation" is available, which is looking like early 2014 with a first draft for review later in 2012.  I highly encourage  everyone to get involved: providing use cases and requirements, reviewing material, providing implementation feedback, writing specification...the more review and collaboration we have on the better changes for success we'll have.  This has proven true in OSLC and W3C.

Monday, April 2, 2012

Linked Data Basic Profile - W3C Member Submission



A W3C member submission titled "Linked Data Basic Profile" has been published at http://www.w3.org/Submission/2012/02/.  This submission highlights some of the key best practices and patterns that have helped leverage Linked Data as an approach to integrating lifecycle tools and applications.  This submission contains a proposed specification, use cases and requirements, and a simple RDF schema document describing the vocabulary.  The outcome of a W3C Workshop in December 2011 on "Linked Enterprise Data Patterns" explored various topics around the usage of Linked Data approach for read-write application integration use cases and concluded with the workshop attendees recommending the formation of a W3C workgroup to develop and endorce a specification around the need for a minimal foundation to build Linked Data application from.  Be on the look out to participate in the evolution of this at W3C and the adoption of it within OSLC.  This submission represents a significant step in the adoption of Linked Data as an integration architecture and thank those who helped contribute.

Sunday, December 11, 2011

W3C Workshop on Linked Enterprise Data Patterns

I attended a W3C Workshop title "Linked Enterprise Data Patterns" on December 6&7, 2011 at MIT in Cambridge, MA. It had many interesting sessions covering a wide variety of topics. The position paper by Rational presenting by Martin Nally covered what we learned in Rational and with OSLC. Responses made it clear that we weren't alone with seeing this need. This included Tim Berners-Lee and many other attendees with years of experience deploying and building Linked Data applications.

Key result of the workshop was consensus on the need for standardization, as logged in IRC:
(1:45:06 PM) sandro: RESOLVED: We want a Working Group to produce a W3C Recommendation which defines a Linked Data Platform -- something that solves IBM Rational's use case (presented yesterday). We expect this to be an enumeration of specs which constitute linked data, with some small additional specs to cover things like pagination, if necessary.

Next steps will involve evolving the IBM Developer Works publication "Towards a Basic Profile for Linked Data" into a member submission for consideration into the W3C Recommendation track.