SOA Consortium Insights

CIO Magazine | SOA Consortium Case Study Contest: Judging Criteria

A few people have asked us how we determined the winners of the case study contest.  Rather than responding privately in email, we thought it made sense to post publicly about the process.  The judging criteria and process were devised by our Executive Suite working group, which is comprised of SOA practitioners, suppliers and me.  This is group also initiated the contest. 

To best explain the judging criteria, I'm going to start with the submission process.

The call for submissions:

"The competition is open to organizations of all sizes, including government agencies, which have successfully delivered business or mission value using a SOA approach.

The goal of the SOA Case Study Competition is to highlight business success stories and lessons learned to provide proof points and insights for other organizations considering or pursuing SOA adoption. To qualify for the competition, the SOA project must be complete with demonstrated business results.

Entries will be judged on the complexity of the business problem addressed, the ROI/Business Value achieved (Agility/Innovation/Flexibility), the level and sophistication of the cross-organizational collaboration (Business/Technical) and the usage of SOA approaches and supporting technology. In addition to one overall winner, organizations will be recognized by industry/government and project size/scope."

During the submission process, entrants were given the following guidelines:

"SOA is focused on a business/IT alignment, and the case study needs to consider how SOA addressed a particular business challenge or problem. This is not intended to be a detailed discussion of technology, but a view of how business and IT can, with SOA, drive improved business value. Answer the questions as if you were trying to convince the CEO of the value of SOA to the organization. NOTE: Questions must be answered by a SOA Project Team Member."

Entrants were asked to answer 6 questions about their projects.  For each question, we provided some guidance and a word count limit.  The questions:

1. What was the business challenge or problem addressed by the SOA project, and why was SOA selected to address this?

2. When was the project started, how large was the project and how was it funded, and how long did it take to see results?

3. What was the planned and achieved ROI/Business Value (ie, improved agility, innovation, flexibility)?

4. How was the SOA Project team organized and what types of business staff were on the team? How was cross-organization collaboration (Business/Technical) achieved? Was a Center of Excellence (CoE) created?

5. What technology or software was used in the project? What vendors were involved? Was service reuse taken into consideration? What was the most complex technical challenge encountered?

6. What were the most significant lessons learned from the SOA project?

The Judging

There were 6 contest judges, all committed and announced as the contest went live.  This prior commitment was a good thing, because the submissions received far exceeded our initial projections.  After the contest closed, the submissions were validated and sent to the judges.  Accompanying the submissions were the judging rules and a tally-keeping workbook.  During the judging period, we held check-in calls to surface any questions on the process or entries and to keep us on task.

We used the following weighted scoring system:

Weight	Question(s)	Aspect
20%	1	Business Challenge
30%	2 & 3	Project Size, ROI/Business Value
30%	4	Collaboration
20%	5	SOA Approach

This gave us our first score, the total weighted score.  Judges could award extra points for lessons learned or any "wow" factor.  The weighted score plus the extra points gave us our second score, the total score. 

We selected the winners by reviewing both the weighted score and the total score.  In other words, we allowed extra credit to augment the scoring, not distort it.

As for the scoring the individual questions...

Question 1 - Business Challenge: 1 - 5 points based on the complexity / scope / criticality of the business challenge

Question 2 - Project Size: 1 - 4 points based on the project size and time to value; highest scores went to "large, quick projects", lowest to "small, slow projects"

Question 3 - ROI/Business Value: 1 - 8 points based on the business value type and span; highest scores went to "new value creation across a wide span", lowest to "SOA readiness at a small span".  Here is the matrix we used:

Question 4 - Collaboration: 1 to 5 points based on the degree of collaboration within IT, between business & IT, with external business partners and/or with customers.   Technology suppliers did not count as external business partners.

Question 5 - SOA Approach: 1 to 5 points based on the sophistication, appropriateness and execution of the SOA approach.

Reconciliation

Astoundingly, the six winning cases were at the top of each judge's scorecard.  From there, it became a simple matter of math to identify the overall winner, Synovus Financial, and the special recognition winners -- Canada Infoway, Con-Way, Penn National, SunGard, and US DoD AT&L.

As I've mentioned previously, there were really no "losers".  We received tons of great cases, full of insights and lessons.  I'll be mining all of those for fun facts and more.  Stay tuned.

Enterprise Architecture 2010 Talk-- Where EA Means Business -- at SAP ASUG on May 5, 2008

On Monday, I'll be representing the SOA Consortium's EA2010 working group at SAP's ASUG conference in, you guessed it, sunny Orlando.  I'll be giving a talk on our EA2010 work.  Areas of discussion include 21st century business, service-orientation, business architecture, enterprise architecture and enterprise architects.  If you are attending the show, please come by, Monday at 9:30-10:30, in 308C. 

If you'd like to connect at the show, I'll be at the Sunday evening ESOA Community Networking Session and taking in sessions and wandering around the expo floor on Monday.  Drop me an email at bmichelson at gmail dot com.

SOA Consortium and CIO magazine launch Case Study Contest

Today, the SOA Consortium and CIO magazine are announcing the launch of a SOA case study contest.  The goal of the SOA case study contest is to highlight business success stories and lessons learned to provide proof points and insights for other organizations considering or pursuing SOA adoption.

Case study submissions must be for completed projects that used a SOA approach to deliver business value.   In keeping with our charter, we are not looking for dissertations on the technical beauty of the architecture and implementation. Rather, we are interested in the business story, the business value generated, the degree of cross-organizational collaboration, and the usages of SOA approaches and supporting technology.

For more information on the contest and participation, please go here.

Thanks to our Executive Suite SOA team and CIO magazine for putting this contest together! 

Enterprise Architecture 2010: Questions for EA, SOA and Business Architecture Communities

Have you had a chance to check out our new released EA2010 webcast and position deck?  If you did, great.  If not, no worries.  You can still participate in the discussion.  In our work, we call out the importance of business architecture as organizations progress with service-orientating their business and IT operations.  (Yes, SOA as a strategy.  No, SOA as a technology.)

As you can imagine, we had a lot of healthy (and some circular) discussion as we were working on the EA2010 material.  In the end, we decided we would publish the frame, and use it to solicit feedback and insight.

Some of the questions we would love the broader EA, SOA and Business Architecture communities to chime in on are:

1. Do you agree that Business Architecture as an important emerging enterprise architecture discipline?  Does your organization already practice business architecture?  If so, formally or informally?

2. Where will/do business architects come from?  Are these business savvy technologists?  Technology savvy business professionals?  Today's business analysts?  Other?

3. Assuming Enterprise Architecture 2010 encompasses not only today's technology laden "enterprise architecture" but also Business Architecture, where will it report?  Is this still an IT function?  Does it report to a business executive?

Please share your thoughts with us, as a comment on this post, via a trackback from your blog, or drop me an email brenda at soa-consortium dot org.  If you have other thoughts on the EA2010 work, please share those as well.

Enterprise Architecture in 2010 Means Business

One of the working groups in the SOA Consortium’s community of practice is the “EA2010” group. This group of seasoned enterprise architects from industry, government, systems integrators and vendors, has been actively discussing and defining the next generation role of enterprise architecture. Specifically, what enterprise architecture looks like – organization, practices and people – in a business-driven, service-oriented world.

This week, the EA2010 working group published the first release of their view of Enterprise Architecture 2010 in a Powerpoint deck.  Here's a sneak peak:

In preparation for this release, I sat down with the EA2010 working group leaders, Ashok Kumar and Yogish Pai, to discuss their working group’s motivation, findings and next steps. That conversation is available to the public as an on-demand webcast or podcast.

During our conversation, Ashok and Yogish touched on a wide range of enterprise architecture concerns, including catalyzing business change, gaining business-smarts, shifting focus to business architecture, managing enterprise architecture, participating in strategy and delivery, and winning enterprise constituents.

After viewing or listening to the webcast, please share your thoughts with us. As we discuss in the webcast, this is a work-in-progress and we really want to hear from the broader enterprise architecture community.  Leave a comment here or send an email to brenda at soa-consortium dot org.

And yes, the full EA2010 presentation deck is available for download at the webcast location).  Feel free to discuss and use any and all materials in your own enterprise architecture programs.

SOA Practitioner'€™s Forum: Service Version Management or Mayhem?

In our Community of Practice's Open Forum, Brenda Michelson shared some comments from CIOs and CTOs on managing SOA environments:

During our February Executive Summits, the CIOs and CTOs expressed concern that there aren't ITIL-like practices and guidelines for managing a SOA based environment.

A specific question was "How many versions of a service should be in production at once?"€ The group stated that "It was naive to believe only one version of a service would be live". But, how many is too many?

Brenda then asked our COP:

What are SOA-C members doing in their organizations? How many versions are active? How many versions are manageable? What are your version limits and practices?

I agree that service versioning is inevitable and must be managed. In this post I'€™d like to share my response to Brenda'€™s question. I focused on "€œwhen and why" to version. At the end, I have some questions for the broader SOA community.

Before I continue, I should introduce myself. I am Surekha Durvasula, Enterprise Architecture Manager for Kohl'€™s and the Leader of the SOA-C'€™s Executing Business-Driven SOA Strategy.

My Service Versioning Perspective

In the broadest sense, there may be two key reasons to support business service versioning.

A) Consumer SLA/ QoS driven

B) Business functionality or business policy driven

Consumer SLA driven service versioning could be supported by just altering the values on the envelope. The envelope level metadata can be used to direct a call from the mediation layer (read ESB) to the appropriate service provider end point. Here the consumer can provide the values for the SLA metadata or a metadata lookup activity for the service consumer can be performed by the mediation layer.

Examples of the Consumer SLAs could be greater audit, more stringent response times, tighter security needs etc. Here the service interface layer may not have to be altered and so the consumer may not be affected. However, the service provider implementation layer may have to support multiple service implementation end points that are in keeping with the QoS requirements.

Business Functionality driven service versions are typically impacted by service interface and/ or the payload level changes.

Here the consumer has to "know" how to construct the new payload and may need to "indicate" to the mediation layer the right business service to invoke. Given this behavior change on the consumer side, service versioning becomes a necessity to insulate existing consumers. One way of limiting the number of service versions being supported may be to leverage the service mediation layer to populate default values on newer payload parameters or to perform payload transformation. The mediation layer can invoke utilities/interceptors on behalf of the service consumer to absorb the impact of changes to the interface or the payload. However, in scenarios where no valid default values are available or else if new consumer context information is required on the payload then a new version of the service would need to be published, thus requiring the enterprise to have to support multiple service versions.

Other considerations for supporting service versioning include management and monitoring costs, efficient use of IT resources, regulatory and central governance policies and the stringency of the contract agreements with the consumers.

Service Versioning Discussion Questions

1. How many versions of a service should be "live" at any given time? How many versions are too many?

2. Do you use the service mediation layer to achieve backward compatibility?

3. Where do you determine the service version - the mediation layer, the service consumer or a service provider?

4. Do you have governance models and policies around transitioning existing consumers to new versions? Who is responsible for executing that transition?

5. What is the most graceful way of "retiring" or "decommissioning" a service?

Please share your comments on this post, your own blog with a trackback, or via email to blog at soa-consortium dot org