SOA Consortium Insights

New SOA Consortium Podcast: SOA & Event Processing Roundtable

The SOA Consortium’s SOA and event processing day concluded with a hot topics roundtable discussion between industry experts and SOA Consortium members on the current and future relationship of SOA and event processing in the context of delivering business capabilities.

The roundtable began with the industry experts, Ian Foster of Cisco, Bruce Henderson of Savant, Ed Lynch of IBM, and Greg Peres of Sun Microsystems sharing their thoughts on SOA & event processing business drivers, technology challenges, human/organizational elements and event processing’s market timing – new or retro-technology strategy.

In their remarks, each roundtable leader called out at least one business scenario for SOA and event processing, these included manufacturing floor management, real-time customer loyalty programs, healthcare and electronic records management and business visibility. In respect to technology challenges, the conversation centered on dealing with massive amounts of information, and the requisite implications on volume, scale, filtering and security. An interesting point on the human aspect is that today’s teenagers, with their ubiquitous use of texting, are already very event aware, quick to respond to problems or opportunities delivered via an event-based mechanism.

After the opening insights, the roundtable leaders engaged in discussion with each other, and meeting attendees on a variety of topics including event-based information proliferation and privacy, event and event emitter standards, recognizing event patterns and business trends, and business confidence in automated, event-driven reactions.

As the session closed, I asked each roundtable leader to provide a design tip to architects and developers whose services need to either emit or respond to events. Their responses follow:

Ian Foster: “I think it starts with the business user, a conversation with the business user about the key events that they would want information on. You know, is it a key event, the location of that part on the manufacturing floor? Is a key event the temperature of that truck coming across the country? What are the key events? And then look at how you measure to instrument.”

Greg Peres: “Make sure that as an architect, you are choosing technologies and platforms that are open, so that the event that you generate and design are open to other technologies and integrate, so that you can process them in a variety of different ways.”

Ed Lynch: “In my experience, the technical aspect is to make the event emission extremely lightweight...Do not make it intrusive, because at the end of the day you do not want to have somebody make a performance trade-off between the emitting of the event and the driving of the transaction. On the consuming side, make sure that the event correlation engine can drive services with a standard interface. Once you do that, I think the standards on the driving side and the standards on the event emissions side – as long as they are lightweight – it becomes a ‘synthesize the infrastructure mechanism’, as opposed to an ‘integrated into the infrastructure’ mechanism.”

Bruce Henderson: “Do not be afraid to create event models. I know it is sort of the bastard child of UML to draw these little blob boxes and put event transitions between them, but I have found in the past that that is a very powerful means to be able to identify and really get a handle on what your events could and should be. And working with the business user is a tremendous leverage in that.

And I would also say that do not treat your event pipes and your event information as stovepipes. Decorate them with meta data, because at some point down the road as it was mentioned before, when you put these things out, you do not really actually know necessarily who is going to consume them or why. So make them as interesting as possible without overloading them so that they can find lots of neat homes and friends to play with.”

To listen to the roundtable podcast or download the roundtable transcript please go here.

The SOA & Event Processing roundtable discussion is the seventh and final podcast recorded at the June meeting.  Earlier, we released Bruce Henderson on SOA & Event Processing via Mashups, Ed Lynch on SOA & Event Processing via BPM, K. Scott Morrison on How to Fail at SOA, Mel Greer's talk on SOA Hard Problems & Spiral Solution Development,  Jim Johnson's talk on SOA Trends & Pipelining and David Butler's talk on SOA Transformation.

New Podcast: Savant's Bruce Henderson on SOA & Event Processing via Mashups

Adding a unique perspective to the SOA Consortium’s SOA & Event processing day, Bruce Henderson, Chief Scientist of Savant, spoke of the connection between services, event processing and mashups.  Bruce shared his thesis that many major business, geo-political and societal problems arise because we don’t know enough to avoid them.  Despite swimming in data from a vast number of sources, the inability to relate and interact with that information has organizations operating out of rear view mirrors.  As Bruce succinctly stated, “What we don’t know can hurt us”. 

Supporting his thesis, Bruce spoke of three major events, the dotcom bubble, the September 11 attacks and the U.S. housing market implosion, whose impact could have been reduced had the available, yet scattered, data been correlated and viewed in totality.   Bruce then shared his prediction, based on applying Savant’s information agility techniques to publicly available FDIC reports, that the US Banking System faces impending failures similar to Northern Rock in the UK.

From that vivid context, Bruce walked through the technical concepts to relate and correlate disparate information via microscale services, events, event classification and metadata decoration, and rich user interfaces. 

Completing the circle, Bruce demonstrated three of Savant’s mashup projects, including the US Banking System mashup that incorporates a trend visual to show mortgage portfolios, related write-offs and balance sheet strength of major US banks.

To listen to an audio recording of Bruces’s presentation and view the slides go here.

Bruce's talk is the sixth podcast recorded at the June meeting, and the second of three from our SOA & Event Processing day. 

Earlier, we released Ed Lynch on SOA & Event Processing via BPM, K. Scott Morrison on How to Fail at SOA, Mel Greer's talk on SOA Hard Problems & Spiral Solution Development,  Jim Johnson's talk on SOA Trends & Pipelining and David Butler's talk on SOA Transformation.

New Podcast: IBM's Ed Lynch on SOA & Event Processing via BPM

Day two of the SOA Consortium’s June Meeting focused on combining SOA and Event Processing to deliver business capability. Ed Lynch, Product Manager for the BPM & Connectivity portfolio, and Integration Exec for the Aptsoft acquisition, IBM Corporation, kicked off the morning by sharing his view that SOA and Event Processing come together within Business Process Management.

In building his case, Ed first called out the findings of IBM’s Enterprise of the Future CEO study, which speaks to a business environment of accelerating change, brought on by globalization, constant connectivity, doubling data and burgeoning competition. To cope with this torrent of change, executives require two things. First, they need visibility into their businesses, in real-time. Second, those businesses need to be agile. Businesses must be able to respond quickly to opportunities and threats, or risk losing market share to existing or new competitors.

With that business backdrop, Ed shared the basic constructs of event processing and then made the connection to services and business processes. In short, Ed told attendees to think of events as the “when” for the “what” of services. Business processes orchestrate the what, the services, to respond to an event, or a series of events.

Bridging the conceptual and reality, Ed described several customer examples, including fleet management, customer loyalty programs, algorithmic trading, business activity monitoring and fraud detection.

Throughout the presentation, Ed engaged in Q&A with attendees on a range of topics including management practices, data swarms, rule management and event processing futures.

To listen to an audio recording of Ed’s presentation and view the slides please go here.

Ed's talk is the fifth of eight podcasts recorded at the June meeting, and the first of three from our SOA & Event Processing day. 

Earlier, we released Mel Greer's talk on SOA Hard Problems & Spiral Solution Development,  Jim Johnson's talk on SOA Trends & Pipelining, David Butler's talk on SOA Transformation and K. Scott Morrison on How to Fail at SOA. 

New SOA Consortium Podcast: Layer 7's Scott Morrison on "How to Fail at SOA"

Renowned security expert, K. Scott Morrison, VP of Engineering and Chief Architect at Layer 7 Technologies, delivered an innovative and example-packed talk on “How to Fail at SOA” at the SOA Consortium’s June meeting.

Calling on Layer 7 Technologies’ six years of experience – “an eternity in the SOA world” – Scott warned meeting attendees about repeated patterns of bad practices, pitfalls and bad decisions. Focused in the security realm, Scott reenacted customer scenarios that highlighted breakdowns with goals, teams, planning, knowledge and physical design.

For each identified problem, or anti-pattern, Scott provided insights on how SOA practitioners can recover from, or better yet, avoid the pitfall. These insights covered a wide range, including proper understanding of security standards, security design tips, skills, team composition, training and business outcome identification. Scott pointed out how SOA based implementations -- with their highly distributed, cross stack natures -- bring together a diverse group of new stakeholders that must collaborate for success.

In closing, Scott encouraged attendees to treat security and management as first class citizens of SOA efforts, rather than falling into the trap of patchwork retrofitting.

To listen to, or download the audio recording of Scott’s presentation, and view the slides, please go here.

Scott's talk is the fourth of several podcasts recorded at the June meeting.  Next up are the SOA & Event Processing day podcasts.  Earlier, we released Mel Greer's talk on SOA Hard Problems & Spiral Solution Development,  Jim Johnson's talk on SOA Trends & Pipelining and  David Butler's talk on SOA Transformation.

New SOA Consortium Podcast: Melvin Greer on SOA Hard Problems & Spiral Solution Development

The SOA Consortium was extremely fortunate to have Mel Greer, Lockheed Martin’s Chief SOA Architect, return to speak at our June meeting in Ottawa. Consistent with his March talk on SOA Competency Centers, Mel delivered a very engaging and thought-provoking talk on the topic of SOA Hard Problems and Spiral Solution Development.

Mel began by defining hard problems and spirals. Hard problems have three characteristics. First, a hard problem doesn’t go away over time. Second, left unresolved, a hard problem will have a significant negative impact on your SOA adoption. Third and most important, resolving a hard problem requires multiple disciplines that come from inside and outside your own organization. A spiral is a technique that breaks a hard problem into a series of small activities, each lasting 30-90 days. Each activity, or spiral, produces an answer that moves the hard problem towards resolution.

Lockheed Martin has identified SOA hard problems across six categories: business, engineering, operations, security, governance and skills development. During his talk, Mel shared examples of hard problems within each category, as well as the inter-relationships between hard problems.

Within the engineering category, Mel spoke of altering existing development processes and methodologies for SOA, designing for context awareness, and designing for runtime discovery and composition. In respect to runtime discovery and composition, Lockheed Martin is trying to determine the best way for a running to composition to become aware of newly delivered capability. As an example, Mel called out how the Mars Land Rover continues to receive new capability without returning to earth.

In closing, Mel spoke of impending challenges as third-party services – SaaS, Applications as a Service (APAS), cloud computing, etc – become the new business models. These changes will require support for service-level agreements, real-time monitoring, end-to-end testing, pricing models and service usability.  Mel encouraged attendees to consider these new hard problems in their work as a multi-discipline, cross-industry consortium.

To listen to, or download the audio recording of Mel’s presentation, and view the slides please go here.

Mel's talk is the third of several podcasts recorded at the June meeting.  Next up is K. Scott Morrison of Layer 7 Technologies on How to Fail at SOA.  Earlier, we released Jim Johnson's talk on SOA Trends & Pipelining and  David Butler's talk on SOA Transformation.

New SOA Consortium Podcast: Jim Johnson on SOA Trends & Pipelining

In the second keynote of the SOA Consortium’s June meeting in Ottawa, Jim Johnson, Chairman of The Standish Group, shared the results of a research study on the top 10 drivers that are influencing decisions on how IT implements SOA. These drivers are increased business agility, business process modeling, fear/fashion/peer pressure, staff coercion, investment reuse, readiness assurance, architecture flexibility/scaling, regulatory compliance, security promise and vendor hype.

For each of the 10 drivers, Jim describes the underlying survey analysis; pointing out related statistics and considerations around investment, risk, yield, project scoping, resourcing and success and failure rates. Citing The Standish Group’s 10 Laws of Chaos, Jim provides insight on how to proactively recognize and prevent situations that typically cause projects to fail. Such as, "The law of the empty chair", which states your best possible person, will leave at worst possible time.

Throughout the presentation, Jim shared how applying pipelining techniques – micro-project stacks, portfolio baselines and resource pools – can increase an organization’s SOA project success rate.

To listen to, or download the audio recording of Jim’s presentation, view the slides, and learn why Jim likens SOA to a clean closet, please go here.

Jim's talk is the second of several podcasts recorded at the June meeting.  Next up is Melvin Greer of Lockheed Martin on SOA Hard Problems & Spiral Solution Development.  Earlier, we released David Butler's talk on SOA Transformation.

New SOA Consortium Podcast: David Butler on SOA Transformation

David Butler, Worldwide SOA Director and Chief Evangelist, HP, kicked off the SOA Consortium’s June meeting in Ottawa with an information-packed presentation on SOA Transformation.

Setting the record straight from the start, David shared “In many respects the SOA market has lost its vision around what SOA is actually trying to do. Not provide a new set of technology, but provide a transformation focused on a design point called a business service. A business service drives measureable business outcome.” David then shared examples of business outcomes from a semiconductor company, a financial services firm and an insurance provider.

David pointed out that successful SOA transformation is more than implementing software. Transformation spans people, organization, process and technology. On the organizational aspect, David called out the criticality of business-IT alignment for proper business service identification and implementation. “…how we map business capabilities in a consumer-provider fashion, and map that into IT systems and technologies. A business person should be able to consume and provide capabilities without knowledge of the underlying technology and platforms, and do that in a changeable manner.”

David shared a 10-point roadmap for SOA transformation, based on customer experience, that covered business service definition, center of excellence, SOA governance, execution architecture and platform, quality and provisioning, service lifecycle, data center operations, IT management processes and line of business engagement.

In closing, David stated, "SOA is a business-oriented architecture". Following his presentation, David engaged in Q&A with the meeting attendees on SOA transformation challenges, including security, business service definition and granularity, service-engineering, service chargeback and economic models, and the changing role of the CIO.

To listen to, or download the audio recording of David’s presentation and view the slides please go here. 

David's talk is the first of several podcasts recorded at the June meeting.  Next up is Jim Johnson of the Standish Group on Trends in SOA.

SOA Hot Topics Roundtable Podcast: Public Sector SOA

The final podcast from the Public Sector SOA day at our March meeting, is now available.  This Public Sector SOA Hot Topics Roundtable podcast features Victor Harrison from CSC, Bill Vass from Sun Microsystems Federal, and Kshemendra Paul of the Office of Management & Budget.  Keeping with our hot topic roundtable format, our objective was to generate a meaningful conversation between thought leaders and meeting attendees on the drivers, issues and available guidance for public sector SOA adoption and management. The podcast starts with brief opening statements from Victor, Bill and Kshemendra, followed by a facilitated Q&A with the audience. 

For the most part, my facilitation consisted of passing the microphone around.  However, I did get one question in, after observing the numerous similarities between public and private sector SOA.  No surprise, I found the biggest difference to be the predisposition for governance in the public sector.  On that theme, I asked the panelists "how do you balance governance with productivity, actually being able to get something done and out the door".  

In his response, Kshemedra Paul spoke to correlating the size, scope, complexity and impact of the problem domain with the governance structure:

"If you are trying to solve a big societal problem, then these formalities and these structures are important, because there are so many stakeholders, there are so many different levels and so many issues.  There are the technical issues, but also in many cases privacy/civil liberty-type issues, or security and protection, personally liable information, I mean, so many different considerations that come into play. And there are plenty of examples where somebody skipped a step or did not do the full due diligence, and it later on kind of came back and torpedoed the program. And forget about the waste of money, and time and effort, you are still not filling that societal need, right. So these structures, they have grown up through a need to have this level of control. And another thing I would add is that sometimes they can be your friend, right? If you can work within those processes, right, and then when things get to be mature like service-oriented architecture, we are able to inject them in the appropriate way through the governance processes and the establishment’s practices, then it becomes easier to get the broad-based adoption, because it is familiar to people"

Bill Vass shared an interesting governance tactic from his CIO days, in respect to managing a systems integrator relationship:

"I put the requirement in their contracts that if they did not follow those things [architecture and programming standards], I did not have to pay them, they would do the work for free. And that tends to herd them in the right direction. Another thing that we did is incentives for re-use...most systems integrators are going to be incentivized not to re-use... because they are time and material, to build everything from scratch that they can for each individual chance, and sell the same thing over and over again. So what we did is we built into our contracts that if a service was created that could be re-used, and it gets re-used, then they get a bonus. And then if they re-use a service, they get a bonus. So they make more money if they re-use existing things. And this changed the behavior within the governance model where they went out and looked in the universal business registry for a get-order status before they wrote it, because they got paid more if they re-used one, because they would have less labor and greater profit. And if they created the get-order status routine, they would generalize it enough so others could use it, so they would make profit off of that. And this herded the cats towards an integrated architecture over time without – again, people are always the problem. It is not the technology. So I would love to see the government look at that kind of incentivized capability inside their contracts."

Later in the roundtable, the topic of skills came up.  In his response, Bill provided one of my (new) all time favorite quotes from a provider of products.  Emphasis is mine:

"It is finding people who can abstract, even people who can build an architecture without putting a product name in it is extremely hard. It is not an architecture if it has a product name in it. I spent a long time explaining that, because architecture lives beyond that, right, and I think that is a challenge across the board."

To listen to the full podcast (54 min) and/or read the transcript, go here.

Next up, we'll be releasing the podcasts from day 2 of the March meeting, which was focused on Centers of Excellence. 

Kshemendra Paul's Practical Guide to Federal Service Oriented Architecture (PGFSOA) podcast now available

 The podcast of Kshemendra Paul's talk on “The Practical Guide to Federal Service Oriented Architecture (PGFSOA)”  recorded at our  March SOA Consortium meeting is now available.  During his talk, Kshemendra spoke of his experience employing an SOA strategy for information sharing at the Department of Justice and the Federal Chief Information Officers Council's initiative to develop a Practical Guide to Federal Service Oriented Architecture. 

The PGFSOA speaks to service-orientation at three levels, service oriented enterprise, service oriented architecture and service oriented infrastructure.  Below, excerpted from version 1.1 of the PGFSOA, are a diagram of the three levels and a summary of key recommendations at each level.  For more information, I encourage you to listen to the podcast, review the PGFSOA and participate in the current  round of public review.

Recommendations for Service Oriented Enterprise

• Treat SOA as a change initiative. It must have strong executive buy-in, adequate resources, organizational visibility and sustained support.
• Create a program plan with goals and objectives, and objective performance measures for the SOA initiative.
• Establish a Center of Excellence (COE) to guide and manage the SOA initiative.
• Adequately fund the COE and the SOA initiative.
• Develop and sustain appropriate and viable Communities of Interest (COI) to help facilitate the development and use of shared standards, platforms and semantics.
• Establish formal funding mechanisms to support the creation and delivery of services, coupled with appropriate charging mechanisms that are tied to service usage.
• Establish a Federated Governance structure that includes a charter defining organizational structure and relationships, scope of responsibility, rules of behavior, conflict resolution processes and the authority and structure of the COE.
• Adopt a twin-track SDLC that facilitates the incremental, innovative refresh of the IT assets on an on-going basis.
• Create and use a services development, test and evaluation laboratory to meet enterprise requirements.
• Adopt procurement policies and processes that encourage vendor competition around service models.

Recommendations for Service Oriented Architecture

• Develop, in collaboration with business units and IT, business models that help to align the EA with business objectives.
• Develop an EA Target Architecture that is service based and focused on business priorities.
• Based on the service-based Target Architecture, develop investment portfolios that are founded on services and solutions focused on fulfilling key business objectives.
• Establish or adopt reference architectures and reference implementations that can be used by project implementation teams to jump start their SOA efforts.
• Use the EA artifacts and the database of information developed during the EA process to mitigate the compliance burden placed on projects.
• Assess all legacy assets in terms of their relationship to Target Architecture objectives, and factor this analysis into decisions for how to provision each service.

Recommendations for Service Oriented Infrastructure

• Adopt a Federated approach to security and privacy for defined Communities of Interest (COI) that identifies common security and privacy solutions based on a risk/reward approach.
• Determine the most effective approach to achieve semantic interoperability, either through processes like NIEM or through semantic technologies.
• Incorporate run time and build time service management functionality to define, monitor, enforce, and adjust Service Level Agreements (SLA).
• Utilize a Registry/Repository to discover needed services during application build and to insure that key criteria such as reliability, efficiency, dependency, and adherence to and/or violations of policies are fulfilled.
• Adopt a trust model that addresses both security and privacy and also quality of service.
• Establish a collaborative test/evaluation and certification/accreditation process for services.
• Evaluate emerging technology against relevant test cases. For example: Internet Business Logic (www.reengineeringllc.com); using better technology can lead to working smarter rather than just working harder.

Ross Altman's Getting to the Service Oriented Enterprise Podcast Available

Ross Altman's talk on Getting to the Service Oriented Enterprise from our March meeting is now available as a podcast.  In this podcast, Ross speaks to the opportunities and challenges of composite application development, the role of open standards and open source in SOA, general SOA challenges, and Ross' view of SOA futures, including real-time SOA, mobile SOA, trusted SOA and more.

To listen to the podcast and grab the slides, go here.  Keep an eye out for additional podcasts from our March meeting.