Community Insights: Services, Sprawl, Marketplaces, Bazaars and Portfolio Management

Note: For practitioners, one of the most valuable aspects of consortium participation is the opportunity to exchange ideas in a trusted peer community. While the diverse group doesn’t always reach a single conclusion, each participant gains new perspective to consider in their unique situations.

One such SOA Consortium conversation thread started with service value, covered service marketplaces, cathedrals and bazaars, and concluded with service portfolio management. What follows are notes from that conversation, published not as an answer, but for consideration as you expand the breadth and depth of your service-oriented architecture. As always, we welcome comments.

CALL #1 – Service Sprawl and Service Marketplaces

In a series of calls, the SOA Consortium’s community of practice engaged in a very interesting discussion on service portfolio management. The conversation began innocently enough, when one member queried the group on visual methods to demonstrate the value of a services approach to business personnel.

This member was looking for insights on creating a business intelligence dashboard for business stakeholders to see the value of service adoption, including key metrics on service use and re-use, and time-to-market improvements. His motivation wasn’t marketing, but an underlying concern that full SOA value wasn’t attained because of service sprawl.

In this case, the service sprawl had two sources. First, and the more common cause, SOA was simultaneously adopted in different business lines. In the early stages, each business line created its own services, unaware of reuse or collaboration opportunities. Second, the organization acquired another organization with similar business lines, which was also pursuing SOA.

Through conversation, members noted other service sprawl causes, such as lack of trust, immature service governance practices, and no mandates for service rationalization.

From here, the group began discussing ways to reduce and prevent service sprawl. A popular idea was the creation of a services marketplace. In a short brainstorming session, the group discussed incorporating prediction markets to gauge service popularity, and social technologies to build trust, as well as to strengthen the service offering with better definition, feature/function, and performance. Detailed brainstorming notes follow.

Services Market Place Requirements

Organizations need to “advertise” their services, advertise as in “marketing”, not UDDI type directory listing
Developers need to learn about “popular services”. These trusted services will continue to receive investment. Less popular, duplicate services should be slated for obsolescence.
Service consumers need ability to provide feedback on service, and suggest improvements
People need ability to “vote with feet” or code
Service sprawl is a people problem, therefore a social solution is required

Services Marketplace Functionality/Technology Ideas

Create a prediction market to understand service popularity (existing or planned service). Each marketplace participant creates an investment portfolio of 10 services. To add a service to your portfolio, one must be removed. The service mix across portfolios will indicate service popularity, as well as pruning trends. Spigit was cited as a technology example, it is used by Microsoft (and others) for release date prediction
Use Amazon model for customer review and even for service recommendations “consumers of this service also use this/these service(s)”
eBay seller rating, equates to service publisher rating, “also available from this publisher…”
Service definition might benefit from Facebook (Lombardi) extended water cooler model, experts/interested parties can add to definition/requirements in public wiki like environment

Services Marketplace Questions

Is there a unit of exchange? If so, what is it?
What is the best way to bring the marketplace into the work stream of potential service consumers, providers? Is it via calendaring – associated with design and review meetings? Modeling tools? IDE?
Is there a similar marketplace opportunity for a business process models?

About this point, the group made the leap to the Cathedral & Bazaar metaphor. Service repository as we know it today is Cathedral, Services marketplace is Bazaar.

Service Bazaar Startup Needs

Advertise the Bazaar
Place close to work stream
Establish minimum entry requirements: Merchants must provide certain attributes for each service: purpose, business process, business process model, nonfunctional attributes, implementation attributes (can and can’t do), chargeback and rates, classes of service (levels, cost), support model (use at own risk or supported)
Find and win early adopters
Incorporate knowledge management, social technology and community practices
Word of mouth marketing – early adopters are best advocates
Incremental introduction plan

At the close of the call, the member originally posing the service sprawl problem decided to explore prediction market technology for service portfolio rationalization. As well, the group decided to continue the bazaar discussion on the next call.

CALL #2 – Service Cathedrals and Bazaars

Continuing the services marketplace conversation theme, but with many new call participants, there was a strong sense that service sprawl needs to be managed both proactively, via portfolio definition/management practices and reactively, via both top-down rationalization efforts and bottoms-up service marketplace advertising and consumption patterns.

Call participants from enterprises expressed concerned that a pure services marketplace – introducing competing code – is not behavior a development organization can afford to participate in.

There was strong support for the advertising aspect of the services marketplace, letting people know a service existed, its purpose, current usage scenarios, community ratings etc.

For a services marketplace to be successful, it needs to be close to where developers work. Oracle’s Enterprise repository (BEA Flashline) was cited as an example of tool with community engagement vision.

It was noted that in organizations where sprawl exists, or M&A activity is underway, the services marketplace can provide a mechanism to contain further sprawl and promote (Darwinian) rationalization, even before formal rationalization assignments/efforts are underway.

Another note was that duplication does not always equate to waste. An organization’s operating model (see Ross & Weil) may call for some separation of services to support a fast growing/changing business unit/activity.

There was a call for distinction in the marketplace’s unit of exchange (used in funding decisions) from service portfolio metrics. Service portfolio metrics should not be number of services; instead use percentage of (unplanned) redundancy & coverage gaps.

The group believes portfolio management – value of IT investment over time – is an important concept. The current project focus in most organizations – on-time and on-budget measurement – is very short sighted. Businesses do look at total IT spend over time, and percent of development versus maintenance, but most often don’t drill into the specific business domains or business capability. This lack of transparency can lead to a behavior of delivering sub-par solutions that require frequent replacement.

In respect to proactive service rationalization, the group spoke of defining enterprise and/or domain service maps. They also spoke of the need to get business analysts involved in service definition, and that business analysts might be better accepted by business units if they (business analysts) reported to Finance or Quality, rather than IT.

Additionally, the group discussed incentives for sharing services. The biggest point/concern here is that the incentives need to map to portfolio metrics and goals. Avoid the trap of putting incentives in place that encourage “gaming the system” to receive the incentive while damaging the overall service portfolio. Some members believe the incentive is keeping your job.

A late discussion point was that sustainable SOA has organizational implications beyond initial centers of excellence. Sustainable SOA requires pervasive practice, cascading changes (service thinking and management) to funding practices, project management, solution delivery, operations, and change management. In other words, SOA cannot be a side practice forever.

CALL #3 – Service Portfolio Management

Reviewing the key discussion points over the past two calls, the group concludes that both models – Services Cathedral and Services Bazaar – are valid.

The adoption of a cathedral and/or bazaar approach may vary based on project lifecycle phase, business criticality of the problem being solved, experience of the project/development team in respect to SOA, and/or time to market considerations.

A discussion on project lifecycle practices leads to a discussion on portfolio management practices:

Organizations most often view portfolio management in respect to managing concurrent projects and managing IT investment.
Organizations also practice asset management in respect to purchased (infrastructure oriented) software and hardware. Vendor contracts, releases, technology advances and the like drive these asset management practices.
Organizations seldom practice asset management practices on their application portfolios, and if they do, it is for purchased rather than custom-built software.

A service-oriented approach fundamentally changes the basic management unit from an application, to services, or a bundle of services that represent a business function (component, segment, flow or capability), which are consumed by many applications, business processes and/or domains.

The multi-purpose aspect of services, and bill-of-materials structure of service compositions and solution assemblies, point more towards product management practices than traditional application management practices. The shift to product, rather than project or application thinking, forces changes to funding, project management, asset management and operations practices.

Most notably, organizations will need to be deliberate in the realm of service portfolio management, understanding what business capabilities are offered in the portfolio (marketplace), how those capabilities are instantiated via software and information assets (model to asset mapping), and how/where those capabilities are packaged into end-products (applications, processes) that solve a business need.

The diagram below, previously published, illustrates services and the various process and portfolio interaction points across the service lifecycle.

[Click on diagram to enlarge]