ESB and BPEL: Changing the Rules of Integration

By Fergal Somers

There's a fundamental change taking place with enterprise software. Complex, expensive, proprietary solutions are being replaced by technology that not only solves problems faster and more efficiently, but brings corporate IT closer to the business function. Nowhere is this more apparent than in the enterprise integration market. What used to take months, is now taking weeks and in some cases days. Web services and service-oriented architecture (SOA) are fueling this phenomenal change.
The SOA philosophy is based on a few simple ideas:

• All functionality should be available as Web services.
• Access to these services should be based on open standards.
• New services can be created from existing services.

The first point is the most important. Whereas in the past we had a mix of competing integration technologies (CORBA, DCOM, JMS, MQ) and proprietary vendor adapters – today there is only one clear choice: Web services. At last a consensus has emerged about an integration technology. The question used to be “how do I get system A to talk to system B”, the question now is “how do I expose system A as a Web service?” Furthermore, the Enterprise Service Bus (ESB) is bringing legacy systems and data stores that can’t natively support Web services onto the network. From an integrators’ point of view this is great news. As vendors start rolling out products and Web service adapters, your life gets easier.

The next important point is open standards. Open standards provide an integrator with two important things: interoperability and product choice. Open standards mean that an integrator can choose between competing vendors on the features they offer (provided those vendors adhere to the standards). Today most integration projects use XML Schema, SOAP (Simple Object Access Protocol) and WSDL (Web Services Description Language). Together they provide the means to describe a service and deliver a message to it. Coming up fast are WS-Security and WS-ReliableMessaging providing signing, encryption, and open reliable message delivery, respectively.

The final principle of SOA is that you can build new services out of existing ones. This really is the “holy grail” of integration. When companies look at their past integration projects they see islands of integration – different technical solutions were used to solve different integration problems. There was little re-use between integration projects and ever increasing maintenance costs. The whole SOA approach is a response to this: you define all your services in one way, as Web services. This means they have at least a potential to be reused. SOA goes further however, enabling you to build new services by composing together existing services. This is new. Previously new services were built by integrating with existing databases and servers. SOA says that you should also be able to build new services from services you've already integrated.

This simple principle leads to a couple of important knock-on effects:

• Services should, where possible, be stateless. State implies a shared context between the client of the service and the service itself. Stateless services make no such assumptions, generally scale better, and can be used in many different situations with little or no cost.
• Services should do one thing but do it well; a service should implement at most, one function. This is implied by re-use. For a service to be re-usable, it should make as few assumptions about the context in which it is going to be used as possible.

Typically, when people begin to look at their existing services they realize a couple of things:

• My existing services don't look like this (stateless and single-function).
• My existing services aren't exposed as Web services (they support differing transports, file formats, and interaction styles).
• If I'm going to create new services by composing existing services, how do I do this and where are these new services going to live?

Where does that leave an organization's existing infrastructure? This is where the Enterprise Service Bus (ESB) comes in. It provides a straightforward way to upgrade an enterprise's assets to the SOA world. The ESB provides:

• Connectivity to your existing assets (application servers, packaged applications, custom-built applications, databases, and message queues).
• Enterprise-level services (digital signing, reliable messaging, authentication and authorization, service discovery and registration, and quality of service guarantees).
• Enterprise-level qualities (clustering, scalability, and failover).
• Tools that enable you to create Web services from your existing assets, host them on the ESB, and manage them (instrumentation and policy management).

The SOA approach emerged in response to the existing problems of integration, namely too many protocols, too many representations, and too many adapters, but early SOA adopters quickly ran into problems migrating their existing systems to a SOA world. The ESB emerged as a technical solution to this problem. An ESB platform works with existing assets in a non-invasive way. It provides the services and qualities you expect from an enterprise deployment platform.

What about the “holy grail” of SOA – the ability to compose new services from existing services? This is where the Business Process Execution Language (BPEL) comes in. BPEL is an XML language for describing business processes. In BPEL, communicating entities, termed partners, support interfaces described using WSDL – that is, partners are Web services. BPEL provides a programming language for specifying the logic of how messages should be routed between Web services. A BPEL process itself is a Web service (it implements a WSDL interface).

It should be evident by now that BPEL at least matches the capabilities required to compose new services from existing services. BPEL, however, has a number of compelling features that really makes it ideal for service orchestration:

• It's an open standard from OASIS. So BPEL scripts are portable between BPEL engines – you are free to choose the tools and engines that best meets your needs.
• It's a programming language that simplifies Web service messaging, control flow, and intelligent (content-based) routing.
• BPEL processes are stateful and potentially long-running. A BPEL process can span milliseconds or minutes, hours, days, or even months. This allows BPEL to model a wide range of business processes.
• It's event-driven. BPEL processes can support multiple threads of execution allowing them to respond to events. For example, a cancel-forward event. This is something that is problematic for traditional RPC-style systems. It allows BPEL to efficiently integrate services that take a long-time to execute. It also allows a BPEL process to integrate easily with both synchronous and asynchronous services.
• It supports fault-handling. This is required for building robust business processes.
• It provides explicit support for message correlation. One standard chore faced by integrators is correlating messages from different systems. BPEL makes this part of the language, freeing developers from having to worry about it.
• It provides for transactional-style recovery, allowing BPEL processes to gracefully back out of failed scripts.
• It is XML native. A lot of integration projects take WSDL and XML Schema descriptions of services and compile bindings (stubs and models) to a specific language. This creates a maintenance problem. When the WSDL or XML Schema change (and they always change) the bindings must be recompiled, which impacts on the integration code that uses these bindings. BPEL works directly with WSDL, XML Schema and XPath. There is no need to create bindings. This minimizes the impact that changes to the WSDL or XML Schema has on a BPEL file. In many cases, there will be no effect on the BPEL script due to changes in WSDL or XML Schema.

BPEL is a language specifically geared towards building new services out of existing Web services. It greatly simplifies this task, allowing new services to be developed and deployed in a matter of hours or days. BPEL processes are being deployed to solve an increasingly diverse set of problems. Some examples are:

• Back-office functions, for example, account provisioning, insurance claim handling, and loan applications.
• B2B integration, for example, service inception and revocation.
• WAN coordination, for example, helpdesk problem ticket routing.
• Test configuration management.
• Web service conformance testing.
• Inter-tool coordination within product suites.

ESBs make it possible to leverage existing assets, deploy, and manage Web services. The addition of BPEL to the ESB makes it possible to compose new services out of existing Web services. Together, ESB and BPEL have changed the rules of integration. They make it easier and faster to develop and deploy Web services, and by doing so, they make the promise of SOA possible.