Business considerations for Enterprise Application Integration

Business considerations for Enterprise Application Integration

David Theriault & Jacqui Cadell
GE Smallworld
1 High Street, Chesterton
Cambridge UK, CB4 1WR

Enterprise Application Integration (EAI) software has enormous potential to reduce Information Technology (IT) costs and introduce immediate benefits to the utility business. The perceived benefits come about, as usual in the IT industry, by making sure the technology offered is well understood and will be applied to improve appropriate business processes. EAI provides the enabling technology that moves away from multiple point-topoint integration between disparate systems, instead providing an ‘integration bus’ that applications can tap into via connectors or adapters. This apparently simple shift from one model to another promises to significantly reduce long term costs.

Most utilities have clearly understood that by introducing e-Business processes they will be establishing an IT infrastructure for future business health. It will be the basis for the interactions between the utility / energy company, their customers, suppliers and partners for years to come. The question is how to make the transition a successful one. The underlying key issue to this is the balance that needs to be struck between striving towards business benefits and the costs of acquiring and long term maintenance of software.

Consider on one hand, Company X that chooses to take the route of establishing a web presence for customers and suppliers and has done the absolute minimal investment in the connection to back-office processes. In addition, those back-office processes are monolithic and internally focussed where most interaction is batch oriented file transfer. Although they will get some instant recognition from being on the Web, the benefits of e- Business are far off for them.

Company Y has decided to go another route by taking a longer-term view. They have chosen to replace their legacy systems with a massive overhaul so that they have all modern web enabled software from start to finish. Although they will have the comfort of knowing they’ve made a good long-term investment, the benefits of e-Business are far off for them.

The scenarios pictured above are not entirely realistic, deliberately so: they represent the extremes of how being too technology focussed can result in poor business decisions. Let’s also consider a third example, Company Z who, by watching and learning from X and Y, has decided to utilize EAI technology to address its business issues.

EAI provides mechanisms, technologies and an integration philosophy to enable organizations to make the right choices to weave the optimal path to e-Business and core systems integration. Spatial technology and systems present special challenges - demanding, complex applications - that make those choices even more difficult although emerging standards for the web and for application integration will make a difference. For utility organizations and vendors making those right choices are key to a rapid return on investment.

This paper will explore these issues further by providing relevant background on the technology but with the focus on how it relates to business considerations.

Why should network organisations be interested in EAI?
The introduction of new business drivers is threatening to erode the planned life cycle of legacy¹ corporate systems. It is a recognized fact that these legacy systems do not easily lend themselves to integration with modern systems. However, most organizations know how to achieve each of the integrations required, on a one-at-a-time basis. The maxim heard at the outset of every IT exercise to integrate yet another system has long been ‘This isn’t rocket science’. But given the costs involved, one could be forgiven for thinking that was the case.

The cost of building and maintaining point-to-point solutions has been the major brake on aggressive, large-scale systems integration. E-Business has meant that this can no longer be put off but the costs of integration have not become smaller.

E-Business has also fostered an obsessive drive for innovation. This and the changes associated with innovation are becoming the key drivers for integration. Constant technical innovation is driving how utility and communication organizations compete.

A key question is then how to both reduce the costs of integration and to provide the right environment and IT platform for incorporating innovation. The answer seems to lie in the direction of EAI technology. Meta Group (Waters, 2000) points out that within Communication companies the use of EAI technology is 35 to 40%. On the other hand, Energy IT organizations have been reluctant to embrace EAI due to perceived complexity and lack of EAI understanding and skills.

What are the carrots and what are the sticks?
The ongoing focus on customer oriented business is one of the key drivers behind EAI integration. The customer is the end beneficiary of integrating CRM and ERP applications with each other and applications from partners and suppliers. The drive is towards better results for improved service levels, ones that get noticed and appreciated by the customer. Other business drivers for application integration include corporate mergers and acquisitions, deployment of best of breed packages applications and organizational changes. The need for integration has increased due to e-Business trends such as corporate self service, virtual supply chains, customer relationship management, application hosting and business-to-business commerce.

Building and owning point-to-point solutions is expensive. Anecdotal evidence show that while figures like $1 million to integrate applications from mainstream ERP systems may not be the norm, it is not unusual either. Meta Group (Waters, 2000) has predicted that early adopters of comprehensive ERP solutions (including back office, operational and front office functionality) will re-evaluate the benefits and cost of tight integration and will transition to a distributed, component based architecture that relies on an EAI backbone across the spectrum of energy business segments. The ability to quickly and economically modify business processes and the business applications that support them will increasingly be a key competitive energy company differentiator.

What are the risks and points of failure?
Reducing costs and leveraging e-Business have clear benefits but what are the further pitfalls that Company X and Y need to avoid? And what about Company Z who has chosen the EAI path? The numbers vary from analyst to analyst but there is general agreement that two thirds of new IT application initiatives fail outright or are seriously overdue on delivery time.

One risk that Company Z will immediately encounter is the up front investment. EAI technology means investing in some rather expensive software to accomplish a grand integration scheme. In contrast, with point-to-point solutions, it is sometimes easier to justify single cost – i.e. you get exactly what you pay for. It can appear that with EAI you pay a lot of overhead first. The justification is that the investment is going into foundations and infrastructure for further work.

The Company Y example has some relevance – there is a real danger of EAI projects getting waylaid by the temptation to go and fix or replace legacy systems to make them more amenable for integration. One analyst from Forecross Corp (Estes, 2000) warns against experimenting with business-critical applications. But where necessary, this process can be contained and made manageable by being able to determine the true condition of legacy systems. Then, by initiating a multi-step process, legacy systems can be brought to a globalized state allowing interaction with systems external to the enterprise.

Assuming our Company Z has a ‘normalized’, maintainable, legacy application, it is possible to address the ‘e-Readiness’ of monolithic legacy systems using XML by retrofitting encoding and decoding logic directly into applications. XSLT can provide an external data mapping that allows the organization to implement a true, three-tier data exchange architecture. By utilizing XML and XSLT, Company Z has not only extended the life of its legacy systems but also extended its reach into e-Business.

There is another technical problem that can plague EAI projects. Data model, system differences and semantic differences can be so great that the middleware is little more than a messaging service and one ends up essentially having a point-to-point solution implemented on top of a very expensive messaging service. There is no quick fix to this situation, however, recognizing that it exists early in the project is obviously a cost saving advantage.

Integration of a spatial system poses special considerations and risks, due to:

The complexity of spatial data structures
Integrating a design environment (long transaction) with other systems
Visual or graphic information content of maps and schematics

In addition, the stability of the major EAI vendors is certainly a risk worth considering. There is currently significant consolidation and shift in the EAI market place. EAI market leaders are acquiring new companies and expanding their offerings (e.g. WebMethods acquired Active Software, TibCo acquired InConcert, Neon partnered with Forte). Overall there is no clear single market leader and there are many players – this is an unstable situation that in most markets is a prelude to consolidation.

Overview of EAI technologies and architectures
Gartner Group (Pezzini, 2000) has pointed out that every application integration problem – either A2A, B2B or B2C – can be reduced to one of three basic forms of integration. Real business situations frequently require a mix of one or more of these.

Data Consistency
The aim of Data Consistency integration is for redundant data in multiple systems to agree with each other. The application systems do not work on a common business process and maintain a loosely coupled relationship.

Multi-Step Process
A Multi-Step Process integration addresses many business needs and can involve either human intervention or re-entering data, or it could be a seamless throughput of data. In either case the systems are logically independent as each step is driven by the work of another system earlier in the process.

Composite Application
Composite applications are rapidly growing in popularity as enterprises seek to provide more customer-focused marketing and service. A composite application generally appears to the end user as a new Web or client/server application but behind the scenes, it may invoke one or more mainframe transactions or calls to packaged Unix or NT applications. Interactive, composite applications represent the most tightly knit integration pattern.

Further EAI key technologies
We can now take a look at the EAI technologies that address the above three integration problems. Message or integration brokers are the most recent components of EAI technology. An integration broker is more than a communication layer (e.g. Messageoriented middleware), it provides value-added services from the application layer.

Integration brokers provide two application level services. These are transformations in order to translate message or file contents and flow automation or control. To enable these two services, a broker has some form of repository that holds metadata descriptions of the input and output message formats and the transformation and routing rules. EAI vendors should offer some related application-specific or technical adapters, along with development tools, gateways and templates for connecting to packaged applications.

An integration broker should contain the following features:

Transport and Messaging – Publish / Subscribe, Message Queuing, Request / Reply
Adapters
Messaging Management and Monitoring
Transformation including XML support, 1-N, N-n transformations
Routing / Workflow – including Business Process automation and workflow
Deployment Management and Monitoring
Integrated Development environment

All of the above are advantageous when integrating spatial systems since these systems will typically be involved in composite application integration – the most demanding form of integration requires the most advanced tools.

The required level of complexity of system integration needs to be supported by the type of Messaging / Transport layer. EAI is often associated with asynchronous (send and forget) messaging however synchronous (request/reply) communications are also important for composite applications. Asynchronous messages are often published on channels and applications can subscribe to certain channels to receive messages of interest (publish/subscribe). Synchronous communication occurs in object oriented message brokers such as the Common Object Request Broker Architecture (CORBA) and Microsoft’s Common Object Model (COM).

XML is a key mechanism for exchanging information in the following situations:

Web site to Web site, e-Business requires seamless integration and transfer of content between disparate web sites
Application to Web Site, interoperation between web-based documents and core business applications is becoming more important. The use of XML and messaging capabilities will enable applications of any type to perform business transactions and conform to existing business process at each location.
Application to Application Integration, XML can provide benefits for streamlining business processes and replacing proprietary APIs for data input and export.

Comparison of EAI to database table level integration
Thankfully the world has moved on from the idea that full integration can be achieved merely by holding all data in tables in the same database. Utility companies are moving towards choosing ‘Best of Breed’ applications which support their business processes. Mainstream databases offer a variety of products to use heterogeneous data residing in databases, file systems or legacy systems.

For example, Oracle are now positioned in the EAI market offering a wide-ranging solution, based on the Oracle Internet platform, to solve e-Business access, application and data integration problems for enterprises. The Oracle Internet platform consists of clients, application servers, databases, integration services, network services and management and development tools.

However, it is important that companies also devise a comprehensive data management strategy. The preferred option is to store key data in a database to maximize manageability, access and security. For e-Business, a consistent data management strategy is the vital factor to successful enterprise portal implementations and successful application integration.

Integration Brokers and Application Servers
Integration brokers and application servers are complementary and converging. Application Servers are for sharing and processing application logic and also connections to back-end resources (including databases and ERP applications) and provide a mechanism to deploy applications on Web platforms. Most application servers use Enterprise Java Beans (EJB) - specialized Java Beans that run on a remote Server. EJBs act similar to distributed objects such as COM and CORBA. Application servers are front- end and application development focussed, integration brokers are back-end and operations / process oriented.

The criteria to best determine long term EAI architecture
The following items are key criteria in evaluating how EAI can best match current and future business considerations:

development of and conformance to emerging standards
ability to support business process and workflow
the required features in an integration / message broker which will provide comprehensive support for the three types of integration - data consistency, multi-step process and composite applications.

It is beyond the scope of this paper to fully cover emerging standards, for more information refer to Sileo (1999) and Batty (1999). Utility organizations are investing effort in defining standards for integration as well as communication protocols and standard business practices. This will bring benefits over the long term as any investment in standards for the semantic layer provides substantial ‘future-proofing’ value. In summary, the International Electromechanical Commission’s (IEC) Technical Committee 57 on Power System Control and Associated Communications includes two working groups focused on developing standards for utility integration requirements. WG13 is developing standards for Energy Management Systems (EMS) integration, while WG14 is looking at standards for Distribution Management Systems (DMS). WG14 is developing the concept of a Utility Integration Bus (UIB) protocol. This protocol could be implemented on many of the emerging integration platforms.

There are also emerging standards for business process and workflow. In summary, on a business process level, various energy industry groups have joined forces to develop uniform business practices for electronic interaction. Utility business processes such as energy trading, outage management, customer service orders, construction work management, transmission capacity planning, metering data management, customer billing, and countless others need to be automated and integrated with an e-Business platform. The emerging standards for business process and workflow will aid this automation and integration process.

EAI vendors are providing an extensive set of features and functionality with integration and message broker technology. As a minimum the following items should be considered essential for long term solutions.

The integration broker should provide transformation and flow automation services. Transformation means messages are syntactically converted or transformed. Flow automation means the broker will implement some routing logic.
A broker should have a metadata repository to maintain data descriptions.
A broker should have development tools and administration and monitoring services.
Brokers should aim to provide business process management (e.g. workflow) and adapters to packaged software product suites (such as SAP, Peoplesoft, Seibel).

Whilst it may not be necessary to utilize all aspects of these features in the first phase of an integration plan – overall these features are essential for e-Business.

A further criterion when selecting EAI technology is the vendors themselves, in particular their positioning in the marketplace. Gartner Group (Pezzini, 2000) classifies integration broker vendors into the following four categories: Challengers, Niche Players, Leaders and Visionaries. Vendors are rated in terms of their overall vision and ability to address all three major forms of integration – data consistency, multi-step process and composite applications. This classification does not indicate hard and fast selection criteria but is a useful indication that the vendor you choose meets your needs. Vendors that appear in Gartner’s Leaders category are IBM, Mercator and Tibco. Vendors that appear in the Visionaries category are Neon, Vitria, BEA, and Oracle. Microsoft is currently classified in the Challengers category. The ability of these vendors to support the three types of integration forms part of the criteria for meeting the aims of long term solutions.

Summary and Conclusions: With EAI, are long term solutions relevant?
A short period of time ago, it would have been impossible to predict the dramatic shift in business drivers and technology brought on by the Internet. The pace at which new technology seems to make yesterday’s solutions obsolete can be both exciting and humbling. By extension, one can cynically reason that planning IT strategies that require return on investment (ROI) spread over more than three to four years will not take into account the next technology revolution. There is certainly a lesson to be learned in that observation. However, the approach taken by Company Z in our example clearly demonstrates that EAI provides safe foundations for the future. They have implemented an e-Ready solution using EAI technology and extended the life of their legacy systems.

It is worth pointing out that overall costs – software, system integration, infrastructure, and customization - are still considerable for EAI technology. However, there are mitigating features of EAI technology. The flexibility to incorporate powerful best of breed application solutions is one of the most potent ways for an organization to get rapid ROI. Spatial technologies are an excellent example of this. In addition, there is the promise of dramatic reduction in the overall cost of ownership of software removing expensive point-to-point solutions. Moving an organization from having a web presence (Company X) to full e- Readiness (Company Z) is a clear advantage.

EAI technology has a demonstrable critical mass of future-proofed features that will still make sense despite shifting business drivers and even more rapidly shifting innovations in technology. The concepts of integration bus, integration broker, transformation, business process, workflow, messaging mean that legacy systems can be easily extended to take significant roles in the most challenging and innovative composite applications for e- Business.

In one form or another, EAI technology will be very significant for some time to come.

References

Batty, P.M., 1999: Technologies for uniting the Enterprise. GITA conference proceedings.
Brown, Tony M., 2000: The Application Archipelago, An Overview of Enterprise Integration. EAI Journal, January 2000.
Brown, Tony M., 2000: Twenty Questions for Active Software’s Jim Green. EAI Journal, June 2000.
Dwyer, Tom, 2000: e-Business Integration Drives EAI. EAI Journal, July/August 2000
Estes, Don, 2000: Disenfranchising Middleware. EAI Journal, November / December 2000.
Kador, John, 2000: Wired for Business, Five Successful e-Business Models. EAI Journal, July / August 2000.
Linthicum, David S., 1999: Enterpise Application Integration. Addison Wesley, 1999.
Pezzini, Massimo, 2000: Selecting Integration Brokers (…by not only tossing the coin). Gartner Group Briefing. 2000
Sileo, Tony, 1999: Business Process Integration, A must for e-Business. GITA conference proceedings.
Waters, Guerry, 2000: Energy Information Strategies. Meta Group, July 2000.

¹ ‘Legacy’ has many connotations for people. In this paper, it is used to mean, although based on previous generation technology, a working, operational system.