Preface

By John Schmidt

I have been practicing Lean Integration for over 15 years. I just didn’t know it.

Over the past 30 years I’ve learned various Lean practices such as kaizen (continuous improvement) and closely related methods such as Six Sigma and agile software development. But I didn’t fully understand Lean as a system until I studied it more thoroughly in 2008. The surprising discovery was that I had in fact been applying Lean principles for many years without formally identifying them as such.

The reality is that Lean principles such as customer intimacy, continuous improvement, seeking quality perfection, and developing effective teams are common sense. David and I don’t have a monopoly on common sense, so we expect that as you read this book you will see examples of Lean principles in your own life. We hope that this book does at least two things for you: first, that it enriches your own experiences with more examples, case studies, and practical advice so that you can improve your level of competence; and second, that it provides a more complete management “system” that makes all these commonsense ideas teachable so that you can improve the level of competence of your team and throughout your value chain.

Contrary to common practices I have always viewed integration as a repeatable ongoing process rather than as a custom one-off project effort. While an integration point between any two systems always looks unique from the perspective of the two systems, if you step back and look at all the information exchanges between systems in an enterprise, what you find is a relatively small number of patterns that are repeated over and over again. The details are different for every instance (at a minimum the data itself that is being exchanged is unique) but the patterns are not.

I have been fortunate over the years in my work at the Integration Consortium, Wells Fargo Bank, Bank of America, Best Buy, American Management Systems, and Digital Equipment Corporation to have the opportunity to help hundreds of companies implement integration practices. The first successful Integration Factory I implemented (several prior attempts didn’t quite take off) was an exercise in mass customization—building customized message adapters on top of vendor platforms with reusable components based on a service delivery model that focused on customer needs. The next successful Integration Factory encompassed not only real-time application-to-application integration, but also high-volume batch-oriented database-to-database integration, external business-to-business (B2B) managed file transfer, and business process integration. The factory also included an internal Web application that allowed users (customers of the integration team) to interact with the factory through a series of role-based user interfaces.

Through these and hundreds of other experiences over the years I have a developed a perspective on what it takes to implement sustainable integration practices. The first book on which David and I collaborated, Integration Competency Center: An Implementation Methodology,¹ articulated the concepts that make the difference between a successful and an unsuccessful integration strategy. This book takes Integration Competency Centers (ICCs) to the next level by adding more specific best practices and a rich collection of case studies, and by leveraging the vast body of knowledge that has developed over the past 50 years on Lean practices. The net result is sustainable integration that begins to turn an art into a science by making it teachable and repeatable.

1. John G. Schmidt and David Lyle, Integration Competency Center: An Implementation Methodology (Informatica Corporation, 2005).

By David Lyle

One of the most rewarding professional periods of my life was leading a remarkable team of developers who put together packaged analytic applications to sell as software products about ten years ago. Little did I realize that our team, at the high-water mark numbering over 80 people, was employing all of the Lean Integration principles discussed in this book while putting together what John and I now describe as an Integration Factory. Like John, I didn’t realize we were employing Lean thinking; I just thought we were continually benefiting from people’s innovative ideas to work smarter, not harder.

Our development team made the realization that the integration logic in those analytic applications followed a relatively small number of integration or processing patterns. Because the time spent on the design, implementation, and testing of integration was such a large proportion of the total development costs, the team realized we became significantly more effective if we thought in terms of developing assembly lines around these integration patterns rather than crafting all integration logic as “unique works of art.” By changing our entire approach over the course of four years, we became far more efficient as a development organization, but most important, we developed higher-quality, more maintainable products for our customers.

Over the past several years, John and I have worked at or talked with numerous companies around the world about how to develop and grow their ICCs. We found several ICCs achieving great success by automating certain processes, using mass customization techniques, or adopting agile development approaches. Several we call out explicitly as case studies in this book, but many of the ideas in this book are the products of conversations and achievements of numerous integration professionals we’ve spoken with over the years. In other words, besides our own experiences, John and I have seen others be successful with many of the ideas we’re pulling together in this book.

That being said, we don’t mean to ever imply that adopting these ideas is easy. Integration is an especially complex, challenging problem, both technically and organizationally. All companies have had to spend significant time continually convincing executive management of the benefits of attacking integration as a discipline that is part of the overall enterprise architecture, rather than as a temporary exercise that is unique to each project. Most IT executives are less aware of the detailed costs of integration or benefits of ICCs.

Lean Integration and the Integration Factory are neither destinations nor vendor products; implementing them is a journey that takes many years. Despite the fact that John and I both now work for Informatica, we worked hard to make this book vendor-neutral. We wanted the book to be broadly useful to integration professionals rather than to be based on a specific vendor’s software offering. Successful ICCs are a product of the synergy of good people, effective processes, and appropriate technology. This book represents what we’ve learned over the years from so many people about how Lean thinking can make ICCs significantly more efficient and effective for their customers.