What is system integration?
Updated December 13, 2022
System integration is essential both for business-to-business communication and internal cooperation within an enterprise. As an iPaaS provider, system integration is something we do on a daily basis.
In this blog, we will explain what system integration is, what types and methods have traditionally been used for implementation, what the challenges are, and how an integration platform with its hybrid capabilities can help enterprises to develop and deploy integrations between their systems.
Table of contents:
What is system integration?
In very broad terms, system integration is the process of connecting different sub-systems (components) into a single larger system that functions as one. With regard to software solutions, system integration is typically defined as the process of linking together various IT systems, services and/or software to enable all of them to work functionally together.
The main reason for organizations to use system integration is their need to improve the productivity and the quality of their operations. The goal is to get the organization’s various IT systems to “talk to each other” through the integration, to speed up information flows and reduce operational costs for the organization. But system integration is not used only to connect an organization’s internal systems but also to third parties that the organization operates with.
Now that we have an understanding of what system integration is, let’s take a look at the different types and methods of integration you can come across and leverage.
The most common types of system integration
There are a lot of different system integration types can be found, but the most common are the following:
Data integration
Data integration is the process of gathering data from disparate sources (e.g., services, platforms, and databases) together to provide businesses with a centralized access point making data more available and easy to process/consume by systems and their users.
Business-to-Business integration
As the name suggests, this type of integration enables companies to automate business communication and processes with all stakeholders to exchange vital for their business data more efficiently. So, with the B2B integration, organizations facilitate real-time data exchange with their business partners.
Legacy system integration
Legacy system integration implies connectivity between outdated legacy systems (still vital for core business) with more modern IT infrastructure to ensure seamless communication between the components and avoid disrupting a company’s day-to-day workflow.
Electronic document interchange (EDI)
Electronic Data Interchange (EDI) is the system-to-system exchange of business documents in a standard electronic format between business partners intended to replace paper-based documents (e.g., invoices or purchase orders). Hence, this type of integration is vital for companies looking to enable paper-based business transactions to save time and avoid costly errors resulting from manual processing.
Enterprise application integration (EAI)
Enterprise application integration is all about bridging gaps between different software programs and applications (ERP, CRM, SCM, etc.) to streamline business processes through smooth data flow from one point to another.
System integration methods
As for typical system integration methods, they can be divided into the following different categories:
- Point-to-Point Integration
- Vertical Integration
- Star Integration
- Horizontal Integration
- Common Data Format integration
Now that we have all the methods listed, let’s discuss each one in detail to understand what they are and how they can be leveraged.
Point-to-Point Integration
One could argue that a point-to-point integration (or point-to-point connection) is not a system integration as such since there are only two system components involved. However, while it lacks the complexity of “true” system integration, it still connects a system to another system for them to function together. Typically, such point-to-point integration handles only one function and does not involve any complex business logic. Many cloud-based applications offer these types of point-to-point integrations as productized, “out of the box” integration modules for the most common IT systems.
Vertical Integration
In vertical integration method, the system components (sub-systems) are integrated by creating functional “silos”, beginning with the basic bottom function upward. This is normally a relatively simple and easy method that only involves a limited number of systems (more than two), but on the other hand, this integration method is quite rigid and more difficult to manage in the long term as any new functionality will require its own functional ”silo”. Still, this method can be used effectively to create simple integrations that only need to address a single function.
Star Integration
Star integration means a system where each sub-system is connected with other sub-systems using point-to-point connections. This allows for more functionality, but as the number of integrated systems increases, the number of integrations also increases significantly, and the management of the integrations becomes very demanding. As an example, connecting ten systems to each other using this method will require 45 separate integrations, and every time there is a change in one system, nine connections may need to be re-done as well. Sometimes Star Integration is there for also referred to as “Spaghetti Integration” as an analogy to “Spaghetti code”.
Horizontal Integration
In horizontal integration, a separate sub-system is used as a common interface layer between all sub-systems. Very often, this layer is referred to as an Enterprise Service Bus (ESB). This method allows each sub-system to have just one single interface to communicate with all the other sub-systems connected to the common interface layer (i.e., with ten systems, there are only ten connections). The benefit of this method is also that each sub-system can be changed or even replaced without having to redo the interfaces of any other systems.
Common Data Format integration
Integrating different IT systems into each other usually requires that the data coming out from one system needs to be transformed into a different data format used by the receiving system. As with the Star Integration, if each transformation needs to be done system-by-system basis, the number of data transformations increases significantly and becomes a high-maintenance task. To overcome this problem, the common data format approach allows each system to do only one data conversion from its native format to the common (and vice versa). This way, the number of required data transformations is just as high as the number of the sub-system.
What is the System Integrator’s role?
On broad terms in the IT world, a Systems Integrator (“SI”) is regarded as a company that specializes in implementing, planning, coordinating, scheduling, testing, improving and sometimes maintaining IT systems. Good examples of SIs are, e.g., Deloitte, IBM, Accenture, TCS, etc. They deliver large IT projects (e.g., ERP projects) trying to manage such projects and the numerous suppliers involved. However, in terms of system integration, the role of the systems integrator is narrowed down to enabling the data integrations between the different existing systems of the end customer defined in the project scope. This may mean anything, starting from simple internal point-to-point connections to very complex many-to-many integrations both internally and with third parties.
The systems integrator’s role in this equation is normally to design, implement and test the integration solution, but the role of the systems integrator may also include continuous management of the solutions as well as contacting third parties to enable connections with them. Most importantly, however, the systems integrator brings to the table integration expertise that the customer is otherwise lacking internally (or has a shortage of available internal resources at hand).
Why is B2B integration more relevant than ever?
Business-to-Business integration is by no means a new concept. Some IT-driven companies started implementing B2B Integration projects almost 50 years ago (and hopefully, by now, some of them have even completed these projects…). B2B Integration basically means the integration, automation, and optimization of business processes that extend outside the company firewall. While these processes may vary significantly between them, the one common thing that they share is that the integration of such external business processes delivers a sustainable competitive advantage to the organization. Such advantages may include, e.g., real-time visibility, improved automation, inventory optimization, and improved customer satisfaction.
Companies have recognized that having good software solutions is simply not enough. They may be using the most functionally-rich software applications within their own firewall (or in the cloud), but without proper B2B connectivity and associated capabilities, they cannot effectively manage, e.g., their end-to-end supply chain process.
While B2B Integration initially began with large enterprises mandating methods of receiving business information, it quite quickly evolved into the adoption of Electronic Data Interchange (EDI) standards and later on to other newer technologies such as XML, JSON, etc. Nowadays, it seems that every new application has some type of API that allows integration with such applications. However, this leaves the task of actually integrating such API with other systems, and more often than enough, most companies simply do not know how to make this happen.
System Integration Challenges
System integration is not rocket science by any means, but there are a number of issues that make system integration challenging for all organizations. According to certain studies, up to 70% of all integration projects fail in some aspect. This is probably more common for complex projects, but sometimes even relatively simple system integration projects can run into trouble. Most of the failures are not due to the chosen integration technology itself or technical difficulties with the systems in the scope but due to project and change management issues.
Typical reasons for system integration project failure include, for example:
- Constant changes in the integration landscape
The longer the project takes, the more significant this issue becomes. To manage this risk, time is of the essence, keeping the integration projects short improves the success rate of the project. Further, an agile working methodology that can cater to changing requirements along the way and also after the project is essential for the success of the systems integration. - Lack of skilled resources
System integration requires expertise that is not easy to come by. Having excellent integration technology is not enough if the required expertise is not there. Most companies struggle to find and retain employees with the required skills set for system integration. The best way to tackle this issue is to use an external third-party provider that can bring the needed integration expertise into the table as required, in addition to providing the integration technology. - Lack of accountability
When you’re integrating many different sub-systems, the accountability for the success of the integration becomes blurred very easily. You may have multiple stakeholders (e.g., vendors, system owners, etc.) in the equation, none of whom is responsible for the entire system integration. They only care and handle their side of integration at best, but they will not venture outside their own territory. But integration always has more than one party. So, when something goes wrong, the situation turns very easily to finger-pointing and blaming the other parties, instead of someone “owning” the integration. If a single party handles the system integration project, that party is also (often contractually) responsible for the success of such a system integration project, and there is no ambiguity over accountability.
Legacy System Integration issue
Most companies that have done business for decades have some old, legacy IT systems in use and running on their own on-premise servers. These systems may be essential to the core business of the organization and cannot be replaced easily with a more modern IT system. Integration with such legacy systems may be tricky as they may completely lack any ready-made interfacing capability. Most systems do, however, have the capability to read or write information into a file folder that can the accessed, e.g., through an FTP connection by another system, but sometimes the only way to integrate such sub-systems with other sub-systems is to read and/or write data directly in their database.
A modern integration solution must be able to handle also these types of integration scenarios. Cloud-based iPaaS solutions typically use local, on-premise adapters that provide the needed functionality for these integrations. Such adapters act as an active local “interface” between the passive legacy system (or its database) and the cloud-based iPaaS solution. If needed, additional business rules and other functionality regarding the legacy system integration will be handled in the iPaaS service, keeping the maintenance of such business logic centralized and easy. This way the customer does not need to make any costly changes in its legacy IT systems, but the system integrator can provide the integration logic outside the company firewall.
How can iPaaS and Hybrid Integration Platform (HIP) help overcome integration challenges?
Modern iPaaS and HIP solutions have various features that help overcome the system integration challenges mentioned above. iPaaS solutions combine technology and services into a service-oriented solution where hardware, software, data management and enrichment, and supporting operations are combined into an overall responsive system that can be monitored and controlled centrally from one user interface. iPaaS solutions provide the ability to share integration resources (e.g., mapping libraries) and other information across multiple applications, deploy system improvements flexibly on the fly, and complete system integration projects at a much faster pace than before.
Hybrid integration platforms allow companies to continue running their core business process on their legacy systems while they can flexibly connect them with complimenting non-core business processes that may run on a cloud application and may change more often. A hybrid integration platform also enables companies to develop their business processes with new technologies, such as IoT and Blockchain, without having to touch their legacy systems. The required new business logic can be built into the system integration layer that is then connected with the legacy systems.
Most importantly, however, an end-to-end iPaaS solution provides the organization with the technical skills and resources needed to deliver the required system integration quickly, effectively and at a lower cost. iPaaS also provides an ongoing development path for the organization to be able to keep up with the ever-changing integration needs of today’s day and age.
Are you interested in learning more about HIP? Check out the post, A Guide to the Hybrid Integration Platform. Also, check out the post on iPaaS: A Comprehensive Guide to learn more about iPaaS.
Also, you can always schedule a virtual session with our integration specialists to ask all the questions you might have regarding system integration in general or specifically for your integration project.
