Integrations
A successful IT ecosystem is characterized by integrations between systems that support processes, data flows, and development efforts.
System integrations (Enterprise Application Integration, EAI) address a wide range of needs. Moving data to data warehouses, real-time synchronization between processes, external interfaces, service bus implementations — a quality cloud-based integration platform handles all of these. The concept of Integration Platform as a Service (iPaaS) refers to a centralized integration platform built with cloud components that manages the flow of an organization’s business data. Typically, an iPaaS solution is centralized, meaning there is a single point through which data passes within the organization. This approach is cost-effective and enables efficient measurement, security, and monitoring of integrations.
While an integration platform is fundamentally a technical component, it should still be developed with business needs in mind. Key considerations include questions related to the reliability, real-time capabilities, and volume of data distribution. A single integration platform can handle everything from IoT event streams to weekly batch jobs, but its structure should be designed based on what delivers the most value to the customer.
Our integration solutions are built on the Azure Integration Services product family (including Azure Logic Apps, API Management, Service Bus, and Event Grid).
The building blocks of an integration platform
Service Bus and Event-Driven Architecture
An integration platform can also function as a service bus, delivering business messages to all interested parties. For instance, when a new order is placed, a message can be sent to the bus without the sender needing to know which systems the message reaches — the bus handles the delivery.
When developed further, this enables an event-driven enterprise architecture, where the core architecture of the organization is based on systems that openly communicate state changes via the bus.
Interface Services and API Management
At its simplest, integration layers consist of predefined data transfer workflows — for example, data retrieval, transformation, and forwarding.
However, more and more integration implementations are built with some form of API architecture from the outset, enabling data streams to be exposed through application programming interfaces for further use.
Designing interfaces for internal use is generally relatively straightforward and technical, while publishing APIs for partners or even open public use brings its own challenges, including API commitments, data governance, and performance monitoring.
Streaming Data
Business operations are primarily real-time, and so should data streams be. An extreme example of this is streaming data, such as continuous measurement data received from sensors.
Handling such data in an integration layer often requires time-series processing; for instance, most systems do not need temperature data every second — an hourly average may be sufficient for business needs.
A well-designed integration platform also enables analytics and refinement of streaming data, so only the necessary portion of data is delivered to target systems. This approach reduces both system load and costs.
Reporting and Monitoring
Whether it's batch jobs, integration workflows, or API calls, monitoring integration traffic is critical. If data does not flow, the impacts are often both unexpected and costly.
We build monitoring solutions on top of Azure Integration Services and Azure Monitor, providing clear monitoring dashboards as well as, if needed, text message alerts or escalations to an existing service management system (e.g., ServiceNow).
In many environments, the best person to monitor integrations is often the process owner. In such cases, we frequently create separate Power BI-based visualizations for monitoring, enabling even non-technical users to identify basic integration issues.
