What is Data Integration?
Data integration is the process of bringing data from disparate sources together to provide users with a unified view. The premise of data integration is to make data more freely available and easier to consume and process by systems and users. Data integration done right can reduce IT costs, free-up resources, improve data quality, and foster innovation all without sweeping changes to existing applications or data structures. And though IT organizations have always had to integrate, the payoff for doing so has potentially never been as great as it is right now.
Companies with mature data integration capabilities have significant advantages over their competition, which includes:
Increased operational efficiency by reducing the need to manually transform and combine data sets
Better data quality through automated data transformations that apply business rules to data
More valuable insight development through a holistic view of data that can be more easily analyzed
A digital business is built around data and the algorithms that process it, and it extracts maximum value from its information assets—from everywhere across the business ecosystem, at any time it is needed. Within a digital business, data and related services flow unimpeded, yet securely, across the IT landscape. Data integration enables a full view of all the information flowing through an organization and gets your data ready for analysis.
The Evolution of Data Integration
The scope and importance of data integration has completely changed. Today, we augment business capabilities by leveraging standard SaaS applications, all while continuing to develop custom applications. With a rich ecosystem of partners ready to leverage an organization’s information, the information about an organization’s services that gets exposed to customers is now as important as the services themselves. Today, integrating SaaS, custom, and partner applications and the data contained within them, is a requirement. These days, an organization differentiates by combining business capabilities in a unique way. For example, many companies are analyzing data in-motion and at-rest, using their findings to create business rules, and then applying those rules to respond even faster to new data. Typical goals for this type of innovation are stickier user experiences and improved business operations.
How does data integration work?
One of the biggest challenges organizations face is trying to access and make sense of the data that describes the environment in which it operates. Every day, organizations capture more and more data, in a variety of formats, from a larger number of data sources. Organizations need a way for employees, users, and customers to capture value from that data. This means that organizations have to be able to bring relevant data together wherever it resides for the purposes of supporting organization reporting and business processes. But, required data is often distributed across applications, databases, and other data sources hosted on-premises, in the cloud, on IoT devices, or provided via 3rd parties. Organizations no longer maintain data simply in one database, instead maintaining traditional master and transactional data, as well as new types of structured and unstructured data, across multiple sources. For instance, an organization could have data in a flat-file or it might want to access data from a web service. The traditional approach of data integration is known as the physical data integration approach. And that involves the physical movement of data from its source system to a staging area where cleansing, mapping, and transformation takes place before the data is physically moved to a target system, for example, a data warehouse or a data mart. The other option is the data virtualization approach. This approach involves the use of a virtualization layer to connect to physical data stores. Unlike physical data integration, data virtualization involves the creation of virtualized views of the underlying physical environment without the need for the physical movement of data. A common data integration technique is Extract Transform and Load (ETL) where data is physically extracted from multiple source systems, transformed into a different format, and loaded into a centralized data store.
Considerations for Improving Simple Integration
The value gained from implementing data integration technology is, first and foremost, the cost of no longer having to manually integrate data. There are other benefits as well including the reduction from avoiding custom coding for the integration. Organizations whenever they can should look to use an integration tool provided by a vendor rather than write custom integration code. Reasons for doing this are a) improved data quality b) optimal performance c) time savings.
Organizations could derive much greater value by adding the following additional goals to their integration maturity roadmaps.
留言