Recent Developments In Theory And Practice Of Application And Data Integration

Key System Components

Data Integration is widely used to merge the data easily .If the data takes from different sources. The main purpose to retrieve the data easily and simply. It merge the data from different database. The integration is used for the merging process (Huang and Zhu, 2013). The different techniques are used for this data integration namely enterprise application and process management. It focus to increasing the number of the customer. The system integration is existing. The important role of the integration system is connected the internet. The system integration easily to satisfy the user requirements. Can’t access all the technology, it only access the trending new technology. The system integration is mainly used for the cloud services.

Data Merging and cleansing:

Data Merging is the process, the aim of the process id to combine the different type of the data folder. It combine the more than one file. The database should contain the table stored the data. The cleansing is the one type of detecting the data and correct it. After completing the work, it should be compare the other system.

PETL python package

Petl is a python package index. The following command is used to describe the pip

$ pip install petl

 And to download manually, extract and run by following command

python setup.py install

To verify the installation following command is used

$ pip install nose

$ nosetests –v pet1

We are using the python version 2.7 and 3.4. The UNIX and WINDOWS operating system is used to perform python.

ETL pipelines

Using this package we can easily avoid the lazy evolution and iterations. The pipeline will not execute accurately, until the data is required.

For instance

>>> example_data = “””foo,bar,baz

… a,1,3.4

… b,2,7.4

… c,6,2.2

… d,9,8.1

… “””

>>> with open(‘example.csv’, ‘w’) as f:

…     f.write(example_data)

…

petl.util.vis.look() is a calling function. Using this function easily write the data and files or database.

Following codes are some examples

petl. Io csv. tocsv()

petl.io.db.todb()

Table containers are used to perform the data extraction. Each table contains table containers and table iterations. First we need to accept the requested data otherwise the actual transformation is not done. All the transformations are run using pipeline.

 RESTful Web Services:

It is one type of architecture. It based on the Web Services. RESTful Web Services is contain the resources of the system and organization. It is the type of protocols. It is the open source protocols. And RESTful web services give the access to the web and for the client access and the resources is the kind of object-oriented contents. And the resources are represented in the way of understandability, completeness and link ability (Wang, Shen and Sun, 2013). The client and server should understand the resource and analyze what kind of resource. And the resources should be complete and can be access by others and in the link ability it should have link with others and in any way finally it deliver the xml format. RESTful web services uses the protocol like HTTP for the data communication. And the request and response are delivered in the form of HTTP response. In the request message verb, uniform resource locator and for that request body and header must be needed. And for the response the version of HTTP and the status and for this also the body and header also needed. And the RESTful web services used for the web applications built in different programming languages and it is used to for the applications that can be reside in the platforms like windows and Linux and it provides the flexibility to the web services can easily communicate with others (Maritime over the Horizon Sensor Integration: High Frequency Surface-Wave-Radar and Automatic Identification System Data Integration Algorithm, 2018). And in the RESTful web services addresses should be like use noun as plural and keep the backward compatibility and the usage of lower case letters and ignore the spaces. HTTP protocol performs the REST architecture style. In RESTful design is based on the following concepts.

• Uniform resource identifiers are used. Basically it’s a hierarchal structure based. Each and every resource must have at least one URI.
• To read and manipulate the resources using uniform interfaces
• There are four basic HTTP operations are applicable. They are GET, POST, PUT, and DELETE.
• And also support other operations such as HEAD, OPTIONS
• Using these operations the meat data is easily performed.
• The accepted messages are represented in different formats like HTML, JSON, or XML. So self-descriptive messages are applicable
• It’s also known as stateless interaction. The session date is does not maintain by the stateless interaction.
• That means all the information’s are meaningful including the HTTP message
• For instance resource name and message

Data Merging and cleansing

Mashups:

It is the one type of web application. The mashup is the Greek word comes from British. The overall mashup contains the Web application and Web pages. The main purpose of the mashup are given namely, they are Combination, Aggregation and visualization. The mashup technologies include the gadgets and widgets and JavaScript and supporting languages like css and html and interfaces like active x controls and the java applets and these are all in the presentation kind of mashups (Estridge, Eveleigh and Tanju, 2016). And in data related mashups it includes the data combination and the sites of webpages and the languages like xml and json and this data related or data oriented services also called as data as a service and it is used to provide the application programming interface and the components and in this data mashups the process is vary in the form in and out and in process data mashups it uses the technology like JavaScript and Ajax and it includes the configuration of data retrieving from the servers and about the user interface and in the out process  mashups it is used the technologies like java and python and used to create new data models. And in the process related mashups it includes the functions and the programming languages like python and java and in that the function is used to the inter process communication (Arputhamary and Arockiam, 2015). And the data is exchanged in between the processes and the results are different in each data oriented process. The mashup mechanism is known as mashup logic. The mashup logic is used to tell about how the mashup operators are created and how the mashup components are created. They are used to specify some of operators between the different components. Control flow, data flow, and data mediation, are types of operations performed by mashup tools.

Combine the two files.

Code explanation:

The python program that is “dataMerger.py” is used to merge given content and the files are imported by using the keyword “import” and every attributes in the codes are used to form the tree and the web service side the python program that is “clinics_locator.py” used to search and locate the address in the nearest tab.

 

Figure 1combine two file

Restful web services:

Code Explain:

 Which was carried out to the execution of python files and save the location on .csv files. For show the results of the operation “import csv” was used. For opening the information “clinicopen()”  was used. To read the file we need to use “clinicFileReader()” was used. If length of the  purpose we need to use the “If(len !=row)”. To increase the no of rows we need to use “ScoreList[]=ScoreList[]+row”. For exiting from the file we need to use “ScoreFile.close”.

 

Figure 2Restful web services

To search the Location:

 

Figure 3location search by user

Displayed the Location (Google Map):

Figure 4Display the location in Google Map

The clinics_html file used to show the geolocation and direction of the clinic wants to know. We can able to see the MAP which contains the direction for the clinic location. It very useful show the location of the clinics services location easily.

Conclusion:

Finally , Integrating the information and demonstrations are also performed. The position of the stores in the MAP was identified. The IT structure are mainly used to access the data centers and based on the Functionality of the dependent on the type of the Infrastructure. Growth of the process is slowly increasing and non – dynamic. The techniques are used to compute the responsibilities of the system. The system integration of various data the final required data was recovered. Scalability of the system was ensured by the virtualizing techniques.

Reference:

Arputhamary, B. and Arockiam, L. (2015). Data Integration in Big Data Environment. Bonfring International Journal of Data Mining, 5(1), pp.01-05.

Estridge, C., Eveleigh, T. and Tanju, B. (2016). Integration of System Modeling and Design Processes, Data and Technology for Streamlining Enterprise Integration. Procedia Computer Science, 95, pp.311-318.

Huang, X. and Zhu, W. (2013). An Enterprise Data Integration ERP System Conversion System Design and Implementation. Applied Mechanics and Materials, 433-435, pp.1765-1769.

Maritime over the Horizon Sensor Integration: High Frequency Surface-Wave-Radar and Automatic Identification System Data Integration Algorithm. (2018). Sensors, 18(4), p.1147.

Wang, X., Shen, J. and Sun, C. (2013). Data Warehouse Oriented Data Integration System Design and Implementation. Applied Mechanics and Materials, 321-324, pp.2532-2538.