Data Mining In The Financial Sector

Background of Data Mining

Data mining is carried out for the purpose of discovering valuable patterns and information from data warehouses or databases (Larose 2014). Financial data plays a significant role in the financial sector for analyzing consumer data in order to find legitimate customers such as the debtors. Data mining has become an integral part of the financial sector as it helps in predicting the future behaviour and trends in the financial markets.

This report gives an overview of the concept of data mining. It discusses the various types of methods or operations of data mining in the financial industry. It also examines and analyzes some of the challenges of data mining in the financial sector. This report gives a brief overview of the future developments that can be done in the banking sectors, investment sectors and other financial institutes by applying data mining techniques and methods.

Data mining is known to be a process that extracts hidden knowledge from large databases that contain raw data (Witten et al. 2016). It can also be defined as the science of extracting valuable information or discovering knowledge from data warehouses (Larose 2014). Knowledge is discovered in the data mining process. There are various steps in data mining process that follows an iterative sequence. The steps of data mining process are:

Name of the step	Analysis Save Time On Research and Writing Hire a Pro to Write You a 100% Plagiarism-Free Paper. Get My Paper
1. Learning	This step focuses on learning and getting prior knowledge about the application domain. The goals and objectives of the application need to be learnt before carrying out the process of knowledge or information discovery.
2. Creating of target dataset	This step focuses on creating and identifying a target dataset. Selection of the dataset must be done correctly in order to apply data mining processes and methods for discovering knowledge.
3. Data cleaning	This is a pre-processing method that carries out basic operations like removal of noise. This stage focuses on removing inconsistent data and errors (Preethi and Vijayalakshmi 2017).
4. Data projection	Data projection is carried out in this stage. This step focuses on discovering valuable features for the purpose of data representation.
5. Data mining function	This step focuses on taking a decision regarding the objective of the data mining model derived. Data mining function is selected in this step.
6. Data mining algorithm	This step focuses on selecting a method for searching for data patterns.
7. Data patterns	This step includes classification trees or rules, clustering, regression, line analysis and sequence modeling for finding out data patterns.
8. Pattern interpretation	The discovered patterns of data are interpreted to find a meaning and the redundant data or patterns are removed in this step.
9. Discovered knowledge	In this step, knowledge is incorporated in the performance system and actions are taken based on the knowledge that is discovered.

Table 1: Steps in Data Mining Process

(Source: Larose 2014, p. 56)

Data mining is gaining importance in almost every sector of the financial industry for the purpose of analyzing data and summarizing the discovered data into valuable information (Charliepaul and Gnanadurai 2014). The main target of the banking sector is to retain customers (Chitra and Subashini 2013). This process is used for analyzing customer details for identifying their tastes and preferences in order to develop new strategies for customer retention. Data mining tools can facilitate automated discovery of unknown patterns and automatic future trend prediction. Data mining is needed in the financial sector for forecasting bank bankruptcies, stock market, and exchange rate of currency, credit rating, money laundering analysis and loan management. Royal Bank of Scotland Group uses data mining in the loan management process to determine the loan risk.

Different Methods and Techniques covered by Data Mining

According to Raval (2012), the four main techniques of data mining are:

Association: This technique focuses on discovering patterns based on the relationship of a specific item with other items that are present in same transaction. Association method can be used in market analysis for identifying the products and services that consumers tend to purchase together. Boolean and quantitative association rules are based on the types of data that are handled by the process. Single-level and multilevel association rules are applied when abstraction levels are involved.

Clustering: This technique is used for identifying objects belonging to similar classes. It can be used as a pre-processing approach for combining similar objects into a single group (Charliepaul and Gnanadurai 2014). An algorithm called k-means algorithm can be used for partitioning data. Financial sectors can use this method to segment the demographic data into groups and clusters. Vaishali (2014) showed how clustering approach can be used for fraud detection in the credit cards of the consumers. Clustering methods can also be used for fraud detection purposes. Naïve Bayes algorithm is also used in the clustering techniques.

Prediction: Regression analysis is used for forecasting and predicting. This technique is useful for discovering patterns and knowledge. Decision trees and neural networks are also used for predicting future values and data. Financial sectors can use this technique for customer retention.

Classification: This method is used for classifying large volume of data. Decision tree algorithm plays a significant role in classifying data. Classification methods can be used for credit approval systems. Decision tree algorithm applies if-then-else rules to predict the future values in the method of classification (Kadam and Raval 2014).

Data mining facilitates a financial institute to analyze customer data and behaviour for the purpose of predicting future trends and activities (Charliepaul and Gnanadurai 2014). Data mining process helps to compare and analyze the change between customer behaviours between two consecutive months. Data mining techniques can be applied to segment the market based on similar characteristics of the customers. It helps to analyze the market trends for predicting the behaviour pattern of the customers. Data mining plays a broad role in cross selling and direct marketing (Pavlovic, Reljic and Jacimovic 2014). Banking industry utilizes data mining techniques for minimizing risk (Miyan 2017). It helps in distinguishing reliable borrowers from non reliable borrowers.

Data Mining Applications in the Financial Sector

Data mining has several applications in the financial sector such as:

1) Customer retention: Financial Institutes collect and analyze customer information like income status and expenditure. Analysis of the customer status helps the financial institutes to offer additional services for customer retention (Baumann, Elliott and Burton 2012). Financial institutes will be able to target new customers by using data mining techniques. Classification methods are used for the purpose of retaining customers and getting new customers.

Decision tree is a graphical representation that shows the relationships among different variables.  It solves classification as well as prediction problems. The customers are classified into two groups: risky and safe group. Value prediction models and methods can be used for predicting the default amount for the application of loan. Regression method plays a significant role in this case. Clustering techniques help in carrying out customer profiling. k-Means technique can be used segment the customer profession into various clusters and identify the customer need so that banks and financial institutes can fulfil their requirements.

2) Fraud detection: Credit card frauds as well as insurance frauds are the most common types of fraud activities that take place in the financial sector. It is important to distinguish between legitimate and fraudulent activities (Sharma and Panigrahi 2013). SPV or support vector machines techniques are used for getting accurate result. Clustering methods along with probability estimation methods can be used for the purpose of detecting fraud activities. Classification methods are also applicable for classifying the fraud activity.

3) Market forecasting: Data mining techniques can be used for predicting any crisis such as fall in the stock price that will occur in the future by using Bayesian network. Financial institutes can analyze past data and trends and determine present demand. These techniques can also be used for predicting customer behaviour in the future by analyzing the historical purchasing details of the customers.

Data mining is considered to be powerful but it also has several challenges and loopholes. These challenges can be related to data methods and performance. Some of the challenges are discussed below.

1) Incompleteness and heterogeneity: The data stored in the databases have different patterns and rules like email, images and pdf documents. Transformation of the heterogeneous data into a structured form is a challenge in data mining (Airccj.org 2017).  Incomplete or missing data gives inaccurate results during the analysis process. Carrying out appropriate analysis is a major challenge in the process of data mining (Verma and Nashine 2012).

Different Data Mining Applications and Methods Used – Case Studies

2) Complexity: It is a challenging issue to manage large volumes of data. Data analysis, retrieval, modelling and data organization are a big challenge because of the complex nature of the data.

3) Timeliness and privacy: Data analysis consumes more time. In some cases immediate analysis results are required. For example, detection of fraudulent transaction must be suspected immediately. But scanning of entire data set requires more time. Privacy is a major issue in data mining. Customer details are analyzed by the banks and financial institutes without their consent which leads to privacy issues. For example, there can be unethical hacking of sensitive customer data.  

 Relational data mining can solve financial problems and bring great advancements in the future for the financial applications (Paidi, 2012). Future development of data mining will be to develop decision support tools for making the operations for financial tasks easier. APIs such as Braintree API allows customers to accept payments via the payment gateways of Braintree.

Conclusion 

This report concludes that data mining plays a significant role in the financial industry such as banking and investment sectors. Data mining techniques like classification and clustering methods have been discussed in this report. Applications of data mining in the finance sector have also been discussed in this report. This report examines the data mining methods used in these applications. According to this report, the main data mining application in the financial sector are customer retention, market analysis and fraud detection. The main challenges in data mining are incomplete and complex data. It can be concluded from this report that the financial sector will become more competent and gain competitive advantage by adopting data mining techniques.  

References

Airccj.org.,2017. Issues, Challenges, and Solutions: Big Data Mining. [online] Available at: https://airccj.org/CSCP/vol4/csit43111.pdf [Accessed 21 Nov. 2017].

Baumann, C., Elliott, G. and Burton, S., 2012. Modeling customer satisfaction and loyalty: survey data versus data mining. Journal of services marketing, 26(3), pp.148-157.

Charliepaul, C. and Gnanadurai, G. , 2014. A Detailed Review on Data Mining in Finance Sector. International Journal On Engineering Technology and Sciences – IJETS™, 1(3), pp.124-131.

Chitra, K. and Subashini, B., 2013. Data mining techniques and its applications in banking sector. International Journal of Emerging Technology and Advanced Engineering, 3(8), pp.219-226.

Kadam, S. and Raval, M., 2014. Data Mining in Finance. International Journal of Engineering Trends.

Larose, D.T., 2014. Discovering knowledge in data: an introduction to data mining. John Wiley & Sons.

Miyan, M., 2017. Applications of Data Mining in Banking Sector. International Journal of Advanced Research in Computer Science, 8(1), pp.108-114.

Paidi, A., 2012. Data Mining: Future Trends and Applications. International Journal of Modern Engineering Research (IJMER), 2(6), pp.4657-4663.

Pavlovic, D., Reljic, M. and Jacimovic, S., 2014. Application of data mining in direct marketing in banking sector. Industrija, 42(1), pp.189-201.

Preethi, M. and Vijayalakshmi, M., 2017. Data Mining In Banking Sector. International Journal of Advanced Networking & Applications, 8(5), pp.1-4.

Raval, K.M., 2012. Data Mining Techniques. International Journal of Advanced Research in Computer Science and Software Engineering, 2(10).

Sharma, A. and Panigrahi, P.K., 2013. A review of financial accounting fraud detection based on data mining techniques. arXiv preprint arXiv:1309.3944.

Vaishali, V., 2014. Fraud Detection in Credit Card by Clustering Approach. International Journal of Computer Applications, 98(3), pp.29-32.

Verma, D. and Nashine, R., 2012. Data Mining: Next Generation Challenges and FutureDirections. International Journal of Modeling and Optimization, pp.603-608.

Witten, I.H., Frank, E., Hall, M.A. and Pal, C.J., 2016. Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann.