Search Paper
  • Home
  • Login
  • Categories
  • Post URL
  • Academic Resources
  • Contact Us

 

Data integration in a Hadoop-based data lake: A bioinformatics case

google+
Views: 41                 

Author :  J´ulia Colleoni Couto

Affiliation :  PUCRS University

Country :  Brazil

Category :  Data Mining

Volume, Issue, Month, Year :  12, 4, July, 2022

Abstract :


When we work in a data lake, data integration is not easy, mainly because the data is usually stored in raw format. Manually performing data integration is a time-consuming task that requires the supervision of a specialist, which can make mistakes or not be able to see the optimal point for data integration among two or more datasets. This paper presents a model to perform heterogeneous in-memory data integration in a Hadoop-based data lake based on a top-k set similarity approach. Our main contribution is the process of ingesting, storing, processing, integrating, and visualizing the data integration points. The algorithm for data integration is based on the Overlap coefficient since it presented better results when compared with the set similarity metrics Jaccard, Sørensen-Dice, and the Tversky index. We tested our model applying it on eight bioinformatics-domain datasets. Our model presents better results when compared to an analysis of a specialist, and we expect our model can be reused for other domains of datasets.

Keyword :  Data integration, Data lake, Apache Hadoop, Bioinformatics

Journal/ Proceedings Name :  International Journal of Data Mining & Knowledge Management Process (IJDKP)

URL :  https://aircconline.com/ijdkp/V12N4/12422ijdkp01.pdf

User Name : tania
Posted 01-10-2026 on 17:32:46 AEDT



Related Research Work

  • Analyzing The Effects Of Different Policies And Strictness Levels On Monthly Corona Virus Case
  • Linking Early Detection/treatment Of Parkinson’s Disease Using Deep Learning Techniques
  • Demand Forecasting Of A Perishable Dairy Drink: An Arima Approach
  • Class Imbalance Handling Techniques Used In Depression Prediction And Detection

About Us | Post Cfp | Share URL Main | Share URL category | Post URL
All Rights Reserved @ Call for Papers - Conference & Journals