25 January 2020 SAT-ETL-Integrator: an extract-transform-load software for satellite big data ingestion
Badr-Eddine Boudriki Semlali, Chaker El Amrani, Guadalupe Ortiz
Author Affiliations +
Abstract

Satellite data are used in several environmental applications, particularly in air quality supervising, climate change monitoring, and natural disaster predictions. However, remote sensing (RS) data occur in huge volume, in near-real time, and are stored inside complex structures. We aim to prove that satellite data are big data (BD). Accordingly, we propose a software as an extract-transform-load tool for satellite data preprocessing. We focused on the ingestion layer that will enable an efficient RSBD integration. As a result, the developed software layer receives data continuously and removes ∼86  %   of the unused files. This layer also eliminates nearly 20% of erroneous datasets. Thanks to the proposed approach, we successfully reduced storage space consumption, enhanced the RS data accuracy, and integrated preprocessed datasets into a Hadoop distributed file system.

© 2020 Society of Photo-Optical Instrumentation Engineers (SPIE) 1931-3195/2020/$28.00 © 2020 SPIE
Badr-Eddine Boudriki Semlali, Chaker El Amrani, and Guadalupe Ortiz "SAT-ETL-Integrator: an extract-transform-load software for satellite big data ingestion," Journal of Applied Remote Sensing 14(1), 018501 (25 January 2020). https://doi.org/10.1117/1.JRS.14.018501
Received: 5 September 2019; Accepted: 7 January 2020; Published: 25 January 2020
Lens.org Logo
CITATIONS
Cited by 14 scholarly publications.
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Satellites

Remote sensing

Data storage

Data modeling

Data acquisition

Electroluminescence

Sensors

Back to Top