Brazil Data Cube Workflow Engine: a tool for big Earth observation data processing

References

• Aiordăchioaie, Andrei, and Peter Baumann. 2010. “PetaScope: An Open-Source Implementation of the OGC WCS Geo Service Standards Suite.” In Scientific and Statistical Database Management, edited by Michael Gertz and Bertram Ludäscher, 160–168. Berlin, Heidelberg: Springer Berlin Heidelberg.
   Google Scholar
• The Apache Software Foundation. July, 2022. “Apache Airflow.” Accessed July 20, 2022. https://airflow.apache.org/.
   Google Scholar
• Argo. July, 2022. “Argo Project.” Accessed July 20, 2022. https://argoproj.github.io/.
   Google Scholar
• Bargnesi, Anthony, Anselmo DiFabio, William Hayes, Georgiy Shibaev, Cristophe Benz, Hugh Pyle, Erik Dao, and Travis Giggy. July, 2022. “JSON Graph Format (JGF).” Accessed July 20, 2022. https://jsongraphformat.info/.
   Google Scholar
• Brown, Molly E.. 2016. “Remote Sensing Technology and Land Use Analysis in Food Security Assessment.” Journal of Land Use Science 11 (6): 623–641. https://doi.org/10.1080/1747423X.2016.1195455.
   Web of Science ®Google Scholar
• Camara, Gilberto, Luiz Fernando Assis, Gilberto Ribeiro, Karine Reis Ferreira, Eduardo Llapa, and Lubia Vinhas. 2016. “Big Earth Observation Data Analytics: Matching Requirements to System Architectures.” In Proceedings of the 5th ACM SIGSPATIAL International Workshop on Analytics for Big Geospatial Data, 6. ACM.New York, NY, USA
   Google Scholar
• Carlos, Felipe Menino. May, 2023. “Storm: Platform to Support The Development of Reproducible and Collaborative Geospatial Applications.” Accessed May 15, 2023. http://mtc-m21d.sid.inpe.br/col/sid.inpe.br/mtc-m21d/2023/01.04.23.36/doc/publicacao.pdf.
   Google Scholar
• de Lima, Thainara Munhoz Alexandre, Claudia Giardino, Mariano Bresciani, Claudio Clemente Faria Barbosa, Alice Fabbretto, Andrea Pellegrino, and Felipe Nincao Begliomini. 2023. “Assessment of Estimated Phycocyanin and Chlorophyll-a Concentration From PRISMA and OLCI in Brazilian Inland Waters: A Comparison Between Semi-Analytical and Machine Learning Algorithms.” Remote Sensing15 (5): 1299. https://doi.org/10.3390/rs15051299.
   Web of Science ®Google Scholar
• Ekim, Burak, and Elif Sertel. 2021. “Deep Neural Network Ensembles for Remote Sensing Land Cover and Land Use Classification.” International Journal of Digital Earth 14 (12): 1868–1881. https://doi.org/10.1080/17538947.2021.1980125.
   Web of Science ®Google Scholar
• Elementl. July, 2022. “Dagster – Cloud-Native Orchestration of Data Pipelines.” Accessed July 20, 2022. https://dagster.io/.
   Google Scholar
• FAO. May, 2023. “SEPAL Repository.” Accessed May 7, 2023. https://github.com/openforis/sepal/.
   Google Scholar
• Ferreira, K. R., G. R. Queiroz, R. F. B. Marujo, and R. W. Costa. 2022. “Building Earth Observation Data Cubes on AWS.” The International Archives of the Photogrammetry, Remote Sensing and Spatial Information SciencesXLIII-B3-2022:597–602. https://doi.org/10.5194/isprs-archives-XLIII-B3-2022-597-2022.
   Google Scholar
• Ferreira, Karine R., Gilberto R. Queiroz, Lubia Vinhas, Rennan F. B. Marujo, Rolf E. O. Simoes, Michelle C. A. Picoli, Gilberto Camara, et al. 2020. “Earth Observation Data Cubes for Brazil: Requirements, Methodology and Products.” Remote Sensing 12 (24): 4033. https://doi.org/10.3390/rs12244033.
   Web of Science ®Google Scholar
• Giuliani, Gregory, Paolo Mazzetti, Mattia Santoro, Stefano Nativi, Joost Van Bemmelen, Guido Colangeli, and Anthony Lehmann. 2020. “Knowledge Generation Using Satellite Earth Observations to Support Sustainable Development Goals (SDG): A Use Case on Land Degradation.” International Journal of Applied Earth Observation 88:102068. https://doi.org/10.1016/j.jag.2020.102068.
   Web of Science ®Google Scholar
• Gomes, Vitor C. F., Felipe M. Carlos, Gilberto R. Queiroz, Karine R. Ferreira, and Rafael Santos. 2021. “Accessing and Processing Brazilian Earth Observation Data Cubes with the Open Data Cube Platform.” ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information SciencesV-4-2021 (4): 153–159. https://doi.org/10.5194/isprs-annals-V-4-2021-153-2021.
   Google Scholar
• Gomes, Vitor C. F., Gilberto R. Queiroz, and Karine R. Ferreira. 2020. “An Overview of Platforms for Big Earth Observation Data Management and Analysis.” Remote Sensing 12 (8): 1253. https://doi.org/10.3390/rs12081253.
   Web of Science ®Google Scholar
• Gorelick, Noel, Matt Hancher, Mike Dixon, Simon Ilyushchenko, David Thau, and Rebecca Moore. 2017. “Google Earth Engine: Planetary-scale Geospatial Analysis for Everyone.” Remote Sensing of Environment 202 (2016): 18–27. http://dx.doi.org/10.1016/j.rse.2017.06.031.
   Web of Science ®Google Scholar
• Harenslak, Bas P., and Julian de Ruiter. 2021. Data Pipelines with Apache Airflow. Shelter Island, New York, USA: Manning.
   Google Scholar
• Ivie, Peter, and Douglas Thain. 2018. “Reproducibility in Scientific Computing.” ACM Computing Surveys (CSUR) 51 (3): 36.
   Web of Science ®Google Scholar
• Kamali Maskooni, Ehsan, Hossein Hashemi, Ronny Berndtsson, Peyman Daneshkar Arasteh, and Mohammad Kazemi. 2021. “Impact of Spatiotemporal Land-use and Land-cover Changes on Surface Urban Heat Islands in a Semiarid Region Using Landsat Data.” International Journal of Digital Earth14 (2): 250–270. https://doi.org/10.1080/17538947.2020.1813210.
   Web of Science ®Google Scholar
• Killough, Brian. 2018. “Overview of the Open Data Cube Initiative.” In IGARSS 2018 – 2018 IEEE International Geoscience and Remote Sensing Symposium, 8629–8632.Valencia, Spain
   Google Scholar
• Killough, Brian, Syed Rizvi, and Andrew Lubawy. 2021. “Advancements in the Open Data Cube and the Use of Analysis Ready Data in the Cloud.” In 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, 1793–1795. IEEE.
   Google Scholar
• LabISA. July, 2022. “Mapaquali.” Accessed July 20, 2022. http://www.dpi.inpe.br/labisa/project/mapaquali/.
   Google Scholar
• Lobo, Felipe de Lucia, Gustavo Willy Nagel, Daniel Andrade Maciel, Lino Augusto Sander de Carvalho, Vitor Souza Martins, Cláudio Clemente Faria Barbosa, and Evlyn Márcia Leão de Moraes Novo. 2021. “AlgaeMAp: Algae Bloom Monitoring Application for Inland Waters in Latin America.” Remote Sensing 13 (15): 2874. https://doi.org/10.3390/rs13152874.
   Web of Science ®Google Scholar
• Maciel, Daniel Andrade, Claudio Clemente Faria Barbosa, Evlyn Márcia Leão de Moraes Novo, Nagur Cherukuru, Vitor Souza Martins, Rogério Flores Júnior, Daniel Schaffer Jorge, Lino Augusto Sander de Carvalho, and Felipe Menino Carlos. 2020. “Mapping of Diffuse Attenuation Coefficient in Optically Complex Waters of Amazon Floodplain Lakes.” ISPRS Journal of Photogrammetry and Remote Sensing 170:72–87. https://doi.org/10.1016/j.isprsjprs.2020.10.009.
   Web of Science ®Google Scholar
• Maciel, Daniel Andrade, Claudio Clemente Faria Barbosa, Evlyn Márcia Leão de Moraes Novo, Rogério Flores Júnior, and Felipe Nincao Begliomini. 2021. “Water Clarity in Brazilian Water Assessed Using Sentinel-2 and Machine Learning Methods.” ISPRS Journal of Photogrammetry and Remote Sensing 182:134–152. https://doi.org/10.1016/j.isprsjprs.2021.10.009.
   Web of Science ®Google Scholar
• Maciel, Daniel, Evlyn Novo, Lino Sander de Carvalho, Cláudio Barbosa, Rogério Flores Júnior, and Felipe de Lucia Lobo. 2019. “Retrieving Total and Inorganic Suspended Sediments in Amazon Floodplain Lakes: A Multisensor Approach.” Remote Sensing 11 (15): 1744. https://doi.org/10.3390/rs11151744.
   Web of Science ®Google Scholar
• Marujo, Rennan F. B., Karine R. Ferreira, Gilberto R. Queiroz, Raphael W. Costa, Jeferson S. Arcanjo, and Ricardo C. M. Souza. 2022. “Generating Analysis Ready Data Collections for Brazil.” In IGARSS 2022 – 2022 IEEE International Geoscience and Remote Sensing Symposium, 6844–6847.Kuala Lumpur, Malaysia
   Google Scholar
• Masser, Ian. 2019. “The Future of Spatial Data Infrastructures.” In Geographic Information Systems to Spatial Data Infrastructure, 227–252. CRC Press.Boca Raton, Florida
   Google Scholar
• Matskin, Mihhail, Shirin Tahmasebi, Amirhossein Layegh, Amir H. Payberah, Aleena Thomas, Nikolay Nikolov, and Dumitru Roman. 2021. “A Survey of Big Data Pipeline Orchestration Tools From The Perspective of The Datacloud Project.” In Proceedings of the 23rd International Conference Data Analytics Management Data Intensive Domains (DAMDID/RCDL 2021), 63–78.Moscow, Russia
   Google Scholar
• Milcinski, Grega, and Primoz Kolaric. 2023. “Sentinel Hub-Federated On-Demand ARD Generation.” In EGU General Assembly 2023, 4160.Vienna, Austria
   Google Scholar
• Müller, Matthias. 2016. “Service-Oriented Geoprocessing in Spatial Data Infrastructures.” PhD diss., Technische Universität Dresden.
   Google Scholar
• Müller, Matthias, Lars Bernard, and Johannes Brauner. 2010. “Moving Code in Spatial Data Infrastructures – Web Service Based Deployment of Geoprocessing Algorithms.” Transactions in GIS14 (SUPPL. 1): 101–118. https://doi.org/10.1111/tgis.2010.14.issue-s1.
   Google Scholar
• openEO. November, 2023. “openEO Processes (1.0).” Accessed November 5, 2023. https://openeo.org/documentation/1.0/processes.html.
   Google Scholar
• OSGeo, S.. July, 2022. “Geoserver.” Accessed July 20, 2022. https://geoserver.org/.
   Google Scholar
• Schramm, Matthias, Edzer Pebesma, Milutin Milenković, Luca Foresta, Jeroen Dries, Alexander Jacob, Wolfgang Wagner, et al. 2021. “The OpenEO API–Harmonising the Use of Earth Observation Cloud Services Using Virtual Data Cube Functionalities.” Remote Sensing 13 (6): 1125. https://doi.org/10.3390/rs13061125.
   Web of Science ®Google Scholar
• Simoes, Rolf, Gilberto Camara, Gilberto Queiroz, Felipe Souza, Pedro R. Andrade, Lorena Santos, Alexandre Carvalho, and Karine Ferreira. 2021. “Satellite Image Time Series Analysis for Big Earth Observation Data.” Remote Sensing 13 (13): 2428. https://doi.org/10.3390/rs13132428.
   Web of Science ®Google Scholar
• Soille, P., A. Burger, D. De Marchi, P. Kempeneers, D. Rodriguez, V. Syrris, and V. Vasilev. 2018. “A Versatile Data-intensive Computing Platform for Information Retrieval From Big Geospatial Data.” Future Generation Computer Systems 81:30–40. https://doi.org/10.1016/j.future.2017.11.007.
   Web of Science ®Google Scholar
• Temporal Technologies. July, 2022. “Temporal – Open Source Durable Execution Platform.” Accessed July 20, 2022. https://temporal.io/.
   Google Scholar
• Wang, Lizhe, Yan Ma, Jining Yan, Victor Chang, and Albert Y. Zomaya. 2018. “pipsCloud: High Performance Cloud Computing for Remote Sensing Big Data Management and Processing.” Future Generation Computer Systems 78:353–368. https://doi.org/10.1016/j.future.2016.06.009.
   Web of Science ®Google Scholar
• Xu, Chen, Xiaoping Du, Xiangtao Fan, Gregory Giuliani, Zhongyang Hu, Wei Wang, Jie Liu, et al. 2022. “Cloud-based Storage and Computing for Remote Sensing Big Data: A Technical Review.” International Journal of Digital Earth 15 (1): 1417–1445. https://doi.org/10.1080/17538947.2022.2115567.
   Web of Science ®Google Scholar