Capturing deprived areas using unsupervised machine learning and open data: a case study in São Paulo, Brazil

References

• Abascal, Á., Rothwell, N., Shonowo, A., Thomson, D., Elias, P., Elsey, H., Yeboah, G., & Kuffer, M. (2021, March). “Domains of deprivation framework” for mapping slums, informal settlements, and other deprived areas in LMICs to improve Urban planning and policy: A scoping review. Preprints, 1–23. https://doi.org/10.20944/preprints202102.0242.v1
   Google Scholar
• Ajami, A., Kuffer, M., Persello, C., & Pfeffer, K. (2019). Identifying a slums’ degree of deprivation from VHR images using convolutional neural networks. Remote Sensing, 11(11), 1282. https://doi.org/10.3390/rs11111282
   Web of Science ®Google Scholar
• Alelyani, S., Tang, J., & Liu, H. (2014). Feature selection for clustering: A review. In Aggarwal, Charu C., Reddy, Chandan K. (Ed.), Data Clustering - Algorithms and Applications 1st ed., Chapman and Hall/CRC. 9781315373515 .
   Google Scholar
• Bastos da Cunha, M., Firpo de Souza Porto, M., Pivetta, F., Zancan, L., Santos Francisco, M., Brum Pinheiro, A., Melo Souza, F., & Calazans, R. (2015). O desastre no cotidiano da favela: reflexões a partir de três casos no Rio de Janeiro [An everyday disaster in favela: reflections based on three cases in Rio de Janeiro]. O Social Em Questão, 33, 95–122.
   Google Scholar
• Bhuyan, K., Westen, C. V., Wang, J., & Raj, S. (2022). Mapping and characterising buildings for flood exposure analysis using open ‑ source data and artificial intelligence. In Natural Hazards Issue 0123456789. Springer Netherlands. https://doi.org/10.1007/s11069-022-05612-4
   Google Scholar
• Brito, P. L., Ku, M., Koeva, M., Pedrassoli, J. C., Wang, J., Costa, F., & Freitas, A. D. D. (2020). The spatial dimension of COVID-19: The potential of earth observation data in support of slum communities with evidence from Brazil. ISPRS International Journal of Geo-Information, 9(9), 557. https://doi.org/10.3390/ijgi9090557
   Web of Science ®Google Scholar
• Cai, D., Zhang, C., & He, X. (2010). Unsupervised feature selection for multi-cluster data. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 333–342. https://doi.org/10.1145/1835804.1835848
   Google Scholar
• Campello, R. J. G. B., Moulavi, D., & Sander, J. (2013). Density-based clustering based on hierarchical density estimates. In J. Pei, V.S. Tseng, L. Cao, H. Motoda, & G. Xu (Eds.), Advances in knowledge discovery and data mining (Vol. 7819, pp. 160–172). Springer. https://doi.org/10.1007/978-3-642-37456-2_14
   Google Scholar
• Chiang, M. M. T., & Mirkin, B. (2010). Intelligent choice of the number of clusters in k-means clustering: An experimental study with different cluster spreads. Journal of Classification, 27(1), 3–40. https://doi.org/10.1007/s00357-010-9049-5
   Web of Science ®Google Scholar
• da Fonseca Feitosa, F., Vieira Vasconcelos, V., Moutinho Duque de Pinho, C., Frizzi Galdino da Silva, G., da Silva Gonçalves, G., Correa Danna, L. C., & Seixas Lisboa, F. (2021). IMMerSe: An integrated methodology for mapping and classifying precarious settlements. Applied Geography, 133, 102494. https://doi.org/10.1016/J.APGEOG.2021.102494
   Web of Science ®Google Scholar
• Dos Santos, B. D., de Pinho, C. M. D., Oliveira, G. E. T., Korting, T. S., Escada, M. I. S., & Amaral, S. (2022). Identifying precarious settlements and Urban fabric typologies based on GEOBIA and data mining in Brazilian amazon cities. Remote Sensing, 14(3), 704. https://doi.org/10.3390/RS14030704
   Web of Science ®Google Scholar
• Duque, J. C., Patino, J. E., & Betancourt, A. (2017). Exploring the potential of machine learning for automatic slum identification from VHR imagery. Remote Sensing, 9(9), 1–23. https://doi.org/10.3390/rs9090895
   Web of Science ®Google Scholar
• Ebert, A., Kerle, N., & Stein, A. (2009). Urban social vulnerability assessment with physical proxies and spatial metrics derived from air- and spaceborne imagery and GIS data. Nat Hazards, 48(2), 275–294. 10.1007/s11069-008-9264-0
   Web of Science ®Google Scholar
• Eckle, M., De Albuquerque, J. P., Herfort, B., Zipf, A., Leiner, R., Wolff, R., & Jacobs, C. (2016). Leveraging OpenStreetMap to support flood risk management in municipalities: A prototype decision support system. Proceedings of the International ISCRAM Conference Krystiansand, Norway, May.
   Google Scholar
• Ferreira, N. D. J., & Feitosa, F. D. F. (2020). Cartografias das Favelas : Uma Análise Comparativa. In Pandemia e Cotidiano, 5, I Seminário Nacional – Urbanismo, Espaço e Tempo. Cidade.
   Google Scholar
• Fränti, P., & Sieranoja, S. (2019). How much can k-means be improved by using better initialization and repeats? Pattern Recognition, 93, 95–112. https://doi.org/10.1016/j.patcog.2019.04.014
   Web of Science ®Google Scholar
• Frazier, A E., & Kedron, P. (2017). Landscape Metrics: Past Progress and Future Directions. Curr Landscape Ecol Rep, 2(3), 63–72. 10.1007/s40823-017-0026-0
   Google Scholar
• Geiß, C., Schauß, A., Riedlinger, T., Dech, S., Zelaya, C., Guzmán, N., Hube, M. A., Arsanjani, J. J., & Taubenböck, H. (2017). Joint use of remote sensing data and volunteered geographic information for exposure estimation: Evidence from Valparaíso, Chile. Natural Hazards, 86(S1), 81–105. https://doi.org/10.1007/s11069-016-2663-8
   Google Scholar
• Gevaert, C. M., Kohli, D., & Kuffer, M. (2019, May). Challenges of mapping the missing spaces. 2019 Joint Urban Remote Sensing Event, JURSE 2019, 2019–2022. https://doi.org/10.1109/JURSE.2019.8809004
   Google Scholar
• Ghaffarian, S., Kerle, N., & Filatova, T. (2018). Remote sensing-based proxies for Urban disaster risk management and resilience: A review. Remote Sensing, 10(11), 1760. https://doi.org/10.3390/rs10111760
   Web of Science ®Google Scholar
• Gram-Hansen, B. J., Helber, P., Coca-Castro, A., Kopackova, V., & Bilinski, P. (2019). Mapping informal settlements in developing countries using machine learning and low resolution multi-spectral data AAAI/ACM Conference on AI, Ethics, and Society (AIES ’19) January 27–28, 2019 Honolulu, HI, USA. 361–368.
   Google Scholar
• Hall-Beyer, M. (2017). Practical guidelines for choosing GLCM textures to use in landscape classification tasks over a range of moderate spatial scales Practical guidelines for choosing GLCM textures to use in. International Journal of Remote Sensing, 38(5), 1312–1338. https://doi.org/10.1080/01431161.2016.1278314
   Web of Science ®Google Scholar
• Han, J., Kamber, M., & Pei, J. (2012). Cluster analysis: Basic concepts and methods. In Data Mining 3rd, (pp. 443–495). https://doi.org/10.1016/B978-0-12-381479-1.00010-1
   Google Scholar
• IBGE. (2010, August). SP Capital. Censo Demográfico.
   Google Scholar
• Instituto Brasileiro de Geografia e Estatística (IBGE). (2019). Aglomerados Subnormais IBGE. https://ibge.gov.br/geociencias/organizacao-do-territorio/tipologias-do-territorio/15788-aglomerados-subnormais.html
   Google Scholar
• Jochem, W. C., Leasure, D. R., Pannell, O., Chamberlain, H. R., Jones, P., & Tatem, A. J. (2020). Classifying settlement types from multi-scale spatial patterns of building footprints. Environment & Planning B: Urban Analytics & City Science, 48(5), 1–19. https://doi.org/10.1177/2399808320921208
   Google Scholar
• Karmitsa, N., Bagirov, A. M., & Taheri, S. (2018). Clustering in large data sets with the limited memory bundle method. Pattern Recognition, 83, 245–259. https://doi.org/10.1016/j.patcog.2018.05.028
   Web of Science ®Google Scholar
• Kit, O., Lüdeke, M., & Reckien, D. (2012). Texture-based identification of urban slums in Hyderabad, India using remote sensing data. Applied Geography, 32(2), 660–667. https://doi.org/10.1016/j.apgeog.2011.07.016
   Web of Science ®Google Scholar
• Kohli, D., Sliuzas, R., Kerle, N., & Stein, A. (2012). Computers, environment and Urban systems an ontology of slums for image-based classification. Computers, Environment and Urban Systems, 36(2), 154–163. https://doi.org/10.1016/j.compenvurbsys.2011.11.001
   Web of Science ®Google Scholar
• Kohli, D., Stein, A., & Sliuzas, R. (2016). Uncertainty analysis for image interpretations of urban slums. Computers, Environment and Urban Systems, 60, 37–49. https://doi.org/10.1016/j.compenvurbsys.2016.07.010
   Web of Science ®Google Scholar
• Kohli, D., Warwadekar, P., Kerle, N., Sliuzas, R., & Stein, A. (2013). Transferability of object-oriented image analysis methods for slum identification. Remote Sensing, 5(9), 4209–4228. May 2014. https://doi.org/10.3390/rs5094209
   Web of Science ®Google Scholar
• Kuffer, M., Pfeffer, K., Baud, I., & Sliuzas, R. (2013). Analysing sub-standard areas using high resolution remote (VHR) sensing imagery, N-AERUS XIV, Enschede, 12 - 14th September 2013. https://ris.utwente.nl/ws/portalfiles/portal/30133835/Kuffer2013analysing.pdf
   Google Scholar
• Kuffer, M., Pfeffer, K., Sliuzas, R., & Baud, I. (2016). Extraction of slum areas from VHR imagery using GLCM variance. IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing, 9(5), 1830–1840. https://doi.org/10.1109/JSTARS.2016.2538563
   Web of Science ®Google Scholar
• Kuffer, M., Thomson, D. R., Boo, G., Mahabir, R., Grippa, T., Vanhuysse, S., Engstrom, R., Ndugwa, R., Makau, J., Darin, E., de Albuquerque, J. P., & Kabaria, C. (2020). The role of earth observation in an integrated deprived area mapping “system” for low-to-middle income countries. Remote Sensing, 12(6), 982. https://doi.org/10.3390/rs12060982
   Web of Science ®Google Scholar
• Leonita, G., Kuffer, M., Sliuzas, R., & Persello, C. (2018). Machine learning-based slum mapping in support of slum upgrading programs: The case of Bandung City, Indonesia. Remote Sensing, 10(10), 1522. https://doi.org/10.3390/rs10101522
   Web of Science ®Google Scholar
• Li, J., Cheng, K., Wang, S., Morstatter, F., Trevino, R. P., Tang, J., & Liu, H. (2017). Feature selection: A data perspective. ACM Computing Surveys, 50(6), 1–45. https://doi.org/10.1145/3136625
   Web of Science ®Google Scholar
• Lilford, R., Kyobutungi, C., Ndugwa, R., Sartori, J., Watson, S I., Sliuzas, R., Kuffer, M., Hofer, T., Porto de Albuquerque, J, & Ezeh, A. (2019). Because space matters: conceptual framework to help distinguish slum from non-slum urban areas. BMJ Glob Health, 4(2), e001267 10.1136/bmjgh-2018-001267
   PubMed Web of Science ®Google Scholar
• Lucci, P., Bhatkal, T., & Khan, A. (2018). Are we underestimating urban poverty? World Development, 103, 297–310. doi:10.1016/j.worlddev.2017.10.022
   Web of Science ®Google Scholar
• Luo, E., Kuffer, M., & Wang, J. (2022). Urban poverty maps - from characterising deprivation using geo-spatial data to capturing deprivation from space. Sustainable Cities and Society, 84(July), 104033. https://doi.org/10.1016/j.scs.2022.104033
   Google Scholar
• Madhulatha, T. S. (2012). An overview on clustering methods. IOSR Journal of Engineering, 2(4), 719–725. https://doi.org/10.9790/3021-0204719725
   Google Scholar
• Mahabir, R., Agouris, P., Stefanidis, A., Croitoru, A., & Crooks, A. T. (2020). Detecting and mapping slums using open data: A case study in Kenya. International Journal of Digital Earth, 13(6), 683–707. https://doi.org/10.1080/17538947.2018.1554010
   Web of Science ®Google Scholar
• Martínez, J., Pfeffer, K., & Baud, I. (2016). Factors shaping cartographic representations of inequalities. Maps as products and processes. Habitat International, 51, 90–102. https://doi.org/10.1016/j.habitatint.2015.10.010
   Web of Science ®Google Scholar
• Martino, A., Rizzi, A., & Mascioli, F. M. F. (2017). Efficient approaches for solving the large-scale k-medoids problem. IJCCI 2017 - Proceedings of the 9th International Joint Conference on Computational Intelligence, Ijcci, 338–347. https://doi.org/10.5220/0006515003380347
   Google Scholar
• McInnes, L., Healy, J., & Astels, S. (2016). The HDBSCAN Clustering Library. IBM.
   Google Scholar
• Merodio, P., Jimena, O., Carrillo, J., Kuffer, M., Thomson, D. R., Luis, J., Quiroz, O., Villaseñor, E., Vanhuysse, S., Abascal, Á., Oluoch, I., Nagenborg, M., Persello, C., & Brito, P. L. (2021). Earth observations and statistics: Unlocking sociodemographic knowledge through the power of satellite images. Sustainability, 13(22), 12640. https://doi.org/10.3390/su132212640
   Web of Science ®Google Scholar
• Nobrega, R. A. A., Hara, C. G. O., & Quintanilha, J. A. (2008). An object-based approach to detect road features for informal settlements near Sao Paulo , Brazil. In Object-Based Image Analysis. Lecture Notes in Geoinformation and Cartography. Springer 978-3-540-77058-9, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77058-9_32
   Google Scholar
• Nugrahita, R., & Surjandari, I. (2020). Identify product families using cluster analysis: Case study in Passenger Car Radial (PCR) tire product. IOP Conference Series: Materials Science and Engineering PAPER. https://doi.org/10.1088/1757-899X/909/1/012057
   Google Scholar
• Olthuis, K., Benni, J., Eichwede, K., & Zevenbergen, C. (2015). Slum upgrading: Assesing the importance of location and a plea for a spatial approach. Habitat International, 50, 270–288. https://doi.org/10.1016/j.habitatint.2015.08.033
   Web of Science ®Google Scholar
• Owen, K. K., & Wong, D. W. (2013). An approach to differentiate informal settlements using spectral, texture, geomorphology and road accessibility metrics. Applied Geography, 38, 107–118. https://doi.org/10.1016/j.apgeog.2012.11.016
   Web of Science ®Google Scholar
• Pasternak, S., & D’Ottaviano, C. (2016). Favelas no Brasil e em São Paulo: avanços nas análises a partir da Leitura Territorial do Censo de 2010*. Cadernos Metrópole, 18(35), 75–100. https://doi.org/10.1590/2236-9996.2016-3504
   Google Scholar
• Patel A, Koizumi, N., & Crooks, A. (2014). Measuring slum severity in Mumbai and Kolkata: A household-based approach. Habitat International, 41, 300–306. 10.1016/j.habitatint.2013.09.002
   Web of Science ®Google Scholar
• Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, E. (2011). 2.3. Clustering — scikit-learn 0.24.2 documentation. Machine Learning in Python.
   Google Scholar
• Pfaffel, O. (2021). Feature importance for partitional clustering (0.1.4). CRAN.
   Google Scholar
• Roy, D., Bernal, D., & Lees, M. (2020). An exploratory factor analysis model for slum severity index in Mexico City. Urban Studies, 57(4), 789–805. https://doi.org/10.1177/0042098019869769
   Web of Science ®Google Scholar
• Sliuzas, R., Kuffer, M., & Masser, I. (2010). The Spatial and Temporal Nature of Urban Objects. In C. Jürgens & T. Rashed (Eds.), Remote Sensing of Urban and Suburban Areas. Springer Science+Business Media. https://doi.org/10.1007/978-1-4020-4385-7
   Google Scholar
• Small, C. (2014). Mapping urban growth and development as continuous fields in space and time. Geography Department University of Sao Paulo, 1(spe), 155–179. https://doi.org/10.11606/rdg.v0i0.555
   Google Scholar
• Taubenböck, H., & Kraff, N. J. (2014). The physical face of slums: A structural comparison of slums in Mumbai, India, based on remotely sensed data. Journal of Housing and the Built Environment, 29(1), 15–38. https://doi.org/10.1007/s10901-013-9333-x
   Web of Science ®Google Scholar
• Taubenböck H, Kraff N & Wurm M. (2018). The morphology of the Arrival City - A global categorization based on literature surveys and remotely sensed data. Applied Geography, 92, 150–167. 10.1016/j.apgeog.2018.02.002
   Web of Science ®Google Scholar
• Taubenböck, H., Kra, N. J., & Wurm, M. (2018). The morphology of the arrival city - a global categorization based on literature surveys and remotely sensed data. Applied Geography, 92(February), 150–167. https://doi.org/10.1016/j.apgeog.2018.02.002
   Google Scholar
• Thomson, D. R., Kuffer, M., Boo, G., Hati, B., Grippa, T., Elsey, H., Linard, C., Mahabir, R., Kyobutungi, C., Maviti, J., Mwaniki, D., Ndugwa, R., Makau, J., Sliuzas, R., Cheruiyot, S., Nyambuga, K., Mboga, N., Kimani, N. W., de Albuquerque, J. P., & Kabaria, C. (2020). Need for an Integrated Deprived Area “Slum” Mapping System (IDEAMAPS) in Low- and Middle-Income Countries (LMICs). Social Sciences, 9(5), 80. https://doi.org/10.3390/socsci9050080
   Google Scholar
• Thrun, M. C., Ultsch, A., & Breuer, L. (2021). Explainable AI framework for multivariate hydrochemical time series Machine Learning and Knowledge Extraction. 3. February. 1–29. doi:10.3390/make3010009
   Google Scholar
• Trento Oliveira, L. (2021). The diversity of deprived areas: Applications of unsupervised machine learning and open geodata (Issue August) [University of Twente]. https://essay.utwente.nl/88986/
   Google Scholar
• UN-Habitat. (2016). World Cities Report 2016: Urbanization and Development–Emerging Futures. In UN-Habitat (Nairobi, Kenya)Accessed 11 07 2022 . Available online: https://unhabitat.org/world-cities-report-2016
   Google Scholar
• Van Dijk, M., Moorthy, I., Nguyen, B., See, L., & Fritz, S. (2019). Tracking poverty using satellite imagery and big data. The International Institute for Applied Systems Analysis, December, 1–16. http://pure.iiasa.ac.at/id/eprint/16240/
   Google Scholar
• Wang, J., Kuffer, M., & Pfeffer, K. (2019). The role of spatial heterogeneity in detecting urban slums. Computers, Environment and Urban Systems, 73(April 2018), 95–107. https://doi.org/10.1016/j.compenvurbsys.2018.08.007
   Google Scholar
• Wurm, M., Taubenböck, H., Weigand, M., & Schmitt, A. (2017). Remote sensing of environment slum mapping in polarimetric SAR data using spatial features. Remote Sensing of Environment, 194, 190–204. https://doi.org/10.1016/j.rse.2017.03.030
   Web of Science ®Google Scholar
• Yeboah, G., de Albuquerque, J. P., Troilo, R., Tregonning, G., Perera, S., Shifat Ahmed, S. A. K., Ajisola, M., Alam, O., Aujla, N., Azam, S. I., Azeem, K., Bakibinga, P., Chen, Y. F., Choudhury, N. N., Diggle, P. J., Fayehun, O., Gill, P., Griffiths, F., Harris, B., Yusuf, R. … Yusuf, R. (2021). Analysis of openstreetmap data quality at different stages of a participatory mapping process: Evidence from slums in Africa and Asia. ISPRS International Journal of Geo-Information, 10(4), 265. https://doi.org/10.3390/ijgi10040265
   Web of Science ®Google Scholar