Machine learning in cartography

ABSTRACT

Machine learning is increasingly used as a computing paradigm in cartographic research. In this extended editorial, we provide some background of the papers in the CaGIS special issue Machine Learning in Cartography with a special focus on pattern recognition in maps, cartographic generalization, style transfer, and map labeling. In addition, the paper includes a discussion about map encodings for machine learning applications and the possible need for explicit cartographic knowledge and procedural modeling in cartographic machine learning models.

KEYWORDS:

• Cartography
• machine learning
• deep learning
• pattern recognition
• map generalization
• style transfer
• map labeling

Introduction

Automation of cartographic processes has been on the research agenda for several decades, with the aim of improving the production and maintenance of maps for specific users and usages as well as for developing new types of maps and map usage. This research has been influenced by research directions in neighboring fields, such as geography, computational geometry, cognitive science, spatial analysis, and image science. During the last years, research in image science has much focused on machine learning, especially since the introduction of deep machine learning models. These models have seemingly surpassed more traditional (top-down) rule-based mechanisms in several domains with impressive improvements in recent years. This rapid development in nearby fields – see, for example, reviews in remote sensing (e.g. Wang et al., 2022) and in geospatial data encoding (e.g. Mai et al., 2022) – has directed researchers in the cartographic domain to study the capability of machine learning models for map applications. To document this development, CaGIS launched a special issue on Machine Learning in Cartography. This paper is an extended editorial of this special issue with the aim of providing the background context for the papers presented in the special issue. It should be noted that this paper is not a comprehensive review paper in the sense that it is not based on a systematic literature review. The papers discussed below have been selected based on work cited by the papers of the special issue, background knowledge of the authors, and some literature search.

Research in automation of cartography has gone through several paradigms. In cartographic generalization, for example, which is a main challenge in the cartographic domain, we can see an evolution from condition-action modeling, to human interaction modeling, to a focus on constraint-based modeling (Harrie & Weibel, 2007). What we learnt from this evolution in paradigms is that none of them has been capable of creating end-to-end solutions, but that they have been useful for solving sub-tasks in the cartographic (generalization) process. To which extent machine learning models will be capable to create end-to-end solutions, or solving sub-tasks, is an open question (discussed in, e.g. Touya et al., 2019). Much of the research described in this paper is still on an exploratory stage, with promising results but not necessarily better performance than what has been achieved using other paradigms. However, there are several aspects that make machine learning interesting, such as its ability to mimic the output of the human cartographer without having to mimic the actions of the cartographer, which has proven to be difficult to model. Its success in neighboring applications, notably image science and natural language processing (NLP) makes the use of machine learning in cartography appear very promising. With the pervasive success of Chat-GPT, the general public has experienced how transformers and attention-based models boosted NLP and achieve unexpected levels of performance. Although the use of off-the-shelf models dedicated to NLP in cartographic applications is not straightforward, the transformers that represent their core network are already put in use for computer vision and have been proven to be efficient, notably with Vision Transformer (Dosovitskiy et al., 2021). In addition, various cartographic applications rely on the learning of context that can be learnt efficiently by such mechanisms. Furthermore, machine learning open ups new directions in cartography that have not been previously possible. Examples of this include style transfer techniques (Chen et al., 2022; Kang et al., 2019), text-to-map generation (cf. products such as Imagen, DALL-E, and Disco Diffusion; for map examples see Jacomy, 2023), and supporting 3D map applications with 2D sketches/images (Schnürer et al., 2023).

The main idea of (supervised) machine learning is to learn from examples; in case of cartographic applications, learning from map examples. It is still under debate whether this is a good approach in cartography. One argument for using machine learning is the difficulty to formalize cartographic knowledge. Cartographic knowledge is a skill of an experienced cartographer, which entails substantial challenges for the knowledge acquisition and formalization (Kilpeläinen, 2000; Sester, 2000; Weibel et al., 1995). Another argument in favor of machine learning is linked to the complexity of the automated rule-based and constraint-based models; this complexity requires substantial work in designing the framework of the models (Brassel & Weibel, 1988; Touya & Duchêne, 2011) and the parameter setting (Harrie, 2003; Zhou & Li, 2016). For the latter, which is often unique for a certain context, much effort is needed in, e.g. pattern recognition and input data enrichment. These are both tasks where machine learning models could facilitate new solutions to guide rule-based and constraint-based models (see e.g. Knura (2023), Li et al. (2023), Fu et al. (2023).

A main argument against machine learning models is that the models are, to a smaller or larger extent, black boxes with a lack of explicitly formulated cartographic rules. Another main drawback is the massive computational demands of deep learning models, which is problematic both from economical and environmental perspectives.

This paper starts with a description of map encodings for machine learning as well as a description of the evolution of machine learning models. These models have been applied to several cartographic applications. In the paper, we describe the following applications: (1) pattern recognition in maps, (2) cartographic generalization, (3) style transfer, and (4) map labeling. Then follows a discussion about an issue that is important in all of these applications: the possible need for explicit cartographic knowledge and procedural modeling in cartographic machine learning models. The paper ends with concluding remarks including directions of further research.

The selection of cartographic applications reviewed in this paper is based on the application areas that the papers in the special issue Machine Learning in Cartography cover. It should be acknowledged that machine learning techniques have been applied to other fields within cartography, such as extracting information from historical maps (Farella et al., 2021; Uhl et al., 2018, 2022), fake maps and ethical issues (Jacomy, 2023; Wu & Biljecki, 2022; Zhao et al., 2021), and map metadata enrichment (Hu et al., 2022a).

Map encodings for machine learning applications

Cartographic data can be encoded by, e.g. raster, vector, and graph data structures. A main question is which encoding is most suitable for machine learning applications, which is elaborated on in several of the papers in this special issue. As an analogy with the location encoding problem (Mai et al., 2020, 2022), we call cartographic location encoding the process of deriving a machine-readable vector that describes the configuration of cartographic symbols. The main difference between a location embedding and cartographic location embedding is the fact that the cartographic embedding also encodes the cartographic symbols and their semantics, and not only the vector geometries. The most usual way to obtain this cartographic location embedding is to rasterize the map (Figure 1b). Though this encoding includes many of the requirements of a cartographic encoding, several problems remain: (1) the geometry is blurred and approximated by the rasterization process; (2) overlaps that can occur when a symbol renders a vector geometry are removed by the raster encoding as the pixel only contains the value of the layer that is on top of all the others (which is a big problem for map labeling for instance); (3) to keep the computation of convolutions manageable, the number of pixels must be limited, which reduces either the resolution or the extent of the processed tile; (4) there are limitations of encoding geometrical and topological relationships explicitly.

Figure 1. Different types of location encoding for cartographic data. a) Cartographic data to encode (vector + symbols); b) raster encoding; c) layered raster encoding; d) graph encoding; e) spatial relations encoding.

To overcome these limitations, other types of encoding are proposed. For instance, multi-dimensional tensors can be used to replace the simple tensor corresponding to an RGB image (Figure 1c), whether it is called a layered representation (Courtial et al., 2022c), or a multi-channel representation (Fu et al., 2023). Additional dimensions of the tensors can also be used to encode semantic information, or information about the neighboring tiles (Courtial et al., 2022c).

To overcome the first and fourth limitation, a graph representation (Figure 1d) can enable a more precise encoding of the geometries. This approach can be coupled with precise local descriptors of the vertices (e.g. the angle between adjacent vertices), and global descriptors of the whole geometry (e.g. the orientation of the polygon) (Knura, 2023), as well as topological and geometrical descriptors to generate cartographic embeddings. These cartographic embeddings can be used to simplify lines (Yan & Yang, 2023), or to detect patterns in road networks (Xiao et al., 2023) and river networks (Yu et al., 2022). This graph representation only encodes one geometry at a time, or lines connected in networks. But there are other spatial relations that should be encoded (Touya et al., 2014) to enable models that properly generate maps. A graph of spatial relations (Figure 1e) may allow to encode additional spatial relations between the different geometries (Iddianozie & McArdle, 2021). There are many location encoding techniques that were proposed in recent yearsin particular, for points or point sets (Mai et al., 2022).

All these encoding techniques provide cartographic embeddings that are slightly different, and can be useful in different cartographic applications. For instance, spatial relations between symbols might not be important to encode for a given application, and then the raster encodings might be sufficient for such an application. But, it appears that the best cartographic encodings for machine learning are still to be designed. A challenge here is to form a regularly structured input, which is required by the machine learning models, from the irregular cartographic data (Knura, 2023). Encoding is not just about the structure of the input data, the way the convolutions are processed is also important. We can introduce new deep layers, which are more adapted to the cartographic nature of the data, such as the Getis Ord Gi* pooling to replace the default max pooling layer (Deng et al., 2021). Furthermore, cartographic data may also include text labels and icons that potentially should be encoded in the machine learning models. Oucheikh and Harrie (2024) use a (layered) raster encoding for the labels, but there are other encodings, such as graph and spatial relations encoding, that are potentially interesting for use with label data.

Machine learning models

Evolution of machine learning techniques

In the early years of deep learning, the focus was on creating deeper and deeper machine learning models, and strategies to learn from larger and larger data sets. However, a change of perspective has emerged, and researchers have worked on less data-demanding methods (Jing & Tian, 2019). Among these, the so-called semi-supervised representation learning (SSRL) methods aim to reduce the need for annotated data in the following way: first, a large unannotated dataset is used to learn the intrinsic structure of the data (pre-task); then a reduced annotated dataset is used to fine-tune the model to perform the desired task (downstream task). These methods can be used for a large variety of data and downstream tasks, for more information readers can refer to two recent reviews on the subject: one on general purposes (Ericsson et al., 2022) and one on geographic information-related tasks (Corcoran & Spasić, 2023). For cartography, a promising approach may be to first use many map images to teach a model how to read and structure a map, and then a downstream model to perform the cartographic tasks without so much annotated data. Moreover, such a method may help to avoid the usage of a fully supervised pre-trained model as often used in image-related tasks. In cartography, this is often not useful as natural images used for the pretraining are too different from map images. In contrast to the semi-supervised scheme that requires a small amount of labeled data, unsupervised methods involve training without any labels and learn to find patterns and structure in the data on their own.

The self-training methods also belong to the semi-supervised methods designed to reduce the amount of annotated data required for learning. This is an iterative process where the model is trained with a growing amount of data. 1) In the first iteration, only annotated data are used to train a supervised model. 2) The first model is used to predict a pseudo-label on a subset of the unlabeled data. 3) A new annotated dataset is created with both the labeled data and pseudo-labeled data. This operation is repeated until all unlabeled data has been pseudo-labeled and used for training a model. Reviewing the literature, these methods appear promising (Amini et al., 2023). Some experiments have even shown that under some conditions they can outperform the quality level of some SSRL techniques (Zoph et al., 2020).

Another interesting trend is meta-learning or learning-to-learn strategies. In such an approach, an outer model is trained to improve the inner model, which performs the task. For instance, the objective of the outer model could be the generalization performance or learning speed of the inner algorithm (Hospedales et al., 2022; Vanschoren, 2019). Such strategies require a set of source tasks each with both training and validation set, and can benefit any type of learning based on prior experience with other tasks. As an example, a model for medium-scale topographic map generation could benefit to be learned from a model that can already generate other maps at this scale or topographic maps at other scales.

In addition to the mentioned techniques, which deal with scarcity of labeled data, transfer learning and active learning techniques can also be employed for similar purposes as well as for boosting model performance. Transfer learning techniques enable the reuse of the knowledge gained from one task or domain in another task or domain with the ultimate goal of improving the performance. Specifically, domain adaptation is a method that can enhance a model’s effectiveness in a target domain that lacks sufficient labeled data by leveraging the knowledge it has gained from a related domain with huge and sufficient labeled data. In the extreme case of labeled data scarcity, human-in-the-loop approaches can be used. Active learning involves iteratively selecting the most informative data points from a large pool of unlabeled data for annotation by an expert or a labeling algorithm. Humans are required to handle low confidence instances and feed them back into the model.

Finally, Ensemble Learning is based on a common concept in artificial intelligence. The assumption is that several models will most likely not fail at the same place, and thus combining efficiently the prediction of several models gives better results than using the prediction from a unique one. Applied to cartography, such a combination could allow dealing with local specificity (e.g. combining a model that performs better in the city center with one for rural areas, etc.). However, applying this idea to deep learning models is not so easy as it requires dealing with variability and cost in the models (Ganaie et al., 2022).

This variety of methods provides a high level of flexibility as it allows to choose the best scheme suitable for each specific problem depending on the task to learn, the availability of labeled data, its complexity, performance requirements, computational resources, and domain expertise, among others. It also allows to increase the performance generalizability, interpretability, and robustness of the models as the kind of method used has a significant impact on the design of the machine learning model architecture.

Machine learning models applied to cartography

After providing an overview of advances in machine learning techniques, we will now focus on machine learning models that are currently used in cartography. The fast development of machine learning, especially deep learning (DL), has resulted in several potential models. How to select suitable DL models for cartographic questions depends on the task, the data structure, the application object, and other conditions. From the perspective of DL studies, researchers mainly focus on large-scale media data, such as web text, images, video, audio, etc. Map data encodings, often vectors, seem not to be of particular interest in this area of research, and therefore suitable DL models are limited. However, some universal DL models can be adapted to conduct cartographic tasks considering map characteristics. For an investigation of opportunities and challenges of large pre-trained (in a task-agnostic manner) models in geospatial applications, see Mai et al. (2023).

The underlying data structure of DL models can be divided into various types depending on the input data, such as linear structure, matrix array structure, and graph structure, that correspond, respectively, to the map encodings vector, raster, and graph. Several researchers have used universal DL models based on all these three structures for cartographic applications. There are also other types of DL models, e.g. based on a tree structure, which are (currently) not much used for map encodings and cartographic machine learning studies.

DL models that utilize a linear structure are, e.g. recurrent neural networks (RNN). RNN models have demonstrated significant potential in the processing of sequential data in the geographic domain. For instance, RNNs and derived LSTM (Long Short-Term Memory) networks have been utilized for time series data analysis, including trajectory data and time series image processing, to perform tasks such as traffic prediction (Li et al., 2020), taxi demand prediction (Wu et al., 2022), pedestrian next-stop prediction (Bao et al., 2021), and urban spatial-temporal event detection (Jin et al., 2022).

Deep learning models for matrix array structures (images, raster maps, etc.) include Convolution neural network (CNN) models. Examples of CNNs are, e.g. U-Net, ResNet (He et al., 2016) and Generative Adversarial Networks (GANs; Goodfellow et al., 2014). In cartography, among others U-net, residual U-net and GAN have been applied to map generalization tasks (e.g. Feng et al., 2019), and ResNet has been used for extracting raster maps metadata from geospatial vector data (Hu et al. 2022a).

A GAN includes two networks trained in contest: the generative network generates new samples and learns to map from a latent space to a given data distribution, while the discriminative network evaluates the generated samples and distinguishes them from the true data distribution. To make the link with the general techniques explained earlier, GANs can be either applied in a supervised manner, such as in the case of the Pix2Pix model, which is trained on pair images, or they can be designed to be unsupervised, such as CycleGAN (that uses unpaired images). In cartography, GANs have been used for, e.g. style transfer (Kang et al., 2019), map labeling (Oucheikh & Harrie, 2024) and map generalization (Courtial et al., 2023). Beyond GANs, recent research introduced Diffusion Models (DM) (Ho et al., 2020; Song & Ermon, 2019), which introduce some noise in the training data, and then learn how to generate some coherent images from the noise. DM can also be combined with a GAN for, e.g. image generation tasks (Wang et al., 2022).

Deep learning models for graphs are called Graph Neural Networks (GNN) (Scarselli et al., 2009) with variants, such as Graph Convolutional Networks (GCN), Graph Attention Networks (GAT), and Graph Generative Networks (GGN). GNNs open a wide door for map applications, especially since they allow for explicit modeling of the connections between spatial objects. A map graph has an irregular structure with varied edge connections and node numbers. This map graph is implemented in a GNN by a set of nodes that represent the datapoints embedded in a multi-dimensional space and edges that capture the relationships linking the datapoints. Through a set of operations that can be convolutional in the case of GCN to nodes and edges in the graph, these models can learn representations for each node and edge based on its local neighborhood in the graph. The main advantage of GNNs is their ability to capture the complex intricate interdependencies and interactions among objects in a graph. This makes them ideal for many cartographic network applications (Jepsen et al., 2020; Xiao et al., 2023; Yan & Yang, 2023).

In the same context, to learn hierarchical relationships between features in complex data, Capsule Networks are interesting candidates. The key innovation of capsule networks is the use of dynamic routing between capsules, which allows to learn spatial relationships between features, for example, between objects in an image. When a higher-level capsule receives input from lower-level capsules, it dynamically routes the input based on how well the input agrees with the capsule’s prediction of the properties of the entity it represents (Mensah et al., 2022).

An interesting feature with the new models (e.g. DMs and GNNs) is the possibility to incorporate prior knowledge of certain fields (e.g. cartography) with data-driven machine learning models. This has emerged as an effective means to increase the performance of the models, guide and speed up the learning process and ensure their plausibility, alleviate the scarcity of training data, and increase the models’ generalizability. Similar to the embedding of knowledge in other domains in machine learning (e.g. Karniadakis et al., 2021; Meng et al., 2022), cartographic knowledge can be integrated on three levels: data enhancement, deep neural network architecture, or cartographic-informed optimization. This is further discussed in Section The need for explicit cartographic knowledge and procedural modeling in machine learning models.

While it is not straightforward to borrow entire (universal) machine learning models from other domains, it is of great interest to reuse some advanced mechanisms to tackle specific issues in cartography. One such issue is the learning of context, which is studied in, e.g. the NLP domain. Cartographic problems suffer from a lack of context when small tiles or graphs are used, which most of the time is the case (see most of the papers presented in this special issue). As can be seen in Figure 2, it is required to consider the surrounding tiles for most cartographic processes (generalize the map, place labels, or transfer the style of the center tile, etc.). In NLP, transformers enable the use of the context of a word (e.g. the sentence it is included in) while encoding or decoding the word (Vaswani et al., 2017). Several propositions have been made to translate the principles of transformers to computer vision problems (Bao et al., 2022; Dai et al., 2021; Dosovitskiy et al., 2021; Liu et al., 2021b). Two recent review papers give a good idea of these flourishing new applications (Guo et al., 2022; Khan et al., 2022) with possible implications for cartography.

Figure 2. A topographic map cut into 15 small tiles in order to be processed by a convolutional neural network. Each tile lacks the context of its neighboring tiles to be fully understandable (source: IGN France).

Examples of machine learning studies in cartography

Pattern recognition in maps

Pattern recognition in cartography refers to identifying, interpreting, organizing, and utilizing the latent knowledge from spatial data. This includes spatial relations, structure, and distribution of features, such as roads, buildings, rivers, and topography. Traditionally, cartographers and remote sensing experts have relied on manual interpretation and rule-based methods to identify these patterns. However, with the emergence of machine learning algorithms, particularly deep learning, pattern recognition in cartography can now be automatically carried out. This enables faster and more efficient map applications including map analysis, map evaluation, and pre-processing in map generalization.

Pattern recognition for map analysis involves examining and interpreting meaningful information from maps to inform decision-making and improve our understanding of the world. With the development of convolutional neural networks and different kinds of spatial data, cluster detection and classification has become a crucial application in map analysis field. Based on POI data and related VGI support data, the special model by graph-based GNN has been applied to partition urban functional areas (Huang et al., 2023; Sun et al., 2022; Xu et al., 2022) and to quantify the spatial homogeneity of urban road networks worldwide (Xue et al., 2022). Detecting landforms and topology is another important application, such as the classification of different types of landforms, e.g. dune patterns (Shumack et al., 2020). The intelligent recognition of map elements is also essential for enhancing map services and analyzing the region’s development. For example, Schnürer et al. (2021) used CNNs to identify pictorial objects in historic and contemporary maps to improve the advanced search of digital map catalogs. Saeedimoghaddam and Stepinski (2020) employed a region-based CNN (RCNN) framework to automatically detect road intersections in historical maps for temporal analysis. Overall, pattern recognition for map analysis utilizing machine learning has significant potential for improving our understanding of geo-phenomena and spatial decision-making.

Pattern recognition for map evaluation focuses on the comparison between two or more data versions to find pattern inconsistency. This could happen in different temporal, theme, and applied domain data. Duan et al. (2020) presented a temporal comparison automatically aligning contemporary vector data and historical maps by reinforcement learning methods. The work of Xi et al. (2023) to quantify the emotional semantics of maps belongs to the evaluation of pattern recognition through different themes. Pattern recognition supported by machine learning includes similarity evaluation, which is an important way to judge the quality after applying map processes. For example, Li et al. (2022) proposed a metric learning model (denoted LineStringNet), based on a Siamese model, that measures the similarity between single lines. Li et al. (2023) extend this by introducing a deep neural network for learning patterns in linear feature sets, such as networks (e.g. rivers and roads) and clusters (e.g. contours). Their approach also works for closed linear features such as buildings. Inputs to the models are the linear features as well as topological and geometrical relationships between the features. In their study, they successfully apply their model for classifying building groups and segmenting interchanges in road networks.

Pattern recognition and detection in map processing is one of the fundamental operations, especially in map generalization. Pattern recognition usually serves as a pre-process in map generalization, making explicit the implicit structures of the map, such as collinear building groups and typical drainage patterns. Traditional rule-based approaches using manually defined rules based on geometric, topological, or both properties fail to capture deeper feature information about the pattern (Touya & Lokhat, 2020). Pattern recognition and later scaling actions are usually combined together in map generalization. Yang et al. (2022) employed a Back Propagation Neural Network (BPNN) based approach to analyze building shapes and then determine the optimal building simplification method, while Yan et al. (2021) utilized a graph autoencoder (a type of GNN) to first recognize building outline patterns and then to simplify templates accordingly (Yan et al., 2017a, 2017b). Additionally, GNNs can facilitate building group clustering (Yan et al., 2022; Zhao et al., 2020) and preserve group features during building aggregation. Knura (2023) perform shape classification as a pre-process to cartographic generalization. In their test they use the neural networks CNN, RNN, and GCNN (graph convolutional neural networks). As input to the networks, they used both closed line (building) and open line (coast line) data with three different encodings: plain coordinates, sketch sequences, and feature descriptors. Their results show, among others, that feature descriptors improve the accuracy for all three types of networks.

As far as the data type is concerned in machine learning of map application, vector pattern recognition shows great potential. Early machine learning methods, such as random forest (Du et al., 2016; Jochem et al., 2018) or Markov random field (Huang et al., 2013), act as a classifier to classify groups into potential patterns based on a feature vector (e.g. building types (regular vs. irregular) (Jochem et al., 2018), building group (collinear, curvilinear, and grid, etc.) (Du et al., 2016). Due to convolution and neural-layer stacking techniques, deep learning is better at information mining than traditional machine learning methods. Directly rasterized vector data can be fed into image-based deep learning models (e.g. VGGNet, AlexNet, GoogleNet, and Mask-RCNN) to conduct building shape classification (Yan & Yang, 2022), drainage pattern classification (Donadio et al., 2021; Li et al., 2020; Touya & Lokhat, 2020) and road intersection detection (Yang et al., 2022). However, given the detailed information lost caused by rasterization, the vector data input into the model is considered to be reorganized from the data structure. For example, Yang et al. (2022) proposed 1D-U-Net fed into a line’s grid shape context descriptors to conduct the segmentation of boundary lines.

Among different model members in the deep learning family, the graph related models specially apply to map pattern recognition. The graph representation of vector data is becoming a more popular data structure due to the advent of graph neural networks (GNNs). The map organization uses a graph to model neighborhood and other spatial relations, and efficient graph convolution operations to process the node features (Yan et al., 2019; Yang et al., 2022; Yu et al., 2022). Well-designed graph structures successfully solve the pattern recognition problem of different elements based on the graph convolutional networks (GCN, such as 1st-chebyshev, GraphSAGE, and GAT). Some examples include skeleton-line-based (Li et al., 2022), triangle-based (Bei et al., 2019; Liu et al., 2021a; Zhao et al., 2020), boundary-point-based (Yan et al., 2021), and minimum-spanning-tree-based GCN (Yan et al., 2019) for building shape classification and group recognition, road-based undirected graphs for complex intersection detection (Yang et al., 2022), and drainage dual graphs for drainage pattern classification (Wang et al., 2023; Yu et al., 2022) and segmentation (Yu et al., 2023). Besides, unsupervised graph-autoencoder-based shape embedding (Yan et al., 2021) and semi-supervised few-shot learning methods (Hu et al., 2022b) have also been introduced and tested in shape recognition to get rid of reliance on supervised learning.

Cartographic generalization

When you design a map at a scale at which your spatial data are too detailed to be legible, you need to abstract, simplify your spatial data to adapt it to the scale of the map. This abstraction process is called map generalization, and it can be a tedious task when performed manually or interactively. From the inception of research on the automation of map generalization, researchers have used novel artificial intelligence methods, such as expert systems, multi-agent systems, early machine learning techniques, or optimization techniques. Deep learning may also be a useful technique to solve map generalization problems (Touya et al., 2019). Early attempts to use deep learning for map generalization focus on the generalization of individual map features, such as buildings (Feng et al., 2019, 2020; Kang et al., 2020), roads (Courtial et al., 2020, 2022b), or coastlines (Du et al., 2021; Du et al., 2022), and they make use of U-Net or GAN architectures, which is common in computer vision. A main difficulty of the automation of map generalization lies in the orchestration of multiple algorithms to generalize all the features contained in a map, but deep models dedicated to map generalization are not there yet (Courtial et al., 2021). All these deep models that process map images provide promising outputs but face limitations due to the raster nature of the input/output of the model (Courtial et al., 2022c). To overcome this limitation, researchers now try to encode the input given to the model differently. Some encode the map image as a multi-dimensional tensor (Courtial et al., 2022c), others encode the map as a graph (Zhou et al., 2022), while yet others transform a line into a vector that can be processed by an autoencoder (Yu & Chen, 2022). Finally, map generalization is usually an iterative process where the intermediate map is evaluated to assess whether the previous transformation actually improved the map. Some recent research using machine learning also tried to address this topic (Courtial et al., 2022a; Yang et al., 2022).

In this special issue, several papers propose to go further with the use of deep learning models for map generalization, and they address the limitations faced by the first deep models cited above. Fu et al. (2023) follow on the idea of the multi-dimensional tensor instead of the plain map image, as the input of a U-Net architecture, where they use one layer for the building to be generalized, while the other layers store the context. This technique improves the rectangularity and parallelism of buildings in the generalization process, which is tested in scale transitions from 1:5,000 to 1:10,000 and 1:15,000. Yan and Yang (2023) propose an approach comparable to Yu and Chen (2022), by modeling the lines and polygons as graphs, which are processed by a self-supervised graph autoencoder to simplify them. To obtain better results, they modeled both shape preservation, area balance, and angle-characteristic enhancement in the loss function. A quantitative evaluation reveals that their method gives low changes in position, area and scale, and visual comparison with simplified lines by common simplification methods also provides good results. Also, Xiao et al. (2023) use graph networks, but in their study for point cluster generalization. Their point selection method is a combination of a data-driven approach (supervised network) and explicit domain knowledge in the input data. The latter includes spatial (generated by, e.g. a Delaunay triangulation) and contextual features. The methodology demonstrates the ability to maintain both local and overall characteristics in the selection process. To address the orchestration of multiple deep models, Courtial et al. (2023) propose a framework where the map is separated into layers, generalized by different models, with a final GAN assembling the layers into the map. The framework is tested on building, water, and road data, where the data is generalized from detailed scale to medium scale (1:50,000). The experimental test shows that sub-diving the generalization problem into sub-processes (solved independently by machine learning techniques) is more promising than using a single machine learning model for the whole generalization task. Finally, Karsznia et al. (2023) show that deep learning is not the only machine learning solution applicable to map generalization. They propose to use the machine learning techniques random forests, support vector machines, decision tree, and neural networks to learn how to properly select roads for maps at smaller scales. They compare their machine learning results with some traditional methods (based on guidelines and graph theoretical measures) where the results indicate that the machine learning models are more similar to an atlas product of the same area.

Style transfer

Cartographic style – including symbols, colors, typography, icon design, etc. – is an important part of the aesthetics and efficiency of maps (Ory et al., 2015). Digital techniques have enabled that a geographic dataset can be visualized by several cartographic styles to generate various maps. In practice, most maps are defined by style regulations based on either international specifications (used in e.g. sea charts), or national specifications (most commonly used in topographic mapping; see, e.g. Kent & Vujakovic, 2009). Cartographic style is represented as stylesheets, stored in specific mapping or GIS software formats, open formats, such as the OGC Symbol Encoding (OGC 2006), or in program code. The latter is, e.g. common for multiscale web maps. Some researchers have introduced machine learning techniques as an alternative to stylesheets or program code for transferring cartographic style. The basic idea is to transfer a style from one map (or even artistic painting) to another map; that is, learning map style from examples. Researchers in image science (e.g. Isola et al., 2017, Zhu et al. 2017a; Park & Lee, 2019) have developed techniques to transfer the style from one image to another image using GAN networks, for example. Early adopters of this technique in cartography were Kang et al. (2019). They utilized the cartographic style from Google Maps to create maps of unstyled OpenStreetMap vector datasets. To facilitate this transfer, they applied image-to-image mapping using the GAN networks Pix2Pix and CycleGAN. This type of transfer is interesting especially in the context of generating specific map styles for certain applications and user groups, or even to create individual designs.

Cartographic style transfer could also be applied to aerial (ortho-) and satellite images. This means that styled maps could be generated directly from images. For example, Christophe et al. (2022) generate maps of historical styles from modern ortho-images, and Ganguli et al. (2019) generate maps from satellite images, both using GAN techniques. Li et al. (2020) use a similar approach in their creation of their MapGAN network. In their comparison with other machine learning techniques, the MapGAN network provides maps with a higher visual quality. Chen et al. (2022) extend previous work by introducing a generation of multi-scale maps from satellite images. This multi-scale mapping could have been implemented by parallel scale-wise mapping for several scales, but this would inevitably lead to inconsistencies between scales. Instead, Chen et al. (2022) designed a machine learning model where high-resolution satellite images are inputted to a GAN generator to output large-scale maps, which are then generalized to multiscale maps through a series of multiscale map generators. One shortcoming of maps generated directly from images is that the topology is not always consistent. To facilitate topologically improved maps, additional information could be added to the machine learning models. This is performed by, e.g. Zhang et al. (2020) who added GPS-traces to the satellite images as input to the GAN models to improve the topology of the road networks in maps. Another approach to improve topology in the generated map, designed and implemented by Xu et al. (2023), is to add topological consistency loss as part of the total loss function.

In the examples given above, the machine learning models are trained for all map layers, or at least the majority of layers. There are also machine learning models developed for specific layers, such as relief layers. Shaded relief has been a challenge in the digital cartographic era, since quantitative methods, such as using the diffuse reflection properties (Yoëli, 1967), do not provide results of the same quality as manually created reliefs. As an alternative to current quantitative methods, Jenny et al. (2021) developed an image-to-image translation, based on a U-net neural network architecture. The network was trained with manually shaded relief images of the Swiss topographic map series and terrain models of the same area. The result was satisfying; the network-generated shaded reliefs were visually similar to the manually made reliefs.

A specific type of cartographic style transfer was performed by Schnürer et al. (2023). They created 3D human figures from 2D images/sketches on maps using machine learning techniques, and then added these 3D figures to 3D visualizations of the maps. This application is interesting for, e.g. creating naturally looking 3D figures (taken directly from e.g. historical maps) in story map applications.

Map labeling

Label placement is an important task in map production. Since this task demands a considerable amount of manual effort and time, several studies have been conducted to automate the label placement process. The majority of research on map labeling relies on quantifying the rules found in the cartographic literature (e.g. Imhof, 1975). In the early stages, the problem was formulated as a geometric independent set problem and considered one of the computational geometry tasks, which are NP-hard (Formann & Wagner, 1991).

Considering the task as an optimization problem and expressing the cartographic requirements (generally legibility, association, readability, and aesthetics) as objective functions, (near-)optimal solutions can be found using several approximation and heuristic methods, such as simulated annealing (Zoraster, 1997), genetic algorithms (Yamamoto & Lorena, 2005) and integer programming (Haunert & Wolff, 2017). Later, some researchers formulated the problem as multi-criteria optimization by specifying and evaluating more detailed cartographic principles as quality functions, particularly for point feature labeling (Rylov & Reimer, 2014). These optimization approaches have led to the development and implementation of rule-based systems and optimization tools (e.g. PAL; Ertz et al., 2009) which provide satisfactory solutions to several labeling applications. However, the level of map labeling automation for production purposes remains relatively low and the resulting quality does not achieve the same level as that of a skilled human cartographer. Therefore, semi-automatic approaches were developed, such that the algorithm can give a good but not perfect solution, which will be a starting point for a cartographer who can then locally refine and improve the labeling and, thus, save much time (Klute et al., 2019).

With the growing utilization of machine learning in various application domains, some cartographic researchers have studied the use of deep learning in map labeling and, implicitly, whether the cartographic knowledge embedded in map examples can be used for generating high-quality map labels. One early study was performed by Pokonieczny and Borkowska (2019) who used machine learning to determine feature labeling in topographic maps. They trained a basic neural network (multilayer perceptron) with input terrain coverage data obtained from military topographic maps to determine in which rectangle a label should be placed around a polygon feature (built-up area). It is worth mentioning that this study does not deal with conflicting labels, and the maps they used had relatively few features that could overlap with the labels.

There are three machine learning approaches that can be used to solve the map labeling problem using an image-to-image transformation (Harrie et al., 2022). The first approach is based on key point detection models, such as stacked hourglass networks, that have to learn the raster key points that indicate the positions where labels should be placed. One example is work by Li et al. (2020) who developed a deep learning approach for placing labels on area features based on a stacked hourglass network to create heatmaps that show good positions of the area labels. This approach differs from the common strategy used in many GIS programs, which involves placing the label on specific candidate positions, particularly on the centroid of the polygon.

The second approach uses generative models that are able to learn the mapping between a source domain that includes the maps without labels and a target domain where the maps are labeled. Investigating the feasibility of this approach, Oucheikh & Harrie (2024) trained GAN models, namely CycleGAN and Pix2Pix, on map examples and evaluated their ability to predict good locations of the labels given unlabeled raster maps. The obtained results were compared with manual map labeling and a state-of-the-art optimization-based method using metrics for legibility, readability, and association. The deep learning models showed similar legibility results, but manual labels are better in terms of association, while labels placed by the optimization tool have higher readability scores.

The third approach is image composition (Niu et al., 2022) in which deep learning models should learn how to generate consistent composite images considering the relationship between the features and their labels as well as the relationship between the labels themselves. To the authors’ knowledge, this approach has not yet been tested in map labeling research.

Using image-to-image deep learning techniques for map labeling has inherent problems. They all focus mainly on synthesis of appearance features (the labels in this case) by learning the style of images of the target domain. This implies that the actual label geometries are not included in the modeling. Ideally, a solution for the label placement problem should incorporate both the label geometries and appearance realism. This could be implemented by, e.g. utilizing a geometry synthesizer to learn the local geometry of background images (maps) on which the labels representing the foreground objects can be transformed and placed. To our knowledge, no research has been conducted in this direction yet. In the last section of this paper, we propose some ideas on how to integrate label geometry or, in general, knowledge related to map labeling in a deep learning framework.

The need for explicit cartographic knowledge and procedural modeling in machine learning models

As noted in the previous sections, machine learning models are increasingly used in cartography. Machine learning models derive their power from their ability to identify patterns (and biases) in data that result in regularities and more general characteristics that go beyond specific data points (samples). By using a substantial number of examples, machine learning models often achieve very good reliability in identifying patterns in the training phase that could be used in applications. For these mechanisms to work, sufficiently many, and sufficiently useful examples need to be provided. The more complex the task is, the more examples are needed.

A disadvantage of the machine learning models is that it is not (fully) predictable what exactly they will learn. Essentially, they solve a classification (pattern recognition, function approximation) task, and will use whatever principles or attributes allow best to discriminate inputs, or offer highest gain in their outputs in case of generative models. The principles employed by these models may well not at all be related to any principles of good cartographic practice. This holds in particular if the models operate directly in the image domain, but the task they are meant to solve may rather reside on another level as, for example, in vector and network simplification. Two shortages of purely data-driven machine learning models for cartographic applications can be identified: a lack of explicit representation of (cartographic) world knowledge (e.g. visual design esthetics or preservation of topology, such as the need for connected road networks), and a lack of pragmatics, i.e. how (and why) to perform specific cartographic operations (Scheider & Richter, 2023).

The lack of explicit modeling of cartographic knowledge can especially be identified in pioneering usage of machine learning in cartography, for example in cartographic generalization of buildings (e.g. Feng et al., 2019) and roads (e.g. Courtial et al., 2020), as well as in map labeling (Li et al., 2020). However, since these deep learning networks are comparably simple models, they can be extended if the user has cartographic expertise. This adding of cartographic knowledge has been utilized in some recent studies. For example, Fu et al. (2023) introduce a multi-channel model for building generalization, where one building to be generalized is given in one channel and the geographic context in another channel. Furthermore, the encoding of cartographic data is important for adding cartographic knowledge. Generally, it is easier to add cartographic knowledge for vector and network data. One example of this is found in Knura (2023) who use cartographic knowledge in the form of shape measures together with a vector encoding as input to a machine learning model for the classification of building and coastline data. Other examples are provided by Xiao et al. (2023) and Yu et al. (2022) who use graph encodings, where several geometrical and topological measures are explicitly added as input to a graph convolutional network.

The lack of any kind of pragmatics in purely data-driven machine learning models would entail that the models do not incorporate any procedural knowledge about good cartographic practices or any cartographic principles. This implies that they may lead to incorrect or at least counter-intuitive results. This may also mean that a lot of effort is spent on training the models to learn principles that everybody (in cartography) knows anyway. Some researchers in the geosciences have introduced well-known geographical constraints and more explicit semantic knowledge in machine learning models (Wolanin et al., 2019; Yan et al., 2017a). Such models are supposed to lead to more targeted results. Since the models have already some knowledge about the domain at hand, they may potentially also require smaller training sets. Also, in cartography, researchers have started to add pragmatics into the machine learning models. Courtial et al. (2023) divide the generalization task into several sub-tasks according to common cartographic practice and then train machine learning models for each sub-task. While this approach will not lead to an end-to-end solution using a single machine learning model, such an approach will allow for a more targeted choice for the machine learning models to solve specific tasks (which would require less training data) and their results may be easier to explain, i.e. contribute to more explainable AI in cartography.

To enhance the interpretability further, integration strategies combining data-driven computation and domain knowledge will play an important role. Map-making rules, map analysis models, spatial cognition principles, and other related cartographic domain knowledge can be embedded into a deep learning model. This integration can be done before, during and after the machine learning computations. Before the machine learning step, the input data (in, e.g. vector encoding) can be extended to add some specific properties of mapping applications, such as geometric measures on Gestalt cognition principles, or constraints in neighboring object aggregation. During the learning step, a machine learning model can be opened to change or add some targeted computation operator suitable for map applications. In a convolution network, for example, we can replace or add new convolution computation cores or pooling cores to detect geometric patterns, or we can add cartographic knowledge into the objective functions that govern the machine learning outcomes (Courtial et al., 2022b; Yan & Yang, 2023). Another possibility is to create a workflow where some sub-tasks are solved by rule-based or constraint-based models based on domain knowledge, while others are solved by machine learning models (e.g. Courtial et al., 2023). After the machine learning step, the output results can be further enhanced with cartographic knowledge in post-processes, such as applying principles of selection, as e.g. formalized by Töpfer and Pillewizer (1966), to enhance the selection results of a GCN model (Xiao et al., 2023). Improving the machine learning models themselves requires knowledge of the basic principles of machine learning. However, adding cartographic knowledge before and after the machine learning computations relies mostly on cartographic domain knowledge.

Currently, there is a general debate in the geosciences about whether or not explicit models are required, i.e. whether “spatial is so special” that standard machine learning models will not suffice (Janowicz et al., 2020; Kuhn, 2012; Zhu et al., 2017b). This debate is interesting also in the cartographic context. Explicit models have several advantages, as discussed in the previous paragraphs. However, they are also clearly more complex than “normal” machine learning models. And in some ways, explicit models have less “freedom” to pick out characteristics of the data that allow them to solve the task at hand. Some claim that given enough data, the need for explicit models will vanish. The question remains whether we will reach a stage where there will ever be enough quality (training) data to make away with the need for explicit cartographic domain and procedural knowledge in machine learning, and whether this would even be desirable.

Concluding remarks

A cartographic dream would be to create high-quality multi-scale maps directly from aerial and satellite images. We are quite far from achieving this today, but machine learning techniques – together with other computing paradigms – could possibly solve this task in the future. What is illustrated in this paper is that some building blocks have emerged toward this end, such as: (1) deriving basic multi-scale maps directly from aerial and satellite images (e.g. Chen et al., 2022), (2) identifying patterns/characteristics of features and group of features, (3) generalizing maps, and (4) labeling maps. A challenge here is that to perform the later steps in the process (e.g. generalization and labeling) we need to enrich the purely geometric data (derived from the images) with semantic information. This is not only important for creating high-quality maps but also for going from a map as a free-standing product to it being part of a digital twin (cf. Lei et al., 2023). To facilitate this linkage, we need to (automatically) establish linkages between the geometries created from the images and semantic information in external databases. If such enriched dataset would exist (possibly as part of a digital twin), it would be interesting to utilize both images and text (linked to the images) as input to (future) multi-modal machine learning models (models that take both images and text as input).

To achieve the vision, we need to improve all the sub-processes. Below follow some ideas, from respective fields, of what could be researched in the near future.

Pattern recognition used to be a traditional decision-making question, and supported by machine learning it becomes an active new topic. Cartography and remote sensing image processing both have pattern recognition tasks but with different focuses. Remote sensing focuses on the identification from pixel to object. On the other hand, cartography focuses on class judgment and characteristic detection from object to high-level clusters. The application of data-driven approaches has promoted the development of rule-based methods. Based on the training of typical sample data, deep learning methods can identify different feature patterns in map space. Currently, the main work of pattern recognition focuses on geometric data, such as graph, network, polygon cluster, etc. This will be extended to include other data, such as social attributes, semantic descriptions, functional data, and others. Based on the integration of different data, generalized pattern recognition can be conducted, such as identifying urban functional zones, CBD regions, and others. Another trend in map pattern recognition is to combine cartographic domain knowledge and deep learning methods, for example, to incorporate spatial cognition or scaling principles. Such integrated deep learning models will enhance map pattern recognition.

Regarding map generalization, deep learning models showed promising results for different tasks, but they are not able to generalize better than traditional automated techniques yet. To go further, deep learning models will have to (1) better encode the vector geometries in the map (using graph encodings?), and (2) to be able to “look around,” and make decisions based on the cartographic context (using transformers?). One last challenge is to better convey with loss functions what a good generalized map is. As the deep models optimize the value of this loss function, we have to make sure that a better loss really corresponds to a better map.

Style transfer, in combination with object extraction, has been successfully applied in generating basic maps, and even multiscale maps, directly from images. To improve these maps – from a cartographic perspective – there are several issues to solve, both in terms of information content and presentation. From an information content perspective, it would be interesting to study the utilization of the automatically generated maps with other information sources. One aspect is that several maps do not only contain visible objects, but also, e.g. functional information (e.g. land use, urban functional regions, etc.). Such information could possibly be derived from images (Zhang et al., 2018) and point of interest (POI) databases (Huang et al., 2023). It would also be interesting to conflate the automatically generated maps with other (map) data sources. As part of this, we need to establish linkages between objects in the two datasets, performed by machine learning techniques and/or geometrical and topological matching. From a presentation perspective, it would be interesting to study generation models that translate textual descriptions to images (or maps). Ideally, these generation models should support generation of maps to user groups with specific needs, such that they translate a textual description to a map representation that can be understood by the user.

As noted above, research in using machine learning for map labeling has been quite limited, and most of it has been purely data-driven. In future research, there is a need to use explicit cartographic and map labeling knowledge, which can be embedded in machine (deep) learning on three different levels. The first level is the data enhancement where one can feed the model with additional features to learn from, such as the text attributes (length, font, etc.) and the attributes of the labeled objects (centroid coordinates, area, vertices coordinates, etc.). The second level is the model architecture where it is possible to include some subnetworks specialized, for example, in learning or evaluating the context of a possible label location determined by the core network or another subnetwork. The third level is the design of loss functions related to the labeling objectives (e.g. legibility, association, and map readability requirements); for instance, the legibility metric can be included in the loss function by computing the intersection of bounding boxes of the labels, which is somehow similar to the Intersection over Union (IoU) or Dice score used in image segmentation.

In conclusion, research in cartography – including the papers in this special issue – has shown the potential of using machine learning in cartographic applications. In some applications, e.g. in style transfer and pattern recognition, machine learning models often already outperform traditional methods, while in other applications they are promising but not necessarily better. To which extent machine learning models will be successful in cartographic applications, and to which degree we need to explicitly model cartographic knowledge in these models, are still open questions.

Acknowledgment

The authors would like to thank the anonymous reviewers for their constructive comments.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

The first and third authors were funded by eSSENCE@LU 7:1: “Data-driven automation of map labeling – enabling affordable high-quality maps” financed by the Swedish research council. The second and fifth authors were financed by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme [grant agreement No. 101003012 - LostInZoom].