Exploring the impact of machine learning on dance performance: a systematic review

ABSTRACT

In recent years, the intersection of machine learning and dance has garnered increasing attention as a means to enhance the aesthetics and creativity of dance performance. Machine learning techniques, such as human-pose detection, have been utilized to analyze body movement and generate visual models in realtime, offering new approaches to traditional art form. However, despite the growing interest in this field, there is a lack of comprehensive research that evaluates the impact of machine learning on dance from multiple perspectives. This systematic review addresses this gap by examining the impact of machine learning on four key aspects of dance performance: choreographic creation support; dataset or network collection and training; improved techniques of human-pose detection; perception and new visual representations of dance movement. This exploration reveals that machine learning contributes both positively and negatively to artistic development, introducing a sense of novelty and heightened interest in the performance while concurrently prompting ethical concerns regarding artistic ownership and authorship. Through a nuanced analysis of machine learning's effects on key aspects of dance, this study aims to provide valuable insights into how these technologies can enhance the artistic practice in dance while signaling areas that demand further research in this rapidly evolving field.

KEYWORDS:

• Dance
• machine learning
• performance
• artificial intelligence
• art

1. Introduction

Dance, as an expressive art form communicating through human emotion, body language, and movement, has evolved with the embrace of technological innovations, establishing dynamic synergies to enhance its artistic essence Stevens and McKechnie (2005); Bartenieff and Lewis (2013). Within this transformative landscape, artificial intelligence, notably machine learning techniques, has seamlessly integrated into the realm of dance, ushering in a new era of creativity and exploration Smoliar and Weber (1977); Calvert and Chapman (1978); Camurri et al. (1986); Maletic (1987); Herbison-Evans (1991); Sagasti (2019). Innovative choreography, facilitated by machine learning, has given rise to new movement sequences that challenge traditional perceptions of dance, as shown Wayne McGregor and Google Arts and Culture Wayne McGregor (2019). Real-time performances, showcasing novel visual compositions Bill T. Jones (2019); Nogueira, Simões, and Menezes (2023), redefine how we observe and comprehend the art of dance. Additionally, ongoing research endeavors focus on refining the detection of the human body within the dance domain, with machine learning, particularly pose detection, at the forefront of this intersection Crnkovic-Friis (2016); Chan et al. (2019); Cao et al. (2019).

In recent years, the impact of machine learning on dance performance has become a central focus, fostering the development of diverse artworks and performances that transcend traditional boundaries. Pioneering instances, such as those led by OpenEndGroup – a visionary company founded by Marc Downie and Paul Kaiser – have demonstrated the transformative potential of integrating machine learning into dance practice. Collaborating with eminent dance artists like Merce Cunningham, Trisha Brown, and Wayne McGregor, OpenEndGroup’s portfolio encompasses live performances, installations, and collaborative projects with dance professionals, showcasing the synergies between technology and artistic expression, see Kaiser, Downie, and Birringer (2008). Further emphasizing this collaboration, research projects such as ‘Martial Arts, Dancing, and Sports dataset: A Challenging Stereo and multi-view Dataset for 3D Human Pose Estimation’ by Zhang et al. (2017) and ‘Weakly-Supervised Deep Recurrent Neural Networks for Basic Dance Step Generation’ by Yalta et al. (2019) underscore the role of dance professionals in advancing datasets and movements. However, not all projects have integrated active engagement from dance professionals, especially noticeable in their limited involvement throughout the research process, particularly in curating dance datasets from videos. In light of these observations, this article embarks on a comparative study of research studies, artistic creations, and performances that leverage machine learning techniques and artificial intelligence within the dance domain. By analyzing the impact of machine learning on dance from both research and artistic perspectives, this review aims to provide a comprehensive understanding of the current landscape and identify areas for future exploration and collaboration.

The research methodology for this systematic review employed a comprehensive search across literature databases, including Web of Science, Scopus, and Google Scholar. Utilizing keywords such as ‘machine learning’, ‘dance’, ‘dance movement’, ‘human-pose detection’, ‘dance performance’, and ‘audience engagement’, the search focused on articles published in English from 2010 to 2023. Out of the initially identified 345 articles, a rigorous application of inclusion and exclusion criteria led to the selection of 23 articles for detailed review.

In essence, this systematic exploration delves into the intricate intersection of machine learning and dance, exploring the dynamics in both the research and artistic realms. It seeks to uncover not just the positive impacts but also the inherent complexities of this fusion. This comprehensive examination establishes a foundation for a deeper understanding of the interplay between technology and artistic expression within the dance domain (Table 1).

Table 1. Detailed summary of results related to Choreographic Creation Support.

Title	Author(s)	Year	Study design	Type of dance	Type of technology	Main findings	Classification	Arguments to classify
Interaction-based Authoring for Scalable Co-creative Agents	Jacob & Magerko	2015	Novel approach called interaction-based authoring	Not specified	Microsoft Kinect depth camera.	Presents a novel approach called interaction-based authoring for co-creative systems. Combines ideas from case-based learning and imitative learning. Relies on user interaction for knowledge authoring. This approach is evaluated within the Viewpoints AI installation. Evaluation focuses on mitigating knowledge-authoring bottleneck, producing high-quality subjective experiences, and supporting equal creative agency.	Choreographic creation support	Development of an interaction-based authoring approach for co-creative systems. The approach combines ideas from case-based learning and imitative learning to support the generation of novel choreographic material in an open-ended co-creative application domain.
Generative Choreography using Deep Learning	Luka and Louise Crnkovic-Friis	2016	Experimental	Contemporary dance	Deep recurrent neural network	Chor-rnn can generate novel choreographic sequences in the choreographic style represented in the corpus. Chor-rnn can understand and generate choreography style, syntax, and, to some extent semantics. Chor-rnn can produce a higher level compositional cohesion than just generating sequences of movement. Chor-rnn can be used for collaborative human machine choreography or as a creativity catalyst for a choreographer. Future work can explore the possibility of tracking multiple dancers and experimenting with variational autoencoders that would allow the automatic construction of a symbolic language for movement that goes beyond simple syntax.	Choreographic creation support	The system ‘Chor-rnn’ generates novel choreographic material using a deep recurrent neural network trained on motion capture data.
Everybody Dance Now	Chan et al.	2018	Video-to-video translation using pose as an intermediate representation	Not specified	Human pose extraction using OpenPose and pose-to-appearance mapping	A method for ‘do as I do’ motion transfer from a source video of a person dancing to a novel (amateur) target after only a few minutes of the target subject performing standard moves. The method produces compelling results and includes a forensics tool for reliable synthetic content detection. The authors also release a first-of-its-kind open-source dataset of videos that can be legally used for training and motion transfer. Limitations include visual artifacts caused by loose clothing or hair, missing limb detections, and texture artifacts in clothing. Future work could focus on improving pose detection systems, mitigating artifacts in loose/wrinkled clothing or hair, and improving training data for extrapolation to radically different poses.	Choreographic creation support and Improve techniques of human-pose detection	A proposed method for motion transfer from a source video to a novel target is primarily focused on the choreographic creation aspect.
Google’s latest experiment teaches AI to dance like a human	Leprince-Ringuet	2018	Collaborative work between Google AI and Wayne McGregor, a contemporary choreographer	Contemporary dance	PoseNet, a deep neural network that estimates human pose from video or images	Developed a tool that utilizes machine learning to assist choreographers in generating new dance movements and variations from existing ones. The tool works by tracking a dancer’s movements using PoseNet and then using machine learning algorithms to identify patterns in those movements and generate new ones. The generated movements can be adjusted and refined by the choreographer using a simple interface. The tool was tested with professional dancers and choreographers and was found to be a useful creative tool.	Choreographic creation support	AI-based generation of new dance movements and variations based on existing choreography. The work can be classified as choreographic creation support, as the focus is on using AI to generate new dance movements and variations based on existing choreography.
Beyond Imitation: Generative and Variational Choreography via Machine Learning	Pettee at al.	2019	Experimental	Contemporary Dance	Recurrent neural network and autoencoder architectures using 53 three-dimensional points at each timestep	The study developed configurable machine-learning tools for generating novel sequences of choreography and variations on input choreographic sequences. The tools provide a mode of documentation that enables externalization of movement into the visual domain for reflection on choreography design and architecture. The study also pointed out the distinction between creative expression and research-based inquiry in dance-making. Future work will involve investigating nonlinear, invertible data-reduction techniques, exploring latent spaces of multiple dancers, and using more sophisticated methods for sampling from latent spaces.	Choreographic creation support	The authors have developed a deep learning model that can generate new variations of dance movements based on existing input sequences, which can assist choreographers in creating new and inventive movements. They have also trained the model using a dataset of human poses, which falls under the category of network collection and training. However, the work does not primarily focus on improving techniques of human-pose detection or on developing new visual representations of dance movement.
Living Archive	Wayne McGregor	2019	Dataset Creation and Evaluation	Contemporary Dance	Deep Convolutional Neural Network	Studio Wayne McGregor collaborated with Google to develop the ‘Living Archive’, a system trained on the choreographer’s archive. The online version by Google enables users to interact with the archive by dancing in front of their webcam and searching for similar movement patterns. Notably, McGregor describes a ‘live dialogue’ in a dance performance resulting from the project. In this context, the system generates dance movements in response to the performers, who, in turn, respond to the system. This dynamic exchange between dancers and the system suggests a form of emergent interaction, aligning with our defined framework. While a more in-depth analysis would be beneficial, exploring the intricacies of this interaction exceeds the scope of our present study.	Choreographic creation support and Dataset or network collection/training	This work involves the collection and utilization of a dataset comprising McGregor’s choreographic movements. The project leverages machine learning algorithms to enable a novel interaction where individuals can dance in front of their webcams, and the system searches the archive for similar movement patterns. The underlying machine learning model is trained on this dataset, allowing it to recognize and respond to a diverse range of choreographic expressions.
Learn to dance with aist++: Music conditioned 3d dance generation	Ruilong Li et al.	2021	Multi-modal dataset and network design	Old School (Break, Pop, Lock and Waack) and New School (Middle Hip-hop, LA-style Hip-hop, House, Krump, Street Jazz and Ballet Jazz).	Multi-view videos with known camera poses	AIST++ is a multi-modal dataset of 3D dance motion and music, covering 10 dance genres with multi-view videos with known camera poses. They proposed a Full-Attention Cross-modal Transformer network called FACT for generating 3D dance motion conditioned on music. They show that their proposed changes in architecture design and supervision overcome shortcomings in generating realistic dance motion attuned to the input music. Their method outperforms recent state-of-the-art methods both qualitatively and quantitatively in user studies.	Choreographic creation support and Dataset or network collection/training	The creation of the AIST++ dataset involves the collection and organization of multi-modal data for various dance genres, which falls under the ‘Dataset or network collection and training’ category. The development of the Full-Attention Cross-modal Transformer network called FACT for generating 3D dance motion conditioned on music is a form of ‘Choreographic creation support’ as it assists in the creation of new dance movements.
Between Us	MotionBank, Staatstheater Mainz – Tanzmainz, Kunsthalle Mainz | Main article by Danae Kleida	2021	Collaborative work between MotionBank, Staatstheater Mainz – Tanzmainz, and Kunsthalle Mainz	Contemporary Dance	Motion-Capture; Marker-less System: Employing machine learning for real-time skeleton calculation; Video documentation; Online score	The ‘Between Us’ project leverages motion capture, a marker-less system, video documentation, and an online score to comprehensively document and analyze the choreographic creation process. It produces rich datasets, including motion capture data, video recordings, and annotations, encapsulating the six-week development of the ‘Effect’ choreography. The project also explores the intersection of choreography, contemporary dance, traditional artistic contexts, data, and creative coding, fostering interdisciplinary collaboration.	Choreographic creation support and Dataset or network collection/training	The project significantly contributes to choreographic creation support by introducing an innovative interaction-based authoring approach. This approach, blending case-based and imitative learning, empowers co-creative systems to generate novel choreographic material. The utilization of motion capture technology and marker-less systems underscores the commitment to enhancing the creative agency of choreographers and dancers. The ‘Online Score’ further amplifies this support, providing a digital arena for comprehensive exploration and interaction.

2. Research aims and questions

The merging of machine learning and dance has sparked innovation lately, with a growing focus on leveraging machine learning techniques to elevate this art form. Yet, despite this surge in interest, there’s a noticeable gap — a lack of thorough research assessing both the positive impacts and potential drawbacks of machine learning on dance performance from various angles. This review aims to fill this void, identifying overlooked areas in current research and offering practical insights into how machine learning can drive innovation in the field of dance. To delve into this exploration, we pose two pivotal research questions:

• Research Question 1: What are the main applications of machine learning techniques in dance performance, and how do they impact choreographic creation support, dataset or network collection and training, improved techniques of human-pose detection, perception, and new visual representations of dance movement?

• Research Question 2: How does the incorporation of machine learning techniques impact the creativity of performers, choreographers, participants, and dance professionals?

The Research Question 1 aims to provide an overview of the different applications of machine learning techniques in the dance domain, as well as the impact they have on different aspects of dance performance. The systematic review will gather and analyze studies that have used machine learning in dance, focusing on four key aspects: choreographic creation support, dataset or network collection and training, improved techniques of human-pose detection, perception, and new visual representations of dance movement. These chosen areas address the core facets of artistic creation, technological integration, and audience experience. The synergy between machine learning and dance, as explored in these dimensions, lays the foundation for a more enriched, diverse, and innovative dance landscape.

The Research Question 2 further probes the real-world implications of this intersection by exploring how machine learning impacts the creativity of key contributors in the dance community. The positive impacts are evident as performers explore new movement possibilities and choreographers experiment with unconventional styles, aligning with the collaborative nature of technology in dance, as seen in the works of OpenEndGroup Kaiser, Downie, and Birringer (2008). However, potential challenges related to authority, ownership, and cultural considerations necessitate a nuanced examination, aligning with ethical discussions in the realm of artificial intelligence and the arts. Beyond immediate creativity, understanding how machine collaboration influences the ownership of choreography and cultural dynamics is imperative for fostering an inclusive and respectful intersection between technology and dance creativity.

3. Methodology

3.1. Literature search strategy

To conduct this review, we rigorously followed systematic review guidelines (see Petticrew and Roberts (2008)) and adhered to PRISMA guidelines to ensure transparency and methodological rigor in our search strategy. The inclusion criteria for this review encompassed publications exploring the intersection between machine learning and dance performance across the four primary areas outlined. We maintained inclusivity by not discriminating between specific dance styles.

Our exclusion criteria were designed to refine the focus of our review and ensure that it remains centered on the artistic and innovative applications of machine learning in dance. Specifically, we excluded publications that did not primarily address the creative aspects of dance or its intersection with technology. Additionally, we excluded publications solely focused on somatics and authentic movement. While these areas are undoubtedly valuable in exploring embodiment and movement practices, they were deemed beyond the scope of our review, which specifically delves into the integration of machine learning into dance performance.

The decision to exclude publications focusing on somatics and authentic movement stems from the broader context of the post-human body, where traditional corporeal notions intersect with digital technologies. In this age of digital transformation, where the distinctions between physicality and virtuality blur, it is imperative to examine the implications of machine learning on dance practice thoroughly. By narrowing our scope to publications addressing the creative and technological dimensions of dance, we aim to provide a focused analysis that reflects the current state of the field and anticipates its potential future trajectories.

We limited the results to publications between 2010 and 2023 and English-language publications due to resource constraints. The screening process was conducted in two stages: identification and screening of peer-reviewed studies. The results were then merged at the eligibility stage of the review process. Overall, the search strategy for this systematic review aimed to ensure comprehensiveness and inclusivity while also maintaining focus on the four primary areas of intersection between machine learning and dance performance.

3.2. Review procedure

The research centers on the four primary areas in which dance performance and machine learning intersect the most:

1. Choreographic creation support – Leveraging machine learning for choreographic creation involves meticulous analysis of a choreographer’s or dance company’s movement vocabulary. This analysis serves as the foundation for generating innovative choreography. For example, machine learning algorithms can be trained to recognize specific movements or sequences and generate new variations based on that knowledge.

2. Dataset or network collection and training – Crucial to machine learning is the availability of extensive datasets. Dance performances offer a rich source of movement data, enabling machine learning algorithms to be trained on diverse collections of dance videos. This training equips algorithms to discern patterns and structures inherent in various dance styles, fostering a deeper understanding of movement diversity.

3. Improve techniques of human-pose detection – This domain focuses on advancing techniques for detecting and tracking human body poses. Machine learning algorithms contribute to a detailed analysis of dancers’ movement patterns, providing more precise representations of dance choreography. With the assistance of machine learning algorithms, researchers can analyze and understand the movement patterns of dancers in greater detail, leading to more accurate representations of dance choreography. This has applications in areas such as performance analysis, training and rehabilitation, and the creation of new dance works.

4. Perception and new visual representations of dance movement – The perception aspect explores how machine learning enhances the audience’s perception of dance movement. By developing novel representations and understandings of movement, machine learning algorithms can discern emotional states conveyed through dance or analyze the interplay of different body parts in motion. This innovation opens avenues for enriching the visual aspects of dance performance.

The selected categories—'Choreographic creation support’, ‘Dataset or network collection and training’, ‘Improved techniques of human-pose detection’, and ‘Perception and new visual representations of dance movement’ – play a pivotal role in the synergy of dance and machine learning.

‘Choreographic creation support’ underscores the convergence of artistic expression and technological innovation. Case studies, such as those conducted by Crnkovic and Pettee Crnkovic-Friis (2016) and Pettee et al. (2019), empower dance professionals with new tools to explore and expand their creative repertoire. The category of ‘Dataset or network collection and training’ is fundamental, as the efficacy of machine learning in dance relies on access to diverse datasets. These datasets enable algorithms to recognize and respond to a broad spectrum of choreographic styles. It is important to note that there is still a shortage of datasets incorporating dance movement data and content, as emphasized in the works conducted by Zhang et al. (2017) and Yalta et al. (2019). The importance of the category ‘Improved techniques of human-pose detection’ lies in refining the nuanced understanding of dance movements, enhancing accuracy, and facilitating applications across biomechanics and artistic exploration. This category, somewhat related to the previous one, emphasizes the significance of obtaining accurate dance movement data when integrating real-time body detection mechanisms, as presented in the studies by Kim and Kim (2018) and Priya and Arulselvi (2019). Lastly, ‘Perception and New Visual Representations of Dance Movement’ explores the perceptual dimensions of dance, fostering innovative visual representations that contribute to both artistic expression and analytical understanding. These categories collectively form the backbone of a symbiotic relationship between dance and machine learning. Works such as those by Lee et al. (2019) and Nogueira, Simões, and Menezes (2023) enrich the artistic and analytical dimensions of movement exploration. These categories are foundational pillars supporting the symbiotic evolution of dance and machine learning. As dance and machine learning continue to intersect, these categories play a pivotal role in advancing both disciplines.

4. Analysis

In this section, we embark on a detailed exploration of the outcomes derived from the systematic review analyses within the four designated categories. Our focus will be on dissecting the studies incorporated in each category, illuminating key findings, and outlining implications for future research. This chapter aims to furnish a thorough panorama of the systematic review analyses, facilitating a profound comprehension of the influence exerted by machine learning on dance performance.

4.1. Choreographic creation support

In 2016, Crnkovic-Friis (2016) created ‘Chorn-rnn’, a choreographic system for collaborative human-machine choreography or as a creativity catalyst for a choreographer. The system was trained with a neural network via RMS propagation through time. They collected contemporary dance movement data, which consisted of five hours of motion capture material. ‘Chorn-rnn’ is able to produce new choreographic sequences, based on the learned movement style represented in the training data. To create this work, the Crnkovic-Friis couple developed a neural network with an architecture based on Recurrent Neural Network (RNN), which is a network developed to process sequential data, see Sherstinsky (2020). To train the network, the couple collected information, through motion capture (MOCAP) mechanisms, namely the Microsoft Kinect v2 tool. With this mechanism, it was possible to record the dancer’s body while performing contemporary dance, over a total time period of five hours. After five hours of training the model, the choreography danced by the system began to resemble human movement. After two days of total training time, the model was not only able to generate new choreography but was also able to perform this creation based on the style danced by the choreographer.

In the same year, Jacob and Magerko (2015) started researching questions related to computational, cognition, and creativity, specifically in the dance field, through this expressive, movement-based interactive experience in the ‘LuminAI’ project. ‘LuminAI’ is an interactive installation that promotes social collaboration between humans and virtual agents projected onto a geodesic dome. It was showcased at a local arts event, where interview and video data were collected for analysis. By combining this analysis with interdisciplinary literature, they developed a taxonomy to guide the design of socially interactive human-agent systems. This taxonomy allowed them to evaluate the success of the installation in facilitating transitions between interaction levels and identify areas for improvement, such as conflicts in digital space, as well as unexplored concepts like agent awareness and collective action. Their work has contributed a comprehensive framework for analyzing socially interactive installations that involve both human movement and artificial agents, while also creating an art installation that explores the social dynamics of the dance movement and human-agent interactions, Long et al. (2017).

In 2019, similarly to Crnkovic-Friis (2016), the concept of generating choreography integrating deep learning methods became better known through the collaboration between Wayne McGregor’s company and Google Arts and Culture Wayne McGregor (2019). This work uses an algorithm that was trained through a collection of different choreographies from McGregor, and after several hours of learning, is then able to suggest new dance movements based on the choreographer’s contemporary repertoire. It generates a stick figure projection and provides real-time choreographic sequence suggestions to dancers based on McGregor’s movements. This tool has the capability to forecast the sequence of steps and movements for an individual dancer. In a manner akin to Crnkovic-Friis (2016), ‘Living Archive’ shares a common artistic concept: leveraging technology as a support for choreographic creation. Furthermore, both works acknowledge drawing inspiration from handwriting prediction techniques by Graves (2013). McGregor discussed this ground-breaking experiment, highlighting its transformative nature: ‘a piece that changes every night. We can generate 24,000 permutations – we don’t have that many shows! It’s a challenge for the dancers, not knowing what they’re doing [very far] in advance, but then making meaning from it. It’s a little experiment that I think speaks directly to the idea of life-writing. Life unfolds without our having control, and we have to deal with those instances. I believe that can be a truly beautiful thing’, as cited by McGregor for The Guardian (2017). One intriguing aspect arising from these projects is the notion of authorship in the context of AI collaboration. As machines actively contribute to the creative process, it prompts a discussion about the dual authorship – human and machine. How do we attribute artistic ownership when the AI system plays a role in generating choreographic elements and influencing the final output?

The study ‘Everybody Dance Now’ by Chan et al. (2019) proposed a method used for motion transfer from a source video to a novel target, which can be used as a creative catalyst or collaborative human-machine choreography. A method for ‘do as I do’ motion transfer from a source video of a person dancing to a novel (amateur) target after only a few minutes of the target subject performing standard moves. The method produces compelling results and includes a forensics tool for reliable synthetic content detection. The authors also release a first-of-its-kind open-source dataset of videos that can be legally used for training and motion transfer. Limitations include visual artifacts caused by loose clothing or hair, missing limb detections, and texture artifacts in clothing.

Also in 2019, ‘Body, Movement and Language’, by Bill T. Jones (2019), is another work that explores the intersection between contemporary dance and human pose, focusing on the work of choreographer Bill T. Jones Pettee et al. (2019). Jones collaborated with Google Arts and Culture to create a performance collection that integrates four experiments exploring the creative possibilities of AI technologies such as speech recognition and PoseNet, Google’s machine-learning model that estimates human poses in real-time, see Bill T. Jones (2019); Pettee et al. (2019). Unlike previous works that use the software as a catalyst for ideation, Jones’s work uses AI in the digital space to enhance performance, creating an effect that goes beyond what could have been achieved without the technology. What distinguishes this work from others is the way it involves the virtual component itself as an integral part of the final result (Table 2).

Table 2. Detailed summary of results related to Dataset or Network Collection and Training.

Title	Author(s)	Year	Study design	Type of dance	Type of technology	Main findings	Classification	Arguments to classify
Martial Arts, Dancing and Sports dataset: A challenging stereo and multi-view dataset for 3D human pose estimation	Zhang et al.	2017	Dataset Creation and Evaluation	Martial Arts (Tai-chi and Karate), Dancing (hip-hop and jazz), Sports (basketball, volleyball, football, rugby, tennis and badminton)	Depth-based Pose Estimation	The proposed method achieved high accuracy in detecting poses and capturing subtle movements for all three types of dance. The method was compared to a color-based pose estimation method and demonstrated its superiority in terms of accuracy and robustness to lighting conditions and occlusions. The results suggest that the depth-based method has the potential to be a useful tool for motion analysis and performance assessment in a variety of dance styles. The study also evaluated a variety of tracking frameworks, including a generative tracker based on the annealing particle filter and robust likelihood function, a discriminative tracker using twin Gaussian processes, and hybrid trackers, such as Personalized Depth Tracker. The results of the evaluation suggest that discriminative approaches perform better than generative approaches when there are enough representative training samples, and that the generative methods are more robust to diversity of poses, but can fail to track when the motion is too quick for the effective search range of the particle filter. The data and the accompanying code were made available to the research community.	Dataset or network collection and training and Improve techniques of human-pose detection	The work described in the article can be classified as a dataset or network collection and training, as well as an improvement in techniques of human-pose detection. The authors created a new dataset that includes challenging actions from different dance styles and sports, and they used this dataset to evaluate and compare different human pose estimation methods. Their proposed depth-based method demonstrated higher accuracy and robustness compared to a color-based method, and could potentially be used for motion analysis and performance assessment in a variety of dance styles.
Let’s Dance: Learning From Online Dance Videos	Castro et al.	2020	Experimental	Ballroom dances	Video, Optical Flow, Multi-Person Pose Estimation	The study article makes three main contributions. Firstly, it provides an analysis of both baseline and state-of-the-art approaches in video classification. Secondly, it proposes a general method for learning from multiple motion parameterizations concurrently in video. Lastly, the study authors introduce a new dataset called ‘Let’s Dance’ consisting of 1000 highly dynamic dance videos. This dataset is unique compared to existing video datasets and is intended to motivate further research and understanding of motion parameterization in video classification.	Dataset or network collection and training	The work can be classified as ‘Dataset or network collection and training’ as it describes the creation of a new dataset, ‘Let’s Dance’, that is focused on motion-heavy categories and requires the use of human motion as a key distinguisher. The work also compares the performance of numerous state-of-the-art techniques on this dataset using three different representations (video, optical flow, and multi-person pose data) and analyzes their approaches to motion parameterization for learning to categorize online dance videos.
Indian Classical Dance Action Identification and Classification with Convolutional Neural Networks	Kishore et al.	2018	Experimental	Indian Classical Dance	Convolutional Neural Networks	Proposes the classification of Indian classical dance actions using convolutional neural networks (CNN). Perform human action recognition on Indian classical dance videos from both offline (controlled recording) and online (live performances, YouTube) data. Achieved a 93.33% recognition rate compared to other classifier models reported on the same dataset.	Dataset or network collection and training	The authors use convolutional neural networks (CNNs) to classify Indian classical dance actions and achieve a high recognition rate on a dataset of dance videos from both controlled recordings and live performances on YouTube.
Deep Learning for Human Pose Classification using Multi View Dataset	Priya and Arulselvi	2019	Experimental	Karate and Bharathanatyam	Deep Convolutional Neural Network	The study aimed to create a multi-view dataset of Karate and Bharathanatyam poses and classify them using a deep learning algorithm. The proposed work achieved a pose classification accuracy of 62%. The authors aim to extend the work for classifying diverse poses in the future. The study also suggested that further tuning of parameters or applying other machine learning techniques can improve the classification of certain poses.	Dataset or network collection and training	The focus of the study is on creating a multi-view dataset of poses from two different dance forms and using deep learning algorithms to classify them. The authors aim to extend this work to classify more diverse poses in the future, indicating a focus on data collection and network training.
Living Archive	Wayne McGregor	2019	Dataset Creation and Evaluation	Contemporary Dance	Deep Convolutional Neural Network	Studio Wayne McGregor collaborated with Google to develop the ‘Living Archive’, a system trained on the choreographer’s archive. The online version by Google enables users to interact with the archive by dancing in front of their webcam and searching for similar movement patterns. Notably, McGregor describes a ‘live dialogue’ in a dance performance resulting from the project. In this context, the system generates dance movements in response to the performers, who, in turn, respond to the system. This dynamic exchange between dancers and the system suggests a form of emergent interaction, aligning with our defined framework. While a more in-depth analysis would be beneficial, exploring the intricacies of this interaction, it exceeds the scope of our present study.	Dataset or network collection and training and Choreographic creation support	This work involves the collection and utilization of a dataset comprising McGregor’s choreographic movements. The project leverages machine learning algorithms to enable a novel interaction where individuals can dance in front of their webcams, and the system searches the archive for similar movement patterns. The underlying machine learning model is trained on this dataset, allowing it to recognize and respond to a diverse range of choreographic expressions.
Exploring Rare Pose in Human Pose Estimation	Hwang et al.	2020	Experimental Study	Not specified	K-means clustering algorithm	The issue of data imbalance between different poses in the human pose estimation problem was tackled by exploring unusual and rare poses. A simple K-means clustering algorithm was applied to identify rare poses without additional learning. Outliers that are far from the nearest cluster center can be defined as rare poses, and the accuracy decreases as the distance between the data point and the cluster center increases. Three methods were proposed to address the problem of data scarcity: addition of rare pose duplicates, addition of synthetic rare pose data, and weighted loss based on the distance from the cluster. The highest increase in score was 13.5 mAP at the rare pose data using the proposed methods.	Dataset or network collection and training	The work can be classified as ‘Dataset or network collection and training’ as it involves exploring and addressing the issue of data imbalance in human pose estimation by developing a clustering algorithm and proposing methods to improve performance on rare poses. It can also be classified as ‘Choreographic creation support’ as rare poses are a valuable component of choreographic expression in dance and improving the detection of these poses can support the creation of more diverse and complex choreography.
Learn to dance with aist++: Music conditioned 3d dance generation	Ruilong Li et al.	2021	Multi-modal dataset and network design	Old School (Break, Pop, Lock and Waack) and New School (Middle Hip-hop, LA-style Hip-hop, House, Krump, Street Jazz and Ballet Jazz).	Multi-view videos with known camera poses	AIST++ is a multi-modal dataset of 3D dance motion and music, covering 10 dance genres with multi-view videos with known camera poses. They proposed a Full-Attention Cross-modal Transformer network called FACT for generating 3D dance motion conditioned on music. They show that their proposed changes in architecture design and supervision overcome shortcomings in generating realistic dance motion attuned to the input music. Their method outperforms recent state-of-the-art methods both qualitatively and quantitatively in user studies.	Dataset or network collection and training and Choreographic creation support	The creation of the AIST++ dataset involves the collection and organization of multi-modal data for various dance genres, which falls under the ‘Dataset or network collection and training’ category. The development of the Full-Attention Cross-modal Transformer network called FACT for generating 3D dance motion conditioned on music is a form of ‘Choreographic creation support’ as it assists in the creation of new dance movements.
Between Us	MotionBank, Staatstheater Mainz – Tanzmainz, Kunsthalle Mainz | Main article by Danae Kleida	2021	Collaborative work between MotionBank, Staatstheater Mainz – Tanzmainz, and Kunsthalle Mainz	Contemporary Dance	Motion-Capture; Marker-less System: Employing machine learning for real-time skeleton calculation; Video documentation; Online score	The ‘Between Us’ project’s primary contribution in dataset and network aspects lies in its comprehensive documentation of a six-week creative process. The dataset includes motion capture data, synchronized videos, and sound recordings, forming a rich resource for training networks and refining choreographic algorithms. The decision to share this dataset freely underscores the project’s commitment to openness and collaboration, making it a valuable asset for researchers and artists exploring the convergence of dance and technology.	Choreographic creation support and Dataset or network collection/training	The extensive documentation of the six-week creation process, incorporating motion capture, video recordings, and annotations, forms a rich dataset. This dataset becomes a valuable resource for training networks and refining choreographic algorithms. The project’s commitment to making the recorded data freely available to the public aligns with principles of dataset sharing and collaborative learning within the dance and technology communities.

The research conducted by Pettee et al. (2019), titled ‘Beyond Imitation: Generative and Variational Choreography via Machine Learning’, introduces a deep learning model designed to generate innovative dance movements by building upon existing input sequences. The study presents configurable machine-learning tools adept at producing new sequences of choreography and variations based on input movements. An essential outcome of this work is the documentation that facilitates the externalization of movement into the visual domain, providing a platform for the contemplation of choreographic design and architecture. Moreover, the research draws attention to the nuanced relationship between creative expression and research-based inquiry within the domain of dance-making. This exploration raises intriguing questions about authorship and collaboration when machines actively contribute to the creative process alongside human choreographers.

In 2021, the research paper ‘AI Choreographer: Music Conditioned 3D Dance Generation with AIST++’, authored by Ruilong et al. (2021), introduced a ground-breaking contribution to the field. The study presents AIST++, a new multi-modal dataset encompassing 3D dance motion and music. AIST++ incorporates a Full-Attention Cross-Modal Transformer Network (FACT) designed for generating 3D dance motion conditioned on music. Notably, the dataset boasts 5.2 h of 3D dance motion across 1408 sequences, spanning 10 dance genres. With multi-view videos and known camera poses, AIST++ stands out as the largest dataset of its kind. The experiments conducted on AIST++, coupled with user studies, demonstrate that the proposed method outperforms recent state-of-the-art techniques both qualitatively and quantitatively.

Despite these achievements, the study acknowledges the inherent limitations of the approach. The authors propose future exploration into the physical interactions between the dancer and the floor, as well as the generation of multiple realistic dances per music piece.

The final work in this section, ‘Between Us’, stands as a ground-breaking model in the realm of choreographic creation support, seamlessly blending dance, fine arts, and digital dance research, see Kleida (2021). A collaborative effort between Staatstheater Mainz, Kunsthalle Mainz, and Motion Bank at the Mainz University of Applied Sciences, the project centers around the creation of the dance piece ‘Effect’ by choreographer Taneli Törmä, see Törmä (2020). What sets this project apart is its commitment to thoroughly documenting and annotating the entire creative process, extending beyond the mere recording of the final performance. A noteworthy contribution comes from the Motion Bank team in the form of a comprehensive ‘Online Score’. This digital space delves into the intricacies of choreography development, providing not only the finished choreography but also a window into the creative journey. Visitors can explore significant moments, annotations, and the evolution of choreographic structures. Motion Bank utilizes motion capture data to emphasize specific movement principles through 3D animations and 2D visuals, Rittershaus et al. (2022). Particularly vital in ‘Effect’ were the dancers’ pathways, graphically rendered to meaningfully reflect the specific nature of the piece (Table 3).

Table 3. Detailed summary of results related to Improved Techniques of Human-Pose Detection.

Title	Author(s)	Year	Study design	Type of dance	Type of technology	Main findings	Classification	Arguments to classify
Martial Arts, Dancing and Sports dataset: A challenging stereo and multi-view dataset for 3D human pose estimation	Zhang et al.	2017	Dataset Creation and Evaluation	Martial Arts (Tai-chi and Karate), Dancing (hip-hop and jazz), Sports (basketball, volleyball, football, rugby, tennis and badminton)	Depth-based Pose Estimation	The proposed method achieved high accuracy in detecting poses and capturing subtle movements for all three types of dance. The method was compared to a color-based pose estimation method and demonstrated its superiority in terms of accuracy and robustness to lighting conditions and occlusions. The results suggest that the depth-based method has the potential to be a useful tool for motion analysis and performance assessment in a variety of dance styles. The study also evaluated a variety of tracking frameworks, including a generative tracker based on the annealing particle filter and robust likelihood function, a discriminative tracker using twin Gaussian processes, and hybrid trackers, such as Personalized Depth Tracker. The results of the evaluation suggest that discriminative approaches perform better than generative approaches when there are enough representative training samples, and that the generative methods are more robust to diversity of poses, but can fail to track when the motion is too quick for the effective search range of the particle filter. The data and the accompanying code were made available to the research community.	Dataset or network collection and training and Improve techniques of human-pose detection	The work described in the article can be classified as a dataset or network collection and training, as well as an improvement in techniques of human-pose detection. The authors created a new dataset that includes challenging actions from different dance styles and sports, and they used this dataset to evaluate and compare different human pose estimation methods. Their proposed depth-based method demonstrated higher accuracy and robustness compared to a color-based method, and could potentially be used for motion analysis and performance assessment in a variety of dance styles.
Dance Pose Identification from Motion Capture Data: A Comparison of Classifiers	Eftychios Protopapadakis et al.	2018	Comparative study of classifiers and data sampling schemes for dance pose identification based on motion capture data	Folk dances and their variations	Kinect sensor	The study aimed to evaluate the effectiveness of classification techniques in identifying dance types based on motion-captured human skeleton data. The focus was on identifying characteristic poses for each dance, using body joint information captured by a Kinect sensor. Six folk dances and their variations were used in the study, and various pose identification methods were tested, including temporal constraints, spatial information, and feature space distributions, to create an optimal training dataset. The study’s findings were then evaluated and discussed.	Improve techniques of human-pose detection	This work focuses on the development of classification techniques to recognize different dance types based on motion-captured human skeleton data acquired by a Kinect sensor. The study explores various pose identification schemes using temporal constraints, spatial information, and feature space distributions to create an adequate training dataset and evaluate the effectiveness of the classification techniques.
Real-time dance evaluation by markerless human pose estimation	Yeonho Kim and Daijin Kim	2018	Case Study	Traditional Korean Dance, K-Pop	RGB-D Camera	The study presents a case study of using an RGB-D camera for motion capture and pose estimation in traditional Korean dance. The proposed method achieved high accuracy in detecting poses and capturing movements, providing a useful tool for both training and performance analysis. The study also highlighted some limitations of the method, such as sensitivity to lighting conditions and occlusions, and suggested potential solutions for future improvements.	Improve techniques of human-pose detection	The paper presents a new method for markerless estimation of human poses that is invariant to complicated dance poses such as full-body rotation and self-occlusion, and also proposes a metric to quantify the similarity between dance sequences. The study then validates the proposed method on several benchmark datasets, achieving high accuracy in pose estimation and dance performance evaluation.
Everybody Dance Now	Chan et al.	2018	Video-to-video translation using pose as an intermediate representation	Not specified	Human pose extraction using OpenPose and pose-to-appearance mapping	A method for ‘do as I do’ motion transfer from a source video of a person dancing to a novel (amateur) target after only a few minutes of the target subject performing standard moves. The method produces compelling results and includes a forensics tool for reliable synthetic content detection. The authors also release a first-of-its-kind open-source dataset of videos that can be legally used for training and motion transfer. Limitations include visual artifacts caused by loose clothing or hair, missing limb detections, and texture artifacts in clothing. Future work could focus on improving pose detection systems, mitigating artifacts in loose/wrinkled clothing or hair, and improving training data for extrapolation to radically different poses.	Improve techniques of human-pose detection and choreographic creation support	This work proposed method is used for motion transfer from a source video to a novel target, which can be used as a creative catalyst or collaborative human-machine choreography. Although the work also involves the use of pose estimation as an intermediate representation, it seems to be primarily focused on the choreographic creation aspect rather than on improving techniques of human-pose detection or perception and new visual representations of dance movement.
Weakly-Supervised Deep Recurrent Neural Networks for Basic Dance Step Generation	Yalta et al.	2019	Experimental	Ballet	RGB + D sensors, motion capture system	Developed a system that can track the movements of ballet dancers and provide real-time feedback on posture and technique. The system was evaluated by professional dancers and showed that it can accurately detect and classify ballet movements.	Improve techniques of human-pose detection	The study proposes an optimization technique for weakly-supervised deep recurrent neural networks for dance generation tasks. The models generated a correlated motion pattern with a motion beat f-score similar to that of a dancer and lower cross entropy, and the models could be used for real-time tasks because of the low latency. However, the motion pattern is constrained to the given dataset, which will be addressed in future research.
A Deep Learning-Based End-to-End Composite System for Hand Detection and Gesture Recognition	Mohammed et al.	2019	Experimental study	Indian classical dance (ICD)	Convolutional neural network (CNN) for hand detection and gesture recognition	Proposed an end-to-end composite system for hand detection and gesture recognition using a convolutional neural network (CNN). The system achieved high accuracy in detecting hands and recognizing gestures, with potential applications in dance analysis and recognition. The proposed architecture is efficient and robust, achieving state-of-the-art performance in hand detection and gesture recognition tasks. The system has potential applications in dance analysis and recognition.	Improve techniques of human-pose detection	The work focuses on developing a deep learning-based architecture for hand detection and gesture recognition, which can be applied to various fields, including dance analysis and recognition. The study also evaluated the proposed approach on publicly available datasets, which demonstrates the potential of the system to improve the accuracy and robustness of human-pose detection and gesture recognition.
Machine learning model – based two – dimensional matrix computation model for human motion and dance recovery	Yi Zhang, Mengni Zhang	2020	Experimental	Not specified	Machine learning model-based two-dimensional matrix computation (MM-TDMC) approach	The MM-TDMC approach demonstrated promising performance in short-term motion recovery problems. The proposed MM-TDMC for human motion and dance recovery showed extensive experimental results and comparisons with existing methods. The study overcame a theoretical guarantee drawback for the recovery of nonlinear movement information in the two-dimensional matrix computation model developed for linear information.	Improve techniques of human-pose detection	The study proposes a machine learning-based approach, MM-TDMC, for human motion and dance recovery, which showed promising performance in short-term motion recovery problems. The study focuses on the recovery of motion from degraded observations and the underlying complete sequence of motion, and it aims to overcome the nonlinear structure and fliming property integrated into the movements. The proposed approach is compared with other existing methods, including auto-conditioned recurrent neural network, multimodal corpus, low-rank matrix completion, and Kinect sensors methods

The previous selection and review have opened up new avenues for choreographic creation, collaboration, and exploration. These works showcase the potential of machine learning to generate innovative choreographic sequences, foster social collaboration, and enhance creativity and performance. Noteworthy examples such as Ruilong et al. (2021) and Chan et al. (2019) also shed light on the challenges and limitations associated with using machine learning in dance, such as visual artifacts and missing body detections. Conversely, works like Crnkovic-Friis (2016) and Wayne McGregor (2019) introduce a compelling question regarding the collaboration of artificial intelligence in art. What are the primary considerations surrounding authorship and ownership when the machine plays a role in creating a dance piece? This prompts a critical examination of the ethical and legal implications of AI involvement in artistic endeavors. The convergence of machine learning and dance performance in choreographic creation presents promising possibilities for the future of dance as an art form. Nevertheless, the integration of AI poses challenges, and artists must be prepared to address potential negative impacts arising from this collaboration. This calls for a nuanced understanding of the ethical, legal, and artistic dimensions of AI-assisted choreography.

4.2. Dataset or network collection and training

This section zooms in on the influence of ‘Dataset or network collection and training’ on the progression of machine learning research in dance performance. We present comprehensive case studies where machine learning algorithms were trained on datasets of videos to grasp the intricacies inherent in various dance styles.

In 2017, Zhang et al. (2017) introduced the ‘MADS dataset (Martial Arts, Dancing, and Sports)’. This dataset is unique in its incorporation of challenging actions spanning diverse dance styles like hip hop and jazz, martial arts including Tai-chi and Karate, and various sports such as basketball, volleyball, football, rugby, tennis, and badminton. The primary focus of this work revolves around evaluating and comparing different human pose estimation methods. The dataset was curated by capturing movements performed by two martial arts masters, two dancers, and an athlete using multiple cameras or a stereo-depth camera. Notably, their depth-based approach exhibited superior accuracy and robustness compared to color-based methods, showcasing its potential applicability in motion analysis and performance assessment across a spectrum of dance styles. It’s worth noting, as acknowledged by the authors themselves, that the dance movements included in the dataset were not executed by dance professionals. Consequently, the system learned empirical movements rather than specific dance techniques from genres like jazz or hip-hop (Table 4).

Table 4. Detailed summary of results related to Perception and new visual representations of dance movement.

Title	Author(s)	Year	Study Design	Type of Dance	Type of Technology	Main Findings	Classification	Arguments to classify
Becoming	Open-Ended Group in collaboration with Wayne McGregor and Nick Rothwell	2013	Experimental	Contemporary dance	Pose estimation and synthesis-by-analysis learning framework	The ‘Becoming’ project delves into the synthesis of artificial intelligence and dance movement by creating an abstract, artificially intelligent entity. Through a meticulous process, the project analyzes each of the 1240 shots from an iconic 1980s science-fiction film, employing computer vision techniques to extract geometry, color, and movement. The abstract agent, functioning heuristically, navigates its own body’s configuration space to replicate the film’s movements. This iterative process, guided by visible captions, culminates in a visually striking representation displayed on a large 3D screen at a human scale.	Perception and new visual representations of dance movement	‘Becoming’ introduces a novel approach to visualizing dance movement by harnessing artificial intelligence, transcending conventional representations. Secondly, the incorporation of human pose detection and machine learning techniques enriches the audience’s visual experience, presenting abstract visual models that redefine the perception of dance. The project’s innovative use of technology aligns with the category’s objectives, contributing to the ongoing discourse on the intersection of dance, technology, and visual representation. Overall, ‘Becoming’ stands as a groundbreaking exploration that challenges traditional boundaries in the realm of dance perception.
Designing Co-Creative AI for Public Spaces	Long et al.	2019	Literature Review, Case Studies	Not specified	AI technology	Design principles for co-creative AI in public spaces can broaden public AI literacy and engage a broad range of participants. This work is focused on AI literacy and co-creative AI in public spaces, it does not involve any specific application of AI to dance or choreography. However, it could be relevant to the development of new ways to represent dance movement and enhance audience perception through the use of AI in public spaces.	Perception and new visual representations of dance movement	Dancers and choreographers could potentially use it to create new and innovative dance performances that incorporate audience interaction and improvisation. Additionally, this work’s emphasis on engaging a broad range of participants in public spaces through co-creative AI could potentially help to broaden the audience for dance performances and increase public engagement with the art form.
Dancing to Music	Lee et al.	2019	Experimental	Not specified	Pose estimation and synthesis-by-analysis learning framework	The proposed method can synthesize realistic, diverse, style-consistent, and beat-matching dances from music. The article proposes a novel framework called ‘synthesis-by-analysis’ to generate dance from music. In the analysis phase, the framework decomposes a dance into a series of basic dance units and learns how to move. In the synthesis phase, the framework composes a dance by organizing multiple basic dancing movements seamlessly according to the input music.The proposed method can synthesize realistic, diverse, style-consistent, and beat-matching dances from music, as demonstrated by both qualitative and quantitative results.	Perception and new visual representations of dance movement	This work can be classified as ‘Perception and new visual representations of dance movement’ since it proposes a framework for generating dance movements from music input, which involves analyzing and synthesizing basic dance units to create a realistic and diverse dance performance that is consistent with the style and beat of the music.
Dance to your own drum: Identification of musical genre and individual dancer from motion capture using machine learning	Carlson et al.	2020	Experimental	Freestyle dance to 8 genres of music	Motion Capture	The study aimed to explore the degree to which music induced movements are genre-specific using motion capture data from participants dancing freely to eight different musical genres. The researchers used a Support Vector Machine model to classify the data by both genre and individual dancer. The study found that individual classification was notably more accurate than genre classification, which was contrary to expectations. These findings suggest that the influence of musical genre on dance movements may not be as strong as previously thought and that individual differences play a significant role in shaping dance movements. The study also highlights the importance of embodied cognition and culture in understanding how music and dance are related.	Perception and new visual representations of dance movement	The study aimed to explore the relationship between music and dance movements and how they are influenced by musical genre. The researchers used motion capture data to analyze the dance movements of participants in response to eight different musical genres. The study used a Support Vector Machine model to classify the data by both genre and individual dancer, providing new insights into the relationship between music and dance. Overall, the study contributes to our understanding of how music and dance are related and highlights the role of embodied cognition and culture in shaping dance movements.
F O R M S – Creating new visual perceptions of dance movement through machine learning	Nogueira et al.	2022	Case study	Contemporary dance	Human pose detection framework	The intersection of dance movement and machine learning techniques through a human pose detection framework can create visual results that enrich the dance performance in real-time. The framework provides different visual forms through human pose detection from dancers, combining geometric and curvilinear forms. The study culminates with a real case study presented in a contemporary dance performance on the stage, where the performance was created with professional young dancers that explored their movement through the framework developed and enriched the experience of the public.	Perception and new visual representations of dance movement	The authors developed a framework that uses human pose detection and machine learning techniques to create new visual forms that enrich the perception of dance movement for the audience. The work is focused on the intersection between dance movement and technology and aims to create abstract or literal visual models that combine geometric and curvilinear forms. The main objective is to enhance a new visual perspective of dance movement during a performance.
Designing Interactive Visuals for Dance from Body Maps: Machine Learning and Composite Animation Approaches	Correia et al.	2022	Multi-stage study	Contemporary dance	Machine learning and Composite Animation	Both prototypes (MLIV and CAIV approaches) were successful in addressing the aim of visualizing non-visible bodily aspects of dancers, with some limitations. Two software systems were developed, released as open-source, and in their design frameworks. The study confirmed that both approaches can successfully transform static body maps into interactive visuals. Both approaches allow for novel pathways to create idiosyncratic and introspective visuals for dance, grounded in the somatic experience of dancers and how they convey it visually through body maps.	Perception and new visual representations of dance movement	This work reflects an exploration of interactive visuals for dance performance through the design of two software systems that use machine learning (MLIV) and composite animation (CAIV) to convert static body maps into interactive visuals. The work aims to make non-visible bodily aspects of dancers apparent to the audience and is grounded in soma design and the somatic experience of dancers. The study involved a multi-stage participatory study with 12 dancers, and the results confirm that both prototypes were successful in addressing the aim of visualizing unseen aspects of the body. The main contributions of the research are the two approaches for designing interactive visuals from body maps and their analysis, which provide new pathways for creating interactive visuals for dance performance from body maps. The work can be useful to designers in the field of dance and technology, soma design, and more broadly in the field of embodied interaction.

In 2018, Kishore et al. (2018) introduced a method for classifying Indian classical dance actions using convolutional neural networks (CNNs) in their study titled ‘Indian Classical Dance Action Identification and Classification with Convolutional Neural Networks’. The primary objective of this research was to identify human actions in Indian classical dance videos, encompassing both controlled offline recordings and dynamic online sources like live performances and YouTube videos. Their CNN architecture exhibited superior training and validation accuracies in comparison to previously proposed models for Indian classical dance classification. Notably, the method reported lower training and validation loss, and achieved an impressive accuracy rate of 93.33% on the dataset, outperforming other classifiers documented in the literature. This work serves as a testament to the potential efficacy of machine learning approaches, particularly CNNs, in recognizing and classifying intricate human poses within the domain of Indian classical dance.

In ‘Let’s Dance’, Castro et al. (2018) introduced an expanding dataset of 1000 videos centered around ten visually overlapping dance categories. These categories, including ballet, flamenco, latin, square, tango, breakdance, foxtrot, quickstep, swing, and waltz, showcase various dance styles. Notably, four of the styles involve paired performances in ballroom dancing, such as tango, foxtrot, quickstep, and waltz. Each video in the dataset is 10 s long, captured at 30 frames per second. The dataset emphasizes the significance of motion for accurate classification, addressing variations in dance patterns arising from factors like music tempo and the captured nuances of the dancer’s body. To prioritize motion as a key classification characteristic, the dataset utilizes three distinct representations – video, optical flow, and multi-person pose data. The research extensively analyses these representations, exploring their approaches to motion parameterization for effective learning in categorizing online dance videos.

In 2019, Priya and Arulselvi (2019) proposed the research ‘Deep Learning for Human Pose Classification using Multi-View Dataset’. They created a multi-view dataset of Bharathanatyam and Karate poses and classified them using a deep learning algorithm. The focus of the study is on creating a multi-view dataset of poses from dance and sport and using deep learning algorithms to classify them. The proposed work achieved a pose classification accuracy of 62%. The study also suggested that further tuning of parameters or applying other machine learning techniques can improve the classification of certain poses. The authors aim to extend this work to classify more diverse poses in the future, indicating a focus on data collection and network training.

As mentioned in the previous category overview, the collaboration between Studio Wayne McGregor and Google resulted in the presentation of ‘Living Archive’. This work is deeply integrated into McGregor’s choreographic realm, functioning as a digital repository for his dance movements. Google’s online manifestation of the archive offers a unique interactive experience, allowing users to interact with McGregor’s choreography dataset by dancing in front of a webcam, initiating a search for analogous movement patterns within the archive, see Murray-Browne and Tigas (2021). In the article authored by Leprince-Ringuet (2018); McGregor articulated his intention to leverage this extensive archive of work intriguingly. The crux of this work lies in the collection and utilization of a specialized dataset encompassing McGregor’s distinctive choreographic movements and expressions. Here, machine learning algorithms play a crucial role, facilitating interaction and empowering the system to recognize and respond to a diverse spectrum of choreography derived from a collection of McGregor’s movements.

In 2020, the research work ‘Exploring Rare Pose in Human Pose Estimation’ by Hwang, Yang, and Kwak (2020) proposed a new criterion for defining rare pose samples and methods to improve the performance of pose estimation for rare poses. The proposed methods have the potential to be further developed and utilized in dance performance analysis, allowing for a more accurate estimation of unique movements and styles. By addressing the challenge of data imbalance and improving accuracy for rare poses, the proposed methods could contribute to the development of more advanced tools for choreographic creation, training, and performance analysis in dance. The experiments conducted on the COCO and MPII Human Pose Dataset, see Andriluka et al. (2014), showed that the methodology improved the performance of rare poses compared to the baseline models. Although the overall pose performance did not significantly improve due to the small percentage of rare poses, the study highlights the importance of addressing data imbalances in human pose estimation, especially in dance, where rare poses can occur frequently.

The ‘Between Us’ revolutionizes the landscape of dance-related datasets and computational tools. The dataset, based on the dance piece ‘Effect’ by Törmä (2020), comprises meticulous six-week rehearsal documentation, culminating in a 60-minute motion capture recording, eight HD video perspectives, and a 4-channel sound recording, see Kleida (2021). This ambitious continuous recording challenges conventional motion capture norms by opting for a single, uninterrupted take of the entire performance, presenting a paradigm shift in data collection methodology, Rittershaus et al. (2022). The utilization of marker less motion capture technology enhances real-time full-body skeleton calculation through machine learning, contributing to the project’s technical innovation. Nevertheless, this ground-breaking approach prompts crucial reflections on data ownership and ethical considerations. As the continuous recording method challenges established practices, questions arise regarding who owns the recorded data and the ethical implications of its free distribution and use. Positioned at the forefront of choreographic exploration, the ‘Between Us’ dataset not only contributes to contemporary dance research but also shapes the future of computational tools in this domain (Table 5).

Table 5. Summary of all final results describing the impact, main application, and disadvantages/limitations of each work.

Q1: What are the main applications of machine learning techniques in dance performance, and how do they impact choreographic creation support, dataset or network collection and training, improved techniques of human-pose detection, perception, and new visual representations of dance movement?

Q2: How does the incorporation of machine learning techniques impact the creativity of performers, choreographers, participants, and dance professionals?

Year	Title	Author(s)	Impact and Main Application	Disadvantages or Limitations
2013	Becoming	Open-Ended Group in collaboration with Wayne McGregor and Nick Rothwell	By subjecting an iconic science-fiction film’s shots to intricate computer vision techniques, the project pioneers a unique synthesis of artificial intelligence and dance, offering a novel lens through which choreographers can derive inspiration. The abstract, heuristically driven agent’s attempts to mirror the film’s movements not only demonstrate the potential of machine learning in dataset and network collection but also provide a powerful tool for choreographers to envision and experiment with new movement possibilities. The incorporation of machine learning techniques, specifically human pose detection and iterative approximations, opens avenues for redefining choreographic creation, pushing the boundaries of what is conceivable in dance performance.	The ambiguity of the form, while artistically intriguing, may pose a barrier to immediate understanding for viewers unfamiliar with the underlying technology. Moreover, the dependence on a specific film’s shots for inspiration could limit the adaptability of the machine learning model to diverse choreographic styles. This limitation might hinder its widespread applicability across different dance genres and themes. Additionally, the reliance on heuristics and approximations may introduce a level of unpredictability in the AI’s output, potentially requiring careful consideration and fine-tuning by choreographers.
2015	Interaction-based Authoring for Scalable Co-creative Agents	Jacob & Magerko	The main contribution of this study is the development of an interaction-based authoring approach for co-creative systems, which combines ideas from case-based learning and imitative learning to support the generation of novel choreographic material in an open-ended co-creative application domain. The study provides a novel approach that could significantly impact the co-creative application domain by providing new tools and techniques for generating and exploring movement data in a collaborative and open-ended way.	One of the main limitations is the small sample size used in the study. Furthermore, some negative feedback suggested that more evidence was required to demonstrate that the agent was doing something other than just mimicking the user. Another limitation is that video analysis showed that novice users had a hard time triggering the system’s rhythmic repeated movement gesture segmentation mechanic, which may limit the system’s applicability in broader co-creative contexts.
2016	Generative Choreography using Deep Learning	Luka and Louise Crnkovic-Friis	Chor-rnn can produce a higher-level compositional cohesion than just generating sequences of movement, which could significantly impact the choreographic creation support. The system can be used for collaborative human-machine choreography or as a creativity catalyst for a choreographer. The impact of this system could be seen in its ability to explore and generate new choreographic ideas and movements that might not have been thought of previously, providing new tools and techniques for choreographic creation.	One of the main limitations is that it is currently limited to generating choreographies for a solo dancer, which may not be suitable for group performances. Another limitation is that the Kinect sensor used in the system cannot directly handle occluded body parts, which may be problematic even with one dancer and make it nearly impossible to capture interactions between multiple dancers. The proposed solution is to use multiple Kinect sensors and combine their data, which may increase the complexity and cost of the system.
2017	Martial Arts, Dancing and Sports dataset: A challenging stereo and multi-view dataset for 3D human pose estimation	Zhang et al.	Development and evaluation of a depth-based method for human-pose detection in dance demonstrated superior accuracy and robustness compared to a color-based method. The method has the potential to be a useful tool for motion analysis and performance assessment in a variety of dance styles. The study evaluated and compared different tracking frameworks, including generative and discriminative approaches, and found that discriminative methods perform better with sufficient training samples, while generative methods are more robust to pose diversity but can fail with quick motion. The authors also created a new dataset that includes challenging actions from different dance styles and sports, which could be useful for future research on human-pose detection.	Errors in tracking are induced by fast motions, such as arm waving or jumping in dance sequences. This is since the tracker’s initialization in the current frame is based on the tracked pose in the previous frame, and when there is a large difference between frames, the initialization can be far from the observation, making it difficult for the tracker’s Gaussian diffusion model to cover such a large search space. Another weakness of the work is the occurrence of self-occlusions in the MADS dataset, which is a common issue when monitoring and estimating human poses.
2018	Everybody Dance Now	Chan et al.	The development of a method for ‘do as I do’ motion transfer from a source video of a person dancing to a novel target after only a few minutes of the target subject performing standard moves. This approach enables the generation of compelling results and includes a forensics tool for reliable synthetic content detection. Furthermore, the authors release a first-of-its-kind open-source dataset of videos that can be legally used for training and motion transfer, which could significantly improve the accessibility of this technology.	The visual artifacts caused by loose clothing or hair, missing limb detections, and texture artifacts in clothing may affect the quality and realism of the generated motion sequences. Future work could focus on improving pose detection systems, mitigating artifacts in loose/wrinkled clothing or hair, and improving training data for extrapolation to radically different poses, which could enhance the reliability and generalizability of the method.
2018	Dance Pose Identification from Motion Capture Data: A Comparison of Classifiers	Eftychios Protopapadakis et al.	The main contribution of this study is the development of classification techniques to recognize different dance types based on motion-captured human skeleton data acquired by a Kinect sensor. The study explores various pose identification schemes using temporal constraints, spatial information, and feature space distributions to create an adequate training dataset and evaluate the effectiveness of the classification techniques. The impact of this work is in the potential application of these techniques in the fields of dance education and performance, as well as in motion capture technology for other fields such as sports and medicine. By accurately identifying characteristic poses for each dance, these techniques can help improve choreography, training, and the analysis of dance performances.	The research did not attain fully satisfactory performance due to the limited capability of the Kinect sensor to capture complex spatiotemporal variations and the low amount of data available for training the classifiers. These limitations may have affected the overall performance of the system.
2018	Real-time dance evaluation by markerless human pose estimation	Yeonho Kim and Daijin Kim	The main contribution of the study is the development of a unified framework that evaluates dance performance using marker less estimation of human poses. The proposed method is able to handle complicated poses such as full-body rotation and self-occlusion using ridge data and data pruning. The proposed method achieved high accuracy in pose estimation, with a mean average precision of 0.9358 and an average pose error of 3.88 cm. Furthermore, the proposed method showed a 98% concordance with experts’ evaluation of dance performance. The findings of the study show that the proposed method provides a useful tool for both training and performance analysis, which can be applied in various dance-related fields.	The method presented in the study was found to have limitations, specifically regarding its sensitivity to lighting conditions and occlusions. The study acknowledged these limitations and suggested potential solutions for future improvements.
2019	Weakly-Supervised Deep Recurrent Neural Networks for Basic Dance Step Generation	Yalta et al.	Development of a system that identifies and classifies ballet movements in real-time. This is a tool for improving posture and technique. Furthermore, the paper presents an optimization strategy for weakly-supervised deep recurrent neural networks for dance generation tasks that creates motion patterns comparable to those of a dancer while having a lower cross-entropy.	The study notes that the motion pattern is constrained to the given dataset, which is an area for future research.The proposed models in the study demonstrated a significant reduction in performance, almost half when an untrained music track of the same or different genres was used as input. Additionally, the proposed models did not outperform a model trained to only process music beats, i.e. MADMOM, which could be due to the music processing approach used in the models. The motion pattern generated by the proposed model is constrained to the given dataset and affected by the diversity of the trained patterns, which will require further research to address.
2019	Beyond Imitation: Generative and Variational Choreography via Machine Learning	Pettee at al.	Development of configurable machine-learning tools for generating novel sequences of choreography and variations on input choreographic sequences. These tools allow for documentation of movement and reflection on choreography design and architecture. The study also highlights the distinction between creative expression and research-based inquiry in dance-making.	The work provides creative tools for documenting and reflecting on choreographic design and architecture, but relying solely on the visual aspect of choreography may be limiting. Generating novel movements allows for evaluating subjective preferences and negotiating the aesthetic dimension. The tools enable documenting the systematic, recursive process of dance-making. Future work will focus on developing methods for pre-processing inputs, using neural network-based models for time-series data, and sampling from latent spaces.
2019	Deep Learning for Human Pose Classification using Multi View Dataset	Priya and Arulselvi	The main contribution of this study is creating a multi-view dataset of Bharathanatyam and Karate poses and using deep learning algorithms to classify them. The proposed work achieved a pose classification accuracy of 62%, which provides a baseline for future improvements. The impact of this work is significant as it provides a framework for collecting and classifying pose data from different dance forms, which can be used in various applications, such as dance education, rehabilitation, and entertainment.	Small number of images to train the network. A deep learning method based on the Keras library and running on top of Tensorflow was used for categorization. Although the suggested study obtained a pose classification accuracy of 62%, certain stances were poorly classified, highlighting the need for future improvements. To solve this issue, the authors want to create a multi-view dataset that includes activities from martial arts, sports, and dancing for diverse positions.
2019	A Deep Learning-Based End-to-End Composite System for Hand Detection and Gesture Recognition	Mohammed et al.	The main contribution of the study is proposing an end-to-end composite system for hand detection and gesture recognition using a convolutional neural network (CNN) with high accuracy in detecting hands and recognizing gestures, which has potential applications in dance analysis and recognition. The impact of this work lies in its potential applications in dance analysis and recognition, as well as in various other fields. The study also evaluated the proposed approach on publicly available datasets, which demonstrates the potential of the system to improve the accuracy and robustness of human-pose detection and gesture recognition.	The study suggests several avenues for future research to address the limitations of the proposed method. One limitation is the need to explore other CNN networks for one-stage hand detection and focus on developing lightweight architectures for embedded vision applications. Although the current architecture focuses on recognizing static hand gestures, there is potential for extension to dynamic hand gesture recognition by incorporating the temporal aspect of the gesture. The study also highlights the need to explore solutions for multi-user interactions, as the current system is limited to single-user interactions.
2019	Designing Co-Creative AI for Public Spaces	Long et al.	This work proposes design principles for co-creative AI in public spaces, with the potential to broaden public AI literacy and engage a diverse range of participants. This could lead to innovative dance performances that incorporate audience interaction and improvisation, benefiting dancers and choreographers. Moreover, the emphasis on engaging a wide range of participants through co-creative AI could broaden the audience for dance performances and increase public engagement with the art form, contributing to its growth and sustainability.	The focus is not on applying AI to dance or choreography, the principles could inspire new ways to represent dance movement and enhance audience perception through AI in public spaces.
2019	Dancing to Music	Lee et al.	Development of a method for generating dance movements from music input that can synthesize realistic, diverse, style-consistent, and beat-matching dances. This work’s main contribution is in the field of ‘Perception and new visual representations of dance movement’, as it provides a framework for analyzing and synthesizing basic dance units to create a realistic and diverse dance performance that matches the style and beat of the music. This work has the potential to impact the dance and music industries by providing a new way to create innovative and synchronized dance performances that enhance the audience’s experience.	Although the entire model considers movement coherence, it may compromise the beat hit rate of individual motions, making it potentially inappropriate for some dancing styles that need strong beat coverage. The model’s performance may also be influenced by the comparatively low beat hit rate of real dances, which may be ascribed to noisy data from the automated collecting procedure, inaccurate posture extraction, and the approximation of the kinematic beat detector utilized. Because the LSTM model used in the study is incapable of multimodal generation, no multimodality score can be assigned to it.
2019	Living Archive	Wayne McGregor	‘Living Archive’ represents a paradigm shift in the application of machine learning techniques to dance performance, particularly in choreographic creation support and the cultivation of a living repository of movement data. By leveraging machine learning for dataset and network collection, McGregor creates an ever-evolving resource that captures and encapsulates the nuances of diverse movements. The incorporation of advanced human-pose detection techniques ensures meticulous and comprehensive cataloging of performers’ gestures, fostering a groundbreaking approach to preserving and accessing dance knowledge. This living archive becomes a reservoir for choreographers, providing not only a wealth of inspiration but also a tool for refining and expanding their creative processes in real-time.	One challenge lies in ensuring the ethical and secure use of the vast amount of movement data collected. Privacy concerns, consent issues, and the responsible management of the archive must be carefully navigated to prevent unintended consequences. Additionally, the reliance on historical movement data may inadvertently perpetuate biases present in the original performances, impacting the diversity and inclusivity of future choreographies. The machine learning techniques employed, while powerful, may not fully capture the subjective and embodied nature of dance, potentially limiting their application in certain creative contexts.
2020	Exploring Rare Pose in Human Pose Estimation	Hwang et al.	The proposed approaches, which included the insertion of rare pose duplicates, synthetic rare pose data, and weighted loss based on cluster distance, were demonstrated to boost accuracy considerably. This research has the potential to have an influence on the field of human pose estimation by increasing the recognition of unusual poses and facilitating the production of more diversified and complicated choreography.	Due to the small percentage of rare pose data, the overall performance did not significantly improve. The authors suggest that increasing the ratio of rare poses may enhance overall performance, and further development and utilization of synthetic data could lead to more improvements. The limitations of the proposed methods include the need for a larger representation of rare poses and more development of synthetic data.
2020	Let’s Dance: Learning From Online Dance Videos	Castro et al.	This work proposes a novel and general method for learning from multiple motion parameterizations concurrently in video. Lastly, the study authors introduce a new and unique dataset called ‘Let’s Dance’, which consists of 1000 highly dynamic dance videos that focus on motion-heavy categories and require the use of human motion as a key distinguisher.	The researchers first assumed that dancing and martial arts would suffice, but quickly recognized that martial arts faced a multiclass challenge and that separating different dance forms was difficult owing to availability limits. Furthermore, background areas in the input data may have an adverse effect on motion parameterization and the accuracy of video classification systems. To remedy this, the researchers tried filtering away background regions and improving motion parameterization in the input data, but this would take substantial work, particularly in independently categorizing posture estimates, which is known to be challenging.
2020	Machine learning model – based two – dimensional matrix computation model for human motion and dance recovery	Yi Zhang, Mengni Zhang	The study’s main contribution is the development of MM-TDMC, a machine learning-based approach for human motion and dance recovery, that has demonstrated promising performance in short-term motion recovery problems. The proposed approach aims to overcome the nonlinear structure and filming property integrated into the movements, and it has the potential to improve the recovery of motion from degraded observations and the underlying complete sequence of motion. The impact of this work could potentially lead to more effective methods for motion recovery in various fields, including dance analysis and recognition, sports analysis, and surveillance.	In the suggested model, the theoretical guarantee for recovering data from nonlinear motion is missing in the two-dimensional matrix computer model for linear data instead of using a current method.
2020	Dance to your own drum: Identification of musical genre and individual dancer from motion capture using machine learning	Carlson et al.	The study found that individual classification was notably more accurate than genre classification, which challenges previous assumptions about the influence of musical genre on dance movements. The study underscores the importance of embodied cognition and culture in understanding the relationship between music and dance movements. Ultimately, the study’s contribution to the field is its advancement of our understanding of the connection between music and dance movements, particularly in the role that individual differences play in shaping dance movements.	One disadvantage of the study is that it only focused on a few musical genres and did not cover a broader range of genres, which may limit the findings’ generalizability. Another problem is that the study depended on people spontaneously dancing to the music, which may not truly reflect how musical genres impact dance motions in a more structured or scripted situation. Furthermore, using a Support Vector Machine model for categorization may not be the best way for studying the complexities and nuance of dancing movements.
2021	Between Us	MotionBank, Staatstheater Mainz – Tanzmainz, Kunsthalle Mainz | Main article by Danae Kleida	This work makes a significant impact on the intersection of machine learning techniques and dance performance, particularly in the realms of choreographic creation support and dataset/network aspects. The primary contribution lies in the comprehensive documentation of a six-week creative process, resulting in a dataset comprising motion capture data, synchronized videos, and sound recordings. This dataset serves as a cornerstone for training networks and refining choreographic algorithms, providing a robust foundation for the development of innovative techniques in dance. By offering such a rich resource to researchers and artists, ‘Between Us’ becomes a catalyst for pushing the boundaries of choreographic possibilities, creating new pathways for experimentation, and influencing the broader landscape of dance performance augmented by machine learning.	One consideration is the potential ethical and privacy concerns associated with the extensive documentation of the creative process. The use of motion capture data and synchronized videos raises questions about the consent and privacy of performers, necessitating careful management and adherence to ethical standards. Additionally, the reliance on machine learning techniques introduces challenges related to bias and interpretation, as algorithms may inadvertently reinforce certain choreographic patterns or overlook diverse movement vocabularies. The openness of the dataset, while promoting collaboration, also requires vigilance in addressing issues related to data security and intellectual property. As machine learning techniques continue to evolve, ongoing efforts are essential to mitigate these limitations, ensuring that the incorporation of technology in dance remains ethical, inclusive, and respectful of the diverse perspectives within the field.
2021	Learn to dance with aist++: Music conditioned 3d dance generation	Ruilong Li et al.	The work’s major contribution is the construction of AIST++, a multi-modal dataset that comprises 3D dance motion and music for 10 dance genres with multi-view movies and known camera postures. They also proposed FACT, a Full-Attention Cross-modal Transformer network that produces 3D dancing motion based on music. This work has had an impact in that their recommended adjustments in architecture design and supervision have surmounted problems in creating realistic dancing motion attuned to the input music, beating previous state-of-the-art approaches in user surveys both subjectively and numerically.	The technology is kinematic-based and does not consider physical interactions between the dancer and the floor, which can lead to artifacts such as foot sliding and floating due to global translation. Additionally, the model is currently deterministic, so it cannot generate multiple realistic dances per music, which is a potential area for further exploration.
2022	F O R M S – Creating new visual perceptions of dance movement through machine learning	Nogueira et al.	The study’s authors created a framework that combines human posture identification and machine learning approaches, resulting in new visual forms that improve the audience’s perception of real-time dance movement. They used an actual case study with eight professional young dancers who explored their movementswithin the framework in a contemporary dance performance on stage in real-time for the audience. The major contribution of the study is the development of a new tool that enhances the audience’s experience during dance performances and investigates the junction of dance movement and technology.	The limitation of the work is the single application in a stage performance, i.e. one could explore the integration of this tool in experiences open to the public. It would be interesting to make this tool available for use by the whole community.
2022	Designing Interactive Visuals for Dance from Body Maps: Machine Learning and Composite Animation Approaches	Correia et al.	The study involved a participatory study with 12 dancers and resulted in the creation of two open-source software systems that successfully visualized non-visible bodily aspects of dancers. Its main contribution is the development of two approaches for creating interactive visuals from body maps. The work can be useful for designers in the field of dance and technology and more broadly in the field of embodied interaction, providing new pathways for creating interactive visuals for dance performances from body maps.	The study identified several limitations, including the lack of sufficient time and budget to adequately develop artistic ideas with the technology. This created a tension between the goals of the research project and the time and budget needed to develop choreography in the intersection of contemporary dance and technology.

The research work ‘Learn to Dance with AIST++: Music Conditioned 3D Dance Generation’, conducted by Ruilong et al. (2021), also discussed in the preceding category, introduces a transformative learning framework for 3D dance generation, conditioned on music. This work falls under the domain of ‘Dataset or Network Collection and Training’ since it entails training a pioneering Full Attention Cross-modal Transformer (FACT) model on an extensive dataset of dance motion paired with corresponding music. The model underwent training to anticipate future motion sequences and generate continuous motion during test phases in an auto-regressive manner. To facilitate this, the authors curated a substantial dataset named AIST++, encompassing over 5000 dance sequences harmonized with corresponding music. This dataset served as the foundation for training the FACT model, enabling the generation of realistic 3D dance motion intricately synchronized with the accompanying music.

The preceding works underscore the pivotal role of datasets in training machine learning algorithms for applications in the dance domain. The potential benefits of utilizing large datasets for machine learning in dance performance are substantial. Nevertheless, there is a perceived need to collect exhaustive and in-depth data from each dance style, as such a dataset could significantly enhance a comprehensive compilation of movements and poses across different dance styles and techniques. However, it is imperative to acknowledge concerns, such as the non-involvement of dance professionals in some research projects, particularly in the collection of dance poses. Studies without the involvement of dance professionals may lack the most accurate dance poses, as the machine is not trained with the expertise of dance professionals. Furthermore, the increased reliance on extensive datasets may pose challenges related to data bias, privacy concerns, and ethical considerations. Striking a balance between the advantages and potential negative impacts is crucial for ensuring the responsible and ethical use of machine learning tools in the realm of dance, representing a future concern and an ongoing struggle to address.

4.3. Improve techniques of human-pose detection

This section delves into the advancements in machine-learning techniques that have played a pivotal role in refining and enhancing the accuracy of human-pose detection in dance. Building upon the preceding discussions, the interest of researchers in this category aligns with the overarching goal of developing innovative approaches to enrich human pose-detection processes. The continuous development and refinement of algorithms in this realm have been providing deeper insights into movement patterns and facilitating the creation of more accurate representations of dance movement.

The research conducted by Zhang et al. (2017) holds particular significance in the realm of refining techniques for human-pose detection. The study introduced a diverse array of movements from varied domains, encompassing dance, martial arts, and sports. Notably, the authors’ proposed depth-based method exhibited superior accuracy and robustness when compared to a color-based method. Despite the study’s merits, the authors acknowledged a limitation in the collection of dance movements, clarifying that they were not performed by professional dancers. This acknowledgement prompts considerations regarding the system’s ability to effectively learn and replicate specific dance techniques, especially those unique to hip-hop or jazz. Nevertheless, the MADS dataset introduced by this research stands as a valuable resource for advancing human pose estimation techniques. Its contribution extends to broadening the spectrum of actions that can be accurately detected and analyzed, marking significant progress in enhancing human-pose detection methodologies.

In 2018, the study ‘Real-time dance evaluation by marker less human pose estimation’ conducted by Kim and Kim (2018) introduced an innovative method for marker less human pose estimation, demonstrating invariance to complex dance poses, including full-body rotation and self-occlusion. The research also proposed a metric to quantify the similarity between dance sequences. The study utilized an RGB-D camera for motion capture and pose estimation in Korean dance genres such as Traditional Korean Dance and K-pop. The proposed method exhibited high accuracy in detecting poses and capturing movements, establishing itself as a valuable tool for both training and performance analysis in the realm of dance. This research further validated the proposed method on multiple benchmark datasets, consistently achieving high accuracy in both pose estimation and dance performance evaluation.

The research project titled ‘Dance Pose Identification from Motion Capture Data: A Comparison of Classifiers’ by Protopapadakis et al. (2018) aimed to assess the efficacy of classification techniques in recognizing dance types based on motion-captured human skeleton data. The study focused on identifying distinctive poses for six folk dances, employing their variations. Various pose identification methods were explored, encompassing temporal constraints, spatial information, and feature space distributions, to construct an optimal training dataset. The outcomes of this research carry practical implications for performance analysis and dance training, offering the potential for more precise recognition of characteristic poses across different dance styles.

The study ‘Everybody Dance Now’ by Chan et al. (2019), mentioned in the ‘Choreographic creation support’ category, significantly contributes to the ‘Improve techniques of human-pose detection’ classification. The proposed method facilitates motion transfer from a source video to a new target, serving as a tool for creative catalysts and collaborative human-machine choreography. The authors provide an open-source dataset of legally usable videos for training and motion transfer, fostering the advancement of more accurate and robust algorithms for human-pose detection. This work marks a substantial stride toward enhancing choreographic representations and refining techniques for human-pose detection in dance performance.

In 2019, the research project ‘Weakly-Supervised Deep Recurrent Neural Networks for Basic Dance Step Generation’ by Yalta et al. (2019) developed a system that can track the movements of ballet dancers and provide real-time feedback on posture and technique. The system was evaluated with professional dancers and showed that it can accurately detect and classify ballet movements this work proposes an optimization technique for weakly-supervised deep recurrent neural networks for dance generation tasks. The generated models exhibited a motion pattern that was strongly correlated with that of a professional dancer, as evidenced by a motion beat f-score comparable to that of a human performer and lower cross-entropy. The models also demonstrated a low forwarding time, making them suitable for use in real-time applications.

In the same year, the research project ‘A Deep Learning-Based End-to-End Composite System for Hand Detection and Gesture Recognition’ by Mohammed, Lv, and Islam (2019) proposed a deep learning-based architecture for gesture recognition and hand detection. The work focused on developing a deep learning-based architecture that can be applied to various fields, including dance analysis. To evaluate the approach, they conducted extensive experiments on four publicly available datasets for hand detection, including Indian classical dance (ICD), Oxford, 5-signers, and EgoHands datasets, along with two hand gesture datasets with different gesture vocabularies for hand gesture recognition, namely, the LaRED and TinyHands datasets. The study demonstrates the potential of the system to improve the accuracy and robustness of human-pose detection and gesture recognition.

The study conducted by Zhang and Zhang (2020), titled ‘Machine learning model-based two-dimensional matrix computation model for human motion and dance recovery’, introduces MM-TDMC, a machine learning-based approach for human motion and dance recovery. This model demonstrates promising performance in addressing short-term motion recovery challenges. The research focuses on recovering motion from degraded observations and reconstructing the underlying complete motion sequence, aiming to navigate the nonlinear structure and inherent filming properties within movements. Their approach is benchmarked against other existing methods, including auto-conditioned recurrent neural network, multimodal corpus, low-rank matrix completion, and Kinect sensor methods. The notable contributions of this work include the development of a computational model for human motion and dance recovery using a machine learning-based approach, showcasing superior accuracy compared to existing methods.

The studies presented in this section have introduced diverse methodologies, including pose estimation, motion transfer, and two-dimensional matrix computation, to effectively address challenges associated with degraded observations, nonlinear structures, and film properties. These approaches have successfully yielded more precise representations of dance performance and choreography, providing valuable insights into movement patterns. Moreover, they hold significant potential for applications in performance analysis, dance training, rehabilitation, and the creation of new works. Despite these advancements, certain limitations, such as visual artifacts and constrained datasets, persist, necessitating further research and development efforts. Nevertheless, the contributions of these studies offer invaluable insights and pave the way for the continued progress of the dance research and performance domain.

4.4. Perception and new visual representations of dance movement

The intersection of machine learning, artificial intelligence, and dance has witnessed a repertoire of fresh visual representations of dance movement. New visual approaches have been improving the comprehension of intricate movements and patterns for both artists and audiences. In this section, we delve into the literature, scrutinizing the utilization of machine learning in crafting innovative visual depictions of dance movements. Additionally, we explore the potential implications of these advancements for the broader field of dance.

In the realm of dance and artificial intelligence, Marc Downie emerges as a pivotal collaborator, bringing a nuanced sensibility and expertise to the fusion of choreography and technology. With a background marked by award-winning digital artworks and a Ph.D. thesis titled ‘Choreographing the Extended Agent’ from MIT’s Media Lab Downie (2005), Downie stands as a crucial contributor to the digital arts collective Open-Ended Group OpenEndedGroup (Online). Open-Ended Group collaboration with renowned choreographers like Trisha Brown, Bill T. Jones, Merce Cunningham, Wayne McGregor, among others, underscores an interdisciplinary approach, treating the creation of agent-based artworks as a software engineering challenge tailored for artistic domains DeLahunta (2017). The ‘Becoming’ project Becoming (On-line), a remarkable offshoot of a collaboration (Marc Downie, Nick Rothwell, and Wayne McGregor), delves into the intersection of artificial intelligence, dance, and aesthetics. Through a heuristic search and iterations, ‘Becoming’ attempts to replicate movements from film clips, evolving into an abstract agent displayed on a 3D screen, Leach and Delahunta (2017). ‘Becoming’ challenges traditional notions of representation, offering an experiential understanding of kinesthetic responsiveness, and expanding the dialogue on the body’s role in contemporary dance. This live, artificially intelligent installation unfolds as an abstract body endeavor to master every movement found in a seminal 1980s science-fiction film. The innovative approach involves subjecting each of the film’s 1240 shots to a battery of computer vision techniques, extracting their geometry, color, and movement. The abstract agent then engages in a heuristic search, navigating the space of configurations and muscle activations unique to its form to emulate each shot’s movement. The iterative process continues until the virtual body is satisfied with its approximation. Displayed on a 6-foot 3D screen in portrait mode, ‘Becoming’ imbues the virtual body with the height of a human body. The visual commands issued by the agent, such as ‘Increase trail’, ‘Into emptiest space, move to side’, and ‘Torque (long) and parachute’, are made visible in captions, providing insights into its decision-making process. The project transcends traditional visual representations, offering a novel way to perceive and interpret dance movements.

The study ‘Designing Co-Creative AI for Public Spaces’ by Long, Jacob, and Magerko (2019) shifts its focus to artificial intelligence (AI) and machine learning literacy, and the application of co-creative AI in public spaces. The design principles outlined in the paper cater to those developing artificial intelligence and machine learning for public spaces, providing valuable insights for co-creativity researchers, museum exhibitors, and artists aiming to integrate artificial intelligence and machine learning into museum and gallery settings. ‘This work delves into the improvisational and collaborative nature of co-creation through the machine, emphasizing its capacity to evoke creative exploration, surprise, awe, joy, play, and social and embodied interaction’, as cited by the authors Long, Jacob, and Magerko (2019). Importantly, these aspects might hold relevance for the evolution of novel visual representations in dance performances. Viewing co-creative artificial intelligence and machine learning as design material opens possibilities for dancers and choreographers to create innovative performances with audience interaction and improvisation. The work’s emphasis on engaging a diverse audience in public spaces, using co-creation through machines, has the potential to broaden the viewership of dance performances and heighten public involvement with this artistic expression.

The research project ‘Dancing to Music’ by Lee et al. (2019) introduces a ground-breaking framework designed to generate dance movements based on music input. This innovative approach involves a meticulous analysis and synthesis of fundamental dance units to craft a diverse and realistic dance performance that seamlessly aligns with the style and beat of the accompanying music. The article unveils a unique framework termed ‘synthesis-by-analysis’, which undertakes the dual processes of dissecting a dance into basic units during analysis and, in synthesis, orchestrating these units harmoniously in response to the input music. The demonstrated method showcases the framework’s ability to synthesize dances that are not only realistic, diverse, and style-consistent but also synchronized with the musical beat, a fact substantiated by both qualitative and quantitative results. Beyond its technical achievements, this work carries significant potential to elevate audience perception and understanding of dance. By generating performances closely attuned to the music, it enhances the conveyance of intended emotions and expressions, fostering a more immersive and resonant experience. In essence, ‘Dancing to Music’ contributes valuable insights into the application of machine learning techniques for generating dance movements and novel representations inspired by the interplay between music and dance.

In 2020, the research project ‘Dance to your drum: Identification of musical genre and individual dancer from motion capture using machine learning’ by Carlson et al. (2020) also explores the relationship between music and dance movements and how they are influenced by musical genres. The researchers used motion capture data to analyze the dance movements of participants in response to eight different musical genres. The study used a Support Vector Machine model to classify the data by both genre and individual dancer, providing new insights into the relationship between music and dance. Similarly, to Lee et al. (2019), this study contributes to our understanding of how music and dance are related but also highlights the role of embodied cognition and culture in shaping dance movements.

Centered on the integration of interactive visuals into contemporary dance, the study undertaken by Correia et al. (2022) scrutinizes audience experiences during performances featuring diverse visual interaction approaches, including motion capture and biosignal sensors. The research spans four dance performances, each employing distinct methods of visual interaction. This endeavor culminated in the development of two innovative software systems, MLIV (Machine Learning Interactive Visualization) and CAIV (Composite Animation Interactive Visualization), harnessing machine learning to translate static body maps into interactive visuals. Rooted in soma design and the somatic experiences of dancers, the primary goal is to make non-visible bodily aspects of dancers perceptible to the audience. A multi-stage participatory study involving 12 dancers corroborates the positive impact of both prototypes in achieving the objective of visualizing previously unseen aspects of the body. The significant contributions of this research manifest in presenting two distinct approaches for crafting interactive visuals from body maps and conducting a comprehensive analysis. These findings chart new methodologies for creating interactive visuals for dance performances, particularly from body maps, with implications extending beyond dance. The research proves beneficial to designers in the realms of dance and technology, soma design, and the broader landscape of embodied interaction.

In the same year, ‘F O R M S – Creating new visual perceptions of dance movement through machine learning’ by Nogueira, Menezes, and de Carvalho (2022) introduced a ground-breaking art concept that seamlessly integrates dance movement with machine learning techniques, offering real-time creation of novel visual representations to enrich on-stage performances. The primary goal of this convergence is to elevate the perception of dance movement by generating abstract or literal visual models that blend geometric and curvilinear forms. The study encompassed the development of a comprehensive framework, exemplified through a real case study presented in a contemporary dance performance featuring professional young dancers. The results unequivocally affirm the success of both the dancers and the audience in this innovative exploration. This work leveraged machine learning techniques, specifically human-pose estimation, to dynamically identify the body skeleton of performers in real-time. This detection process facilitated the creation of unique visual representations for each dancer. Moreover, insights gleaned from the dancers themselves provide valuable reflections on the impact of this technological intersection. Dancers expressed revelations such as ‘Observing my movement through the technology made me realize that I need to expand the range of my movements’, ‘Seeing myself from a different perspective helped me to understand the need for clearer and more precise movements, as well as increased focus on the extremities of my body’, and ‘The technology highlighted the potential for more expressive and expansive movements in my arms’. These testimonials, cited in the article Nogueira, Simões, and Menezes (2023), underscore the transformative influence of machine learning on the dancers’ understanding of their own movements, further substantiating the positive impact of such collaborations. The development of a robust framework and its successful application in a real-world dance performance underscores the potential of this approach to significantly contribute to the dance domain.

Research has shown that the integration of artificial intelligence (AI) and machine learning contributes to enhancing dance performance and connecting dance to other artistic practices, such as music by Lee et al. (2019) and Carlson et al. (2020), or visual art by Correia et al. (2022) and Nogueira, Simões, and Menezes (2023). For instance, a study revealed that the utilization of machine learning algorithms for motion tracking in contemporary dance significantly heightened the audience’s perception of the performance, particularly during segments featuring close interactions among performers, Correia et al. (2022). Another study demonstrated the feasibility of generating abstract and dynamic visual representations of dance, using machine learning techniques, thereby captivating the audience’s attention and interest, Nogueira, Menezes, and de Carvalho (2022). While these advancements underscore the potential of machine learning and related technologies to enrich the field of dance by creating innovative and engaging visual representations of movement, it is crucial to recognize potential impacts. Challenges include the risk of overreliance on technology, which may shift focus from the artistic expression of the performers to technological aspects. Striking a balance between the positive impacts and these potential challenges is vital for ensuring the responsible and ethically sound integration of machine learning in the realm of dance.

5. Discussion

In this comprehensive exploration, our primary goal was to illuminate the intricate implications of machine learning on dance performance, probing key research questions regarding the assimilation of machine learning techniques into the dance domain. Our investigation focused on unraveling the primary applications of machine learning in dance, scrutinizing its influence on choreographic creation, dataset refinement, human-pose detection techniques, and the emergence of new visual representations of dance movement. Simultaneously, we sought to comprehend the intricate dynamics of how machine-learning techniques infuse and shape the creativity of performers, choreographers, participants, and dance professionals. From aiding choreographers in conceiving novel movement sequences to enabling the analysis and display of intricate movement data, machine learning emerges as a valuable tool in the choreographer’s arsenal. Additionally, it facilitates the collection and evaluation of extensive movement datasets, contributing to the refinement of human-pose detection and perception techniques.

This dualistic nature emphasizes the necessity for a nuanced understanding of the interplay between machine learning and the creative processes inherent in dance performance. While acting as a catalyst for innovation and experimentation, the transformative power of machine learning raises ethical considerations, particularly regarding ownership and authorship in the context of machine-generated movements and choreography. As the boundaries between human creativity and machine-generated output blur, questions of attribution and intellectual property become increasingly complex. The dual impact of machine learning on dance performance underscores the need for a balanced and thoughtful approach. Achieving a harmonious integration of technology and artistic expression requires careful consideration of ethical implications, ensuring that the benefits of innovation do not compromise ethical standards or the integrity of the creative process in dance.

5.1. Addressing the first research question

1. What are the main applications of machine learning techniques in dance performance, and how do they impact choreographic creation support, dataset or network collection and training, improved techniques of human-pose detection, perception, and new visual representations of dance movement?

The integration of machine learning and artificial intelligence for choreographic creation support is a recurring theme in various works, exemplified by the ‘Chorn-rnn’ system by Crnkovic-Friis (2016), Wayne McGregor and Google’s ‘Living Archive’, and Pettee et al. exploration in ‘Beyond Imitation: Generative and Variational Choreography via Machine Learning’ Pettee et al. (2019). These endeavors employ deep learning approaches to construct new choreography based on existing movement data. However, each project differs significantly in method and purpose. For instance, the ‘LuminAI’ project by Jacob and Magerko (2015) creates an interactive installation with a taxonomy that detects and evaluates visitors’ interactions based on body movement. In contrast, Chan et al. ‘Everybody Dance Now’ Chan et al. (2019) offers a method for transferring motion from a source video to a fresh target, facilitating collaborative human-machine choreography. Bill T. Jones and Google Arts and Culture’s initiative, ‘Body, Movement, and Language’, uniquely integrates AI technologies such as voice recognition and PoseNet into live performances, involving professional choreographers and dancers in the production process. This stands in contrast to other works primarily focused on using technology as a creative catalyst. The research of Ruilong et al. (2021) introduces a novel multi-modal dataset of 3D dance motion and music, showcasing superior performance compared to recent methods. Despite this success, the study acknowledges the need to explore physical interactions between the dancer and the floor for future improvements. Conversely, ‘Between Us’ introduces an innovative interaction-based authoring approach, blending case-based and imitative learning to empower co-creative systems in generating novel choreographic material. Collectively, these projects share a common goal of leveraging technology to enhance choreographic creativity, each making distinctive contributions aligned with its artistic and technical ethos.

Several works have concentrated on enhancing dataset collection and machine training to improve dance performance, as detailed in the ‘Dataset or network collection and training’ section. Zhang et al. (2017) built the MADS dataset to test human pose estimation algorithms, comparing depth-based and color-based methods. Kishore et al. (2018) proposed a CNN-based technique for identifying and categorizing Indian classical dance motions, achieving a high classification rate. Castro et al. (2018) introduced the ‘Let’s Dance’ dataset, exploring motion parameterization algorithms for learning to classify internet dancing videos. Priya and Arulselvi (2019) employed a deep-learning method to classify Bharathanatyam and Karate stances in a multi-view dataset. Wayne McGregor and Google Arts and Culture innovatively approached dataset creation, enabling the generation of new choreographies using knowledge derived from McGregor’s dance pieces. Hwang, Yang, and Kwak (2020) introduced new criteria and methods to enhance pose estimation performance for unusual positions. Ruilong et al. (2021) presented a music-conditioned transformer-based learning framework for 3D dance creation. These works underscore the advantages and benefits of advancements in dataset collection and unique deep-learning architectures, contributing to the accuracy and resilience of movement and pose estimation in dance performance. These findings may pave the way for more advanced tools for choreographic invention, training, and performance analysis in dance.

Projects employing machine learning to improve human-pose detection algorithms in dance performance include Zhang et al. (2017), which outperformed color-based techniques in terms of accuracy and resilience. Protopapadakis et al. (2018) investigated classification algorithms for detecting dance styles based on motion-captured human skeletal data. Kim and Kim (2018) introduced a markerless human pose estimation method with high accuracy in recognizing postures and recording motions. Chan et al. (2019) motion transfer approach from a source video to a novel target enables the creation of more accurate and robust human-pose identification systems. Yalta et al. (2019) developed a system to track ballet dancers’ movements, delivering real-time feedback on posture and technique. Mohammed, Lv, and Islam (2019) introduced a deep learning-based system for human posture estimation that outperformed competitors on benchmark datasets. These studies offer far-reaching benefits to the dance world, enabling comprehensive performance analysis, dance training, rehabilitation, and the production of new works. By utilizing machine learning to better understand how the human body moves and interacts with its surroundings, researchers can enhance our understanding of the art of dance.

In the subsequent reflection, we analyze and compare findings from six research projects that apply machine learning in dance, focusing on ‘Perception and new visual representations of dance movement’. Long, Jacob, and Magerko (2019) present a design framework for co-creative AI in public spaces, emphasizing audience participation and improvisation. This project has the potential to enhance audience involvement, improve perception and comprehension of dance, and promote public engagement with the art form. Lee et al. (2019) established a framework for generating dance movements from music input, shedding light on using machine-learning approaches to produce dance motions closely linked with music, thereby improving audience perception and comprehension of dance. Carlson et al. (2020) investigated the link between music and dance motions, emphasizing embodied cognition and culture’s role in defining dance motions. Leach and Stevens (2020) studied the impact of collaboration during improvisation among professional contemporary dancers, contributing to our understanding of social ties in dance improvisation and creativity. Nogueira, Simões, and Menezes (2023) proposed a performance concept that intersects dance movement with machine learning techniques to produce new visual representations, introducing novel ways of seeing and depicting dance movements. Correia et al. (2022) research integrated machine learning algorithms to produce interactive images for contemporary dance performances, investigating the audience’s reaction to these visuals. These studies collectively demonstrate machine learning algorithms’ potential to provide new insights into the perception of dance movement, create innovative ways of visualizing dance performances, and improve audience understanding and engagement with the art form.

The practical benefits derived from these advancements underscore positive outcomes in performance analysis, dance instruction, and innovative choreography. While each work operates independently, they collectively contribute to pioneering novel approaches at the intersection of dance and machine learning, reflecting substantial progress and transformative potential.

5.2. Addressing the second research question

(2)	How does the incorporation of machine learning techniques impact the creativity of performers, choreographers, participants, and dance professionals?

The incorporation of machine learning techniques into dance processes has indeed propelled creativity to new heights for performers, choreographers, participants, and dance professionals. This infusion of technology serves as a wellspring of inspiration, granting choreographers the ability to push traditional boundaries and explore uncharted territories of movement. The generation of avant-garde choreography through machine learning techniques offers fresh perspectives, allowing for experimentation with diverse motion variants and the computational analysis of choreographic effectiveness. However, amidst these positive impacts, the integration of machine learning into dance has also ushered in challenges resonating across ethical dimensions. The transformative power of machine learning raises complex questions related to ownership and authorship, particularly concerning machine-generated movements and choreography. As technology continues to blur the boundaries between human creativity and algorithmic output, navigating the attribution of credit becomes increasingly complex, thereby influencing traditional concepts of authorship and ownership within the realm of dance, as elucidated in the subsequent section.

5.3. Ethical and artistic implications

The integration of machine learning into the domain of dance, while ushering in innovative possibilities, is not without its share of challenges and ethical concerns. This subsection explores the negative impacts stemming from the intersection of machine learning and dance, drawing insights from both research questions.

1. Limited Engagement with Dance Professionals – A notable limitation is observed in studies that lack direct engagement with dance professionals, potentially resulting in a superficial examination of dance styles and execution rigor. This gap raises concerns about the depth of examination in understanding various dance styles and the rigor of execution. The technical and qualitative prerequisites essential for training machine learning networks may be overlooked, potentially leading to a superficial analysis of dance dynamics. The reliance on videos sourced directly from platforms like YouTube may further deviate from perspectives integral to dance, risking a compromise in technical precision and artistic excellence.

2. Artistic Ownership and Authorship Challenges – A pressing concern lies in the potential compromise of artistic ownership and authorship within machine-generated choreography. Instances, where machine learning algorithms play a pivotal role in crafting dance sequences, introduce complexities in crediting. Decisions about attributing credit to the algorithm, the programmer, the dancers, or a combination of these entities can spark disputes over intellectual property and artistic credit, challenging traditional notions.

3. Cultural Impact and Unintended Biases – The use of machine learning algorithms in generating movements introduces potential impacts on the origin, culture, and identity embedded in dance projects. Inadvertent biases or cultural insensitivities in algorithmic outputs raise questions about the authenticity and representation of diverse dance forms. Maintaining sensitivity to cultural nuances and identities is essential to ensure ethical and inclusive practices in the evolving intersection of art and technology.

4. Diminishing Human Expression – While machine learning sparks innovation in choreography, risk surfaces in potentially diminishing the unique qualities of human expression in dance. The reliance on machine-generated movements, while enabling avant-garde choreography, may compromise the authenticity and emotional depth inherent in human-created choreography. This challenge questions the essence of dance as a deeply human and emotive form of artistic expression.

5. Striking a Delicate Balance – Achieving a harmonious integration of machine learning into the dance domain requires careful consideration of ethical implications. Striking a delicate balance involves harnessing the potential for creative enhancement while addressing associated challenges. The dance community, technology developers, and ethicists must collaborate to navigate these intricacies and foster a responsible and sustainable coexistence of art and technology in the realm of dance. Recognizing challenges, such as limited engagement with dance professionals and the risk of deviating from authentic perspectives, is crucial for a comprehensive exploration of the complex interplay between dance and machine learning. This balanced approach ensures that the benefits of innovation elevate the art of dance without compromising its authenticity and integrity.

6. Conclusion

Our analysis sheds light on the transformative potential that machine learning introduces to the dance domain, unlocking innovative avenues for creative expression and pushing the boundaries of artistic innovation. This exploration into the intersection of art and technology underscores both the promises and challenges that come with incorporating machine learning into dance. While our findings showcase machine learning as a catalyst for creative breakthroughs, providing invaluable support in choreographic endeavors, and enhancing human-pose recognition capabilities, it is crucial to address the discerned limitations and ethical considerations within the studies under scrutiny. As technology blurs the lines between human creativity and algorithmic output, attributing credit becomes intricate, impacting traditional notions of authorship and ownership within the dance domain, as reflected in the next section.

The delicate balance between technological augmentation and the preservation of the authentic human touch in the dance domain necessitates ongoing attention and thoughtful exploration. Collaboration between the dance community, technology developers, and ethicists becomes imperative to navigate these challenges successfully. By fostering interdisciplinary dialogues and a commitment to ethical scrutiny, we can ensure that the incorporation of machine learning enhances rather than overshadows the fundamental role of human creativity and expression in the art of dance.

Looking ahead, the promising frontier of machine learning and dance beckons further research dedicated to addressing challenges head-on and uncovering novel ways to harness their full potential. This demands not only a nuanced understanding of the intricate dynamics at play but also a celebration of the positive impacts. Machine learning stands as a powerful ally, offering unprecedented support for choreographers, enriching performance analysis, and contributing to the evolution of dance creation.

As we navigate this evolving landscape, our commitment to maximizing the benefits while mitigating the limitations becomes paramount. The intersection of machine learning and dance is not just a convergence of technologies; it is an opportunity to amplify the artistry of dance, fostering new possibilities for creativity and expression. By delving deeper into these complexities and fostering collaboration, we can forge a path toward a harmonious integration that propels dance into a new era where technology and human ingenuity dance together in seamless choreography.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Correction statement

This article has been corrected with major corrections, but these changes do not impact the academic content of the article.

Additional information

Funding

The research conducted by the first author during the submission of this article was undertaken as part of her Ph.D. studies at the University of Coimbra, specifically at the Institute of Systems and Robotics and the College of Arts. The Ph.D. studies were supported by the Fundação para a Ciência e Tecnologia (FCT – Portugal) through grant number 2020/09137/BD and project FCT/MCTES UIDB/00048/2020.

Notes on contributors

Maria Rita Nogueira

Maria Rita Nogueira, originally from Coimbra, is a lecturer and researcher in the Department of Digital Art at Manchester Metropolitan University in England. As a researcher, she has specialized in interactive art, integrating artificial intelligence into performative and visual art through motion-based interaction. She completed her bachelor's and master's degrees in Design and Multimedia at the Department of Computer Science of the University of Coimbra before pursuing her PhD in Contemporary Art at the College of Arts and the Institute of Systems and Robotics, both institutions belonging to the University of Coimbra. Maria Rita's artistic and research practice is driven by a passion for the intersection of art, design, and technology, focusing primarily on creating visual graphic compositions that respond to human interaction in real-time through body movement. Her scientific and artistic contributions have been recognized in various contexts. In May 2023, Maria Rita received the Best Presentation and Performance Award at the Technarte Conference in Bilbao for her work ‘FORMS'. Her doctoral research was also highlighted for presentation at the Massachusetts Institute of Technology (MIT-Boston) in April of the same year. In 2022, her work ‘Move In Tempo' was exhibited at the Czong Institute for Contemporary Art in South Korea. This same piece, ‘Move In Tempo’, was presented as an individual exhibition at the National Museum of Machado deCastro, recognized as a UNESCO World Heritage site. In 2021, it received the Best Artwork Award at the Criatech Artistic Residencies. Throughout her journey in dance, Maria Rita received various scholarships and internships from institutions such as the Institute of Arts in Barcelona, Girne American University, Malandain Ballet Biarritz, Instável Dance Company, and École Supérieur de Danse de Cannes, significantly contributing to shaping her vision of artistic practice. Currently, she teaches various areas of interaction design and conducts research to foster new conceptual and artistic dialogues between art, interaction through body movement, and technology.

Paulo Menezes

Paulo Menezes is an associate researcher at the Institute of Systems and Robotics (ISR) of the University of Coimbra. His research activities encompass various areas, including Robot Architectures, Computer Vision, Human-Robot Interaction, Human Emotional and Activity Analysis, Augmented Reality, Telepresence Systems, and Assistive Technologies. He has authored articles such as “Telepresence Social Robotics towards Co-Presence: A Review” (Switzerland, 2022) and “Gamifying motor rehabilitation therapies: Challenges and opportunities of immersive technologies” (Switzerland, 2020).

José Maçãs de Carvalho

José Maçãs de Carvalho is an artist, lecturer, and curator, currently serving as a Professor at the Department of Architecture and the College of Arts of the University of Coimbra. He also coordinates the master's program in Curatorial Studies at the university. As an artist, he works primarily in the fields of photography and video art. Since 2020, he has held the position of curator at the Contemporary Art Center of Coimbra. Notable exhibitions include “My own private pictures” (Plataforma Revólver, Lisbon Photo Biennial, 2005), which led to his nomination for the BES Photo Prize in 2005, and his shortlisting for the Pictet Prix in 2008.