Digi-flash pedagogy confronts new emerging technologies - Maturity level evaluation case study

Abstract

There is significant interest in emerging technologies. Universities, companies and other adopters are researching how and when these technologies should be implemented. However, this research area still lacks ways to evaluate the maturity level of the technologies and rapid experimentation concepts for testing the new technologies. This paper addresses the research gap by providing a new evaluation method and a rapid experimentation concept. This method and concept were successfully used in twenty-five Flash-projects, where five emerging digital Industry 4.0 themes evaluated.

Keywords:

• cobotics
• autonomous robotics
• digital twin
• machine learning
• immersive technologies
• Industry 4.0
• technology assessment

1. Introduction

The ongoing fourth industrial revolution combines the physical and digital worlds with cyber-physical systems. In industrial revolutions like Industry 4.0, industrial digitalization revolutionizes the whole industrial field from production to operation. In such situations, it is typical that new and partly immature technologies are available, but often companies and industry players are either unaware of their existence or are uncertain about how and at what stage they could and should be adopted (Aheleroff et al., 2022; Grunwald, 2009). This is echoed by Rotolo et al. when they state, “emerging technologies are technologies whose development, practical applications, or both are still largely unrealized, such that they are figuratively emerging into prominence from a background of nonexistence or obscurity” (Rotolo et al., 2015). Moreover, Rotolo et al. (Rotolo et al., 2015) highlight the ability of new technologies to change the status quo.

The main aim of this research was to create a project concept to research five emerging technologies using rapid, iterative experiments with small and medium sized companies (SME) to gain new knowledge and understanding about Industry 4.0 technologies. In addition, the research aimed to evaluate the maturity level of the technologies in order to determine how and at what stage these new technologies could and should be adopted (Fenn & Raskino, 2008; Konstantinidis et al., 2022). The main research and project method was based on the Rapid Iterative Experimentation Process (Figure 2).

Figure 1. Digi-Flash technologies and clusters.

The rapid experimentation model we created was named Flash-project. Twenty-five Flash-projects were implemented: five projects under five different selected themes (Metropolia, 2021). This research was conducted in collaboration with the Metropolia University and City of Vantaa as part of the project Digi-Flash. This paper begins with an introduction to the selected emerging technologies and Digi-Flash project. In section two, the research methods are described. In section three and four, the results are presented and discussed. In the final section, conclusions are drawn, followed by suggestions for future research directions.

1.1. Selected emerging technologies

For the Digi-Flash project, five emerging digital Industry 4.0 themes were selected; Cobotics, Autonomous Robotics, Digital Twin Technology, Artificial Intelligence and Machine Learning in Automation as well Virtual and Augmented Reality. Technologies were selected based on expert interviews when the project was being prepared in 2018.

Cobotics, Collaborative robots are robots that work with humans, contrary to traditional industrial robots, which are isolated from human contact. Cobots do not replace humans; rather, the work of a cobot involves physically heavy and repetitive work, after which the responsibility for the work is transferred to the human again.

Autonomous Robotics, Unmanned Ground Vehicle (UGV) and Unmanned Aerial Vehicle (UAV) are robotic vehicles traveling on land and in the air without a driver. UAVs are commonly referred to as drones. The rapid development of navigation methods, computational algorithms and various detection devices has enabled the development and use of various autonomous robotic vehicles. In addition to cars and drones, autonomous ships, mining machines and transport equipment are being developed.

Digital Twin Technology, Digital Twin refers to a physical machine, device, or even an entire industrial production—a virtual model, a visual copy in the digital world that responds as far as possible to its physical twin. In earlier researches the technology is classified as follows (Kritzinger et al., 2018; Liljaniemi & Paavilainen, 2020):

• Digital Model, a digital representation of an existing or planned physical object, no automated data exchange between the physical—digital objects.

• Digital Shadow, if there further exists an automated one-way data flow between existing physical—digital objects.

• Digital Twin, the data flows between the physical—digital objects are fully integrated in both directions.

The use of Digital Twins can be categorized into three phases: 1) Project Design and Commissioning, 2) Operational and Maintenance, and 3) Decommissioning and Ending of asset life cycle. Digital Twin technology can be utilized in, e.g., product and production design, virtual deployment, training, and maintenance (Autiosalo et al., 2021, 2019; Leng et al., 2021).

Artificial Intelligence and Machine Learning in Automation, Artificial intelligence is a machine or computer program that is capable of performing functions that require adaptive, human-like cognitive abilities. Most of the applications of artificial intelligence are so-called machine learning. Machine learning is an area of artificial intelligence where activities are not pre-programmed. The machine learns independently from the data assigned to it, and no operating instructions are defined for it for each individual situation.

Virtual and Augmented Reality, Virtual reality (VR), augmented reality (AR) and holodeck are technologies that can be used to expand the physical world through digital layers or take the user to a fully virtual environment. Although the best-known applications of virtual reality are in the gaming world, VR, AR and holodeck technologies can also be adopted in industry and workplaces.

1.2. Digi-flash project

This research was conducted as part of the project Digi-Flash—Speeding Up the Use of Industry 4.0 Technologies in Small and Medium Sized Companies in the Uusimaa region by Digi-Flash Projects under Five Verified Themes and Clusters. The Digi-Flash project was implemented in 25 small and medium sized companies (SME) in Uusimaa (Helsinki Metropolitan region) under five digital Industry 4.0 themes: 1) Cobotics, 2) Autonomous Robotics, 3) Digital Twin Technology, 4) Artificial Intelligence and Machine Learning in Automation, and 5) Virtual and Augmented Reality.

For each SME, one theme was selected, and a proof-of-concept level project implemented. The projects that were implemented were strongly related to the Vantaa City Economic Development Programme and the regional Top Clusters of Vantaa City, which are Logistics, Food, Circular Economy, Real Estate and the High Competence Production Cluster (Figure 1). The Digi-Flash project was implemented by Metropolia University of Applied Sciences and the City of Vantaa in cooperation with the Finnish Automation Society and the Finnish Robotics Association (Rotolo et al., 2015).

Figure 2. Rapid Iterative Experimentation Process (RIEP).

The aim of the project was to create competence, competitiveness, and networks in the Vantaa and Uusimaa region. To achieve this, 25 Lightning projects for 25 companies in five different technologies in five different industries were implemented. The chosen technology themes represent an opportunity for companies to try and pilot new technologies. The broader goal was for the experiments to lead to investment and thus to an increase in the competitiveness of companies. In addition, the goal was for the project companies to be the core group for the formation of the new ARI (Automation, Robotics and Industry 4.0) network.

2. Materials and methods

2.1. Rapid iterative experimentation process (RIEP)

The main research and project method was based on Rapid Iterative Experimentation Process (Figure 2). RIEP is a “Lean Startup-style” approach to innovation. It typically includes three iterative steps: pre-examination and conceptualization of the solution, simulation of the chosen concept, and construction of a prototype. The purpose is to evaluate and improve the potential of innovations.

The created rapid experimentation model was named the Flash project. Twenty-five Flash projects were implemented; five projects under five different selected themes. Project implementation was managed by the theme master together with innovation project groups, the project assistant, and the theme mentor. Altogether, over 300 agents of change were involved in projects (GIM Institute, 2015; Ries, 2011; Rogers, 2003).

For each project, a preliminary study and a requirement definition were produced. A preliminary study of the projects was conducted by the theme master, if necessary assisted by an innovation project team. The projects were 1–6 months long. One project included 100 hours of theme master work, 300 hours of project assistant work, and one innovation project (3–4 students, 400–500 h). If required, expert support was supplied by technology providers and the Robotics Association in the form of consultations and mentoring.

Based on the results obtained, the project actors, together with each SME, assessed whether the project was viable and scalable to profitable operations. For each project, an extended profitability assessment was conducted (e.g., payback time, impact on energy costs, material costs, and overall efficiency). The overall aim of the projects was to evaluate the technical and financial feasibility as well as the maturity level of the technologies.

2.2. Maturity level evaluation

The maturity level of the selected five emerging technologies were evaluated based on the combination of four different evaluation methods: Gartner’s Hype Cycle, Roger’s Bell Curve, S-curve concept, and published articles analysis. For these four criteria, a relative weight of 25% was assigned to each, resulting in each technology having a maturity level from 0% to 100%.

The Roger’s Bell Curve technology adoption model describes the adoption or acceptance of a technology according to the adopters. The Bell Curve model divides the technologies into five categories: Innovators, Early adopters, Early majority, Late majority, and Laggards.

The Bell Curve was employed as the framework for the evaluation, and a Likert scale from one to five was used with a value of one corresponding to Innovators and a value of five for Laggards (Figure 3). Accordingly, all four evaluation methods were mirrored towards it. The final maturity evaluation was similarly completed based on this scale and Roger’s Bell Curve (Guidolin & Manfredi, 2023; Ries, 2011; Rogers, 2003).

Figure 3. Roger’s Bell Curve technology adaptation model.

2.3. Customer interviews

The first evaluation component was the customer interviews. After completing each project, a one-to-one customer interviews were carried out. The interviews contained a different set of questions. One question was related to the maturity level of the technology utilised in the project. For this, Roger’s Bell Curve was used as the frame-work, and customers were asked to place the technology in one of the five sections (Figure 4). Participation in the customer interviews included 19 out of 25 customer companies of the project.

Figure 4. Customer interview question related to maturity level, Roger’s Bell Curve.

2.4. Gartner hype cycle

The second evaluation component was the Gartner Hype cycles between the years 2010–2021. “Gartner Hype Cycles (Figure 5) provide a graphic representation of the maturity and adoption of technologies and applications, and how they are potentially relevant to solving real business problems and exploiting new opportunities” (Dedehayir & Steinert, 2016; Gartner, 2022). The Hype Cycle includes five phases:

• Technology Trigger: A potential new technology is starting to emerge to the public.

• Peak of Inflated Expectations: The new technology has risen to the top of the hype.

• Trough of Disillusionment: Interest drops after technology experiments fail.

• Slope of Enlightenment: Companies start to comprehend the ways in which the technology could benefit them, and increasingly more people are beginning to understand the technology.

• Plateau of Productivity: The technology is beginning to spread to the general public.

Figure 5. Gartner Hype Cycle provides a graphic representation of technology maturity levels.

2.5. S-curve concept

The third evaluation component required the Digi-Flash project group members to individually place all five technologies along the S-curve (Figure 6). The S-curve concept can be described as an instrument of strategic innovation management. It is a widely known tool for estimating the future development of different technologies. It provides a method for knowing the current stages of the technology life cycles (TLCs) and thus assists in making important decisions, such as whether to invest or not in technology or development (Nieto et al., 1998; Woo & Magee, 2022). The evaluation was made by six project group members out of eight core persons involved in project.

Figure 6. S-curve concept as an instrument of strategic innovation management.

2.6. Publishes articles analysis

The fourth and final evaluation component was to research the amount of the published scientific articles. For this purpose, Aalto University Primo search portal was utilized. The key words used in the search were the name of the Digi-Flash technologies. The searches were made for every year since the origin of the first published article so that the number of articles compared to 2021 was less than 1%. This was assumed to be the starting point. The most recent year was for the Cobotics and Digital Twin in 2007, and the earliest for ML, AI, and VR in 1987. The search results were then aligned into the scatter diagram format. These diagrams were then fitted along the sigmoid function (Figure 7). The diagram position of 2021 was taken as the maturity point for each technology (Fire & Guestrin, 2019; Revkin et al., 2008).

Figure 7. Basic sigmoid function (S-curve) formula.

2.7. One quarter evaluation method (OQEM)

For the evaluation of the technologies, a new method was developed, one-quarter evaluation method (OQEM). The intention with this was to make the evaluation versatile and reduce the uncertainty compared to one method. Each component was graded from one to five and fitted to match Roger’s Bell Curve (Figure 8). The final maturity level was presented as an average of all four components. Finally, the results were allocated into the five different adaptation categories (Castro et al., 2017; Gkoumas & Tsakalidis, 2019; Hein-Pensel et al., 2023; Steinert & Larry, 2010).

Figure 8. Fitting principle of the Gartner Hype Cycle, S-curve, and Roger’s Bell Curve.

2.8. Official evaluation criteria

Digi-Flash project was funded by the Helsinki-Uusimaa Regional Council. The official evaluation criteria from the funder are presented in Table 1. The utilised criteria were as follows:

Table 1. Official evaluation criteria

16	Companies initiating r&d activities or cooperation with universities, colleges or research institutes
20	A company that creates a new business based on renewable energy solutions or support for low carbon
21	Competence-intensive start-ups with significant readiness to develop a new product, service or production method
22	New applications implemented by companies using open data and interfaces
CO04	Companies receiving non-financial support
CO28	Companies that are launching a new or clearly improved product on the market (the product is new to the market)
CO29	Companies that are launching a new or clearly improved product on the market (the product is not new to the market)

The funder selected eight official evaluation criteria for the project: 1) ID15 The total number of companies that participated in the project, 2) ID16 The number of companies that initiate cooperation with universities, 3) ID20 The number of companies that create new business based on renewables or carbon neutral, 4) ID21 The number of new startup companies incorporated as a result of the project, 5) ID22 Number of open data interfaces used, 6) IDCO04 Number of companies that received De Minimis (non-financial) support, 7) IDCO28 Number of the companies that are launching a new or improved product with the help of the project, and 8) IDCO29 Number of the companies that are launching a new or improved product with the help of the project (product new for the company, but not new to the market). ID meaning funder’s official code for each criteria.

3. Results

This chapter presents the results and a discussion of the results.

3.1. Customer interviews results

Figure 9 presents the results of the customer interviews. All evaluations are positioned within the Early Majority category. Customers ranked the maturity of the technology used in their project on a scale of one to five. A mean was calculated for each technology based on the responses. For this, Roger’s Bell Curve was used as the frame-work.

Figure 9. Customer interviews results placed along Bell Curve.

3.2. Maturity levels based on Gartner hype cycle

Figure 10 shows the different technologies and their hype status. The indicator ball and triangle colors indicate the years to mainstream adaptation (i.e., Plateau of Productivity). Table 2 presents the maturity scores with respect to the current hype status and the years to mainstream adaptation in the situation year 2021.

Figure 10. Technologies and hype status along Gartner Hype Cycle.

Table 2. Scores based on Gartner’s Hype Cycle

	Score
Cobotics	2.8	Early Majority
UGV/UAV	2.63	Early Majority
Digital Twin	2.75	Early Majority
ML/AI	3.5	Late Majority
VR/AR	4.28	Laggards

3.3. Maturity levels based on project group member analysis

Figure 11 shows the evaluation of the Digi-Flash project group. Each member of the project team placed the different technologies on the S-curve based on their maturity level. Average values, standard deviation and variance were calculated from the values. A scale of one to five was used as the scale of the S-curve.

Figure 11. Project group analysis along S-curve.

3.4. Published articles results

Figure 12 indicates the maturity level of the Digital Twin technology measured based on the published articles. This method results in a maturity level of 2.9.

Figure 12. Digital Twin article numbers 2007–2021 and curve fitting along S-curve.

The first published article was found from 2007 (2 pcs), and the amount was 5155 in 2021. When the cumulative number is put into the form of a curve and fitted with the S-curve, the maturity level can be read from the endpoint of the curve. The same method was used for all other technologies. The overall results of this method are presented in Table 3. Parameter c1 indicates the development speed of the technologies (2.0 rapid—0.5 slow).

Table 3. Scores based on article numbers

	Score
Cobotics	2.9	Early Majority
UGV/UAV	2.8	Early Majority
Digital Twin	2.9	Early Majority
ML/AI	2.85	Early Majority
VR/AR	2.85	Early Majority

3.5. Overall maturity level of the technologies

Figure 13 and Table 4 conclude the overall maturity level of the technologies. The presented score is calculated with the OQEM being the average value of the four sub-methods. For ML and AI as indeed with VR and AR, the average of these technologies was used.

Figure 13. Final results presented in Roger’s Bell Curve.

Table 4. Scores based on one-quarter evaluation method

	M1	M2	M3	M4	OQEM ∑
Cobotics	2.8	3.25	3.43	2.9	3.1
UGV/UAV	2.63	2.5	3.33	2.8	2.82
Digital Twin	2.75	2	3.13	2.9	2.7
ML/AI	3.5	3.2	2.79	2.85	3.09
VR/AR	4.28	3	3.75	2.85	3.47

3.6. Official Results of the digi-flash project

Table 5 capsulizes the official results of the Digi-Flash project. The last column shows the aims (Aim) and the final implementation number (Act) for different indicators. The desired number of projects were successfully implemented. In most cases, an existing product was improved or a new one created. The only minor debacle was the amount of start-ups.

Table 5. Official results of the Digi-Flash project

		Aim/Act.
15	Companies participating in project led by research and development institutions	30/53
16	Companies initiating r&d activities or cooperation with universities, colleges or research institutes	35/52
20	A company that creates a new business based on renewable energy solutions or support for low carbon	5/15
21	Competence-intensive start-ups with significant readiness to develop a new product, service or production method	3/0
22	New applications implemented by companies using open data and interfaces	5/5
CO04	Companies receiving non-financial support	25/25
CO28	Companies that are launching a new or clearly improved product on the market (the product is new to the market)	0/3
CO29	Companies that are launching a new or clearly improved product on the market (the product is not new to the market)	5/14

4. Discussion and other remarks

The key findings of this research indicate that to be able to research new emerging technologies, it is important to have the expertise and ability to adapt rapid experimentation techniques as well methods to evaluate the maturity level of the technologies. This provides and enables users to gain concrete knowledge, to identify the stage of these new technologies, and to assess the way in which they should be adopted. The maturity level of the technologies has been traditionally evaluated with different methods as follows: Gartner Hype Cycle, Roger’s Bell Curve, and S-curve concept as well as market share, and the amount of patents and published articles. Nevertheless, it is extremely important to recognize the right moment when to start developing and adopting technologies. Early adopters have a possibility to achieve advances, and late adopters can fall behind. Following only the hype or relying on only one method can be hazardous, the developed new OQEM thus lowers the risk and strengthens the certainty of the evaluation.

An unexpected finding was that the slope of the published articles curve strongly corresponded to the development perspective of the technologies. Obviously, the newly created evaluation method has its limitations. Roger’s Bell Curve and S-curve methods were conducted by experts and customers, and the participation was quite limited in the number of respondents. Contingently, a weight coefficient could be used for sub methods; as well, the amount of patents and market share could be taken into account.

If the emerging technologies would have been chosen e.g., during 2022 they might be different. The Digi-Flash project was prepared during 2018 and from stakeholders perspective these five were the most interesting. New technologies arise constantly and actors must be awake to observe these developments. New generation IT technologies like ABCD—Artificial Intelligence, Blockchain, Cloud Computing, and Data Analytics could have been chosen and it would be interesting and important to study them in the future as well (Akter et al., 2020; Leng et al., 2020, 2022).

From the university point of view, it is vital that the curricula and the content of the education are kept up to date (Liljaniemi & Paavilainen, 2020). For companies, it is essential to follow new technologies and react rapidly when required. The newly conceived OQEM is an excellent tool for universities and companies to perform maturity level evaluations.

5. Conclusions

The study concludes that the selected five technologies are ready to be industrially used; nevertheless, they are rapidly evolving. Cobotics, Autonomous Robotics, Digital Twin Technology, and Artificial Intelligence and Machine Learning technologies can be placed in the Early Majority section, and Virtual and Augmented Reality in the Late Majority section.

The weaknesses and restrictions of the OQEM developed in the research have introduced the following subjects as suggestions for further work. The technology market share and amount of patents could be added to published article component, it is expected that these indicators would follow the S-curve in a similar manner. Another interesting area for research would be the slope of the S-curve of the technologies, i.e., to determine if it would correspond to the development momentum and if machine learning could be implemented to forecast future technological progress.

As a result of this research, a Flash-project concept was successfully created and used in Digi-Flash projects. The concept could be further developed, productized, and integrated in Metropolia’s service business. The successful proof of concept (POC)level demos implemented during the Digi-Flash project could scaled in full-scale investment projects.

Acknowledgements

The research reported in this paper was supported and made possible by the Digi-Flash (ERDF A75137) and Big-Flash (ERDF A77490) projects. This support is gratefully acknowledged.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Uudenmaan liitto [ERDF A77490]; Uudenmaan liitto [ERDF A75137].

Notes on contributors

Antti Liljaniemi

Antti Liljaniemi was born in Hämeenlinna, Finland in 1970. He received the M.Sc. degree in electrical engineering from the Tampere University of Technology, in 1998. He is currently pursuing the Ph.D. degree in mechanical engineering (digital twin technology) at Aalto University. From 1996 to 2006, he worked in various Finnish technology companies. Since 2006, he has been a Senior Lecturer and Project Manager with the Mechanical Engineering Department, Metropolia University. His current research interests include the IoT and virtual reality linked automation areas, such as cloud-based data acquisition, digital twin technology and virtual commissioning.

Heikki Paavilainen

Heikki Paavilainen was born in Viljakkala, Finland 1958. He received the M.Sc. degrees in hydraulics and automation from the Tampere University of Technology, in 1986 and the Lic. degree in hydraulic and automation engineering from Tampere University of Technology, in 1999. From 1986 to 1992, he worked in Valmet Factory Automation and IP-Products. Since 1992, he has been a Senior Lecturer with the Mechanical Engineering Department, Metropolia University.

Timo Tuominen

Timo Tuominen was born in Utajärvi, Finland in 1961.He received the M.Sc. degrees in mechanical engineering from the University of Oulu, in 1988 and the Lic. degree in automation engineering from Helsinki University of Technology, in 2008. From 1988 to 2008, he worked in various Finnish technology companies. Since 2008, he has been a Senior Lecturer and Project Manager with the Automation Engineering Department, Metropolia University. His current research interests include the cobotics and digital twin technology and virtual commissioning.