Designing as trading-off: a practice-based view on smart service systems

Figures & data

Table 1. Meta-requirements and initial design principles.

#	Initial design principles	Meta-requirements from knowledge base	Evaluation focus
1	Human-centred smart service systems require technologies (e.g., sensors, machine learning models, wearable devices etc.) that predict practices based on measuring bodily activities comprehensively, accurately, and reliably.	MR1: Provide effective “backstage analytics” Beverungen, Müller, et al. (2019) to make a practice capable of self-organising and reconfiguring.	Accuracy and reliability of instant feedback as delivered by machine learning model and sensors. Captured via: summative artificial evaluation (see below)
2	Human-centred smart service systems integrate sensors with bodily activities in ways that users perceive as unobtrusive.	MR2: Smart service systems need to operate as “invisible computers” Beverungen, Müller, et al. (2019), that are unobtrusive to users.	Whether, and if so, how, smart service systems’ operations that generate instant feedback become “invisible computers” (Beverungen, Müller, et al., 2019). Captured via: formative naturalistic evaluation (see below)
3	Human-centred smart service systems provide feedback that users can easily understand.	MR3: Feedback and advice on changing practices must be easy for users to understand.	Whether users understand instant feedback and consider it valuable to change their practice of sitting. Captured via: formative naturalistic evaluation (see below)
4	Human-centred smart service systems provide a functional setup that avoids triggering negative emotions in users.	MR4: Smart service systems need to avoid stimulating negative emotions from their usage.	Affective reactions of users to the instant feedback provided by smart service system. Captured via: formative naturalistic evaluation (see below)

Figure 1. The logic of our DSR study.

Table 2. Overview of our DSR project (following Peffers et al., 2007).

Identify problem and motivate	Sitting is a practice with a relationship to posture-related problems. It demands the design of human-centred smart service systems that focus on practices.
Define objectives of a solution	To improve the practice of sitting and to develop design principles for human-centred smart service systems that are useful also in other contexts.
Design and development	Design and development of a shirt, a machine learning algorithm, and a mobile app that provide real-time feedback for improving posture to humans.
Demonstration and evaluation	Naturalistic approach to evaluate the artefact in a human-centred manner through several rounds of expert interviews and user testing. The machine learning model was also evaluated via artificial evaluation.
Communication	Findings were communicated via research papers, workshops with designers and health insurance providers, as well as public relations and social media more broadly.

Table 3. Choice process for ‘human risk and effectiveness strategy” (Venable et al., 2016).

(1) Explication of goals	(2) Choice of evaluation strategy	(3) Determine properties to evaluate	(4) Design individual evaluation episodes
Rigour:Major: improvement of social practice upon intervention (MR2-MR4) Minor: technical considerations, i.e., reliability and accuracy (MR1) Uncertainty and risk reduction: Major: social Risk residing in the breakdown of social practice (MR2-MR4) Minor: technical risk rooted in the functionality of the artefact (MR1) Ethics: substantial risk to users (MR2-MR4) Efficiency: easy access to students continuously engaged in sitting	Prioritization of risks: Risks associated with MR2-MR4 are social risks outweighing technical risks associated with MR1 This calls for human risk and effectiveness strategy	Major (MR2-MR4): Interactions between smart objects and (1) bodily activity, (2) knowledge, and (3) emotions of humans Minor (MR1): Accuracy and reliability of machine learning model and sensors	Major (MR2-MR4): Naturalistic evaluation via seven workshops with physiotherapists (2017/18); 12 user tests at our institution (2018), 19 field interviews (2018) Minor (MR1): Artificial evaluation via conceptualisation of our artefact together with technology experts (2016); user tests and experiments (2017); field experiments via nine workshops with physiotherapists

Figure 2. Timeline of our demonstration and evaluation.

Table 4. Details about demonstration and evaluation, including primary data collected for these purposes.

Time	Role	Primary Data Sources
2016	Demonstration	Conceptualisation with technology experts (multiple interviews with researchers in the domain of posture management to understand the solution space)
2017		Prototype testing within own institution. User tests with three different members of the team (two male, one female) who assessed general functionalities for a proof-of-concept
2017		User tests on multiple prototypes across nine workshops with two physiotherapists (41 hrs of material)
2017/18	Evaluation	User tests on multiple prototypes across seven workshops with two physiotherapists (41 hrs of material)
2017		12 user tests on multiple prototypes with users in an office environment of a co-working space involving observatory techniques, think-aloud method and subsequent interviews
2018		19 field interviews
2019		Founding a venture in order to conduct user tests with prospective clients, as well as to improve posture models and mobile app.

Figure 3. Front-end of our mobile application.

Figure 4. Screens of our mobile application.

Figure 5. Push notifications sent by our mobile application.

Figure 6. 3D image produced by our mobile application.

Table 5. Revised design principles and justification for revisions.

Revised design principle	Trade-offs	Justification for revision
DP1rev (initial DP 1 and DP 2 were merged into DP1) Human-centred smart service systems should balance between the goals of producing complete data and disrupting practices with data collection.	Performance trade-off: Trading off the necessary completeness and accuracy of required data with possibilities to collect that data.	Evaluation showed that the pursuit of DP2 constrained pursuit of DP1. The solution was to merge DP1 and DP2 in order to address a trade-off that matters for designing smart service systems.
DP2rev (formerly DP 3) Human-centred smart service systems should provide a digital representation of practice resembling a user’s visual perception of that practice instead of providing encompassing information requiring expert knowledge.	Knowledge representation trade-off: Trading-off subjective visual perception of practice with objective assessment through complete and accurate data.	Evaluation showed that users demanded a digital representation without expert terminology. This was striking in our context as it contradicted our initial DP3, where it seemed important to tell users exactly what the healthcare-related problem is.
DP3rev (formerly DP 4) Human-centred smart service systems require triggering positive human emotions in the short-run in order to avert adverse consequences in the long-run.	Temporal trade-off: Trading off the long-term avoidance of negative emotions with triggering positive emotions in the short term.	Evaluation showed that without prior encounters with posture-related issues, users do not typically perceive the avoidance of posture-related problems as important in the long run. Instead, our app should be fun to use in the short-run in order to avoid posture-related consequences in the long-run.

Beverungen, D., Müller, O., Matzner, M., Mendling, J., & Vom Brocke, J. (2019). Conceptualizing smart service systems. Electronic Markets, 29(1), 7–18. https://doi.org/10.1007/s12525-017-0270-5

 Web of Science ®Google Scholar

Peffers, K., Tuunanen, T., Rothenberger, M. A., & Chatterjee, S. (2007). A design science research methodology for information systems research. Journal of Management Information Systems, 24(3), 45–78. https://doi.org/10.2753/MIS0742-1222240302

 Web of Science ®Google Scholar

Venable, J., Pries-Heje, J., & Baskerville, R. (2016). FEDS: A framework for evaluation in design science research. European Journal of Information Systems, 25(1), 77–89. https://doi.org/10.1057/ejis.2014.36

 Web of Science ®Google Scholar