Graphical Abstract Figure
Abstract
The rise of generative artificial intelligence (GAI) applications, epitomized by ChatGPT, has reshaped design processes by enhancing idea generation and conceptual depth for designers. However, the facilitating effects on novice designers' thinking remain uncertain, particularly in the context of sustainable service concept generation. This study examines these impacts of ChatGPT on design thinking process and outcomes through controlled experiments with 36 novice designers with ChatGPT, Tiangong AI, and no tools under a sustainable service design task. Through the protocol analysis, this study visualizes the design thinking by network-based cognitive maps, then evaluates design outcomes and systematically analyzes characteristics of design thinking development under different tool interventions. Findings indicate that ChatGPT enhances design concept novelty and systematicity but has limited impact on originality and sustainability. Furthermore, ChatGPT plays an active role in fostering thinking divergence and fluency, especially in providing relevant guides for developed ideas and accelerating the evaluation and creation process. The network-based cognitive maps reveal distinct shifts and styles influenced by ChatGPT, providing references for novice designers using such tools to enhance inspiration and design fluency, and also effectively employ diverse tools during specific concept generation stage. The study also provides insights for enhancing the relevance of educational curricula and enabling bottom-up sustainable service innovations.
References
1.
Muller
, M.
, Chilton
, L. B.
, Kantosalo
, A.
, Martin
, C. P.
, and Walsh
, G.
, “GenAICHI: Generative AI and HCI
,” Proceedings of the CHI Conference on Human Factors in Computing Systems Extended Abstracts
, New Orleans, LA
, Apr. 29–May 5
, pp. 1
–7
.2.
Han
, A.
, and Cai
, Z.
, “Design Implications of Generative AI Systems for Visual Storytelling for Young Learners
,” Proceedings of the 22nd Annual ACM Interaction Design and Children Conference
, Chicago, IL
, June 19–23
, pp. 470
–474
.3.
Schmidt
, A.
, Elagroudy
, P.
, Draxler
, F.
, Kreuter
, F.
, and Welsch
, R.
, 2024
, “Simulating the Human in HCD With ChatGPT: Redesigning Interaction Design With AI
,” Interactions
, 31
(1
), pp. 24
–31
. 4.
Cai
, A.
, Rick
, S. R.
, Heyman
, J. L.
, Zhang
, Y.
, Filipowicz
, A.
, Hong
, M.
, Klenk
, M.
, and Malone
, T.
, “DesignAID: Using Generative AI and Semantic Diversity for Design Inspiration
,” Proceedings of The ACM Collective Intelligence Conference
, Delft, Netherlands
, Nov. 6–9
, pp. 1
–11
.5.
Zhou
, C.
, Zhang
, X.
, and Yu
, C.
, 2023
, “How Does AI Promote Design Iteration? The Optimal Time to Integrate AI Into the Design Process
,” J. Eng. Des.
, 34
(12
), pp. 1
–28
. 6.
Klockmann
, V.
, von Schenk
, A.
, and Villeval
, M. C.
, 2022
, “Artificial Intelligence, Ethics, and Intergenerational Responsibility
,” J. Econ. Behav. Organ.
, 203
, pp. 284
–317
. 7.
Rane
, N.
, 2023
, “Roles and Challenges of ChatGPT and Similar Generative Artificial Intelligence for Achieving the Sustainable Development Goals (SDGs)
,” SSRN 4603244.8.
Prasad Agrawal
, K.
, 2023
, “Organizational Sustainability of Generative AI-Driven Optimization Intelligence
,” J. Comput. Inf. Syst.
, pp. 1
–15
. 9.
Ooi
, K.-B.
, Tan
, G. W.-H.
, Al-Emran
, M.
, Al-Sharafi
, M. A.
, Capatina
, A.
, Chakraborty
, A.
, Dwivedi
, Y. K.
, et al, 2023
, “The Potential of Generative Artificial Intelligence Across Disciplines: Perspectives and Future Directions
,” J. Comput. Inf. Syst.
, pp. 1
–32
. 10.
Alasadi
, E. A.
, and Baiz
, C. R.
, 2023
, “Generative AI in Education and Research: Opportunities, Concerns, and Solutions
,” J. Chem. Educ.
, 100
(8
), pp. 2965
–2971
. 11.
Simon
, H. A.
, 1969
, “The Science of the Artificial
,” Emotion Rev.
, 4
(1
), pp. 266
–268
. 12.
Gero
, J. S.
, and Milovanovic
, J.
, 2020
, “A Framework for Studying Design Thinking Through Measuring Designers' Minds, Bodies and Brains
,” Des. Sci.
, 6
, pp. e19
. 13.
Van Someren
, M.
, Barnard
, Y. F.
, and Sandberg
, J.
, 1994
, The Think Aloud Method: A Practical Approach to Modelling Cognitive
, Vol. 11, No. 6, Academic Press
, London
, pp. 19
–30
.14.
Kruger
, C.
, and Cross
, N.
, 2006
, “Solution Driven Versus Problem Driven Design: Strategies and Outcomes
,” Des. Stud.
, 27
(5
), pp. 527
–548
. 15.
Tang
, H.-H.
, Lee
, Y. Y.
, and Gero
, J. S.
, 2011
, “Comparing Collaborative Co-Located and Distributed Design Processes in Digital and Traditional Sketching Environments: A Protocol Study Using the Function–Behaviour–Structure Coding Scheme
,” Des. Stud.
, 32
(1
), pp. 1
–29
. 16.
Dorst
, K.
, 2004
, “On the Problem of Design Problems-Problem Solving and Design Expertise
,” J. Des. Res.
, 4
(2
), pp. 185
–196
. 17.
Yu
, R.
, and Gero
, J. S.
, 2016
, “An Empirical Basis for the Use of Design Patterns by Architects in Parametric Design
,” Int. J. Archit. Comput.
, 14
(3
), pp. 289
–302
. 18.
Hou
, G.
, and Lu
, G.
, 2019
, “The Influence of Design Proposal Viewing Strategy: Design Aesthetics and Professional Background
,” Int. J. Technol. Des. Educ.
, 29
(3
), pp. 543
–564
. 19.
Zou
, N.
, Zhang
, X.
, Lou
, J.
, Liao
, J.
, and Chai
, C.
, 2024
, “Exploring the Precedents Retrieval Styles of Industrial and Mechanical Design Students During Brainstorming
,” Int. J. Technol. Des. Educ.
, 34
(1
), pp. 57
–73
. 20.
Filippi
, S.
, 2023
, “Measuring the Impact of ChatGPT on Fostering Concept Generation in Innovative Product Design
,” Electronics
, 12
(16
), pp. 3535
. 21.
Buonamici
, F.
, Carfagni
, M.
, Furferi
, R.
, Volpe
, Y.
, and Governi
, L.
, 2020
, “Generative Design: an Explorative Study
,” Comput.-Aided Des. Appl.
, 18
(1
), pp. 144
–155
. 22.
Elgammal
, A.
, 2019
, “AI is Blurring the Definition of Artist: Advanced Algorithms Are Using Machine Learning to Create Art Autonomously
,” Am. Sci.
, 107
(1
), pp. 18
–22
. 23.
Chan
, J.
, Dow
, S. P.
, and Schunn
, C. D.
, 2015
, “Do the Best Design Ideas (Really) Come From Conceptually Distant Sources of Inspiration?
” Des. Stud.
, 36
, pp. 31
–58
. 24.
Goel
, T.
, Shaer
, O.
, Delcourt
, C.
, Gu
, Q.
, and Cooper
, A.
, “Preparing Future Designers for Human-AI Collaboration in Persona Creation
,” Proceedings of the 2nd Annual Meeting of the Symposium on Human-Computer Interaction for Work
, Oldenburg, Germany
, June 13–16
, pp. 1
–14
.25.
Kim
, J.
, and Maher
, M. L.
, 2023
, “The Effect of AI-Based Inspiration on Human Design Ideation
,” Int. J. Des. Creat. Innov.
, 11
(2
), pp. 81
–98
. 26.
Kwon
, E.
, Huang
, F.
, and Goucher-Lambert
, K.
, 2022
, “Enabling Multi-Modal Search for Inspirational Design Stimuli Using Deep Learning
,” AI EDAM
, 36
, pp. e22
. 27.
Kasneci
, E.
, Seßler
, K.
, Küchemann
, S.
, Bannert
, M.
, Dementieva
, D.
, Fischer
, F.
, Gasser
, U.
, et al, 2023
, “ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
,” Learn. Individ. Differ.
, 103
, pp. 102274
. 28.
Wu
, R.
, and Yu
, Z.
, 2024
, “Do AI Chatbots Improve Students Learning Outcomes? Evidence From a Meta-Analysis
,” Br. J. Educ. Technol.
, 55
(1
), pp. 10
–33
. 29.
Tlili
, A.
, Shehata
, B.
, Adarkwah
, M. A.
, Bozkurt
, A.
, Hickey
, D. T.
, Huang
, R.
, and Agyemang
, B.
, 2023
, “What If the Devil Is My Guardian Angel: ChatGPT as a Case Study of Using Chatbots in Education
,” Smart Learn. Environ.
, 10
(1
), pp. 15
. 30.
Wang
, X.
, Anwer
, N.
, Dai
, Y.
, and Liu
, A.
, 2023
, “ChatGPT for Design, Manufacturing, and Education
,” Procedia CIRP
, 119
, pp. 7
–14
. 31.
Hu
, X.
, Tian
, Y.
, Nagato
, K.
, Nakao
, M.
, and Liu
, A.
, 2023
, “Opportunities and Challenges of ChatGPT for Design Knowledge Management
,” Procedia CIRP
, 119
, pp. 21
–28
. 32.
Exintaris
, B.
, Karunaratne
, N.
, and Yuriev
, E.
, 2023
, “Metacognition and Critical Thinking: Using ChatGPT-Generated Responses as Prompts for Critique in a Problem-Solving Workshop (SMARTCHEMPer)
,” J. Chem. Educ.
, 100
(8
), pp. 2972
–2980
. 33.
Hák
, T.
, Janoušková
, S.
, and Moldan
, B.
, 2016
, “Sustainable Development Goals: A Need for Relevant Indicators
,” Ecol. Indic.
, 60
, pp. 565
–573
. 34.
Calabrese
, A.
, Costa
, R.
, Ghiron
, N. L.
, Tiburzi
, L.
, and Pedersen
, E. R. G.
, 2021
, “How Sustainable-Orientated Service Innovation Strategies Are Contributing to the Sustainable Development Goals
,” Technol. Forecast. Soc. Change
, 169
, pp. 120816
. 35.
Cook
, L. S.
, Bowen
, D. E.
, Chase
, R. B.
, Dasu
, S.
, Stewart
, D. M.
, and Tansik
, D. A.
, 2002
, “Human Issues in Service Design
,” J. Oper. Manage.
, 20
(2
), pp. 159
–174
. 36.
Aksoy
, L.
, Alkire
, L.
, Choi
, S.
, Kim
, P. B.
, and Zhang
, L.
, 2019
, “Social Innovation in Service: A Conceptual Framework and Research Agenda
,” J. Serv. Manag.
, 30
(3
), pp. 429
–448
. 37.
Chowdhury
, M. M. H.
, and Quaddus
, M. A.
, 2016
, “A Multi-Phased QFD Based Optimization Approach to Sustainable Service Design
,” Int. J. Prod. Econ.
, 171
, pp. 165
–178
. 38.
Vezzoli
, C.
, Ceschin
, F.
, Diehl
, J. C.
, and Kohtala
, C.
, 2015
, “New Design Challenges to Widely Implement 'Sustainable Product–Service Systems
',” J. Cleaner Prod.
, 97
, pp. 1
–12
. 39.
J. Luca
, E.
, and Ulyannikova
, Y.
, 2020
, “Towards a User-Centred Systematic Review Service: The Transformative Power of Service Design Thinking
,” J. Aust. Libr. Inf. Assoc.
, 69
(3
), pp. 357
–374
. 40.
Xie
, X.
, 2023
, “The Cognitive Process of Creative Design: A Perspective of Divergent Thinking
,” Think. Ski. Creat.
, 48
, pp. 101266
. 41.
Hay
, L.
, McTeague
, C.
, Duffy
, A. H.
, Pidgeon
, L. M.
, Vuletic
, T.
, and Grealy
, M.
, 2017
, “A Systematic Review of Protocol Studies on Conceptual Design Cognition
,” Des. Comput. Cogn.
, 16
, pp. 135
–153
. 42.
Panwar
, T.
, and Khan
, K.
, 2021
, “Integrating Design Thinking in Service Design Process: A Conceptual Review
,” J. Des. Think.
, 2
(1
), pp. 23
–36
. 43.
Behrens
, T. E.
, Muller
, T. H.
, Whittington
, J. C.
, Mark
, S.
, Baram
, A. B.
, Stachenfeld
, K. L.
, and Kurth-Nelson
, Z.
, 2018
, “What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior
,” Neuron
, 100
(2
), pp. 490
–509
. 44.
Kim
, E. J.
, and Kim
, K. M.
, 2015
, “Cognitive Styles in Design Problem Solving: Insights From Network-Based Cognitive Maps
,” Des. Stud.
, 40
, pp. 1
–38
. 45.
Zhou
, C.
, Chai
, C.
, and Liao
, J.
, 2021
, “Analysis of Problem Decomposition Strategies of Novice Industrial Designers Using Network-Based Cognitive Maps
,” Int. J. Technol. Des. Educ.
, 32
(2
), pp. 1
–23
. 46.
Lu
, C.
, Lu
, H.
, Chen
, D.
, Wang
, H.
, Li
, P.
, and Gong
, J.
, 2023
, “Human-Like Decision Making for Lane Change Based on the Cognitive Map and Hierarchical Reinforcement Learning
,” Transp. Res. Part C Emerg. Technol.
, 156
, pp. 104328
. 47.
Liedtka
, J.
, 2015
, “Perspective: Linking Design Thinking With Innovation Outcomes Through Cognitive Bias Reduction
,” J. Prod. Innov. Manage.
, 32
(6
), pp. 925
–938
. 48.
Crilly
, N.
, and Cardoso
, C.
, 2017
, “Where Next for Research on Fixation, Inspiration and Creativity in Design?
,” Des. Stud.
, 50
, pp. 1
–38
. 49.
Hu
, Y.
, Du
, X.
, Bryan-Kinns
, N.
, and Guo
, Y.
, 2019
, “Identifying Divergent Design Thinking Through the Observable Behavior of Service Design Novices
,” Int. J. Technol. Des. Educ.
, 29
(5
), pp. 1179
–1191
. 50.
Quintana-Cifuentes
, J.
, and Purzer
, S.
, 2022
, “Semantic Fluency in Design Reasoning
,” Int. J. Eng. Educ.
, 38
(6
), pp. 1891
–1903
.51.
Goucher-Lambert
, K.
, Moss
, J.
, and Cagan
, J.
, 2019
, “A Neuroimaging Investigation of Design Ideation With and Without Inspirational Stimuli—Understanding the Meaning of Near and Far Stimuli
,” Des. Stud.
, 60
, pp. 1
–38
. 52.
Shah
, J. J.
, Millsap
, R. E.
, Woodward
, J.
, and Smith
, S.
, 2012
, “Applied Tests of Design Skills—Part 1: Divergent Thinking
,” ASME J. Mech. Des.
, 134(4), p. 021005
. 53.
Lee
, J. H.
, and Ostwald
, M. J.
, 2022
, “The Relationship Between Divergent Thinking and Ideation in the Conceptual Design Process
,” Des. Stud.
, 79
, pp. 101089
. 54.
Grace
, K.
, Maher
, M. L.
, Fisher
, D.
, and Brady
, K.
, 2015
, “Data-Intensive Evaluation of Design Creativity Using Novelty, Value, and Surprise
,” Int. J. Des. Creat. Innov.
, 3
(3–4
), pp. 125
–147
. 55.
Christensen
, B. T.
, and Ball
, L. J.
, 2016
, “Dimensions of Creative Evaluation: Distinct Design and Reasoning Strategies for Aesthetic, Functional and Originality Judgments
,” Des. Stud.
, 45
, pp. 116
–136
. 56.
Shen
, J.
, Erkoyuncu
, J. A.
, Roy
, R.
, and Wu
, B.
, 2017
, “A Framework for Cost Evaluation in Product Service System Configuration
,” Int. J. Prod. Res.
, 55
(20
), pp. 6120
–6144
. 57.
Manz
, C. C.
, Skaggs
, B. C.
, Pearce
, C. L.
, and Wassenaar
, C. L.
, 2015
, “Serving One Another: Are Shared and Self-Leadership the Keys to Service Sustainability?
,” J. Organ. Behav.
, 36
(4
), pp. 607
–612
. 58.
Liu
, X.
, Werder
, K.
, and Maedche
, A.
, 2020
, “Novice Digital Service Designers' Decision-Making With Decision Aids—A Comparison of Taxonomy and Tags
,” Decis. Support Syst.
, 137
, pp. 113367
. 59.
60.
61.
Raffel
, C.
, Shazeer
, N.
, Roberts
, A.
, Lee
, K.
, Narang
, S.
, Matena
, M.
, Zhou
, Y.
, Li
, W.
, and Liu
, P. J.
, 2020
, “Exploring the Limits of Transfer Learning With a Unified Text-to-Text Transformer
,” J. Mach. Learn. Res.
, 21
(140
), pp. 1
–67
. 62.
Eccles
, D. W.
, and Arsal
, G.
, 2017
, “The Think Aloud Method: What Is It and How Do I Use It?
,” Qual. Res. Sport Exerc. Health
, 9
(4
), pp. 514
–531
. 63.
Charters
, E.
, 2003
, “The Use of Think-Aloud Methods in Qualitative Research an Introduction to Think-Aloud Methods
,” Brock Educ. J.
, 12
(2
), pp. 68
–82
. 64.
Xie
, L.
, Liu
, C.
, Li
, Y.
, and Zhu
, T.
, 2023
, “How to Inspire Users in Virtual Travel Communities: The Effect of Activity Novelty on Users' Willingness to Co-Create
,” J. Retail. Consum. Serv.
, 75
, pp. 103448
. 65.
Acar
, S.
, Burnett
, C.
, and Cabra
, J. F.
, 2017
, “Ingredients of Creativity: Originality and More
,” Creat. Res. J.
, 29
(2
), pp. 133
–144
. 66.
Wu
, J.
, Zhang
, L.
, and Ren
, X.
, 2022
, “Incorporating the Co-Creation Method Into Social Innovation Design to Promote Intergenerational Integration: A Case Study of a Public Square
,” Int. J. Environ. Res. Public Health
, 19
(19
), pp. 12908
. 67.
Chou
, C.-J.
, Chen
, C.-W.
, and Conley
, C.
, 2012
, “A Systematic Approach to Generate Service Model for Sustainability
,” J. Cleaner Prod.
, 29
, pp. 173
–187
. 68.
Kjaer
, L. L.
, Pagoropoulos
, A.
, Schmidt
, J. H.
, and McAloone
, T. C.
, 2016
, “Challenges When Evaluating Product/Service-Systems Through Life Cycle Assessment
,” J. Cleaner Prod.
, 120
, pp. 95
–104
. 69.
Ciasullo
, M. V.
, Troisi
, O.
, Grimaldi
, M.
, and Leone
, D.
, 2020
, “Multi-Level Governance for Sustainable Innovation in Smart Communities: An Ecosystems Approach
,” Int. Entrep. Manag. J.
, 16
(4
), pp. 1167
–1195
. 70.
Manzini
, E.
, and Vezzoli
, C.
, 2003
, “A Strategic Design Approach to Develop Sustainable Product Service Systems: Examples Taken From the ‘Environmentally Friendly Innovation’Italian Prize
,” J. Cleaner Prod.
, 11
(8
), pp. 851
–857
. 71.
Scott
, G. M.
, Lonergan
, D. C.
, and Mumford
, M. D.
, 2005
, “Conceptual Combination: Alternative Knowledge Structures, Alternative Heuristics
,” Creat. Res. J.
, 17
(1
), pp. 79
–98
. 72.
Kashapov
, M. M.
, and Leybina
, A. V.
, 2009
, “Motivation of Professional Creative Thinking
,” Psychol. Russia
, 2
(1
), pp. 585
–602
. 73.
Perez
, B.
, Hilburn
, S.
, Jensen
, D.
, and Wood
, K. L.
, 2019
, “Design Principle-Based Stimuli for Improving Creativity During Ideation
,” Proc. Inst. Mech. Eng., Part C J. Mech. Eng. Sci.
, 233
(2
), pp. 493
–503
. 74.
Eckert
, C.
, and Stacey
, M.
, 2000
, “Sources of Inspiration: A Language of Design
,” Des. Stud.
, 21
(5
), pp. 523
–538
. 75.
Kelly
, N.
, and Gero
, J. S.
, 2014
, “Interpretation in Design: Modelling how the Situation Changes During Design Activity
,” Res. Eng. Des.
, 25
(2
), pp. 109
–124
. 76.
Abbas
, W.
, Laszka
, A.
, Vorobeychik
, Y.
, and Koutsoukos
, X.
, “Improving Network Connectivity Using Trusted Nodes and Edges
,” Proceedings of the 2017 American Control Conference (ACC)
, Seattle, WA
, May 24–26
, IEEE, pp. 328
–333
.77.
Besta
, M.
, Memedi
, F.
, Zhang
, Z.
, Gerstenberger
, R.
, Blach
, N.
, Nyczyk
, P.
, Copik
, M.
, Kwaśniewski
, G.
, Müller
, J.
, and Gianinazzi
, L.
, 2024
, “Topologies of Reasoning: Demystifying Chains, Trees, and Graphs of Thoughts
,” arXiv preprint arXiv:2401.14295.78.
Rocha
, C. S.
, Antunes
, P.
, and Partidário
, P.
, 2019
, “Design for Sustainability Models: A Multiperspective Review
,” J. Cleaner Prod.
, 234
, pp. 1428
–1445
. 79.
AtKisson
, A.
, 1996
, “Developing Indicators of Sustainable Community: Lessons From Sustainable Seattle
,” Environ. Impact Assess. Rev.
, 16
(4–6
), pp. 337
–350
.80.
Lanzi
, P. L.
, and Loiacono
, D.
, “Chatgpt and Other Large Language Models as Evolutionary Engines for Online Interactive Collaborative Game Design
,” Proceedings of the Genetic and Evolutionary Computation Conference
, Lisbon, Portugal
, July 15–19
, pp. 1383
–1390
.81.
Agarwal
, M.
, Raju
, V.
, Dey
, R.
, and Nanda
, I.
, 2022
, “Descriptive Research on AI-Based Tools to Aid Personalized Customer Service: Case of ChatGPT
,” J. Reproducible Res.
, 1
(1
), pp. 140
–146
.82.
Guo
, Y.
, and Lee
, D.
, 2023
, “Leveraging Chatgpt for Enhancing Critical Thinking Skills
,” J. Chem. Educ.
, 100
(12
), pp. 4876
–4883
. 83.
Carlgren
, L.
, Rauth
, I.
, and Elmquist
, M.
, 2016
, “Framing Design Thinking: The Concept in Idea and Enactment
,” Creat. Innov. Manag.
, 25
(1
), pp. 38
–57
. 84.
Lawson
, B.
, 2004
, “Schemata, Gambits and Precedent: Some Factors in Design Expertise
,” Des. Stud.
, 25
(5
), pp. 443
–457
. 85.
dos Santos
, A. B. A.
, Bianchi
, C. G.
, and Borini
, F. M.
, 2018
, “Open Innovation and Cocreation in the Development of New Products: The Role of Design Thinking
,” Int. J. Innov.
, 6
(2
), pp. 112
–123
. 
