Ecosystem-Driven Design Thinking
A Milestone-Based Progression and Attrition Analysis of Innovation Competence
DOI:
https://doi.org/10.3991/ijep.v16i4.61279Keywords:
Challenge-Based Learning, Design Thinking, Engineering Education, Mixed Methods, Innovation Competency, Sustainable Development GoalsAbstract
Universities increasingly adopt SDG-anchored challenge-based learning (CBL) to develop innovation competencies, yet evaluations often rely on self-reports that cannot distinguish enthusiasm from demonstrated competence. This study evaluates the Innovation Challenge, an eight-week ecosystem-driven CBL (Ed-CBL) program integrated with Design Thinking (DT), and is reported using the TIDieR checklist to enhance replicability. We employed a convergent mixed-methods design: primary evidence comprised team-level performance across four mandatory deliverables and a summative pitch evaluated by an expert jury; secondary evidence assessed individual skill gains using the Youth Innovation Skills Measurement Tool (YISMT) in a non-equivalent groups pre- and post-test design. A milestone-based attrition analysis reveals a rigorous selection filter: only 55.5% (15/27) of verified teams overcame ecosystem constraints to complete the full cycle. Statistical analysis of skill gains (Welch’s t-test with Holm-Bonferroni correction) identified an initial differential gain solely in Creativity. However, a sensitivity analysis adjusting for significant baseline imbalances rendered this effect non-significant (p = .267); other dimensions also showed no significant changes. Consequently, we conclude that while Ed-CBL generates verifiable design outputs and technical performance, it does not yield robust short-term gains in self-reported innovation skills, highlighting the necessity of prioritizing performance evidence over self-efficacy in short-term programs.
References
[1] S. Perna, M. P. Recke, and M. Nichols, Challenge Based Learning. A Comprehensive Survey of Literature, 2023, https://www.challengeinstitute.org/CBL_Literature_Survey.pdf.
[2] J. Larsson, A. Johansson, and O. H. Svensson, “New Engineers for a Bright Future: Deconstructing the Promises of Challenge Based Learning,” Studies in Engineering Education, vol. 6, no. 1, pp. 129–151, 2025, https://doi.org/10.21061/SEE.172.
[3] A. A. Del Savio, K. G. Velarde, L. C. Montero, and M. A. V. Olivera, “Research Training: Unraveling the Research Methodological Design Challenge in Engineering Programs,” International Journal of Engineering Pedagogy (iJEP), vol. 14, no. 7, pp. 30–53, Oct. 2024, https://doi.org/10.3991/IJEP.V14I7.48987.
[4] J. Membrillo-Hernández, M. J. Ramírez-Cadena, M. Martínez-Acosta, E. Cruz-Gómez, E. Muñoz-Díaz, and H. Elizalde, “Challenge based learning: the importance of world-leading companies as training partners,” International Journal on Interactive Design and Manufacturing, vol. 13, no. 3, pp. 1103–1113, Sep. 2019, https://doi.org/10.1007/S12008-019-00569-4.
[5] K. M. Henao-Romero and I. S. E. van der Zande, “Global–Local Learning Ecosystems for Societal Purpose: A Latin American Perspective,” Educ. Sci. (Basel)., vol. 15, no. 12, Dec. 2025, https://doi.org/10.3390/EDUCSCI15121693/S1.
[6] L. C. Bowling et al., “Addressing Water Resources and Environmental Quality Programming Needs in Arequipa, Peru,” J. Contemp. Water Res. Educ., vol. 173, no. 1, pp. 1–12, Sep. 2021, https://doi.org/10.1111/J.1936-704X.2021.3354.X.
[7] G. Salmoral et al., “Water-related challenges in nexus governance for sustainable development: Insights from the city of Arequipa, Peru,” Sci. Total Environ., vol. 747, Dec. 2020, https://doi.org/10.1016/J.SCITOTENV.2020.141114.
[8] K. Doulougeri, J. D. Vermunt, G. Bombaerts, and M. Bots, “Challenge-based learning implementation in engineering education: A systematic literature review,” Journal of Engineering Education, vol. 113, no. 4, pp. 1076–1106, Oct. 2024, https://doi.org/10.1002/JEE.20588.
[9] D. Dunning, K. Johnson, J. Ehrlinger, and J. Kruger, “Why People Fail to Recognize Their Own Incompetence,” Curr. Dir. Psychol. Sci., vol. 12, no. 3, pp. 83–87, Jun. 2003, https://doi.org/10.1111/1467-8721.01235.
[10] K. Lavidas, S. Papadakis, D. Manesis, A. S. Grigoriadou, and V. Gialamas, “The Effects of Social Desirability on Students’ Self-Reports in Two Social Contexts: Lectures vs. Lectures and Lab Classes,” Information 2022, Vol. 13, no. 10, Oct. 2022, https://doi.org/10.3390/INFO13100491.
[11] V. Robledo-Rella, L. Neri, R. M. G. García-Castelán, A. Gonzalez-Nucamendi, J. Valverde-Rebaza, and J. Noguez, “A comparative study of a new challenge-based learning model for engineering majors,” Front. Educ. (Lausanne)., vol. 10, p. 1545071, Mar. 2025, https://doi.org/10.3389/feduc.2025.1545071.
[12] C. L. Garay-Rondero, A. Castillo-Paz, C. Gijón-Rivera, G. Domínguez-Ramírez, C. Rosales-Torres, and A. Oliart-Ros, “Competency-based assessment tools for engineering higher education: a case study on complex problem-solving,” Cogent Education, vol. 11, no. 1, Dec. 2024, https://doi.org/10.1080/2331186X.2024.2392424.
[13] R. J. Mislevy and G. D. Haertel, “Implications of Evidence-Centered Design for Educational Testing,” Educational Measurement: Issues and Practice, vol. 25, no. 4, pp. 6–20, Dec. 2006, https://doi.org/10.1111/J.1745-3992.2006.00075.X.
[14] A. Fridrich, G. J. Jenny, and G. F. Bauer, “The Context, Process, and Outcome Evaluation Model for Organisational Health Interventions,” Biomed Res. Int., vol. 2015, p. 414832, 2015, https://doi.org/10.1155/2015/414832.
[15] A. Leles, L. Zaina, and J. Roberto Cardoso, “Challenge-Based Learning for Competency Development in Engineering Education, a Prisma-Based Systematic Literature Review,” IEEE Transactions on Education, vol. 67, no. 5, pp. 746–757, 2024, https://doi.org/10.1109/TE.2024.3417908.
[16] K. Helker, M. Bruns, I. M. M. J. Reymen, and J. D. Vermunt, “A framework for capturing student learning in challenge-based learning,” Active Learning in Higher Education, vol. 26, no. 1, pp. 213–229, Mar. 2025, https://doi.org/10.1177/14697874241230459.
[17] J. Höffken and J. Lazendic-Galloway, “Engaging for the future: challenge-based learning and stakeholder partnerships in sustainability education,” Sustainable Earth Reviews 2024, vol. 7, no. 1, pp. 20-, Jun. 2024, https://doi.org/10.1186/S42055-024-00087-6.
[18] T. Van Dang, P. T. G. Bao, V. D. Minh, and N. T. T. Tu, “The Application of Problem-Based Learning in Soft Skills Courses: An Experiment in Classes with Multidisciplinary Students in Vietnam,” International Journal of Engineering Pedagogy (iJEP), vol. 15, no. 5, pp. 20–41, Jul. 2025, https://doi.org/10.3991/IJEP.V15I5.53663.
[19] G. Zacharis and S. Papadakis, “Can AI Grade Like a Human? Validity, Reliability, and Fairness in University Coursework Assessment,” Educational Process International Journal, vol. 19, no. 1, 2025, https://doi.org/10.22521/EDUPIJ.2025.19.591.
[20] L. Fallin and C. Turton, “The transformative potential of Design Thinking in Learning Development,” Journal of Learning Development in Higher Education, vol. 34, 2025, https://doi.org/10.47408/jldhe.vi34.1314.
[21] M. Kochanowska and W. R. Gagliardi, “The Double Diamond Model: In Pursuit of Simplicity and Flexibility,” Springer Series in Design and Innovation, vol. 16, pp. 19–32, 2022, https://doi.org/10.1007/978-3-030-79879-6_2.
[22] S. Zhang et al., “How to Assess AI Literacy: Misalignment Between Self-Reported and Objective-Based Measures,” LAK26: 16th International Learning Analytics and Knowledge Conference (LAK 2026), April 27-May 01, 2026, Bergen, Norway, vol. 1, Jan. 2026, https://doi.org/10.1145/3785022.3785088.
[23] Q. Meng and Q. Zhang, “The Influence of Academic Self-Efficacy on University Students’ Academic Performance: The Mediating Effect of Academic Engagement,” Sustainability (Switzerland), vol. 15, no. 7, 2023, https://doi.org/10.3390/su15075767.
[24] C. S. Reichardt, “Experimental and Quasi‐Experimental Designs for Generalized Causal Inference,” Social Service Review, vol. 76, no. 3, pp. 510–514, Sep. 2002, https://doi.org/10.1086/345281.
[25] A. Rangel-Pacheco and A. Witte, “NeMTSS Research Brief Essential Components and Applications of Implementation Fidelity,” 2020, https://nemtss.unl.edu/wp-content/uploads/2022/06/20-Essential-Components-and-Applications-of-Implementation-Fidelity.pdf.
[26] L. Simón-Chico, A. González-Peño, E. Hernández-Cuadrado, and E. Franco, “The Impact of a Challenge-Based Learning Experience in Physical Education on Students’ Motivation and Engagement,” Eur. J. Investig. Health Psychol. Educ., vol. 13, no. 4, pp. 684–700, Apr. 2023, https://doi.org/10.3390/EJIHPE13040052.
[27] J. F. Jiménez-Parra, N. Belando-Pedreño, J. López-Fernández, A. J. García-Vélez, and A. Valero-Valenzuela, “‘ACTIVE VALUES’: An Interdisciplinary Educational Programme to Promote Healthy Lifestyles and Encourage Education in Values—A Rationale and Protocol Study,” Applied Sciences 2022, Vol. 12, no. 16, Aug. 2022, https://doi.org/10.3390/APP12168073.
[28] J. Lönngren, “Wicked problems in engineering education: preparing future engineers to work for sustainability,” Environ. Educ. Res., vol. 25, no. 12, pp. 1808–1809, Dec. 2019, https://doi.org/10.1080/13504622.2019.1639038.
[29] R. Bender-Salazar, “Design thinking as an effective method for problem-setting and needfinding for entrepreneurial teams addressing wicked problems,” J. Innov. Entrep., vol. 12, no. 1, pp. 24-, Dec. 2023, https://doi.org/10.1186/S13731-023-00291-2.
[30] Sh. M. Hassan, “SCAMPER as a Creative Idea Generation Method: Case Study on Graphic Design Students,” Information Sciences Letters, vol. 12, no. 4, Apr. 2023, Accessed: Feb. 14, 2026. [Online]. Available: https://digitalcommons.aaru.edu.jo/isl/vol12/iss4/53
[31] N. Nascimento, A. R. Santos, A. Sales, and R. Chanin, “Enablers and inhibitors in Agile Teams - A Case Study Using Challenge Based Learning for Mobile Application Development,” Proceedings - 5th International Workshop on Software-Intensive Business: Towards Sustainable Software Business, IWSiB 2022, pp. 67–74, Jan. 2023, https://doi.org/10.1145/3524614.3528623.
[32] F. J. Llamas Fernández and J. C. Fernández Rodríguez, “La metodología Lean startup: desarrollo y aplicación para el emprendimiento,” Revista Escuela de Administración de Negocios, no. 84, Apr. 2018, https://doi.org/10.21158/01208160.N84.2018.1918.
[33] E. A. López-Guajardo, R. A. Ramirez-Mendoza, A. Vargas-Martinez, W. Jianhong, A. Roman-Flores, and G. Zavala, “Argumentative-driven assessments in engineering: a challenge-based learning approach to the evaluation of competencies,” International Journal on Interactive Design and Manufacturing, vol. 17, no. 1, pp. 79–91, Feb. 2023, https://doi.org/10.1007/S12008-022-01188-2.
[34] S.-H. Jin, K.-I. Song, D. Shin, and S. Shin, “A Performance-Based Evaluation Rubric for Assessing and Enhancing Engineering Design Skills in Introductory Engineering Design Courses,” International Journal of Engineering Education, vol. 31, pp. 1007–1020, Jan. 2015.
[35] P. E. Shrout and J. L. Fleiss, “Intraclass correlations: uses in assessing rater reliability,” Psychol. Bull., vol. 86, no. 2, pp. 420–428, 1979, https://doi.org/10.1037//0033-2909.86.2.420.
[36] T. K. Koo and M. Y. Li, “A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research,” J. Chiropr. Med., vol. 15, no. 2, pp. 155–163, Jun. 2016, https://doi.org/10.1016/J.JCM.2016.02.012.
[37] E. Chell and R. Athayde, “The identification and measurement of innovative characteristics of young people: development of the Youth Innovation Skills Measurement Tool,” 2009, https://researchke.kingston.ac.uk/ws/portalfiles/portal/48219303/Chell-E-5985.pdf.
[38] B. G. Tabachnick and L. S. Fidell, “Using Multivariate Statistics,” Journal of Applied Psychology, vol. 87, no. 4, pp. 611–628, 2017, [Online]. Available: https://lccn.loc.gov/2017040173
[39] P. D. Ellis, “The Essential Guide to Effect Sizes: Statistical Power, Meta-Analysis, and the Interpretation of Research Results,” The Essential Guide to Effect Sizes, Jul. 2010, https://doi.org/10.1017/CBO9780511761676.
[40] S. E. Maxwell, H. D. Delaney, and K. Kelley, “DESIGNING EXPERIMENTS AND ANALYZING DATA: A Model Comparison Perspective, Third Edition,” Designing Experiments and Analyzing Data: A Model Comparison Perspective, Third Edition, pp. 1–1056, Jan. 2017, https://doi.org/10.4324/9781315642956.
[41] S. Gupta, “Can Innovation Be Taught? Evidence from a School-Based RCT in India,” Stanford | King Center of Global Development, Apr. 2025, https://kingcenter.stanford.edu/sites/g/files/sbiybj16611/files/media/file/wp2076.pdf.
[42] G. Özdemir Aydın, S. Küçükaydınoğlu, A. Palloş, E. Y. Coşkun, and N. Turan, “The relationship between first-year nursing students’ innovation skills and attitudes toward artificial intelligence: A multicentre study,” Nurse Educ. Pract., vol. 86, p. 104429, Jul. 2025, https://doi.org/10.1016/J.NEPR.2025.104429.
[43] H. Z. Altınışık, T. Adıgüzel, and Y. G. Gençer, “Adaptation of Youth Innovational Skills Measurement Tool for Turkish Usage,” Kastamonu Education Journal, vol. 31, no. 1, pp. 155–164, Jan. 2023, https://doi.org/10.24106/KEFDERGI.1246467.
[44] D. L. Head, “A Novel Game-Based Assessment of Risk-Taking Propensity: Development and Psychometric Evidence,” UNLV Theses, Dissertations, Professional Papers, and Capstones, May 2025, https://doi.org/10.34917/39206727.
[45] B. Lira et al., “Large studies reveal how reference bias limits policy applications of self-report measures,” Sci. Rep., vol. 12, no. 1, p. 19189, Dec. 2022, https://doi.org/10.1038/S41598-022-23373-9.
[46] T. Karakose, T. Tülübaş, S. Papadakis, and R. Yirci, “Evaluating the Intellectual Structure of the Knowledge Base on Transformational School Leadership: A Bibliometric and Science Mapping Analysis,” Education Sciences 2023, vol. 13, no. 7, Jul. 2023, https://doi.org/10.3390/EDUCSCI13070708.
[47] J. P. Doh, “Can Leadership Be Taught? Perspectives From Management Educators,” Academy of Management Learning & Education, vol. 2, no. 1, pp. 54–67, Nov. 2017, https://doi.org/10.5465/AMLE.2003.9324025.
[48] A. L. Duckworth and D. S. Yeager, “Measurement Matters: Assessing Personal Qualities Other Than Cognitive Ability for Educational Purposes,” Educ. Res., vol. 44, no. 4, p. 237, May 2015, https://doi.org/10.3102/0013189X15584327.
[49] P. Pantzos, A. Pears, and E. Ampadu, “Students’ Motivation in Challenge-Based Learning in Higher Engineering Education: A Scoping Review,” International Journal of Engineering Pedagogy (iJEP), vol. 15, no. 6, pp. 4–34, Nov. 2025, https://doi.org/10.3991/IJEP.V15I6.54571.
[50] A. G. L. Romme, “Against All Odds: How Eindhoven Emerged as a Deeptech Ecosystem,” Systems 2022, vol. 10, no. 4, Aug. 2022, https://doi.org/10.3390/SYSTEMS10040119.
[51] C. Fang, G. R. Dantes, K. R. Dantes, and D. S. Wahyuni, “Design-Based Learning in Undergraduate Engineering Education: A Systematic Conceptual Review of Implementation, Impact, and Challenges,” PPSDP International Journal of Education, vol. 4, no. 2, pp. 1–16, Nov. 2025, https://doi.org/10.59175/PIJED.V4I2.774.
[52] E. Y. L. Au and R. S. Goonetilleke, “Leveraging Design Thinking to Enhance Engineering Teaching: An Operational Model,” IEEE Transactions on Education, vol. 68, no. 1, pp. 95–102, 2025, https://doi.org/10.1109/TE.2024.3467387.
[53] J. Power and D. Tanner, “Peer assessment, self-assessment, and resultant feedback: an examination of feasibility and reliability,” 2023, https://doi.org/10.1080/03043797.2023.2185769.
[54] P. Dawson, “Assessment rubrics: towards clearer and more replicable design, research and practice,” Assess. Eval. High. Educ., vol. 42, no. 3, pp. 347–360, Apr. 2017, https://doi.org/10.1080/02602938.2015.1111294.
[55] S. I. Marino, “Enhancing graduation outcomes through challenge-based learning: A strategic proposal,” J. Educ. Elearn. Res., vol. 12, no. 3, pp. 499–506, Sep. 2025, https://doi.org/10.20448/JEELR.V12I3.7397.
[56] A. I. Zourmpakis, M. Kalogiannakis, and S. Papadakis, “Adaptive Gamification in Science Education: An Analysis of the Impact of Implementation and Adapted Game Elements on Students’ Motivation,” Computers 2023, vol. 12, no. 7, Jul. 2023, https://doi.org/10.3390/COMPUTERS12070143.
[57] J. A. Palma-Mendoza, D. E. Salinas-Navarro, I. A. Arana-Solares, and F. Franco-Herrera, “Challenge-Based Learning for Active Learning in Industrial Engineering Education,” The Emerald Handbook of Active Learning For Authentic Assessment, pp. 313–335, Apr. 2025, https://doi.org/10.1108/978-1-83797-857-120251016.
[58] Y.-C. Ching and Y.-C. Ho, “AI Virtual Human–Augmented Game-Based Teaching to Enhance Emotional Intelligence in Nursing Students: Protocol for a Single-Group Pretest-Posttest Action Research Study,” JMIR Res. Protoc., vol. 14, p. e80290, Jul. 2025, https://doi.org/10.2196/80290.
[59] S. Sulaiman and K. Z. Mamaraimovna, “Enhancing professional competence through ESP,” American Journal of Philological Sciences, vol. 5, no. 06, pp. 182–185, Jun. 2025, https://doi.org/10.37547/AJPS/VOLUME05ISSUE06-49.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Dely-Marysheck Lazo-Barreda, Elizabeth-Susan Mamani-Machaca, Karina Rosas-Paredes, Midwar Ancco, Carla Madeleine Graciela Cuya-Zevallos
This work is licensed under a Creative Commons Attribution 4.0 International License.
