Evaluating the usability and practicality of a proposed carbon footprint methodology and tool for National Olympic Committees
Article Sidebar
Main Article Content
Abstract
Robust carbon footprint methodologies and tools are increasingly required across the sport sector, yet National Olympic Committees lack a standardised, sector-specific approach. A bespoke carbon footprint methodology and accompanying calculation tool were developed for this purpose; however, technical rigour alone does not ensure that intended users can apply them. This study evaluates the usability and practicality of the proposed methodology and tool through direct engagement with end-users. Fourteen staff members from different National Olympic Committees applied the methodology and tool within their organisations over a four-to-six-week period, then completed the System Usability Scale and a semi-structured interview. The tool achieved a mean System Usability Scale score of 68.4, corresponding to a grade C and a marginal-high acceptability rating. Usability was rated positively, whereas learnability scored below the neutral midpoint, and scores correlated strongly with participants' years of experience (r = .99). Interviews indicated that effectiveness was the strongest dimension and learnability the weakest, with data collection identified as the main source of effort. Participants judged the methodology rigorous and credible but recommended improved onboarding, validation, and guidance to support less experienced users and wider adoption.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
References
Abbott, J., Gao, G., & Shih, P. (2019, March). Creen: a carbon footprint calculator designed for calculation in context. In International Conference on Information (pp. 769-776). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-15742-5_72 DOI: https://doi.org/10.1007/978-3-030-15742-5_72
Abran, A., Khelifi, A., Suryn, W., & Seffah, A. (2003). Usability meanings and interpretations in ISO standards. Software Quality Journal, 11(4), 325-338. https://doi.org/10.1023/A:1025869312943 DOI: https://doi.org/10.1023/A:1025869312943
Adams, W. C. (2015). Conducting semi-structured interviews. In K. E. Newcomer, H. P. Hatry, & J. S. Wholey (Eds.), Handbook of practical program evaluation (4th ed., pp. 492-505). Wiley. https://doi.org/10.1002/9781119171386.ch19 DOI: https://doi.org/10.1002/9781119171386.ch19
Adeoye‐Olatunde, O. A., & Olenik, N. L. (2021). Research and scholarly methods: Semi‐structured interviews. Journal of the American College of Clinical Pharmacy, 4(10), 1358-1367. https://doi.org/10.1002/jac5.1441 DOI: https://doi.org/10.1002/jac5.1441
Adhikari, N., Li, H., & Gopalakrishnan, B. (2025). A bibliometric and systematic review of carbon footprint tracking in cross-sector industries: emerging tools and technologies. Sustainability, 17(9), 4205. https://doi.org/10.3390/su17094205 DOI: https://doi.org/10.3390/su17094205
Aldoihi, S. (2020). Product Usability Driven System Engineering. The Centre for Direct Scientific Communication.
Almazroi, A. A. (2021). A systematic mapping study of software usability studies. International Journal of Advanced Computer Science and Applications, 12(9). https://doi.org/10.14569/IJACSA.2021.0120927 DOI: https://doi.org/10.14569/IJACSA.2021.0120927
Aresta, M., Beça, P., Siso, D., & Quintero, E. (2024, November). Promoting sustainable behaviours in organizations: The IDEA System. In International Conference on Design and Digital Communication (pp. 163-174). Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-77566-6_12 DOI: https://doi.org/10.1007/978-3-031-77566-6_12
Asam, D., Jürgen, E., & Friedl, G. (2025). How carbon accounting supports corporate decarbonization. Foundations and Trends® in Accounting, 19(3-4), 46-77. https://doi.org/10.1561/1400000080-3 DOI: https://doi.org/10.1561/1400000080-3
Augoye, O., Muyiwa-Ajayi, T. P., & Sobowale, A. (2024). The effectiveness of carbon accounting in reducing corporate carbon footprints. International Journal of Multidisciplinary Research and Growth Evaluation, 5(1), 1364-1371. https://doi.org/10.54660/.IJMRGE.2024.5.1.1364-1371 DOI: https://doi.org/10.54660/.IJMRGE.2024.5.1.1364-1371
Azarkamand, S., Ferré, G., & RM, D. (2021, September). Calculating Carbon Footprint in ports through a new Standard Tool: case study applications. In 17th International Conference on Environmental Science and Technology. Athens, Greece.
Azhari, A. E. A., Chai, J., & Anderson, C. (2022). reflections of conducting online interviews during the pandemic: benefits, challenges and practical solutions. Malaysian Journal of Qualitative Research, 8(1). https://doi.org/10.61211/mjqr080103 DOI: https://doi.org/10.61211/mjqr080103
Baehr, J., Zenglein, F., Sonnemann, G., Lederer, M., & Schebek, L. (2024). Back in the driver's seat: how new EU greenhouse-gas reporting schemes challenge corporate accounting. Sustainability, 16(9), 3693. https://doi.org/10.3390/su16093693 DOI: https://doi.org/10.3390/su16093693
Ball, K. R., Burke, I. C., Collins, D., Kruger, C. E., & Yorgey, G. (2023). Digging deeper: Assessing the predictive power of common greenhouse gas accounting tools for soil carbon sequestration under organic amendment. Journal of Cleaner Production, 429, 139448. https://doi.org/10.1016/j.jclepro.2023.139448 DOI: https://doi.org/10.1016/j.jclepro.2023.139448
Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the system usability scale. Intl. Journal of Human-Computer Interaction, 24(6), 574-594. https://doi.org/10.1080/10447310802205776 DOI: https://doi.org/10.1080/10447310802205776
Bangor, A., Kortum, P., & Miller, J. (2009). Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of Usability Studies, 4(3), 114-123.
Barendregt, W., Biørn-Hansen, A., & Andersson, D. (2020). Users' experiences with the use of transaction data to estimate consumption-based emissions in a carbon calculator. Sustainability, 12(18), 7777. https://doi.org/10.3390/su12187777 DOI: https://doi.org/10.3390/su12187777
Bearman, M. (2019). Eliciting rich data: A practical approach to writing semi-structured interview schedules. Focus on Health Professional Education: A Multi-Professional Journal, 20(3), 1-11. https://doi.org/10.11157/fohpe.v20i3.387 DOI: https://doi.org/10.11157/fohpe.v20i3.387
Bekaroo, G., Roopowa, D., & Bokhoree, C. (2019, September). Mobile-based carbon footprint calculation: insights from a usability study. In 2019 Conference on Next Generation Computing Applications (NextComp) (pp. 1-6). IEEE. https://doi.org/10.1109/NEXTCOMP.2019.8883622 DOI: https://doi.org/10.1109/NEXTCOMP.2019.8883622
Bevan, N., Carter, J., & Harker, S. (2015, July). ISO 9241-11 revised: What have we learnt about usability since 1998?. In International conference on human-computer interaction (pp. 143-151). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-20901-2_13 DOI: https://doi.org/10.1007/978-3-319-20901-2_13
Binyamin, S., Rutter, M., & Smith, S. (2016). The utilization of system usability scale in learning management systems: a case study of Jeddah Community College. In ICERI2016 Proceedings (pp. 5314-5323). IATED. https://doi.org/10.21125/iceri.2016.2290 DOI: https://doi.org/10.21125/iceri.2016.2290
Birnik, A. (2013). An evidence-based assessment of online carbon calculators. International Journal of Greenhouse Gas Control, 17, 280-293. https://doi.org/10.1016/j.ijggc.2013.05.013 DOI: https://doi.org/10.1016/j.ijggc.2013.05.013
Blattgerste, J., Behrends, J., & Pfeiffer, T. (2022, June). A web-based analysis toolkit for the system usability scale. In Proceedings of the 15th International Conference on PErvasive Technologies Related to Assistive Environments (pp. 237-246). https://doi.org/10.1145/3529190.3529216 DOI: https://doi.org/10.1145/3529190.3529216
Bowen, F., & Wittneben, B. (2011). Carbon accounting: Negotiating accuracy, consistency and certainty across organisational fields. Accounting, Auditing & Accountability Journal, 24(8), 1022-1036. https://doi.org/10.1108/09513571111184742 DOI: https://doi.org/10.1108/09513571111184742
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. https://doi.org/10.1191/1478088706qp063oa DOI: https://doi.org/10.1191/1478088706qp063oa
Braun, V., & Clarke, V. (2021). Thematic analysis: A practical guide. Sage. https://doi.org/10.4135/9781036232078.n9 DOI: https://doi.org/10.4135/9781036232078.n9
Breton, M. L., & Aggeri, F. (2018). Counting before acting? The performativity of carbon accounting called into question - Calculation acts and dispositifs in a big French construction company. HAL (Le Centre Pour La Communication Scientifique Directe).
Brook, H. (2025). Greenhouse gas (GHG) inventory development and management. In Cases and Exercises on Sustainability Accounting (pp. 129-140). Edward Elgar Publishing. https://doi.org/10.4337/9781035330607.00020 DOI: https://doi.org/10.4337/9781035330607.00020
Brooke, J. (1996). SUS: A "quick and dirty" usability scale. In P. W. Jordan, B. Thomas, I. L. McClelland, & B. Weerdmeester (Eds.), Usability evaluation in industry (pp. 189-194). Taylor & Francis.
Brooke, J. (2013). SUS: a retrospective. Journal of Usability Studies, 8(2).
Büchs, M., Bahaj, A. S., Blunden, L., Bourikas, L., Falkingham, J., James, P., ... & Wu, Y. (2018). Promoting low carbon behaviours through personalised information? Long-term evaluation of a carbon calculator interview. Energy Policy, 120, 284-293. https://doi.org/10.1016/j.enpol.2018.05.030 DOI: https://doi.org/10.1016/j.enpol.2018.05.030
Cairns-Lee, H., Lawley, J., & Tosey, P. (2022). Enhancing researcher reflexivity about the influence of leading questions in interviews. The Journal of Applied Behavioral Science, 58(1), 164-188. https://doi.org/10.1177/00218863211037446 DOI: https://doi.org/10.1177/00218863211037446
Castillo-Montoya, M. (2016). Preparing for interview research: The interview protocol refinement framework. The Qualitative Report, 21(5). https://doi.org/10.46743/2160-3715/2016.2337 DOI: https://doi.org/10.46743/2160-3715/2016.2337
Chatty, T., Qu, Y., Ba-Sabaa, H. H., & Murnane, E. L. (2021). Examining the user experience of life cycle assessment tools and their ability to cater to Ecodesign in early-stage product development practice. Proceedings of the Design Society, 1, 1441. https://doi.org/10.1017/pds.2021.405 DOI: https://doi.org/10.1017/pds.2021.405
Christy, A. P., Elnahass, M., Amézaga, J. M., Browne, A., & Heidrich, O. (2023). A dynamic framework to align company climate reporting and action with global climate targets. Business Strategy and the Environment, 33(4), 3103. https://doi.org/10.1002/bse.3635 DOI: https://doi.org/10.1002/bse.3635
Chyung, S. Y., Barkin, J. R., & Shamsy, J. A. (2018). Evidence‐based survey design: The use of negatively worded items in surveys. Performance Improvement, 57(3), 16-25. https://doi.org/10.1002/pfi.21749 DOI: https://doi.org/10.1002/pfi.21749
Ciocoi-Pop, D. R., Boros-Balint, I., & Baciu, M. A. (2025). Sport and Sustainability Reporting: Assessing the Readiness of Sports Clubs in the Balkans. Amfiteatru Economic, 27(70), 922–938. https://doi.org/10.24818/EA/2025/70/921 DOI: https://doi.org/10.24818/EA/2025/70/921
Clifford, N., Cope, M., Gillespie, T., & French, S. (Eds.). (2016). Key methods in geography. Sage.
Corsini, M., Scopigno, R., Calori, L., Graf, H., Pletinckx, D., Pescarin, S., ... & Denard, H. (2012). Analysis of Visualisation and Interaction Tools Authors. The Centre for Direct Scientific Communication.
Cridland, E. K., Jones, S. C., Caputi, P., & Magee, C. A. (2015). Qualitative research with families living with autism spectrum disorder: Recommendations for conducting semistructured interviews. Journal of Intellectual and Developmental Disability, 40(1), 78-91. https://doi.org/10.3109/13668250.2014.964191 DOI: https://doi.org/10.3109/13668250.2014.964191
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319-340. https://doi.org/10.2307/249008 DOI: https://doi.org/10.2307/249008
Dirin, A., Nieminen, M., & Laine, T. H. (2023). Feelings of being for mobile user experience design. International Journal of Human-Computer Interaction, 39(20), 4059-4079. https://doi.org/10.1080/10447318.2022.2108964 DOI: https://doi.org/10.1080/10447318.2022.2108964
Du, J., Zhong, Z., Shi, Q., Cao, Y., Yang, C., & Wang, L. (2024). A review of household carbon footprint: measurement methods, evolution and emissions assessment. Carbon Footprints, 3(3), N-A. https://doi.org/10.20517/cf.2024.08 DOI: https://doi.org/10.20517/cf.2024.08
Eleftheriadis, I., & Anagnostopoulou, E. (2024). Developing a tool for calculating the carbon footprint in SMEs. Sustainability, 16(5), 1905. https://doi.org/10.3390/su16051905 DOI: https://doi.org/10.3390/su16051905
Eppich, W. J., Gormley, G. J., & Teunissen, P. W. (2019). In-depth interviews. In Healthcare simulation research: A practical guide (pp. 85-91). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-26837-4_12 DOI: https://doi.org/10.1007/978-3-030-26837-4_12
Espinoza Pérez, L., Mauricio, C., Araya, C., Contreras, R., Suazo, F., Velastín, N., ... & Vásquez, Ó. C. (2024, October). Advancing Food Sustainability: A Prototype Platform for Informed Decision-Making. In International Conference on Advanced Research in Technologies, Information, Innovation and Sustainability (pp. 250-263). Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-83210-9_19 DOI: https://doi.org/10.1007/978-3-031-83210-9_19
Faisal, M., & AlQouz, E. H. (2023, July). Utilizing Crowdsourced Heuristic Evaluation in the Assessment of User Experience for Online Tools. In International Conference on Human-Computer Interaction (pp. 132-137). Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-49212-9_18 DOI: https://doi.org/10.1007/978-3-031-49212-9_18
Fathiyyah, D., Diani, M. D. S., & Saputri, Z. A. (2022, November). Usability Evaluation on Life Insurance Application Using System Usability Scale and ISO 9241-11. In 2022 8th International HCI and UX Conference in Indonesia (CHIuXiD) (Vol. 1, pp. 94-99). IEEE. https://doi.org/10.1109/CHIuXiD57244.2022.10009774 DOI: https://doi.org/10.1109/CHIuXiD57244.2022.10009774
Felipe, D. A., Kalinowski, M., Graziotin, D., & Natividade, J. C. (2023). Psychometric instruments in software engineering research on personality: Status quo after fifty years. Journal of Systems and Software, 203, 111740. https://doi.org/10.1016/j.jss.2023.111740 DOI: https://doi.org/10.1016/j.jss.2023.111740
Finstad, K. (2010). The usability metric for user experience. Interacting with Computers, 22(5), 323-327. https://doi.org/10.1016/j.intcom.2010.04.004 DOI: https://doi.org/10.1016/j.intcom.2010.04.004
Forsey, H., Leahy, D., Fields, B., Minocha, S., Attfield, S., & Snell, T. (2025). Designing for learnability: Improvement through layered interfaces. Ergonomics in Design, 33(3), 135-141. https://doi.org/10.1177/10648046241273291 DOI: https://doi.org/10.1177/10648046241273291
Gao, M., Kortum, P., & Oswald, F. L. (2020). Multi-language toolkit for the system usability scale. International Journal of Human-Computer Interaction, 36(20), 1883-1901. https://doi.org/10.1080/10447318.2020.1801173 DOI: https://doi.org/10.1080/10447318.2020.1801173
Golondrino, G. E. C., Alarcón, M. A. O., & Martínez, L. M. S. (2023). Determination of the satisfaction attribute in usability tests using sentiment analysis and fuzzy logic. International Journal of Computers Communications & Control, 18(3). https://doi.org/10.15837/ijccc.2023.3.4901 DOI: https://doi.org/10.15837/ijccc.2023.3.4901
Goodell, L. S., Stage, V. C., & Cooke, N. K. (2016). Practical qualitative research strategies: Training interviewers and coders. Journal of Nutrition Education and Behavior, 48(8), 578-585. https://doi.org/10.1016/j.jneb.2016.06.001 DOI: https://doi.org/10.1016/j.jneb.2016.06.001
Goundar, M. S., Kumar, B. A., & Ali, A. S. (2024). Development of usability guidelines: a systematic literature review. International Journal of Human-Computer Interaction, 40(5), 1298-1316. https://doi.org/10.1080/10447318.2022.2141009 DOI: https://doi.org/10.1080/10447318.2022.2141009
Graziotin, D., Fagerholm, F., Wang, X., & Abrahamsson, P. (2017, May). Consequences of unhappiness while developing software. In 2017 IEEE/ACM 2nd International Workshop on Emotion Awareness in Software Engineering (SEmotion) (pp. 42-47). IEEE. https://doi.org/10.1109/SEmotion.2017.5 DOI: https://doi.org/10.1109/SEmotion.2017.5
Greene, K. K., Kelsey, J. M., & Franklin, J. M. (2016). Measuring the usability and security of permuted passwords on mobile platforms. US Department of Commerce, National Institute of Standards and Technology. https://doi.org/10.6028/NIST.IR.8040 DOI: https://doi.org/10.6028/NIST.IR.8040
Hair Jr, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis. In Multivariate data analysis (pp. 785-785).
Hamm, J., Money, A., & Atwal, A. (2017). Fall prevention self-assessments via mobile 3D visualization technologies: community dwelling older adults' perceptions of opportunities and challenges. JMIR Human Factors, 4(2), e7161. https://doi.org/10.2196/humanfactors.7161 DOI: https://doi.org/10.2196/humanfactors.7161
Harangozo, G., & Szigeti, C. (2017). Corporate carbon footprint analysis in practice-With a special focus on validity and reliability issues. Journal of Cleaner Production, 167, 1177-1183. https://doi.org/10.1016/j.jclepro.2017.07.237 DOI: https://doi.org/10.1016/j.jclepro.2017.07.237
Hennink, M., & Kaiser, B. N. (2022). Sample sizes for saturation in qualitative research: A systematic review of empirical tests. Social Science & Medicine, 292, 114523. https://doi.org/10.1016/j.socscimed.2021.114523 DOI: https://doi.org/10.1016/j.socscimed.2021.114523
Hertzum, M. (2026). System Usability Scale: A Meta-Analysis of How SUS Relates to Workload, Task Time, and Error Rate. International Journal of Human-Computer Interaction, 1-14. https://doi.org/10.1080/10447318.2026.2625260 DOI: https://doi.org/10.1080/10447318.2026.2625260
Hodrien, A., & Fernando, T. (2021). A Review of Post-Study and Post-Task Subjective Questionnaires to Guide Assessment of System Usability. Journal of Usability Studies, 16(3).
Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2), 65-70.
ISO, I. (2018). Ergonomics of human-system interaction-Part 11: Usability: Definitions and concepts (ISO 9241-11: 2018).
ISO/IEC 25010. (2024). Systems and Software Engineering: Systems and Software Quality Requirements and Evaluation (SQuaRE): Measurement of System and Software Product Quality.
Johnson, S. G., Potrebny, T., Larun, L., Ciliska, D., & Olsen, N. R. (2022). Usability methods and attributes reported in usability studies of mobile apps for health care education: scoping review. JMIR Medical Education, 8(2), e38259. https://doi.org/10.2196/38259 DOI: https://doi.org/10.2196/38259
Kallio, H., Pietilä, A. M., Johnson, M., & Kangasniemi, M. (2016). Systematic methodological review: developing a framework for a qualitative semi‐structured interview guide. Journal of Advanced Nursing, 72(12), 2954-2965. https://doi.org/10.1111/jan.13031 DOI: https://doi.org/10.1111/jan.13031
Karapanos, E., Zimmerman, J., Forlizzi, J., & Martens, J. B. (2009, April). User experience over time: an initial framework. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 729-738). https://doi.org/10.1145/1518701.1518814 DOI: https://doi.org/10.1145/1518701.1518814
Kennelly, C., Berners-Lee, M., & Hewitt, C. N. (2019). Hybrid life-cycle assessment for robust, best-practice carbon accounting. Journal of Cleaner Production, 208, 35-43. https://doi.org/10.1016/j.jclepro.2018.09.231 DOI: https://doi.org/10.1016/j.jclepro.2018.09.231
Klug, B. (2017). An overview of the system usability scale in library website and system usability testing. Weave: Journal of Library User Experience, 1(6). https://doi.org/10.3998/weave.12535642.0001.602 DOI: https://doi.org/10.3998/weave.12535642.0001.602
Kong, M., Lee, M., Kang, H., & Hong, T. (2021). Development of a framework for evaluating the contents and usability of the building life cycle assessment tool. Renewable and Sustainable Energy Reviews, 150, 111475. https://doi.org/10.1016/j.rser.2021.111475 DOI: https://doi.org/10.1016/j.rser.2021.111475
Kortum, P. T., & Bangor, A. (2013). Usability ratings for everyday products measured with the system usability scale. International Journal of Human-Computer Interaction, 29(2), 67-76. https://doi.org/10.1080/10447318.2012.681221 DOI: https://doi.org/10.1080/10447318.2012.681221
Kortum, P., & Peres, S. C. (2014). The relationship between system effectiveness and subjective usability scores using the System Usability Scale. International Journal of Human-Computer Interaction, 30(7), 575-584. https://doi.org/10.1080/10447318.2014.904177 DOI: https://doi.org/10.1080/10447318.2014.904177
Kortum, P., Acemyan, C. Z., & Oswald, F. L. (2021). Is it time to go positive? Assessing the positively worded system usability scale (SUS). Human Factors, 63(6), 987-998. https://doi.org/10.1177/0018720819881556 DOI: https://doi.org/10.1177/0018720819881556
Kujala, S., Miron-Shatz, T., & Jokinen, J. J. (2019). The cross-sequential approach: a short-term method for studying long-term user experience. Journal of Usability Studies, 14(2), 105-116.
Lee, S., & Sah, Y. J. (2020). Development of an approach to measuring learnability based on Ngoms from perspectives of extended learnability. International Journal of Human-Computer Interaction, 36(2), 199-209. https://doi.org/10.1080/10447318.2019.1625569 DOI: https://doi.org/10.1080/10447318.2019.1625569
Lewis, J. R. (1995). IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. International Journal of Human‐Computer Interaction, 7(1), 57-78. https://doi.org/10.1080/10447319509526110 DOI: https://doi.org/10.1080/10447319509526110
Lewis, J. R. (2014). Usability: lessons learned… and yet to be learned. International Journal of Human-Computer Interaction, 30(9), 663-684. https://doi.org/10.1080/10447318.2014.930311 DOI: https://doi.org/10.1080/10447318.2014.930311
Lewis, J. R. (2018). The System Usability Scale: Past, present, and future. International Journal of Human-Computer Interaction, 34(7), 577-590. https://doi.org/10.1080/10447318.2018.1455307 DOI: https://doi.org/10.1080/10447318.2018.1455307
Lewis, J. R., & Sauro, J. (2009, July). The factor structure of the system usability scale. In International conference on human centered design (pp. 94-103). Berlin, Heidelberg: Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-02806-9_12 DOI: https://doi.org/10.1007/978-3-642-02806-9_12
Lewis, J. R., & Sauro, J. (2017). Can I Leave This One Out? The Effect of Dropping an Item From the SUS. Journal of Usability Studies, 13(1).
Lewis, J. R., & Sauro, J. (2021). Usability and user experience: Design and evaluation. Handbook of human factors and ergonomics, 972-1015. https://doi.org/10.1002/9781119636113.ch38 DOI: https://doi.org/10.1002/9781119636113.ch38
Lewis, J. R., Utesch, B. S., & Maher, D. E. (2013, April). UMUX-LITE: when there's no time for the SUS. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 2099-2102). https://doi.org/10.1145/2470654.2481287 DOI: https://doi.org/10.1145/2470654.2481287
Lewis, J. R., Utesch, B. S., & Maher, D. E. (2015). Measuring perceived usability: The SUS, UMUX-LITE, and AltUsability. International Journal of Human-Computer Interaction, 31(8), 496-505. https://doi.org/10.1080/10447318.2015.1064654 DOI: https://doi.org/10.1080/10447318.2015.1064654
Liang, J., Xian, D., Liu, X., Fu, J., Zhang, X., Tang, B., & Lei, J. (2018). Usability study of mainstream wearable fitness devices: feature analysis and system usability scale evaluation. JMIR mHealth and uHealth, 6(11), e11066. https://doi.org/10.2196/11066 DOI: https://doi.org/10.2196/11066
Longhurst, R., & Johnston, L. (2023). 10 semi‐structured interviews and focus groups. Key methods in geography, 168. https://doi.org/10.4135/9781036233679.n10 DOI: https://doi.org/10.4135/9781036233679.n10
Lu, Q., Tao, L., Peng, X., & Chen, J. (2024). Signs of front‐line healthcare professionals' information anxiety during the COVID‐19 pandemic: Grounded theory study in a Wuhan hospital. Nursing Open, 11(6), e2207. https://doi.org/10.1002/nop2.2207 DOI: https://doi.org/10.1002/nop2.2207
Malakahmad, A., Albakri, N. A. B. H., & Shafiq, N. (2015, May). Assessment and Reduction of Carbon Footprint: An Approach via Best Management Practices in a Construction Site. In InCIEC 2014: Proceedings of the International Civil and Infrastructure Engineering Conference 2014 (pp. 1019-1030). Singapore: Springer Singapore. https://doi.org/10.1007/978-981-287-290-6_89 DOI: https://doi.org/10.1007/978-981-287-290-6_89
Manouchehri, E., Taghipour, A., Ebadi, A., Homaei Shandiz, F., & Latifnejad Roudsari, R. (2022). How do I deal with breast cancer: a qualitative inquiry into the coping strategies of Iranian women survivors. BMC Women's Health, 22(1), 284. https://doi.org/10.1186/s12905-022-01865-0 DOI: https://doi.org/10.1186/s12905-022-01865-0
Maqbool, B., & Herold, S. (2024). Potential effectiveness and efficiency issues in usability evaluation within digital health: A systematic literature review. Journal of Systems and Software, 208, 111881. https://doi.org/10.1016/j.jss.2023.111881 DOI: https://doi.org/10.1016/j.jss.2023.111881
Marinelli, E., Radini, S., Foglia, A., Lancioni, N., Piasentin, A., Eusebi, A. L., & Fatone, F. (2021). Validation of an evidence-based methodology to support regional carbon footprint assessment and decarbonisation of wastewater treatment service in Italy. Water Research, 207, 117831. https://doi.org/10.1016/j.watres.2021.117831 DOI: https://doi.org/10.1016/j.watres.2021.117831
Marlowe, J., & Clarke, A. (2022). Carbon Accounting: A Systematic Literature Review and Directions for Future Research. Green Finance, 4(1), 71. https://doi.org/10.3934/GF.2022004 DOI: https://doi.org/10.3934/GF.2022004
Mashuri, S., Sarib, M., Rasak, A., Alhabsyi, F., & Syam, H. (2022). Semi-structured interview: A methodological reflection on the development of a qualitative research instrument in educational studies. Journal of Research and Method in Education, 12(1), 22-29.
Maxwell, J. A. (2013). Qualitative research design: An interactive approach: An interactive approach. Sage.
McGlinn, K., Yuce, B., Wicaksono, H., Howell, S., & Rezgui, Y. (2017). Usability evaluation of a web-based tool for supporting holistic building energy management. Automation in Construction, 84, 154-165. https://doi.org/10.1016/j.autcon.2017.08.033 DOI: https://doi.org/10.1016/j.autcon.2017.08.033
Melville, N. P., & Whisnant, R. (2014). Energy and carbon management systems: organizational implementation and application. Journal of Industrial Ecology, 18(6), 920-930. https://doi.org/10.1111/jiec.12135 DOI: https://doi.org/10.1111/jiec.12135
Mol, M., van Schaik, A., Dozeman, E., Ruwaard, J., Vis, C., Ebert, D. D., ... & Smit, J. H. (2020). Dimensionality of the system usability scale among professionals using internet-based interventions for depression: a confirmatory factor analysis. BMC Psychiatry, 20(1), 218. https://doi.org/10.1186/s12888-020-02627-8 DOI: https://doi.org/10.1186/s12888-020-02627-8
Mulrow, J., Machaj, K., Deanes, J., & Derrible, S. (2018). The state of carbon footprint calculators: An evaluation of calculator design and user interaction features. Sustainable Production and Consumption, 18, 33. https://doi.org/10.1016/j.spc.2018.12.001 DOI: https://doi.org/10.1016/j.spc.2018.12.001
Mustaffa, N. K., Mat Isa, C. M., Ekundayo, D., & Joseph, V. R. A. (2022). Barriers and strategies for improving carbon emissions management approaches in Malaysian construction. Construction Economics and Building, 22(3), 99-123. https://doi.org/10.5130/AJCEB.v22i3.7909 DOI: https://doi.org/10.5130/AJCEB.v22i3.7909
Naz, N., Gulab, F., & Aslam, M. (2022). Development of qualitative semi-structured interview guide for case study research. Competitive Social Science Research Journal. 3, no. 2 (2022): 42-52.
Newcomer, K. E., Hatry, H. P., & Wholey, J. S. (Eds.). (2015). Handbook of practical program evaluation. https://doi.org/10.1002/9781119171386 DOI: https://doi.org/10.1002/9781119171386
Nielsen, J. (1994). Usability engineering. Morgan Kaufmann. https://doi.org/10.1016/B978-0-08-052029-2.50009-7 DOI: https://doi.org/10.1016/B978-0-08-052029-2.50009-7
Nielsen, J. B. T.-U. E. R.-M. (2010). CHAPTER 1 - What Is Usability? (pp. 3–22). Morgan Kaufmann. https://doi.org/10.1016/B978-0-12-375114-0.00004-9 DOI: https://doi.org/10.1016/B978-0-12-375114-0.00004-9
Nielsen, J. (2012). How many test users in a usability study? Nielsen Norman Group. https://doi.org/10.1042/CS20120633 DOI: https://doi.org/10.1042/CS20120633
Okon, D. U., & Marauche, O. (2025). The Drivers and Barriers of Corporate Carbon Accounting Adoption: A Systematic Review of Managerial, Organizational, and Policy Factors. International Journal of Advances in Engineering and Management, 7(7), 717. https://doi.org/10.35629/5252-0707717727 DOI: https://doi.org/10.35629/5252-0707717727
Onat, N. C., Mandouri, J., Kucukvar, M., Kutty, A. A., & Al‐Muftah, A. A. (2025). Driving sustainable business practices with carbon accounting and reporting: a holistic framework and empirical analysis. Corporate Social Responsibility and Environmental Management, 32(2), 2795-2814. https://doi.org/10.1002/csr.3096 DOI: https://doi.org/10.1002/csr.3096
Orfanou, K., Tselios, N., & Katsanos, C. (2015). Perceived usability evaluation of learning management systems: Empirical evaluation of the System Usability Scale. The International Review of Research in Open and Distributed Learning, 16(2). https://doi.org/10.19173/irrodl.v16i2.1955 DOI: https://doi.org/10.19173/irrodl.v16i2.1955
Oshidero, D. F. O., & Coley, D. A. (2026). Talking carbon: a lexical approach to predictive embodied carbon analysis via machine learning. Architectural Engineering and Design Management, 1-34. https://doi.org/10.1080/17452007.2026.2613773 DOI: https://doi.org/10.1080/17452007.2026.2613773
Paskaran, S., Gamage, A., & Chandrasiri, S. (2022, November). A novel ranked emission-factor retrieval for emission calculation. In 2022 International Conference on Computer Engineering, Network, and Intelligent Multimedia (CENIM) (pp. 1-8). IEEE. https://doi.org/10.1109/CENIM56801.2022.10037450 DOI: https://doi.org/10.1109/CENIM56801.2022.10037450
Qiang, Y., & Pa, N. C. (2024). Sustainable software solutions: a tool integrating life cycle analysis and ISO quality models. International Journal on Advanced Science, Engineering & Information Technology, 14(5). https://doi.org/10.18517/ijaseit.14.5.11268 DOI: https://doi.org/10.18517/ijaseit.14.5.11268
Quaglio, B., Gallina, F., & Giordano, R. (2024, October). An Infrastructure's emissions assessment tool: AMICO-Account Method of Infrastructures embodied CarbOn. In IOP Conference Series: Earth and Environmental Science (Vol. 1402, No. 1, p. 012063). IOP Publishing. https://doi.org/10.1088/1755-1315/1402/1/012063 DOI: https://doi.org/10.1088/1755-1315/1402/1/012063
Rabbani, R. M., & Wibowo, A. (2025, December). Implementating Mobile-Based Vehicle Carbon Footprint and Transportation Emission Calculation System to Enhance Users Environmental Awareness. In 8th International Conference on Applied Engineering (ICAE 2025) (pp. 614-651). Atlantis Press. https://doi.org/10.2991/978-94-6463-982-7_37 DOI: https://doi.org/10.2991/978-94-6463-982-7_37
Rodriguez-Jimenez, L., Romero-Martín, M., & Gómez-Salgado, J. (2025). The emergency department carbon footprint calculator: Design and validation. Medicine, 104(22), e41652. https://doi.org/10.1097/MD.0000000000041652 DOI: https://doi.org/10.1097/MD.0000000000041652
Rogers, E. M. (2003). Diffusion of innovations (5th ed.). Free Press.
Rosales, M. V., & Dhakal, S. (2024, October). A Comparative Analysis of Carbon Accounting Methodologies and Tools. In 2024 International Conference on Sustainable Energy: Energy Transition and Net-Zero Climate Future (ICUE) (pp. 1-7). IEEE. https://doi.org/10.1109/ICUE63019.2024.10795543 DOI: https://doi.org/10.1109/ICUE63019.2024.10795543
Rotaru, O. A., Vert, S., Vasiu, R., & Andone, D. (2020, October). Standardised questionnaires in usability evaluation. applying standardised usability questionnaires in digital products evaluation. In International Conference on Information and Software Technologies (pp. 39-48). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-59506-7_4 DOI: https://doi.org/10.1007/978-3-030-59506-7_4
Sagar, K., & Saha, A. (2017). A systematic review of software usability studies. International Journal of Information Technology, 1-24. https://doi.org/10.1007/s41870-017-0048-1 DOI: https://doi.org/10.1007/s41870-017-0048-1
Sauro, J. (2011). SUStisfied? Little-known system usability scale facts. User Experience Magazine, 10(3).
Sauro, J. (2018). 5 Ways to Interpret a SUS Score - MeasuringU. Measuringu.Com. https://measuringu.com/interpret-sus-score/
Sauro, J., & Lewis, J. R. (2009, April). Correlations among prototypical usability metrics: Evidence for the construct of usability. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1609-1618). https://doi.org/10.1145/1518701.1518947 DOI: https://doi.org/10.1145/1518701.1518947
Sauro, J., & Lewis, J. R. (2011, May). When designing usability questionnaires, does it hurt to be positive?. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 2215-2224). https://doi.org/10.1145/1978942.1979266 DOI: https://doi.org/10.1145/1978942.1979266
Sharfina, Z., & Santoso, H. B. (2016, October). An Indonesian adaptation of the system usability scale (SUS). In 2016 International conference on advanced computer science and information systems (ICACSIS) (pp. 145-148). IEEE. https://doi.org/10.1109/ICACSIS.2016.7872776 DOI: https://doi.org/10.1109/ICACSIS.2016.7872776
Shea, S. C., & Barney, C. (2015). Teaching clinical interviewing skills using role-playing: conveying empathy to performing a suicide assessment: a primer for individual role-playing and scripted group role-playing. Psychiatric Clinics, 38(1), 147-183. https://doi.org/10.1016/j.psc.2014.10.001 DOI: https://doi.org/10.1016/j.psc.2014.10.001
Siedlecki, S. L. (2022). Conducting interviews for qualitative research studies. Clinical Nurse Specialist, 36(2), 78-80. https://doi.org/10.1097/NUR.0000000000000653 DOI: https://doi.org/10.1097/NUR.0000000000000653
Smulowitz, S. (2017). Interview guide. The International Encyclopedia of Communication Research Methods, 1-2. https://doi.org/10.1002/9781118901731.iecrm0124 DOI: https://doi.org/10.1002/9781118901731.iecrm0124
Spandonidis, C. (2026). Designing for Impact: Evidence-Based Guidelines to Improve Inclusivity, Transparency, and Engagement in Carbon Footprint Tools. Sustainable Development 34(3), 3720–3731. https://doi.org/10.1002/sd.70538 DOI: https://doi.org/10.1002/sd.70538
Sternberg, Z. (2023). Information Design Considerations for Effective Communication of Sustainability Metrics. Massachusetts Institute of Technology.
Strohmeier, S., Becker, M., & Scheer-Weller, E. (2025). Beyond aversion-principles of appropriate algorithmic decision-making in human resource management. Expert Systems with Applications, 128954. https://doi.org/10.1016/j.eswa.2025.128954 DOI: https://doi.org/10.1016/j.eswa.2025.128954
Sweller, J. (2011). Cognitive load theory. In Psychology of learning and motivation (Vol. 55, pp. 37-76). Academic Press. https://doi.org/10.1016/B978-0-12-387691-1.00002-8 DOI: https://doi.org/10.1016/B978-0-12-387691-1.00002-8
Terry, G., Hayfield, N., Clarke, V., & Braun, V. (2017). Thematic analysis. In C. Willig & W. Stainton Rogers (Eds.), The Sage handbook of qualitative research in psychology (2nd ed., pp. 17-37). Sage. https://doi.org/10.4135/9781526405555.n2 DOI: https://doi.org/10.4135/9781526405555.n2
Thelwall, M., & Nevill, T. (2021). Is research with qualitative data more prevalent and impactful now? Interviews, case studies, focus groups and ethnographies. Library & Information Science Research, 43(2), 101094. https://doi.org/10.1016/j.lisr.2021.101094 DOI: https://doi.org/10.1016/j.lisr.2021.101094
Tuomisto, H. L., De Camillis, C., Leip, A., Nisini, L., Pelletier, N., & Haastrup, P. (2015). Development and testing of a European Union‐wide farm‐level carbon calculator. Integrated environmental assessment and management, 11(3), 404-416. https://doi.org/10.1002/ieam.1629 DOI: https://doi.org/10.1002/ieam.1629
Vasileiou, K., Barnett, J., Thorpe, S., & Young, T. (2018). Characterising and justifying sample size sufficiency in interview-based studies: Systematic analysis of qualitative health research over a 15-year period. BMC Medical Research Methodology, 18, 148. https://doi.org/10.1186/s12874-018-0594-7 DOI: https://doi.org/10.1186/s12874-018-0594-7
Vlachogianni, P., & Tselios, N. (2022). Perceived usability evaluation of educational technology using the System Usability Scale (SUS): A systematic review. Journal of Research on Technology in Education, 54(3), 392-409. https://doi.org/10.1080/15391523.2020.1867938 DOI: https://doi.org/10.1080/15391523.2020.1867938
Wagner, A. M. (2023). Perceived Usability Evaluation of Hands-On and Virtual Science Laboratories: Using the System Usability Scale (SUS) to Determine Adult Learners? Preferred At-Home Laboratory Experience (Doctoral dissertation, Franklin University).
Wang, Y., Zhang, Y., & Chen, Y. (2023). A longitudinal study of children's perceived usability: Methodology and measurements. International Journal of Human-Computer Interaction, 39(18), 3604-3619. https://doi.org/10.1080/10447318.2022.2100065 DOI: https://doi.org/10.1080/10447318.2022.2100065
Weiner, B. J. (2009). A theory of organizational readiness for change. Implementation Science, 4(1), 67. https://doi.org/10.1186/1748-5908-4-67 DOI: https://doi.org/10.1186/1748-5908-4-67
Wetzlinger, W., Auinger, A., & Dörflinger, M. (2014, June). Comparing effectiveness, efficiency, ease of use, usability and user experience when using tablets and laptops. In International Conference of Design, User Experience, and Usability (pp. 402-412). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-07668-3_39 DOI: https://doi.org/10.1007/978-3-319-07668-3_39
Whiting, L. S. (2008). Semi-structured interviews: guidance for novice researchers. Nursing Standard (through 2013), 22(23), 35. https://doi.org/10.7748/ns2008.02.22.23.35.c6420 DOI: https://doi.org/10.7748/ns2008.02.22.23.35.c6420
Wutich, A., Beresford, M., & Bernard, H. R. (2024). Sample Sizes for 10 Types of Qualitative Data Analysis: An Integrative Review, Empirical Guidance, and Next Steps. International Journal of Qualitative Methods, 23, 16094069241296206. https://doi.org/10.1177/16094069241296206 DOI: https://doi.org/10.1177/16094069241296206
Xia, Z., Liu, K., Li, J., Gu, Y., Hu, B., Zhong, Y., ... & Lam, K. L. (2025). Evaluating carbon footprint calculators: a comprehensive assessment framework. Clean Technologies and Environmental Policy, 1-12. https://doi.org/10.1007/s10098-025-03169-x DOI: https://doi.org/10.1007/s10098-026-03492-x
Xu, J., & MacAskill, K. (2024). Carbon data and its requirements in infrastructure-related GHG standards. Environmental Science & Policy, 162, 103935. https://doi.org/10.1016/j.envsci.2024.103935 DOI: https://doi.org/10.1016/j.envsci.2024.103935
Yonata, R., & Purwarianti, A. (2017, November). Inveo, a management information system for emissions inventory e-administration in Indonesia. In 2017 2nd International conferences on Information Technology, Information Systems and Electrical Engineering (ICITISEE) (pp. 78-83). IEEE. https://doi.org/10.1109/ICITISEE.2017.8285564 DOI: https://doi.org/10.1109/ICITISEE.2017.8285564
Zapata, B. C., Fernández-Alemán, J. L., Idri, A., & Toval, A. (2015). Empirical studies on usability of mHealth apps: a systematic literature review. Journal of medical systems, 39(2), 1. https://doi.org/10.1007/s10916-014-0182-2 DOI: https://doi.org/10.1007/s10916-014-0182-2