On August 31 at 11:00 a.m. in Room 320, Gideon Seidel will defend his master’s thesis on the topic “Multidimensional Comparison of Large Language Models Taking Company-Specific Objectives into Account.” The examiners are Prof. Kurt Sandkuhl and Eric Müller from the Chair of Business Informatics.
Abstract
Large language models are increasingly being used in business contexts. However, the growing number of available models and their differences in terms of technical, economic, and strategic characteristics make it difficult to select a model suitable for a specific use case. The goal of this thesis is therefore to develop a multidimensional evaluation framework that takes company-specific objectives into account and provides structured decision support for the selection and long-term evaluation of large language models.
To this end, relevant evaluation methods and concepts are examined based on a systematic literature review. Building on this, an evaluation concept is developed that combines the Balanced Scorecard with the Fuzzy Delphi method and the Fuzzy Analytic Hierarchy Process. While the Balanced Scorecard serves to structure and link corporate objectives in a multidimensional way, the Fuzzy Delphi method and the Fuzzy Analytic Hierarchy Process support the selection and weighting of relevant evaluation criteria. The concept is applied in the context of a case study, taking into account the requirements of a partner organization.