Federal Ministry of Education and Research funds two AI junior research groups at the Paderborn University.
Artificial intelligence (AI) is considered one of the key technologies of the future. Whether voice assistants, smart homes or models for predicting pandemics: AI supports us in countless areas and it is impossible to imagine our everyday life without it. In order to comprehensively understand the rapid technical developments, to further develop the technology in a targeted manner and to bring it into application, research in the field is more important than ever. Since September, two new AI junior research groups at the Paderborn University have been investigating how machine learning can be optimized and used more effectively than before. The goal of the young scientists is to improve the quality of models of dynamic systems by combining machine learning and expert knowledge, and to make the training of deep neural networks more robust, efficient and interactive by using so-called multi-objective optimization methods. The German Federal Ministry of Education and Research (BMBF) is funding both groups for three years with a total of about 1.8 million euros.
The two teams plan to work closely together to bundle their competencies in the AI field in the best possible way. Paderborn offers the optimal infrastructure for this: Through the Paderborn Centre for Parallel Computing (PC2), the scientists have the opportunity to use the latest HPC hardware (High Performance Computing) for functional and demonstration tests. The methods, software tools and data of the junior research groups will eventually be made available to a broad user community free of charge as open- access-material.
Combining the benefits of expert and data knowledge for model building.
From temperature estimation in electric motors to predicting COVID-19 dispersion dynamics or the unemployment rate, a wide variety of dynamical systems in engineering, economics, and social sciences, as well as physics, biology, chemistry, and medicine, can be described by mathematical models. However, modelling these complex dynamical systems is often challenging. "In recent years, there has been a clear trend away from expert models toward black-box models developed using machine learning. Both have many advantages, but also disadvantages," says Dr Eng. Oliver Wallscheid from the Paderborn Institute of Electrical Engineering and Information Technology and head of one of the new AI junior research groups. The goal of Wallscheid's team is to combine the advantages of expert-driven and data-driven modelling in order to significantly improve model quality in terms of accuracy, robustness and complexity for various applications. In the project "Automated model building and validation of dynamic systems using machine learning as well as a priori expert knowledge" (ML-Expert), they are researching how the basically different modelling paradigms - developed by experts and by AI, respectively - can be united in a hybrid model building.
Wallscheid explains the problem so far: "While expert-based approaches reproduce system behaviour in a robust and interpretable way, they often require a lot of time and human resources and exhibit systematic deviations due to simplifications. In contrast, black-box models, i.e., data-driven models, can sometimes be generated quickly and without significant prior knowledge. While this allows accurate and scalable models to be generated, they lack interpretability and robustness to outliers, among other things."
Efficient and resource-oriented data generation, model building and validation should lead to faster development and application cycles in the future. "Particularly for cost-relevant industrial applications, e.g. in the automotive, energy or automation industries, this represents significant added value, also against the background of a worsening shortage of skilled workers. However, our work will not be limited to these industries, but will have a cross-domain focus," Wallscheid emphasizes.
Optimal compromise solutions for machine learning processes
Machine learning is also used in numerous other applications. So-called deep neural networks (DNNs), for example, enable intelligent image recognition or speech processing. The increase in available computing capacity allows the construction of ever larger, deeper and more complex DNNs. As these advances increase, so do the challenges: Optimally, DNN design and training should simultaneously satisfy numerous, sometimes conflicting, goals in the best possible way. "In contrast to many areas of technology and society in which the consideration of multiple criteria is taken for granted, e.g., in cancer treatment, the enormous potential of a multi-objective approach in machine learning has so far been largely untapped," says Jun.-Prof. Dr. Sebastian Peitz of the Department of Computer Science. The head of the new AI junior research group wants to change that with the project "Multicriteria Machine Learning - Efficiency, Robustness, Interactivity and System Knowledge" (MultiML). The goal is to make the training of DNNs more robust, efficient and interactive by developing multi-objective optimization methods, and thus to improve it decisively. Furthermore, the additional incorporation of system knowledge should enable the construction of extremely efficient methods tailored to specific problem classes.
Peitz cites simple examples: "In almost all areas of technology, business and society, the dilemma arises that competing criteria are of similar importance: Electric vehicles should drive fast and have a long range at the same time, a product to be manufactured should have high quality and low production costs, and political decisions should take both economic and ecological aspects into account." The challenge is always to identify and select optimal trade-off solutions, so-called Pareto optima. "Even in machine learning, there are numerous criteria that should be met simultaneously in the best possible way, such as robustness to incomplete input data, generalization beyond the training data, or the best possible compliance with physical laws," the computer scientist continues.
The development of multi-objective optimization methods for machine learning should enable algorithms that allow the set of optimal trade-offs to be determined in a highly efficient manner. "Among other things, knowing all the trade-off solutions enables users to make much more informed and deliberate trade-offs and to adapt learning procedures according to the situation by re-prioritizing individual goals," Peitz explains.
