PhD Position (100%, E13, m/f/d) | Automating Large Language Models

PhD Position (100%, E13, m/f/d) | Automating Large Language Models

Universität Leipzig

Universität Leipzig, 04105 Leipzig

Stellenangebot, Vollzeit

01.09.2024

About research group

A recurring criticism of machine learning (ML) is that, in practice, training a model is usually more of an art than a science. It often relies on a process of trial and error that consumes substantial amounts of human and computational time. Automated machine learning (AutoML) is a young branch of machine learning that aims to streamline the ML workflow by progressively automating parts that typically rely on human expert intuition. AutoML has been quite successful; for example, it has outperformed human experts in Kaggle competitions and configured the Monte-Carlo Tree Search of AlphaGo. The Automated Machine Learning group at ScaDS.AI explores approaches for machine learning systems that can configure themselves automatically by learning from past data. We focus on hyperparameter optimization to automatically configure machine learning models and neural architecture search to design better and more efficient neural network architectures.

About the project

Large language models mark the beginning of a new era in artificial intelligence. These models have achieved several breakthroughs in text understanding, code generation, and machine translation. Conceptually quite simple, most of their success can be attributed to the continuous scaling of compute, datasets, and model size. Unfortunately, the computational demands of LLMs make it challenging to deploy and train them in practice. The goal of this project is to develop new AutoML approaches that enhance the efficiency and effectiveness of LLMs to democratize and increase their widespread adoption. More specifically, the project aims to address the following research questions:
- How can we compress LLMs for faster inference?
- How can we accelerate the training process for faster convergence?
- How can we overcome current shortcomings of the transformer architecture?
- Can we exploit scaling laws for LLMs to enable large scale hyperparameter and neural architecture search?

About ScaDS.AI

ScaDS.AI (Center for Scalable Data Analytics and Artificial Intelligence) Dresden/Leipzig is one of the permanently established national center for artificial intelligence at the University of Leipzig and the TU Dresden, which is financed by the federal government and the Free State of Saxony. The Leipzig sub-center will be established as a central facility of Leipzig University and in the medium term will bring together more than 200 staff members at the Leipzig site alone. In ScaDS.AI, various research topics will be worked on within the framework of a graduate school on the fundamentals and applications of data science and artificial intelligence. In addition, service-oriented solutions are developed and there is close cooperation with a large number of partner organizations from science and industry. The center offers an excellent working environment with access to state-of-the-art technologies and an outstanding high-performance computing infrastructure.

Requirements

• An excellent MSc degree (or about to finish) in Artificial Intelligence, Machine Learning, Deep Learning, Computer Science or a related discipline
• Solid knowledge of (and experience with) machine learning and deep learning methods
• Python knowledge with good working knowledge in applying/evaluating machine learning & deep learning methods
• Knowledge in one or more of the following is beneficial: Hyperparameter Optimization | Neural Architecture Search | Large Language Models |

To apply, please upload the following documents (as one PDF) to the plattform

• Preferred starting date (including earliest and latest possible date)
• CV & Transcript of records
• Research Statement (max one page) including why you want to start a PhD and why you want to do research in AutoML

Deadline for applications: none

For further questions, please also do reach out to

Dr. Aaron Klein

Dr. Eric Peukert

hr-scads@uni-leipzig.de.