What makes ML models tick? How do we attribute model behavior to the training data, algorithm, architecture, or scale used in training?...
Recently-developed algorithmic innovations and large-scale datasets have given rise to machine learning models with impressive capabilities. However, there is much left to understand in how these different factors combine to give rise to observed behaviors. For example, we still do not fully understand how the composition of training datasets influence downstream model capabilities, how to attribute model capabilities to subcomponents inside the model, and which algorithmic choices really drive performance...
Data attribution and selection: How can we (efficiently) attribute model outputs back to specific training examples? How can we select data to optimize downstream performance/capabilities?
Also known as: LLMs