Google DeepMind introduced SkillSmith, a system allowing AI models to learn new skills on the fly without requiring retraining. This innovative approach combines stored textual knowledge with parametric model weights through prefix-tuning. All seven authors of the paper, all affiliated with Google, demonstrated that this method significantly outperforms baseline models using either text alone or weight-space alone. SkillSmith enables prefix weights to be treated as composable building blocks that can be mixed and matched based on instructions.
Source: Read the original article
