Nn sequential pytorch11/7/2023 ![]() ![]() ➡️ Easy task-switching in deployment - all we need to change is a handful of weights as compared to the full model. ➡️ No additional inference latency - (unlike adaptors) just add the learned matrix to the pre-trained one. Checkpoint size is greatly reduced with reduction in trainable parameters. ➡️ Storage efficiency - No need to store huge checkpoints for different downstream tasks. Saving is more when using stateful optimizers like Adam, Adadelta etc. ➡️ Time and memory efficiency - With a large percentage of the parameters being frozen, the training time and the GPU memory is saved. This yields significant benefits as compared to full-fine tuning. In one line - All of the pre-trained weights W are kept frozen and the rank decomposition matrices of the “change in weight matrix”, B and A are optimized. THAT’S IT! All we’re now left to optimize is the new matrices B and A that contain a very smaller number of parameters (combined) than the full matrix due to their dimensions. ✅ LoRA hypothesizes that “change in weights” during adaptation has a “low intrinsic rank” -> ΔW is non full rank and so can be written as ΔW = BA (see the figure).ĭuring training, the outputs from W and ΔW are added component wise, like so. Let’s talk about LoRA (low rank adaptation of LLMs), one such PEFT (parameter efficient fine-tuning) technique that relies on a simple concept - decomposition of non-full rank matrices. reduce the model’s usable sequence length. Adaptors - introduce inference latency that becomes significant in online low batch size inference settings. Heavily Parameterized Large Language Models + Basic Linear Algebra Theorem = Save GPU memory! □įine-tuning of large pre-trained LMs on downstream tasks yields impressive performances - there are many techniques to go about this concept called “transfer learning”. High GPU memory costs? Fine-tuning an LLM? Read on! The original paper experimentally investigates that the performance remains fairly stable across varying adaptor sizes m and hence for a given model a fixed size can be used for all downstream tasks. ![]() of optimizable parameters and hence poses a parameter vs performance tradeoff. The size m in the adaptor module determines the no. The adapter is applied directly to the outputs of each of the sub-layers (attention and feedforward) and its output passed into the following layernorm the parameters of which are also optimizable. This is required for stable fine-tuning and is intuitive as with it, we essentially do not disturb the learning from pre-training. ➡️As can be seen, the module features a skip-connection - With it in place, when the parameters of the projection layers are initialized to near-zero which eventually leads to near identity initialization of the module. ➡️The adapter module (right) first projects the original d-dimensional features into a smaller m-dimensional vector, applies a nonlinearity, and then projects it back to d dimensions. We’ll understand by looking at its application in the transformer architecture in 3 points. Keeping the full PT model frozen, these modules are the only optimizable ones while fine-tuning - This means only a very few parameters are introduced per task yielding “compact” models. al) simply inserts new modules called “Adaptor Modules” between the layers of the pre-trained network. ✅ The simplest way out for efficient fine-tuning could be to freeze the networks’ lower layers and adapt only the top ones to specific tasks.Ĭan we do it more efficiently? In a way that trains lesser parameters and hence saves VRAM while also yielding similarly good performance?Īnother PEFT (Parameter Efficient Fine-tuning) technique called “Adaptors” is shown to achieve similar performance as compared to tuning the top layers while requiring as fewer parameters as two orders of magnitude. Full fine-tuning pre-trained LLMs on downstream tasks is a common, effective but an inefficient approach to transfer learning. ![]()
0 Comments
Leave a Reply.AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |