What does LoRA fine-tuning actually train?
"LoRA trains about 1% of the model" is the usual one-line explanation. It's true, but it hides the parts that decide whether a run fits on your machine and how long it takes: which layers get adapters...
"LoRA trains about 1% of the model" is the usual one-line explanation. It's true, but it hides the parts that decide whether a run fits on your machine and how long it takes: which layers get adapters...
Discussion
Log in to join the discussion