Small-Scale Personalization of Language Models
How do LoRA adapters affect internal representation geometry of embeddings?
Status
Publication
NeurIPS Workshop on Parameter-Efficient Learning
Date
March 10, 2024
Authors
1

Abstract
This research investigates the impact of parameter-efficient fine-tuning (LoRA adapters) on language model behavior and internal representations. We analyze whether adopting a conversational persona affects core reasoning capabilities and explore the geometry of embedding space changes during adaptation. Results show minor (4%) degradation on general reasoning while successfully adopting target personas.
- 1.Fine-tuned a base LLM with a small dataset of stylized, first-person text using LoRA
- 2.Evaluated the adapted model on both in-domain (persona) and out-of-domain (general knowledge) question sets
- 3.Analyzed embedding vectors for key concepts before and after fine-tuning
- ✓The model successfully adopted the target persona in conversational tasks
- ✓A minor degradation (~4%) in performance was observed on general reasoning benchmarks
- ✓Embedding space showed a slight but measurable shift for persona-related concepts
- →Investigate methods to 'freeze' or insulate core knowledge layers during adaptation
- →Test if this technique can be used for controlled, reversible 'mode switching' in models
Keywords
References
Hu et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685.
Zhang et al. (2023). Scaling Instruction-Finetuned Language Models. arXiv:2210.11416.
Devlin et al. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL.