Research Entry

Small-Scale Personalization of Language Models

How do LoRA adapters affect internal representation geometry of embeddings?

Status

in revision

Publication

NeurIPS Workshop on Parameter-Efficient Learning

Date

March 10, 2024

Authors

1

Small-Scale Personalization of Language Models

Abstract

This research investigates the impact of parameter-efficient fine-tuning (LoRA adapters) on language model behavior and internal representations. We analyze whether adopting a conversational persona affects core reasoning capabilities and explore the geometry of embedding space changes during adaptation. Results show minor (4%) degradation on general reasoning while successfully adopting target personas.

Motivation
To understand if models can adopt a 'persona' without corrupting their core reasoning capabilities.
Experiments
  • 1.Fine-tuned a base LLM with a small dataset of stylized, first-person text using LoRA
  • 2.Evaluated the adapted model on both in-domain (persona) and out-of-domain (general knowledge) question sets
  • 3.Analyzed embedding vectors for key concepts before and after fine-tuning
Findings
  • ✓The model successfully adopted the target persona in conversational tasks
  • ✓A minor degradation (~4%) in performance was observed on general reasoning benchmarks
  • ✓Embedding space showed a slight but measurable shift for persona-related concepts
Future Inquiries
  • →Investigate methods to 'freeze' or insulate core knowledge layers during adaptation
  • →Test if this technique can be used for controlled, reversible 'mode switching' in models

Keywords

large language models
fine-tuning
LoRA
representation learning
persona adaptation
transfer learning

References

Hu et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685.

Zhang et al. (2023). Scaling Instruction-Finetuned Language Models. arXiv:2210.11416.

Devlin et al. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL.