FinLoRA

Introduction

• Overview
  • Motivation
  • Performance
• Demo

Tutorials

• Setup
  • GPU Requirements
  • RunPod Setup (Optional)
  • Package Installation
    • Using setup.sh
    • Using conda
  • Login to Hugging Face
• Data Processing
  • Dataset Overview
  • Dataset Categories
  • Dataset Directories
  • Dataset Formats
  • Data Processing Scripts
    • process_xbrl_extract.py
    • process_finer.py
    • process_fnxl.py
    • process_sentiment_train.py
    • process_xbrl_agent_data.py
    • process_multitask_data.py
    • Additional Processing Scripts
  • Dataset Statistics
  • Working with Custom Datasets
  • Related Documentation
• Fine-Tuning
  • Fine-Tuning Process
    • Step-by-Step Fine-Tuning
    • Configuration File Structure
  • Using Your LoRA Adapter
  • LoRA Adapters Directory Structure
  • Available Datasets and LoRA Methods
• Federated Learning with LoRA (FedLoRA)
  • Setup and Installation
    • Navigate to FedLoRA Directory
    • Install Dependencies
  • Running FedLoRA
    • Basic FedLoRA Simulation
    • Custom Configuration Options
    • Configuration Parameters
  • Related Documentation
• Evaluation
  • Evaluation Process
    • Using run_test.sh
    • Using test.py Directly
    • Using run_all_adapters.sh
    • Using run_openai.sh
  • Evaluation Results
  • Available Datasets and LoRA Adapters for Evaluation
• Buffett Agent
  • Motivation
  • Building the Buffett Brain
    • Data Sources and Questions
    • Data Processing Steps
  • Fine-tuning with FinLoRA
    • Environment Setup
    • Fine-Tuning Pipeline
    • Using the LoRA Adapter for Inference
  • Head‑to‑Head with Other Models
  • Putting Buffett Agent to the Test
  • Buffett Agent Links
  • Related Documentation

Tasks

• Overview
  • Task Categories Summary
  • Why These Financial Tasks Matter
  • Why Financial AI Tasks Are Critical
  • Related Documentation
• General Financial Tasks
  • Financial Phrase Bank (FPB) (Sentiment Analysis) [fpb]
  • FiQA SA (Sentiment Analysis) [fiqa]
  • Twitter Financial News Sentiment (TFNS) (Sentiment Analysis) [tfns]
  • News with GPT (NWGI) (Sentiment Analysis)
  • Financial Headline Analysis (Headline Analysis) [headline-tasks]
  • Named Entity Recognition (NER) (Named Entity Recognition) [ner-tasks]
  • Fine-tuning for General Financial Tasks
• Financial Certification Tasks
  • CFA Level I (Analyst Exam)
  • CFA Level II (Analyst Exam)
  • CFA Level III (Analyst Exam)
  • CPA REG (Accountant Exam)
  • Fine-tuning for Financial Certification Tasks
• Financial Data Reporting
  • FiNER-139
  • FNXL
  • Fine-tuning for Financial Reporting Tasks
• Financial Statement Analysis
  • Financial Math
  • Tag Extraction
  • Value Extraction
  • Formula Construction
  • Formula Calculation
  • FinanceBench
  • Fine-tuning for Financial Statement Analysis Tasks

LoRA Methods

• Vanilla LoRA
  • Background
    • Quick Facts
    • Algorithmic Idea
    • Key Equations
    • Implementation in FinLoRA
    • Usage Example
  • References
    • Why This Method?
    • Useful Links
• QLoRA
  • Background
    • Quick Facts
    • Algorithmic Idea
    • Key Equations
    • Implementation in FinLoRA
    • Usage Example
  • References
    • Why This Method?
    • Useful Links
• DoRA
  • Background
    • Quick Facts
    • Algorithmic Idea
    • Key Equations
    • Implementation in FinLoRA
    • Usage Example
  • References
    • Why This Method?
    • Useful Links
• rsLoRA
  • Background
    • Quick Facts
    • Algorithmic Idea
    • Key Equations
    • Implementation in FinLoRA
    • Usage Example
  • References
    • Why This Method?
    • Useful Links
• FedLoRA
  • Background
    • Quick Facts
    • Algorithmic Idea
    • Key Equations
    • Implementation in FinLoRA
    • Usage Example
  • References
    • Why This Method?
    • Useful Links

Benchmark Results

• Overview
  • Benchmark Angles
    • Angle I: LoRA Methods’ Performance on Financial Datasets
    • Angle II: LoRA Suitability for Financial Tasks
    • Angle III: Resources of LoRA Fine-tuning and Inference
    • Angle IV: Practical Considerations for LoRA Deployment in Finance
  • Take-home Message
    • Angle I: LoRA Methods Performance on Financial Datasets
      • Comparative Performance of LoRA Variants
      • rsLoRA Performs Better at High Ranks
      • DoRA Benefits from Two Learning Rates
      • LoRA-Tuned Llama 3.1 8B vs. Baseline Models and Gemini Fine-Tuned
    • Angle II: Financial Task LoRA Suitability
    • Angle III: Resource Usage and Performance Trade-offs of LoRA methods
    • Angle IV: Practicability of Applying LoRA in Real-world Financial Scenarios
      • Federated LoRA
      • Catastrophic Forgetting
• Results on General Financial Tasks
  • Detailed Results
  • Results Analysis
    • Angle I: LoRA Methods’ Performance on Financial Datasets
    • Angle II: LoRA Suitability for Financial Tasks
• Results on Financial Certification Tasks
  • Angle I: LoRA Methods’ Performance on Financial Datasets
  • Angle II: LoRA Suitability for Financial Tasks
• Results on Financial Reporting Tasks
  • Angle I: LoRA Methods’ Performance on Financial Datasets
  • Angle II: LoRA Suitability for Financial Tasks
• Results on Financial Statement Analysis Tasks
  • Angle I: LoRA Methods’ Performance on Financial Datasets
  • Angle II: LoRA Suitability for Financial Tasks
• Results on Federated LoRA
  • Performance Comparison: Central vs FedLoRA
  • Key Findings
• Resource Usage and Cost Analysis
  • Fine-tuning Costs
  • GPU Memory Requirements
  • Memory Usage Insights
• Sample Outputs
  • XBRL Tagging
  • XBRL Extraction
    • Formula Calculation

References

• References
  • Datasets
  • Large Language Models
  • LoRA Methods
  • Other Papers