KodeXv0.1: A Family of State-of-the-Art Financial Large Language Models

While powerful, current cutting-edge LLMs may not meet the needs of specialized sectors. We introduce KodeXv0.1, a family of language models that surpass GPT-4 in financial question answering. We use Llama 3.1 8B and 70B variants, adapting them to finance with a custom training regime. We collect publicly available financial documents like earnings calls and business reports to create a high-quality synthetic dataset of Context-Question-Answer triplets closely mirroring real-world tasks. Using this dataset, we perform RAG-aware 4bit LoRA instruction tuning with Llama 3.1 base variants to produce KodeX-8Bv0.1 and KodeX-70Bv0.1. We then conduct evaluations using FinanceBench, FinQABench, and a withheld test set. Our results show that KodeX-8Bv0.1 is more reliable in financial contexts than other models, surpassing them by up to 9.24%, and even outperforming GPT-4 by up to 7.07%. KodeX-70Bv0.1 further improves on this, exceeding GPT-4's performance in every benchmark tested.

‍

Co-Authors

Neel Rajani, PhD Candidate in Responsible NLP, University of Edinburgh; BSc in Computing Science, University of Glasgow

Lilli Kiessling, MSc Candidate in Computational Neuroscience, Bernstein Center for Computational Neuroscience (BCCN) & Technische Universität Berlin; BSc in Physics, Technische Universität Berlin

Aleksandr Ogaltsov, Data Scientist (AI & ML), Kodex AI‍

Claus Lang, CTO & Co-Founder, Kodex AI

Kodex AI_A Family of State-of-the-Art Financial LLMs.pdf

Download PDF • 827KB