In this liveProject, you’ll take on the role of a full-stack developer working for a school district. Your district wants to further student learning by developing its own custom large language model (LLM) to assist students, parents, and teachers. Your goal is to develop an MVP of this by creating a chatbot that can answer chemistry questions and provide follow-up answers and conversations. You’ll utilize the open source Llama LLM from Meta AI to do this. Your model will need to run with very low resources—it needs to work on the school’s outdated computers—and be equipped with an intuitive user interface. Let’s get started!

Fine-tuning an LLM is time-consuming and expensive! That’s why your local school district has tasked you with using RAG (Retrieval Augmented Generation) to help improve the capabilities of a chemistry chatbot based on Meta AI’s Laama. RAG allows an LLM to search a database to help answer questions, avoiding any unfortunate hallucinations. You’ll create a vector database for external knowledge for your chatbot’s RAG, establish an RAG API server for it to use, and then deploy your new bot to both the web and Discord.

Your local school district has a basic LLM chatbot based on Llama—now it needs fine-tuning! That’s where you come in. In this liveProject, you’ll utilize the Supervised Fine Tuning (SFT) approach to customize a chemistry chatbot for teaching children. SFT uses question-and-answer pairs to train a model to answer questions with a given answer. To achieve this, you will need to prepare your training data, utilizellama.cpp tools for LORA fine-tuning, and CLI tools to test inference on a fine-tuned model.