🧠 Complete End-to-End Guide: Setting Up LLaMA-Factory on macOS

Date Created: 2025-05-11
By: 16BitMiker
[ BACK.. ]

Running large language models locally on your Mac might sound intimidating—but thanks to projects like LLaMA-Factory, it’s more feasible than ever. This guide walks you through every step required to get LLaMA-Factory up and running on macOS, from Python installation to launching the WebUI and chatting with powerful LLMs like Mistral and TinyLlama.

But why install LLaMA-Factory in the first place?

🧩 Let’s take a moment to understand the motivation behind using it.

👀 Why Use LLaMA-Factory?

📋 A Foundation For Fine-Tuning and Experimentation

LLaMA-Factory is a powerful training and inference framework built on top of Hugging Face Transformers. It supports:

🧪 Fine-tuning pre-trained models using LoRA (Low-Rank Adaptation)
🖥️ Local inference with a friendly WebUI
🔁 Exporting models in formats compatible with Ollama, llama.cpp, or Hugging Face Hub

This makes it an ideal tool if you're looking to:

Customize open-weight models like Mistral or Llama 2 using your own datasets
Experiment with lightweight fine-tuning techniques like LoRA on consumer-grade hardware
Deploy the result to low-latency inference engines like Ollama

🚀 Why This Matters for Ollama + Mistral Users

Ollama is a fast, local-first runtime for running LLMs using quantized model formats like GGUF. However, Ollama doesn’t currently support training or fine-tuning models directly.

That’s where LLaMA-Factory comes in:

You use LLaMA-Factory to train or LoRA-adapt a base model like Mistral-7B
Then, you export the result to GGUF format
Finally, you load it into Ollama for fast, local inference

This workflow gives you the best of both worlds: flexible training and blazing-fast deployment.

We'll cover the full LoRA training and export process in a future post—but for now, let’s get your environment ready.

👀 System Setup

📋 Installing Python 3.10 with Homebrew

macOS ships with a system Python, but we’ll install Python 3.10 separately using Homebrew to avoid conflicts and ensure compatibility with LLaMA-Factory.


x
# Install Homebrew (if not installed)
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

# Update Homebrew
brew update

# Install Python 3.10
brew install python@3.10

# Confirm installation
python3.10 --version

📋 Prioritizing Python 3.10


xxxxxxxxxx
# Ensure Python 3.10 is used in your shell
echo 'export PATH="/opt/homebrew/bin:$PATH"' >> ~/.zshrc
source ~/.zshrc

👀 Project Environment

📋 Creating a Virtual Environment


xxxxxxxxxx
# Create a directory for virtual environments
mkdir -p ~/venvs

# Create one for llama-factory
python3.10 -m venv ~/venvs/llama-stable

# Activate it
source ~/venvs/llama-stable/bin/activate

# Confirm version
python --version

📋 Cloning the Repository


xxxxxxxxxx
mkdir -p ~/git
cd ~/git
git clone https://github.com/hiyouga/LLaMA-Factory.git
cd LLaMA-Factory

👀 Dependency Installation

📋 Upgrade Core Tools


xxxxxxxxxx
pip install --upgrade pip setuptools wheel

📋 Install LLaMA-Factory (Editable Mode)


xxxxxxxxxx
pip install -e .

📋 Resolve Package Conflicts


xxxxxxxxxx
pip install pydantic==2.10.6
pip install "gradio>=4.38.0,<=5.21.0"
pip uninstall -y llamafactory
pip install llamafactory==0.9.2

📋 Install ML Libraries


xxxxxxxxxx
pip install torch torchvision torchaudio
pip install transformers datasets accelerate
pip install einops sentencepiece protobuf

👀 Verifying the Setup

📋 Confirm Installation


xxxxxxxxxx
# CLI tool location
find ~/venvs/llama-stable/bin -name "llama*"

# Inspect package
ls -la ~/venvs/llama-stable/lib/python3.10/site-packages/llamafactory/

👀 Running the WebUI

📋 Launch It


xxxxxxxxxx
# From repo root
llamafactory-cli webui --share --no-inbrowser

Output should include:


xxxxxxxxxx
Running on local URL: http://0.0.0.0:7860
Running on public URL: https://xxxx.gradio.live

📋 Access the UI

Open the public .gradio.live URL
Or visit http://localhost:7860

👀 Using the LLaMA-Factory WebUI

📋 Downloading a Model

Go to the “Model” tab
Select and download a model like:
- 🐎 TinyLlama
- 🧠 Phi-2
- 🌍 Mistral-7B

📋 Set Up Chat

Switch to the “Chat” tab
Load the downloaded model
Adjust parameters:
- Temp: 0.7
- Top P: 0.9
- Max Tokens: 1024

📋 Start Interacting

Type a prompt and hit Enter to see your model respond.

👀 Troubleshooting Common Issues

🔧 Dependency Errors


xxxxxxxxxx
python3.10 -m venv ~/venvs/llama-fresh
source ~/venvs/llama-fresh/bin/activate
pip install --upgrade pip
pip install pydantic==2.10.6
pip install llamafactory==0.9.2
pip install torch

🔧 macOS Performance Tips


xxxxxxxxxx
# Apple Silicon
export PYTORCH_MPS_ENABLE=1

# Intel Macs
export OMP_NUM_THREADS=4

🔧 WebUI Networking


xxxxxxxxxx
llamafactory-cli webui --server-name 0.0.0.0 --port 7860 --share --no-inbrowser

👀 Advanced Configuration and Next Steps

📋 Custom Model Directory


xxxxxxxxxx
mkdir -p ~/models/llama
llamafactory-cli webui --model-path ~/models/llama --share --no-inbrowser

📋 Fine-Tuning with LoRA (Preview)

Here’s a high-level outline of what we’ll cover in a follow-up post:

✅ Prepare a dataset (JSON or CSV format)
✅ Use the WebUI’s “Train” tab
✅ Configure training parameters for LoRA
✅ Monitor training progress
✅ Export the model as GGUF or Hugging Face format
✅ Load it into Ollama for local inference

This approach enables fast, memory-efficient tuning of models like Mistral on your Mac—even without a dedicated GPU.

🏁 Conclusion

You now have LLaMA-Factory fully installed and ready on macOS, complete with dependency fixes and a working WebUI. More importantly:

🧠 LLaMA-Factory is your launchpad for fine-tuning open LLMs
🔧 It supports LoRA, the most practical tuning method for laptops
📦 You can export your results to formats usable with Ollama

In our next post, we’ll dive deep into the full LoRA training workflow—including dataset formatting, training configs, and GGUF export. Stay tuned!

📚 Read More

Happy tuning! 🧠🛠️