Zephyr-7B Preference tuning

Inspiration

I've seen a lot of companies trying to build smaller models that perform as well as LLama

What it does

This trains Zephyr-7B to act as a customer service chatbot

How we built it

Used a GPTQ-quantized version ofZephyr-7B to optimize with LoRA

Challenges we ran into

Getting it to run with the 3B version

Accomplishments that we're proud of

Got the 7B version to run

What we learned

Huggingface project don't always have the best documentation

What's next for Zephyr-7B Preference tuning

Code cleaning, adapting for Zephyr-3B

Built With

• google-cloud
• huggingface
• llama
• mistral
• python
• transformers
• zephyr