Investigations:
- Simple prompt-engineering with few-shot exemples to have a first benchmarks with Mistral Large on API
- Exploration on fine-tuning Llama on specific French medical datasets (FrenchMedMCQA) to improve performance in French. Using LoRA and Unsloth for fast fine-tuning
- Using a open-source framework with RAG from research https://github.com/Teddy-XiongGZ/MedRAG/tree/main
Our next steps involve gathering more data and further fine-tuning, particularly using the MedMCQA dataset, to refine model performance. The goal is to create a more reliable and efficient system for handling diverse medical QA types.
Built With
- huggingface
- llama
- lora
- mistral
- nebius
- pytorch
- unsloth
Log in or sign up for Devpost to join the conversation.