thewindmom / palmyra-med-70b-32k

Writer/Palmyra-Med-70B-32K in Ollama

63 2 months ago

4 Tags

Readme

Model Details

Palmyra-Med-70b-32k, created by Writer, builds upon the foundation of Palmyra-Med-70b and offers an extended context length and meets the needs of the healthcare industry. The leading LLM on biomedical benchmarks, with an average score of 85.87%, outperforming GPT-4, claude Opus, Gemini and Med-PaLM-2 base model and a medically trained human test-taker.

Resources and Technical Documentation:

Specialized for Biomedical Applications

Palmyra-Med-70B-32k is meticulously designed to meet the unique linguistic and knowledge demands of the medical and life sciences sectors. It has been fine-tuned on an extensive collection of high-quality biomedical data, ensuring it can comprehend and generate text with precise domain-specific accuracy and fluency.

Our system integrates the DPO dataset and a well-crafted fine-tuning recipe along with a custom diverse medical instruction dataset, making it highly adept at handling the specific needs of this field. Key components of our training pipeline include:

  • Policy Optimization: Utilizing Direct Preference Optimization to enhance the model’s performance. DPO.
  • Fine-tuning dataset: Custom Medical Instruct dataset (Writer in-house build)

Model Description

  • Developed by: Writer
  • Model type: Llama
  • Language(s) (NLP): English
  • License: Writer open model
  • Finetuned from model: Palmyra-X-004
  • Context window: 32768

Intended Use

Intended Use Cases Palmyra-Med-70b-32k is intended for non-commercial and research use in English. Instruction tuned models are intended for assistant-like chat, whereas pretrained models can be adapted for a variety of natural language generation tasks.

Out-of-scope Use in any manner that violates applicable laws or regulations (including trade compliance laws). Use in any other way that is prohibited by Writer’s Acceptable Use Policy and the Writer open model license. Use in languages other than English.

Note: Developers may fine-tune Palmyra-Med-70b-32k models for languages beyond English provided they comply with the Writer open model license and the Acceptable Use Policy.

Watermarks: All models built by Writer.com contain watermarks to detect and prevent misuse and illegal use.

More

For the entire model description, read here: https://huggingface.co/Writer/Palmyra-Med-70B-32K

Uploading on Ollama granted

https://huggingface.co/Writer/Palmyra-Med-70B-32K/discussions/3