heredos/ rwkv7:latest

462 2 months ago

an rwkv repo that tries to be less confusing, more up to date and with some QOL features

tools thinking 0.1b 0.4b 1.5b 2.9b 7.2b 13.3b 
ollama run heredos/rwkv7

Details

2 months ago

3796f45c9c9f · 2.6GB ·

model
rwkv7
·
2.95B
·
Q6_K
params
{ "frequency_penalty": 0.5, "presence_penalty": 0.5, "repeat_penalty": 1, "stop": [
template
### System{{- if .System }} {{ .System }}{{- else }} You are RWKV, a helpful assistant.{{- end }}{{-

Readme

Heredos’s RWKV repo

Now with tool calling!

Mollysama is doing an amazing job uploading different models, but it being split across repos makes me hard to recommend to my grandma. So I made this repo that I can continuously update and pull regularly with a cron job on all of my (and my grandma’s) devices. I hope you will find it useful!

original repo

  • 0.1b is mostly for testing purposes. (g1d)
  • 0.5b is able to give somewhat coherent answer, but it cannot be super useful (g1d)
  • 1.5b can chat with you and answer some questions. it is not great at reading context/rag, but t runs well on my phone, which is a huge plus (g1e)
  • 2.9b would be my overall pick. it is decent at answering questions and is able to take into account some context (not too long a context though). It also runs on half decent phones (g1e)
  • 7.2b is just really good, but i recommend you to use it only if you have an ollama-compatible gpu, or a really beefy CPU. Be mindful of your battery life if you run it on a laptop, chances are it won’t trigger your fans, but the power draw is still there. (g1e)
  • lonely-neko (1.5b) is for those who are able to read and write chinese. it is fun to talk to and makes for a great quirky assistant. For some reason it does not appear on this page. maybe ollama is the one who sent her in the void she dreads so much? (base)

about Tool Calls

They have mostly been tested to work, but llms trained with such wildly different datasets cannot be exact science, so there might be incompatibilities i don’t know of yet. Also, this repo has heavily modified the original modelfile (more specifically, the prompt template) to allow the model to run tool calls. This means that even with reproducible parameters, you will likely not get the same output as the models in Mollysama’s repo. It’s not a big deal for my use case, but it might be for you, IDK.

about Text/Code completions

maybe later.