Integrate with Ollama

[andr6]

It would be great to expand this to use local Ollama

[morpheuslord]

I am looking into it cause the API version of Llama I am using sucks 😅 but it totally works thanks to the contributor who wrote the code. If anyone knows a good way to integrate it in all 3 Windows, Linux and mac can post their views and resources cause Ollama is still not supported in Windows.

[andr6]

Ollama Windows support would be via WSL .

[morpheuslord]

I am working on that but it might take some time cause I need to figure out how I am going to execute the Ollama installation from my Windows machine over on Linux if that's sorted 50% of the work is done. I know there is a python integration module but I cant do anything without actually running it.

[morpheuslord]

hey bro check the latest update @andr6

[andr6]

Heya. I can see that you implemented via docker. Can we integrate it with local ollama instead of docker too ? Don't want to be asking too much :)

[morpheuslord]

I know and that was my first intention but it was a lot of work so I took a few shortcuts besides this works out well on all systems so that's a win 😉. I am working on GPU integration so performance won't be a hindrance.