Santa coder can almost fit on a Pine Phone and gpt-neox-20b only needs 40gb of ram. Are any of you guys using them? I've found Santa even is good enough that it can help me find things without breaking my flow to look at documentation as long as I'm only doing python.
Why are you running sophisticated chatbot AI on a phone? Shouldn't you run it on something with more computing power?
>>2
The point is that you don't need that much. I have salesforce's model running on my largest machine.