4GB to 8GB of RAM? That's not large by any standard?! And a bit less than 2 tokens per second for a 6B parameter model isn't fast at all.
this post was submitted on 15 Aug 2024
11 points (86.7% liked)
Single Board Computers
1215 readers
8 users here now
A community for the discussion of all single board computers. Raspberry Pi is ok, but there are so many other boards now that get looked over that deserve attention.
Post news, questions, your setups, guides, anything that has to do with SBCs. Server wide rules apply
founded 1 year ago
MODERATORS
Who downvotes this