this post was submitted on 30 Sep 2024

11 points (100.0% liked)

The Linux Lugcast Podcast

171 readers

4 users here now

website: https://www.linuxlugcast.com/

mumble chat: lugcast.minnix.dev in the lugcast room

email: [email protected]

matrix room: https://matrix.to/#/#lugcast:minnix.dev

youtube: https://www.youtube.com/@thelinuxlugcast/videos

peertube: https://nightshift.minnix.dev/c/linux_lugcast/videos

founded 1 year ago

MODERATORS

[email protected]

[email protected]

11

Exploring Llamafile: Mozilla's Attempt in the World of Open Source AI (itsfoss.com)

submitted 1 month ago by [email protected] to c/[email protected]

4 comments fedilink hide all child comments

top 4 comments

sorted by: hot top controversial new old

[–] [email protected] 2 points 1 month ago (1 children)

The issue isnt if its easy to use the issue is a matter of compute. A majority of people on the internet only have mobile access. We need a way to let the masses utilise distributed compute in a secure and private way.

[–] [email protected] 1 points 1 month ago (1 children)

You can run many 7B models on phones with 8GB RAM

[–] [email protected] 3 points 1 month ago (1 children)

I find the 7b models dont have the capabilities id like and i cant image the tokens per second would be very good.

[–] [email protected] 2 points 1 month ago

Yeah running it and it actually being useful are two very different things.