this post was submitted on 26 Feb 2024
68 points (84.7% liked)
Technology
59107 readers
5344 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Big privacy advocate so I was curious what it takes to self host something like that, more so just wanting a very flexible personal assistant for product, weather alerts all in one.
Takes a lot of RAM and GPU power, more than I have sitting around.
Have you been looking at quantised models? You can get pretty good ones at the 20 gig RAM+VRAM level which is very reasonable if you have a gaming PC and are ok with responses not being instant.