this post was submitted on 31 Mar 2024
36 points (82.1% liked)
Technology
57473 readers
3596 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
After using AI chat stuff like this and chatgpt, ive come to the conclusion that building prompts is akin to building search queries for search engines. Wherein using the right terms leads to better results.
The following system prompt has worked great for me. I will soon test it with llama2.
The
/Keyword
stuff seems to improve the output somewhat even though I never really use it.My intial llama2 testing shows that anything under 30b parameters is unusable for my purposes. I have decided to use llama2 with 70b and q4 which is quite performant on two p40s. I get about 6 tokens/s.