Technology

59174 readers

2128 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago

MODERATORS

[email protected]

435

AI trained on AI garbage spits out AI garbage. (www.technologyreview.com)

submitted 3 months ago by [email protected] to c/[email protected]

62 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 2 points 3 months ago* (last edited 3 months ago) (1 children)

because that phrase doesn’t ever appear in the training data.

Eh but LLMs abstract. It has seen " have feathers" and " have fur" quite a lot of times. The problem isn't that LLMs can't reason at all, the problem is that they do employ techniques used in proper reasoning, in particular tracking context throughout the text (cross-attention) but lack techniques necessary for the whole thing, instead relying on confabulation to sound convincing regardless of the BS they spout. Suffices to emulate an Etonian but that's not a high standard.

[–] [email protected] 1 points 3 months ago

Workarounds for those sorts of limitations have been developed, though. Chain-of-thought prompting has been around for a while now, and I recall recently seeing an article about a model that had that built right into it; it had been trained to use tags to enclose invisible chunks of its output that would be hidden from the end user but would be used by the AI to work its way through a problem. So if you asked it whether cats had feathers it might respond "Feathers only grow on birds and dinosaurs. Cats are mammals. No, cats don't have feathers." And you'd only see the latter bit. It was a pretty neat approach to improving LLM reasoning.