this post was submitted on 30 Jun 2024
1110 points (96.9% liked)
Fuck AI
1097 readers
143 users here now
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
founded 5 months ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
In English yes. But the less popular the language is, the less materials there are. With this you can take any book and simplify it to your level. Unlike mass-produced books, AI can be very flexible.
Unfortunately that popularity directly translates to the AIs ability to digest and paraphrase a book. LLMs have been trained on what is available in computer text format, which means mostly internet sources. English has an outsized presence on the internet compared the to actual number of native speakers, so there's magnitudes more training data for it than any other language. The models of other languages will be severely limited, if AI companies have spent the resources to train them at all.
There are many AI companies, including those that are based in countries where people communicate in other languages. What you are saying is not an insurmountable problem.
Yes it is insurmountable. There is not enough non-english text in the world to be able to train an LLM.