this post was submitted on 28 May 2024
141 points (100.0% liked)
chapotraphouse
13392 readers
896 users here now
Banned? DM Wmill to appeal.
No anti-nautilism posts. See: Eco-fascism Primer
Vaush posts go in the_dunk_tank
Dunk posts in general go in the_dunk_tank, not here
Don't post low-hanging fruit here after it gets removed from the_dunk_tank
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Since LLMs essentially decide on one character at a time, I wonder if they would have better accuracy if asked to tell you the sum backwards. That's how we teach kids to add, right to left, carry the 1.
I think this is essentially what they did. The point of the paper is they made an architecture to make the llm more aware of an individual digit's position in a number. It helped with addition, multiplication, and even sorting.
Its technically true that it decides token at a time but it also takes previous tokens into account.
That's why it's easier. if you're going left to right you have to not only figure out the sum of the first number position, but also if there's a 1 to carry or not. Going right to left you only have to focus on one 1 digit add at a time and you already know if there's a carry by looking at the last addition.