this post was submitted on 07 Aug 2024
130 points (93.9% liked)
Technology
58012 readers
3173 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Do you know if current LLM models use static neural networks (like where each node is connected with each of the next layers)or if they can rearrange their connections into other layers?
They are indeed. It has multiple steps, but pretty much. 3Blue1Brown has a nice visual explanation: https://www.youtube.com/watch?v=wjZofJX0v4M
So if you want it more like a brain, you would have to have nodes that are able to form the connections while learning letting each node decide in what "direction” it wants to grow it’s connection in some sort, rather than having fixed connections where you only adjust the correlation of the nodes. And you would need multiple transformer (and most likely some hard logic algorithms as well) for different inputs as well as a main "thinker” that decides through which transformer (or algorithm) a input has to go and if the output of that transformer needs to be feeded in again as no input.
In theory. Then comes the question of how exactly are you gonna teach/train it. I feel our current approach is too strict for proper intelligence to emerge, but what do I know. I honestly have no clue how such a model could be trained. I guess it would be similar to how people train actual braincells? Tho that field is very immature atm... The neat thing about the human brain is, that it's already preconfigured for self learning, tho it does come with its own bias on what to learn due to its unique needs and desires.
😁🥳 I’d say, you would then need something that takes the role of hormones in that system (like hardcoded reactions to events in and outside of the AI brain/body(so called emotions I would say)) that trigger the connections to grow, shrink, get their values adjusted etc.
At least that would be my approach.
Calling the reward system hormones, doesn't really change the fact that we have no clue where to even start. What is a good reward for general intelligence? Solving problems? That's our current approach, which has the issue of the AI not actually understanding the problems and just ending up remembering question answer pairs (patterns). We need to figure out what defines inteligence and "understanding" in an easily measurable way. Which is something people knew almost a hundred years ago when we came up with the idea of neural networks, and why I say we didn't get any closer to AGI with LLMs.