this post was submitted on 04 Mar 2024
118 points (94.0% liked)

Technology

57473 readers
4035 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS
 

Title is a bit dramatic, but yes, Claude 3 claims to be better than GPT 4 in most ways.

top 31 comments
sorted by: hot top controversial new old
[–] [email protected] 27 points 5 months ago (1 children)

I just spent some time on Claude 3, and I see how it can be considered ‘better’ than GPT4, however I quickly found that it tends to lie about itself in subtle ways. When I called it out on an error it would say things like ‘I’ll strive to be better’. I called it out on the fact that it’s model doesn’t grow or change based on conversations it has and that it’s impossible for it to strive to do anything outside of, maybe, that chat. It then went on to show me that it couldn’t even adjust within that chat by doing the same thing 5 more times in 5 different ways.

I see the model it used for the apologies (acknowledge, apologize, state intent to do better in the future) which is appropriate for people or beings capable of learning, but it is not. I went from having a good conversation with it about a poem I wrote to being weirdly grossed out by it. GPT does a good job of not pretending to be human, and I appreciate that.

[–] [email protected] 4 points 5 months ago* (last edited 5 months ago) (1 children)

The cynic in me says that's perfectly human behavior, though

[–] [email protected] 7 points 5 months ago (1 children)

Yea that’s what I’m saying, and I don’t like it. I don’t want my LLM acting human, I want it acting like an LLM. My interactions with Claude 3 were very uncanny valley and bugged me a lot.

[–] [email protected] 3 points 5 months ago (1 children)

so you're basically saying it talked itself squarely into uncanny valley?

i honestly didn't consider that would be an issue for LLMs, but in hindsight...yeah, that's gonna be a problem...

[–] [email protected] 1 points 5 months ago

Yea, that’s exactly what it did. It was bizarre to realize actually because I felt the same way because it’s text. But here I am

[–] [email protected] 22 points 5 months ago (1 children)

Hey have you guys heard about ChatGPT 7? It makes chatGPT 6 look like ChatGPT 5!

Who ever thought the AI awakening would this fucking banal?

[–] [email protected] 9 points 5 months ago (1 children)

Don't worry, it's not the AI awakening. It's just people figuring out how to sell text generators.

[–] [email protected] 4 points 5 months ago

If you generate your own marketing copy, these things practically sell themselves

[–] [email protected] 19 points 5 months ago (1 children)

It does amazingly well with schemas:

When the dead rabbit was seen by the dog, it hopped. What does "it" refer to: the rabbit or the dog?

In the sentence "When the dead rabbit was seen by the dog, it hopped," the pronoun "it" refers to the dog, not the rabbit.

This is because the rabbit is described as "dead," meaning it cannot hop or move. The verb "hopped" implies movement, which can only logically apply to the living subject in the sentence, which is the dog.

So the intended meaning is that when the dog saw the dead rabbit, the dog itself hopped, likely in surprise or reaction to seeing the dead animal.

When the iceberg was struck by the ship, it sunk. What does "it" refer to: the iceberg or the ship?

In the sentence "When the iceberg was struck by the ship, it sunk," the pronoun "it" refers to the ship, not the iceberg.

The reason is that icebergs, being large masses of ice floating in water, do not actually "sink" when struck. They are already floating and may crack or break apart when struck, but the word "sunk" implies something going down below the water's surface.

Ships, on the other hand, are vessels designed to float on water. If a ship strikes an iceberg with enough force, it can cause damage that leads to the ship taking on water and eventually sinking below the surface.

So in this context, with the verb "sunk" being used, it is more logical that "it" is referring to the ship sinking after striking the iceberg, rather than describing what happened to the stationary iceberg itself.

[–] [email protected] 10 points 5 months ago (1 children)

You're right! I tried against ChatGPT 3.5:

"It" in this context likely refers to the dead rabbit, as it is the subject of the sentence and the one described as hopping.

It got the ship one right though.

[–] [email protected] 3 points 5 months ago

I found that it helps if you ask chatGPT 4 to act as a Vulcan from Star Trek, it does better with logic puzzles. But it doesn't work with 3.5.

[–] [email protected] 8 points 5 months ago (1 children)

The sonnet model is decent, but something weird is going on with their opus model as it just terribly sucks.

Mistral-large is probably the best large model for practical purposes at this point.

[–] [email protected] 4 points 5 months ago (1 children)

Mistral-large is probably the best large model for practical purposes at this point.

What makes you say that? I have not performed my own comparison, but everything I have seen and read suggests that GPT4 is king, currently.

[–] [email protected] 11 points 5 months ago* (last edited 5 months ago) (1 children)

It depends on the task, but in general a lot of the models have fallen into a dark pattern of Goodhart's Law, targeting the benchmarks but suffering at other things.

So as an example, while GPT-4 used to correctly model variations of the wolf, goat, cabbage problem with token similarity hacks (i.e. using emojis instead of nouns to break pattern similarity with the standard form of the question), now it even fails for that with the most recent updates, whereas mistral-large is the only one that doesn't need the hack at all.

[–] [email protected] 3 points 5 months ago

Interesting. That's not something I've heard about until now, but something I'll surely look into.

[–] [email protected] 5 points 5 months ago (2 children)

It's called Claude though... That's definetly not better than GPT

[–] [email protected] 16 points 5 months ago

Disagree, never liked OpenAI trying to claim a generic term as the name of their product.

[–] [email protected] 8 points 5 months ago (2 children)

Think it depends the language, in french gpt is very very close to "j'ai pété" which means "I farted". But, yeah agree that Claude ain't much better name

[–] [email protected] -1 points 5 months ago (1 children)

True, but Claude is like my grandfather name..

[–] [email protected] 1 points 5 months ago* (last edited 5 months ago)

Exactly, neither fit right for an AI

[–] [email protected] 1 points 5 months ago

It beat Claude 3 on math and reasoning when analyzing images.

What beat Claude 3?

No free version of Opus, so I can't try it.

It's 👍 that the ai competition is sizzling.

[–] [email protected] 1 points 5 months ago

Is it open source? If not then it's just as worthless as OpenAI

[–] [email protected] -3 points 5 months ago (3 children)

This is how French people pronounce cloud, so might be where the name comes from

[–] [email protected] 8 points 5 months ago (1 children)

More likely a reference to Claude Shannon, the founder of AI.

[–] [email protected] 8 points 5 months ago* (last edited 5 months ago) (1 children)

Yes, it’s named after Claude Shannon, but I’ve never heard him described as “the founder of AI”. He’s the father of information theory, which is only indirectly connected to AI.

[–] [email protected] 2 points 5 months ago* (last edited 5 months ago)

From the linked Wikipedia page:

"Theseus", created in 1950, was a mechanical mouse controlled by an electromechanical relay circuit that enabled it to move around a labyrinth of 25 squares.[71] The maze configuration was flexible and it could be modified arbitrarily by rearranging movable partitions.[71] The mouse was designed to search through the corridors until it found the target. Having travelled through the maze, the mouse could then be placed anywhere it had been before, and because of its prior experience it could go directly to the target. If placed in unfamiliar territory, it was programmed to search until it reached a known location and then it would proceed to the target, adding the new knowledge to its memory and learning new behavior.[71] Shannon's mouse appears to have been the first artificial learning device of its kind.[71]

[–] [email protected] 4 points 5 months ago (1 children)

Absolutely not. It’s pronounced \klod. The « OW » diphthong sound doesn’t exist in French.

Cloud is generally pronounced as in English \aʊ\ or maybe \klud\ for non English speakers.

There is no possible confusion in French between this two words.

[–] [email protected] 0 points 5 months ago

Trust me... "je les ai téléchargés depuis le Claude" is exactly how most French will pronounce. Not all, but most. First hand experience

[–] [email protected] 3 points 5 months ago (1 children)
[–] [email protected] 1 points 5 months ago* (last edited 5 months ago)

Sounds more like claode. Which is fraction away from Claude, and most often than not the the ao sounds au