this post was submitted on 01 Apr 2024

223 points (98.3% liked)

Programming

17524 readers

631 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Follow the programming.dev instance rules
Keep content related to programming in some way
If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities [email protected]

founded 2 years ago

MODERATORS

[email protected]

223

What we know about the xz Utils backdoor that almost infected the world (arstechnica.com)

submitted 7 months ago by [email protected] to c/[email protected]

35 comments fedilink hide all child comments

top 35 comments

sorted by: hot top controversial new old

[–] [email protected] 56 points 7 months ago (2 children)

Specifically, SSH logins were consuming too many CPU cycles and were generating errors with valgrind, a utility for monitoring computer memory.

Through sheer luck and Freund’s careful eye, he eventually discovered the problems were the result of updates that had been made to xz Utils. On Friday, Freund took to the Open Source Security List to disclose the updates were the result of someone intentionally planting a backdoor in the compression software.

It is lucky that Andres Freund checked and found the issue in valgrind that was maybe intentionally or maybe unintentionally.

https://www.openwall.com/lists/oss-security/2024/03/29/4

I'm interested in figuring out what happened and more information on the contributor behind the attack.

[–] [email protected] 64 points 7 months ago (3 children)

Everybody keeps saying it was found by luck, but this seems like it was found by a guy who maintains his repository properly and monitors his CPU cycles diligently just to prevent this sort of thing. I guess I would call it lucky he found it so quickly, but it was definitely not found by luck.

[–] [email protected] 32 points 7 months ago

The fact that that person happened to be looking on a system downstream to this one, while also having the context needed to pin it back to xz in particular is the lucky part. The same attack in any of countless other places wouldn't have gotten spotted the same way, or as quickly. That's not to say diligence on Freund's part wasn't a big factor here, but it's important to identify that luck was a big factor.

[–] [email protected] 20 points 7 months ago

It's more so lucky that there was someone diligently doing that. It could've easily gone unnoticed had there not been someone like him.

[–] [email protected] 16 points 7 months ago

Yep. It could also be found by luck a bit later, or caught in a strict process sooner or later that we won't know.

No process is perfect, but the fact this attack is super complex, with many levels of indirection on run time, detecting if it's in debug mode, forcing maintainers to disable Oss fuzzer (a tool that scans source code for bugs and vulnerabilities) and also involved social engineering and took years to get close to widespread release is to show what it takes to plant a back door on 100% Foss systems.

As opposed to closed source, where all it takes is a conversation between govt and a CEO.

[–] [email protected] 11 points 7 months ago (2 children)

to me: the most chilling thing is that someone involved in the open source ecosystem introduced this vulnerability and, if it was intentional, what else did they do?

[–] [email protected] 14 points 7 months ago (1 children)

That's what I'm most curious about. Was it government? Was it Microsoft, Apple or Google? Was it some lone hacker or group looking for money? Was it just an OSS developer that wanted revenge? It would make for a spicy story.

[–] [email protected] 20 points 7 months ago

It looks like a state actor, but hard to tell who. The way they did it was by bullying the original overworked maintainer into making someone else a co-maintainer, and that new co-maintainer introduced the backdoor. The accounts that pressured him into adding another maintainer all appear to have been sock puppets.

It just shows how little support there is for the lone maintainers of basic utilities that we all use, and it's really something we need to do something about.

[–] [email protected] 9 points 7 months ago

It isn't if it was intentional. It was intentional. Otherwise the exploit chain wouldn't be so convoluted.

[–] [email protected] 41 points 7 months ago* (last edited 7 months ago) (2 children)

At the moment, it’s unknown if there was ever a real-world person behind this username or if Jia Tan is a completely fabricated individual.

Well, I'm pretty sure it wasn't a platypus. There was definitely one or more real-world people behind this. I doubt anyone thinks that their name is actually Jia Tan, though.

[–] [email protected] 34 points 7 months ago (1 children)

The industry has too long neglected the true threat posed to information security by the platypi.

[–] [email protected] 11 points 7 months ago (2 children)

Platypodes

[–] [email protected] 8 points 7 months ago (1 children)

Platypoxen

[–] [email protected] 9 points 7 months ago (1 children)

Platyhaxor

[–] [email protected] 9 points 7 months ago (2 children)

Pl4tyh4x0r

[–] [email protected] 2 points 7 months ago

Pl0x

[–] [email protected] 0 points 7 months ago

Pretty sure I've seen that one on IRC in the 90's

[–] [email protected] 5 points 7 months ago (1 children)

Platypoodles

[–] [email protected] 1 points 7 months ago

Dammit, this attacker is sounding more and more adorable!

[–] [email protected] 5 points 7 months ago (1 children)

I saw someone trying attribute this to Hong Kong based on the possible origins of the name.

Seems like a stretch to me

[–] [email protected] 4 points 7 months ago

Failing that hard to not know what is, and isn't knowable, is a good reason to not give a single f about what they think

[–] [email protected] 21 points 7 months ago* (last edited 7 months ago) (1 children)

Incredible, just incredible. I am looking forward to the upcoming time when we know more about exactly how it works.

[–] [email protected] 15 points 7 months ago (2 children)

Yeah it sounds pretty wild already with some kind of, like, door knock mechanism using certificates? So you can't scan for it. And some reverse engineering countermeasures.

Like everyone else, I have to wonder what libraries have been compromised in a way that nobody has noticed yet.

[–] [email protected] 5 points 7 months ago (2 children)

Yeah, that's the scariest part. This was caught, but are there other projects out there that have been attacked with similar methods that no one knows about?

[–] [email protected] 5 points 7 months ago

I have a feeling there are a lot of busy people trying to answer that question, now. Yikes.

[–] [email protected] 1 points 7 months ago

There's also the ones known by very few people. There's companies and unofficial groups of people who collect and sell this information. Usually to state governments, off records.

I don't think it's the case for Linux, but I occasionally follow the state of things for bounties offered by Google and Apple to white hat hackers. Though this case is clearly malicious, I understand most vulnerabilities can easily pass as a bug/mistake.

[–] [email protected] 2 points 7 months ago

From what I've read is not authentication bypass, it's a RCE using certificates to deliver the payload. If a specific signature is found it runs the code that was sent in place of the signing public key. It also means that only someone who has the ability to generate that specific key signature could use the RCE.

There were some other bits that looked like they could have been placed to enable compromising other build systems in the future when they checked for xz support.

[–] [email protected] 8 points 7 months ago

This is the best summary I could come up with:

They were likely very close to seeing the backdoor update merged into Debian and Red Hat, the two biggest distributions of Linux, when an eagle-eyed software developer spotted something fishy.

Andres Freund, a developer and engineer working on Microsoft’s PostgreSQL offerings, was recently troubleshooting performance problems a Debian system was experiencing with SSH, the most widely used protocol for remotely logging in to devices over the Internet.

Through a combination of sheer luck and Freund’s careful eye, he eventually discovered the problems were the result of updates that had been made to xz Utils.

On Friday, Freund took to the Open Source Security List to disclose the updates were the result of someone intentionally planting a backdoor in the compression software.

Malicious code added to xz Utils versions 5.6.0 and 5.6.1 modified the way the software functions when performing operations related to lzma compression or decompression.

For instance, Tan replaced Collins' contact information with their own on oss-fuzz, a project that scans open source software for vulnerabilities that can be exploited.

The original article contains 640 words, the summary contains 171 words. Saved 73%. I'm a bot and I'm open source!

[–] Portable4775 8 points 7 months ago* (last edited 7 months ago)

It's crazy how they pressured/manipulated the maintainer. Especially fucked up considering he wasn't in a good mental state and was still helping the community by maintaining FOSS software.

[–] [email protected] 4 points 7 months ago (2 children)

So if I don't use SSH am I fine? Because my distro doesn't let me remove the package. Other packages depend on it.

[–] [email protected] 7 points 7 months ago (1 children)

Unless you're running Debian testing you're safe. If ssh isn't open to the internet you're safe. Just make sure everything is up-to-date.

[–] [email protected] 8 points 7 months ago

Debian testing has 'updated' to 5.6.1+really5.4.5-1 anyway, so as long as you've updated within the past few days it will have been downgraded to 5.4.5.

[–] [email protected] 2 points 7 months ago

You can't remove it but you can downgrade.

[–] [email protected] 2 points 7 months ago (1 children)

How quickly was it caught?

[–] [email protected] 6 points 7 months ago* (last edited 7 months ago)

After a month from what I read, most distro hadn't included it yet

Edit: it had been prepped for 2 years or so and the malicious code was introduced a while ago, but from release it took about a month to detection