this post was submitted on 20 Oct 2024
34 points (97.2% liked)

datahoarder

6763 readers
35 users here now

Who are we?

We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.

We are one. We are legion. And we're trying really hard not to forget.

-- 5-4-3-2-1-bang from this thread

founded 4 years ago
MODERATORS
 

I know for photos i could throw them through something like Converseen to take them from .jpg to .jxl, preserving the quality (identical visially, even when pixel peeping), but reducing file size by around 30%. What about video? My videos are in .h265, but can i reencode them more efficiently? im assuming that if my phone has to do live encoding, its not really making it as efficient as it could. could file sizes be reduced without losing quality by throwing some processing time at it? thank you all

top 12 comments
sorted by: hot top controversial new old
[–] [email protected] 29 points 4 weeks ago (1 children)

This is not the helpful answer you were looking for, sorry. I'm fairly certain that the consensus among data hoarders is that the answer is more storage space not smaller files.

[–] [email protected] 9 points 4 weeks ago

And utilities for identify the eventual duplicates to save space (while still ensuring you don’t have only 1 copy that can be corrupted)

Like anything else it’s always trade offs.

[–] [email protected] 11 points 4 weeks ago (2 children)

You can absolutely re-encode h265 video, but you can't do it losslessly. In the end, it's always a balance between quality and filesize.

I decided for myself, that 1080p30 crf28 h265 is good enough for home video, which lead to a 50% to 80% storage space reduction on videos from my phone.

If you don't obsess over quality, I would highly recommend just messing around with ffmpeg a little bit and decide how much quality you're willing to lose in order to save disk space. When you're happy with your settings, you can either use ffmpeg itself or some fancy batch program like Tdarr to transcode all (or parts of) your video library.

My goto command is:
for file in *.mp4; do ffmpeg -i "$file" -movflags use_metadata_tags -map_metadata 0 -vcodec libx265 -crf 28 -vf scale=1920:-1 -r 30 "${file%.*}_transcoded.mp4"; done

[–] [email protected] 1 points 4 weeks ago

what I might end up doing is offloading the 4k videos to my computer, keep those as backup, and then put 1080p versions on my phone

[–] [email protected] 1 points 4 weeks ago

The phones are too good!

[–] [email protected] 8 points 4 weeks ago (1 children)

You will not gain much from reencoding or compressing these kinds of files. They are already compressed in a way purpose-built for their application. If you have source materials, like blu-ray discs, you can reencode with more aggressive h.265 settings. But this will probably not do better than shrink 5-30%.

It is usually a better use of time and resources to get more storage.

[–] [email protected] 4 points 4 weeks ago

This is what I always think. The real movie was hundreds of gigs, maybe TBs. What you get on the Blu-ray is already compressed to all hell but perfectly mastered to hide as much compression as possible. Why would you want compress it even worse? Hard drives are cheap, if you already want to store 1080p/4k copies of movies, just admit you're like us and pick up a couple 20TB drives

[–] [email protected] 6 points 4 weeks ago

Offhand I think AV1 is supposed to be a bit better than H265, but I think the improvement would be pretty marginal. Also, that’s a newer codec with less support everywhere, so you might find yourself slowing down a lot doing live transcoding to a format with better support like H265 or H264, depending on your devices. Add in all the time transcoding your current files from H265 to AV1 and it might just be worth adding more storage space.

[–] [email protected] 4 points 4 weeks ago

I have a lot of artworks I downloaded over years that were saved in png files and after converting them loslessly to avif I still was able to regain some space.

For videos you cant afford lossless if you want to recover space but visually lossless results are usually good enough on AV1

[–] [email protected] 3 points 3 weeks ago

I converted my old phone photos to JXL for this reason. It is malpractice that Android isn’t doing this by default for all users right now.

[–] [email protected] 3 points 3 weeks ago* (last edited 3 weeks ago)

I will use AVC and HEVC terminology for h264/x264 and h265/x265 respectively.

For video, you get jumps of compression quality from AVC to HEVC, from 8-bit to 10-bit, and from HEVC to AV1.

Depending on your source material, changing compression settings, like target quality, variable bitrate, etc, can also have significant gains. It will depend on your source and target though, and may need some testing to get "right". If you're looking for the best compression, that may be on a file by file basis, because different kinds of video have significantly different compression behavior or concerns. That's likely not feasible for a mass of files though.

Playback compatibility should also be considered. AVC mp4 is the most compatible, right now, if you consider all kinds of and older mobile and embedded devices. If you're fine with modern or desktop, you can go for the best compression codecs.


To get an idea of encoding time investment and quality, you can use ffmpeg with default quality settings, and target the different encoding targets.

AV1 10-bit, Opus audio:

ffmpeg -i input.mp4 -c:a libopus -c:v libsvtav1 -pix_fmt yuv420p10le out_av1-10bit-opus.mkv

AVC mp4 (when targeting mp4 these codec settings are the default, so in fact don't have to be specified):

ffmpeg -i input.mp4 -c:a aac -c:v libx264 -pix_fmt yuv420p out_avc-aac.mp4

HEVC 10-bit, opus:

ffmpeg -i input.mp4 -c:a libopus -c:v libx265 -pix_fmt yuv420p10le out_hevc-10bit-opus.mkv
[–] [email protected] 2 points 4 weeks ago* (last edited 4 weeks ago)

I'm a big fan of AV1, I've reencoded TiBs of video files to AV1. Reencoding is always going to lead to some quality loss, it just depends how much you notice it. For me I, don't see any quality loss going from h265 to AV1 in any videos I have done with my settings. But the biggest screen I use is 32inch 1080 monitor from about 5 years ago.

Its totally subjective so it may be worth taking a couple of your favourite video files and testing out a bunch of different settings and comparing the quality to see if theres a noticeable quality loss. Then you can weigh up the quality vs time taken vs storage cost to see if its worth it to you.

It can be slow, depending on what hardware you're working with but I've seen massive speed improvements over the past few months. ATM, I'm getting between 10 - 20 times speed up. So if I encode a 10 minute video, it takes between 30-60 seconds. This is with a 7700x, 12 cores. And I'm getting anywhere from ~10 times smaller file size to ~70% the original size.

Another option I use sometimes is converting to 720p from 1080. I do this on some videos that my parents take because their cameras aren't great and they have shaky hands so they're pretty blurry at the best of times anyway lol.