this post was submitted on 17 Aug 2023
31 points (97.0% liked)

datahoarder

6792 readers
1 users here now

Who are we?

We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.

We are one. We are legion. And we're trying really hard not to forget.

-- 5-4-3-2-1-bang from this thread

founded 4 years ago
MODERATORS
 

I'm wondering how to effectively keep track of the data I hoard. I'm almost at 80TB and currently use very basic folders structure to keep track of my data. I'm looking for something like a full text index maybe even a like a search engine for local data? Is there such a tool?

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 9 points 1 year ago (1 children)

Where do you store 80TB of data, is it backed up?

[–] hyper 7 points 1 year ago (1 children)

Yes, I have two rackmounted synology nas servers and one diy running unraid

[–] [email protected] 2 points 1 year ago (1 children)

To answer your question, I search my (albeit smaller) amount of data with Kiwix if it's a zim file, if not, the search function in Nautilus allows me to search the contents of text files.

[–] hyper 3 points 1 year ago

Kiwix sounds interesting, thanks. Still looking for something that indexes my NAS though.
If found https://www.diskoverdata.com/ but that looks a bit overkill to me.