datahoarder

6528 readers

12 users here now

Who are we?

We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.

We are one. We are legion. And we're trying really hard not to forget.

-- 5-4-3-2-1-bang from this thread

founded 4 years ago

MODERATORS

[email protected]

How do you keep track of you data? (self.datahoarder)

submitted 1 year ago by hyper to c/[email protected]

9 comments fedilink hide all child comments

I'm wondering how to effectively keep track of the data I hoard. I'm almost at 80TB and currently use very basic folders structure to keep track of my data. I'm looking for something like a full text index maybe even a like a search engine for local data? Is there such a tool?

top 9 comments

sorted by: hot top controversial new old

[–] [email protected] 9 points 1 year ago (1 children)

Where do you store 80TB of data, is it backed up?

[–] hyper 7 points 1 year ago (1 children)

Yes, I have two rackmounted synology nas servers and one diy running unraid

[–] [email protected] 2 points 1 year ago (1 children)

To answer your question, I search my (albeit smaller) amount of data with Kiwix if it's a zim file, if not, the search function in Nautilus allows me to search the contents of text files.

[–] hyper 3 points 1 year ago

Kiwix sounds interesting, thanks. Still looking for something that indexes my NAS though.
If found https://www.diskoverdata.com/ but that looks a bit overkill to me.

[–] [email protected] 6 points 1 year ago (1 children)

I just organize it properly in a folder structure that makes sense to me. No need to do anything more complicated if you're organized.

[–] [email protected] 6 points 1 year ago

I do this too, and avoid as much as possible having folders within folders, eg;
me-photos
me-music
me-documents
john-photos
john-music
work-documents
work-letter-templates

And so on. Sort by name.

[–] [email protected] 4 points 1 year ago (1 children)

VVV https://vvvapp.sourceforge.net/

[–] hyper 1 points 1 year ago

Thanks, I'll check it out

[–] [email protected] 0 points 1 year ago

Use some kind of hierarchical folder-structure like the Usenet does.
Something like: unix.desktop.theming for all your desktop ricing/theming stuff, unix.lx.debian.doc for debian documentation, win.win10.winget for everything related to winget on windows 10, rl.bureau.finances for your finances, accounting, etc. ...

You can use the Browser Extension "QuickCut" to save your bookmarks in folders. Its really helpful when you work on a bigger project and have all the documentation weblinks at hand.