Technology

57473 readers

3609 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago

MODERATORS

[email protected]

I easly got the reddit r/place data into a 4.3GIB sqlite database, where can I post it? (lemmy.world)

submitted 1 year ago* (last edited 1 year ago) by [email protected] to c/[email protected]

9 comments fedilink hide all child comments

cross-posted to: https://lemmy.world/post/2499861

As I said, I made a lossy reformat of the database and a lossless one for 6.0 Gib (6,477,905,920). compared to ~26GIB from Reddit, where fields are almost intentionally anti-compressed to take up more room.

If there is somewhere I can host it, let me know.

also, I couldn't figure this out, do sqlite databses store any information on the creator or editor of a document?

why it's lossy

It's missing a large table of base64 urandom technically required to recreate the document fully

top 9 comments

sorted by: hot top controversial new old

[–] [email protected] 20 points 1 year ago (1 children)

[email protected] looks active and seems like a good place for it

[–] [email protected] 5 points 1 year ago* (last edited 1 year ago) (1 children)

thanks, how do I crosspost/ move this one?

[–] [email protected] 7 points 1 year ago (1 children)

Using the web-ui, on this post there is an icon made up of two squares. It's right next to the star for saving the post. That's the cross post button.

[–] [email protected] 3 points 1 year ago* (last edited 1 year ago)

thanks, made updates to the post

[–] [email protected] 10 points 1 year ago* (last edited 1 year ago)

here are a few options that I see but never actually use.

contact https://wiki.archiveteam.org/; they have some reddit related archival activities going on
more academic type and will be very helpful for researchers:
- https://zenodo.org/
- https://figshare.com/ (this I’ve used and very easy)
- https://datadryad.org/
- https://academictorrents.com/ (pushshift archives are also on here)
- https://socialmediaarchive.org/

Your data don’t seem to be massive compared to the types of data people store on there. So I don’t think it’s gonna be an issue. Plus, if you deposit your data in 1 archivist place + 1 research place, the data may be used by more people. Don’t forget about licenses btw.

EDIT: added https://socialmediaarchive.org/ to the list, just found out about that.

[–] [email protected] 1 points 1 year ago (1 children)

Is this derived directly from the data reddit stored/created or is it a reconstruction of some kind from observing the r/place output? I'm tempted to look at the table structures but not tempted enough to download 4 gigs of it just yet.

[–] [email protected] 2 points 1 year ago* (last edited 1 year ago)

rebuilt from reddit, still messing with optomizations, is adding a color defs table worth it?

[–] [email protected] 1 points 1 year ago

You could post a magnet link here -- assuming that there's no rule prohibiting magnet links, that is.

[–] [email protected] -3 points 1 year ago

Last I heard with large projects like this, people usually upload them to DEEZ