this post was submitted on 28 Nov 2024
960 points (99.3% liked)

Technology

60031 readers
2602 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 2 years ago
MODERATORS
 

Similar vibes than Reddit api pricing

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 7 points 3 weeks ago* (last edited 3 weeks ago) (1 children)

Yeah guys. Downvote me. This is literally part of my day job.

In our industry we call not using an official API a dumpster fire API. Because more than once it has completely broken eventually, and there are a few manufacturers warned will break in the upcoming future

[–] [email protected] 3 points 3 weeks ago (1 children)

Is it illegal to scrape? I mean most companies and governments scrape huge amounts of data all the time without getting anyones permission, hell the entire business of a search engine is basically WeScrape4U.

[–] [email protected] 6 points 3 weeks ago (1 children)

If you extract a key from a proprietary client to decrypt a private API, it definitely is illegal. If you do this as a corporation, it would also be easy to detect by abnormal usage patterns tied to your account and presents you as a nice big set of pockets to get sued.

Just screen scraping isn't illegal, but is fragile as fuck and will break at the worst time and you have no ability to implement version control to manage it.

[–] [email protected] 2 points 3 weeks ago

Personally I wouldn't want to be the guy forced to write scrapers with how much they break on any modern site