Luu Tuyen@lemmy.world to Technology@lemmy.worldEnglish · 2 months agoTikTok’s parent launched a web scraper that’s gobbling up the world’s online data 25-times faster than OpenAIfortune.comexternal-linkmessage-square35fedilinkarrow-up12arrow-down10
arrow-up12arrow-down1external-linkTikTok’s parent launched a web scraper that’s gobbling up the world’s online data 25-times faster than OpenAIfortune.comLuu Tuyen@lemmy.world to Technology@lemmy.worldEnglish · 2 months agomessage-square35fedilink
minus-squareBlackEco@lemmy.blackeco.comlinkfedilinkEnglisharrow-up1·2 months agoAlso it doesn’t respect robots.txt (the file that tells bots whether or not a given page can be accessed) unlike most AI scrapping bots.
Also it doesn’t respect
robots.txt(the file that tells bots whether or not a given page can be accessed) unlike most AI scrapping bots.