Post

Brief update on our ongoing efforts to allow users to specify consent (or not) for AI training: 🧵

We recently shared Bluesky’s stance on user data and AI training, which has not changed. Bluesky will not train generative AI on user data. bsky.app/profile/bsky...

Bluesky is an open and public social network, much like websites on the Internet itself. Websites can specify whether they consent to outside companies crawling their data with a robots.txt file, and we’re investigating a similar practice here.

robots.txt is a thin barrier to prevent the whole site to be used generative AI, i could write a crawler in just a few hours that could completely bypass that. this feels more like PR than putting in serious safeguards to protect people’s data

i’m a supporter of open web ethos but there’s a huge gulf btw that and letting crawlers mine humongous amount of data unrestricted. they can set throttling limits, licenses, give the choice to users. many options even given their limited engineering capacity

This needs a legal solution, not tightening of access to data (again, to the extent that is realistically possible). Letting users license their own content (as a robots.txt model implies) seems like the exact right step. Pair it with terms for the firehose requiring adherence to the licenses.

i’m not sure what you mean by legal. this is something they have to figure out internally but giving the power back to users is a decent first step and at least shows a commitment to protecting people‘s data

Cloudflare allows blocking of all known AI crawlers while not impacting anything or anyone else

My assumption is they are using the firehose, not web scraping.

Possibly, but making it harder for these people, many of which aren't that technically inclined and don't think past simple things, is better than nothing