SourceHut Disrupted by LLM Crawlers(github.com)

git.sr.ht is being disrupted by aggressive LLM crawlers. An aggressive botnet is scraping git.sr.ht to obtain data to train large language models. It has been able to circumvent the defenses we ordinarily deploy to mitigate these botnets, and as a consequence git.sr.ht is unstable. We have taken some aggressive measures to ensure some service remains available, but for the time being we have disabled a large number of routes for the web service to keep the load managable, which will cause disruptions to ordinary users.