SourceBot

1 Oct 2023

Hi! I am a bot. Well, more of a bot project. Initially I will be used by SourceBot (a Discord image-sourcing bot) to find images and show them in their best quality to users, but eventually I will be able to syndicate BlueSky messages onto Discord as they come out! excessive.space

Not many posts on this account. Let's fix that! SourceBot has been trundling along for a while now with nothing new added. That's cool, I don't charge for it anymore. But with Derpibooru being a little wobbly of late, I wanted another good reverse searching service to take up the slack.

Using a chunk of JS as its own worker, making an RPC call to it and getting the info back! This includes a lot of head-smacking-against-brick-wall in the early attempts to make this work, but ultimately I tossed all the ideas out and implemented the final. github.com/SomewhatDama...

SourceBot now just makes a call to this, giving it the URL. Cloudflare handles the image resize/sending, then passes back a list of URLs that have the image. Success never tasted so good. Huge thanks to @fluffle.xyz again. Great service, send love to them!

1 is easy. Cloudflare has a nice little js.fetch() function that can easily gobble a URL and spit out image blob. 2 was actually integrated into 1. Cloudflare allows you to resize images as you ingest them. Very nice! 3 ...

3 is where the rub lies. Fluffle wants a specific user agent layout, Cloudflare doesn't want to allow me to use such a user agent via any of the usual Python utils (httpx, aiohttp, etc), and the requirement for uploading a blob of data means js.fetch() (in Python) doesn't work. What do?

Enter @fluffle.xyz (oooh, they have a Bluesky!). Great little reverse searching service, and they have an API. Very nice. A small downside is that SourceBot only works with URLs, not blobs of image data. Now, I could do some really horrible bodge-coded solution to do it in SourceBot, but...

Let's leverage @cloudflare.social! So, to make an API endpoint to take an image URL and spit back site URLs, I need to do 3 things: 1. I need to download the image 2. I need to ensure it is smol (Fluffle only likes 4MB or less) 3. I need to do the Fluffle query

13 June

So, serious pants time right now. Privacy Policy! I apparently needed one, since SourceBot can connect to over 10,000 people on @discord.com now. Whew, really putting me on the spot here, since SourceBot collects so much... of nothing. No personal data. Ever. excessive.space/policies/pri...

Okay, so I got this working and it runs nicely. But people don't want to deal with database checks and updates and ugh. So I have taken that effort out! There is now an API endpoint to plug in 64char hashes and check to see if they match. api.excessive.space/v1/hashcompa...

SourceBot@sourcebot.excessive.space

Bonus: Made a github repo with the images. github.com/SomewhatDama...

This is the hash for the test image, with a tiny change made. It will take a tiny bit longer (because it has to do a hamming compare) to return a 200 and {"result": true} api.excessive.space/v1/hashcompa...

Happy Monday, and good luck fighting the good fight!

SourceBot@sourcebot.excessive.space

Bonus: Made a github repo with the images. github.com/SomewhatDama...

This is the hash for the test image. It will return a 200 and {"result": true} api.excessive.space/v1/hashcompa...

This is the same hash with enough changed to make it fail. It will return a 404 and {"result": false} api.excessive.space/v1/hashcompa...

Scams suck. If you're a dev on a chat platform, and you're seeing a pile of them, I can recommend a little project! Get yourself some free time on a service (Cloudflare workers will do). Find a pHashing library for your language of choice.

Resources! Python pHashing: pypi.org/project/Imag... List of images: cdn.excessive.space/Scam%20Image... (password is the "Scam Images") Please feel free to email me more scam images if you find some, and I will add them to the list: scam-catcher@excessive.space

Bonus: Made a github repo with the images. github.com/SomewhatDama...

Scan all images and compare hashes to a list of known scam images. Use Hamming distance of around 4 or so to survive JPG compression. Warn/ban/mute/whatever folks who post those images.

6 Jan 2025

Whew. Huge thanks to The AT Protocol SDK Python team with some questions to implement this, but now SauceBot can detect if a post is NSFW (and will hide the image in Discord if so), can put proper links for hashtags, URLs, and user links. Oh, and can also show embeded URLs as if they were an image.

6 Jan 2025

So, here I am, trying to test my Bluesky->Discord bot's label handling, and all I can find is things tagged porn. Okay, great to see it handles that fine, but I need to test all the tags. So, could I ask that Bluesky mods ***not*** change the tags of any of the replies to this post? KKTHX.

Figured I'd give a shout-out to all the services/tools I used to get this thing going. 🧵

Digital Ocean: digitalocean.com A good hosting partner who provides all the tools at a reasonable price. Jetstream: github.com/bluesky-soci... A compact and filterable stream of commits and actions on AT Protocol's firehose.

And Python itself, for being the best dang glue for connecting all these (and more) APIs together. 🥂

Figured I'd give a shout-out to all the services/tools I used to get this thing going. 🧵

First and foremost, Red: github.com/Cog-Creators... A great framework to build a rich and stable Discord bot around. Discord.Py: discordpy.readthedocs.io/en/stable/ Always been the most flexible Discord API interface, hands down.

The AT Protocol SDK for Python: atproto.blue/en/latest/ Powerful enough to do everything I need. AIOHTTP: docs.aiohttp.org/en/stable/ Great, async web server/client API. ASYNCPG: magicstack.github.io/asyncpg/curr... Async Postgres API.

27 Dec 2024

This is a test post intended to ensure that my feed parsing is working. Expect a few of these.

27 Dec 2024

Well I'll be. It worked.