Wouldn't the token-finding script be even more of a risk?

If the token is XYZ and the script is searching https://github.com/search?utf8=%E2%9C%93&q=XYZ&type=Commits&...:

1. It's sharing the token with GitHub.

2. It's embedding the token as query-string parameter in a GET request, which is much more likely to be logged (than sending it as data in a POST request), and more likely to be available to less-privileged/less-trusted staff.

3. If the request is sent to a non-HTTPS endpoint, the query can be MITMd, revealing the token.

I'd be very wary of setting up a token-finding script, it feels like it adds more risk than it saves.

You can scrape the issues without exposing the token. You could probably do it by just subscribing to all of them and parsing emails. No one(especially in security) should be using a third party search to match sensitive data. It's like searching Google for your social security number.

It was a pattern based script, all the tokens had the same length.

You just search that some token was uploaded by your people, not specifically yours.

Maybe they search the tokens public key and not the token itself. Then if the public key is found, then they download the repo and do scanning for the private key.

A group at BigCorp Inc. was sharing a tool they'd written to ThirdParty Ltd. As part of this, they transferred documentation, including how to configure the tool. Including an example. With a real AWS key. For a dev too, so the key had no restrictions.