It seems you're thinking of the old Go dependency management.
The current Go dependency management, Modules, means that you put in the unique name of the Module (which is its URL) and the Semver-compatible version. Then go mod can resolve the exact code you need.
Sure, it's still source-code based, so you don't need to build a JAR file. Of course, that also means you can't have external dependencies that you pull in - you must have everything required to build your go module in that source code.
Go mod also checks version incompatibilities based on Semver and chooses the lowest specific version which matches everyone's Semver dependency specification.
Your source-control server must also know how to act as a go mod repository (it needs to respond to some go mod specific HTTP calls, as far as I could tell).
Now, when you want to publish a new version in Go, you don't build and publish a specific build to some extra repository. Instead, you need to tag some commit in your repo with a Semver version.
If you want to publish a new major version, you need to do much more than that, since any version higher than v1 will impact the name of your package in import statements in Go code (import "github.com/mymod/mypack" will become import "github.com/mymod/mypack/v2" in any code using your module, including internally).
But as a dev? I can drill down to the core (C-b in Goland, jump to definition) of ANY import. Even the entire go toolchain is in go.
ABI/version management of artifacts is a nightmare, every single time.
Using binary dependencies doesn't preclude having access to source code if desired.
Dynamic linking only became mainstream in the mid-90s.
One of the mayor benefits of maven/cargo repositories is that they are configured to be immutable. No issues with deleted repos or deleted/overwritten tags. Once your dependency is published it's there forever.
That said, Gradle has git source dependency support as well.
Obviously that comes at the cost of coupling with the VCS. Maybe its not worth it. Interesting idea though.
With Maven, you can define your own versioning scheme and easily include the branch as a component of the version "number".
In Go mod, as far as I can tell, you have to have a Semver vMAJOR.MINOR.PATCH version, which is much more difficult to adjust for short-lived branches.
I personally found Maven much more friendly, since there are no odd interactions with your source-control solution, you get all relevant details in the Maven pom.xml. I also find this idea of relying on semver, especially with Go's insistence on renaming packages for major version changes, to be very unpleasant and brittle, especially for internal packages.