There probably aren't many languages where that's different, whether a hashset on a hashmap or a treeset on a treemap.

However sets tend to provide support for… set operations. Which currently can't be done generically in go except through interface{} or specialised, neither of which is a great option.