Much like the Y combinator comes out somewhat unexpectedly from the pure lambda calculus, my suspicion is that any sufficiently rich data-construction language will allow, intentionally or unintentionally, for some sort of self-reference.
The acyclic heap is a great property for most data but it's annoying for representing programs. It might be reasonable to restrict letrec to creating functions, and ensure they're all statically allocated, which would give an acyclic data heap with direct representation of functions.
It does mean you can't have an anonymous function which recurses, but that's a relatively rare use case. And actually, if you introduce some kind of special syntax which refers to "this lambda", you can even permit direct recursion of anonymous functions. Mutually recursive anonymous functions would be harder to support–maybe not impossible to support, but such an obscure use case it probably isn't worth thinking about.
In order to print this #1=(... #1#) you have to determine that the given object occurs inside itself. In order to print (#1=(whatever) #1#), you have to determine that the given object occurs later in a sibling expression. Knowing that #1# will not occur inside (whatever) doesn't help all that much with anything; it doesn't help you decide whether or not the #1= prefix needs to be printed.
If anything if the restrictions were reversed, it would be simpler: if the only substructure sharing that were allowed was backward references, that helps. Because then we know that the (whatever) object can only occur again inside (whatever), and not after it. To detect nothing but cycles, without caring about other sharing, we just have to maintain an object path across the recursion so that for every visited object we can ask "is this object already in the path from root to here?" If so we have a backreference.
The sibling case is harder. You can have situations like:
((((#1=(foo)))) (((#1#))))
Deep inside the left item in the list there is an object which occurs again deep in the right item. We need to put the objects we are visiting into a hash, which is pretty expensive.> Just by making sure your primitives for constructing values don't permit cycles to be constructed.
But lambda calculus teaches us that if all we have is functions being passed to functions and binding arguments we can build a circular reference. See the middle word of the domain in the browser address bar. So you have to further dumb things down from there if you want to absolutely eliminate cycles in the object graph.
(You can have it so that you don't care about function-induced cycles (only the garbage collector does) by not having any printed notation or other API that traverses the internals of a function. The application can still build a graph using functions, but is "on its own" as far as providing its own functions for operating on it, and serializing it and whatnot.)
If a language has only immutable data structures, it is easy to make it so that self-reference becomes impossible, just by making sure your primitives for constructing values don't permit cycles to be constructed.
Even if you allow mutable data structures, it is still possible. For mutable compound heap objects (i.e. heap objects which possibly container pointers to other heap objects), they can have a back-pointer pointing to their parent. If you think about it, you can flesh out some rules about how to manipulate these objects which will make cyclic references impossible. When the rules are violated, rather than failing the operation, you could just add a pointer to a deep copy instead. (Not completely deep, you can still share rather than copy non-compound heap objects, such as strings–sharing non-compound objects can't produce cycles.)
Banning direct self-reference doesn't mean nothing can ever refer to itself, just not directly. So, you couldn't have #1='(employee :name "Frank Smith" :supervisor #1#). But you could have something like '(employee :id 1 :name "Frank Smith" :supervisor (ref :to employee :id 1)). At a physical level there is no self-reference, but at a logical level there still is.
Similarly, one would want to allow cyclic references via a symbol, so a symbol can occur in its own binding. So not completely banning self-reference, just limiting it, in the hope that limiting it will limit the issues it causes. I think, using the strategies I've outlined above, reference counting will work as a garbage collection strategy, without the possibility of memory leaks from cyclic references. (Allowing a symbol's binding to refer to itself will not leak memory, since a bound symbol should still exist even if nobody refers to it; to deallocate it, you have to explicitly unbind it, which will remove the binding, breaking the self-reference and allowing the reference count to fall to zero.)