They wouldn't use pure ice. But in cold places, with ice mixed with some other materials you can actually make quite good materials. Consider that in most places gravity is much lower then on earth so it doesn't need to be carbon fiber to be useful.
Yes things still have mass but if you are building a robot that moves around there is a big difference in what kind of quality structural materials you need for the robot to be viable.
Part of the research that would go into such project would be to look at what local resources are, and how to make them into useful materials. For example, using ice in combination with some filler material has been shown to be quite usable in cold temperatures.
The exact materials you would use depend on where you would want to use this kind of system. Maybe in the far future these kind of system would look around to analyses the environment and make smart choices about what materials to use to build themselves.