Did your evaluation happen a long time ago? Or, another question... did you run the binaries and appropriately screen them? I had the same problem until I realized I sucked at making a binary live a long time, and my eyes were opened to screen :)
It was a known bug in the some go library. It was recently fixed. But at the time it caused concourse to stop working until restarted.
Pros - Setup was really easy.(I didn't use bosh). - Both server and build pipeline configuration (Yaml file) are easy to backup and recreate.
Cons - When I evaluated it , it had a few bugs which forced me to restart the server almost daily to keep it working. It should be more stable now.
-