There are a lot of possibilities as to why it's failing. For debugging error codes, it is easy - you can attach something to your error handling code, or in some frameworks you can just install a module/gem/etc that sends your error data off to a service like NewRelic or Opbeat (there's tons of these providers). Using these kinds of things you can understand not just the stack traces of errors but sometimes also the data the user submitted. You can also set up your own infrastructure that does the same, of course.

For timeouts, it may be tougher to do. If you use some kind of reverse proxy to spread the load to instances of your API, then this might be a nice place to start investigating your problems, and there is where you're going to want some tooling. If you're using Node.js, it could be possible the code for a specific endpoint does not send a response (maybe only under certain circumstances) - so you will need to debug that particular aspect of it and ensure that a response is sent, otherwise timeouts will happen.

As for actually sending test requests to your API, the difficultly with this method is that if, for example, you're posting a new user and your tool posts once every 5 mins, then you will have 12 garbage users after an hour. Not to mention that you will need to write a test for each error possibility as well, since it could be that a validation error is causing the timeout. So for this it probably makes sense to write a test suite, and include a test for each of the possible scenarios, or paths through your code.