Isn't that somewhat of a survivor bias? People with ADHD on stimulants would typically have little to no side-effects because if they had moderate to severe side-effects, then I doubt a medical professional would keep them on stimulants.
I treat my ADHD with stimulants, and I would say there are definitely side-effects, but the side-effects are mostly manageable.
While I do benefit from stimulants, I personally believe the benefits are exaggerated, and apparently the Cochrane Library does not disagree with me:
https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD...
I'm interested in knowing why you're reluctant to try stimulants. what you describe in your OP sounds a lot like the symptoms of ADHD and stimulant medication helps many people with the exact symptoms you list with little to no side effects. why leave that on the table?