In 1980, at the end of the Carter administration, the US was suffering from a bad recession that double-dipped in 1981. From 1982 onwards, the economy grew without a hitch until 1990-91. You could blame that recession on Reagan and Bush, but it didn't happen during the Reagan years.
And yeah, it was a booming almost-three-decades of growth between 1982 and 2008, with blips in 1991 and 2001.
* The FED has more direct influence and control (although the FED chair is appointed by the President, so I guess you can blame him indirectly)
* Congress is responsible for most domestic policies, even the ones championed by the President.
* Tons of economic factors are outside of the US government's direct control, especially when it comes to science and technology. (Some of these factors are even outside the control of any American actors: many of the recessions between 1970 and 1990 were at least partly caused by disruptions in the oil market.)
* Even factors that you can attribute to government actions are more long-term in focus: ARPANET was established in the Johnson administration and didn't yield macroeconomic gains until the Clinton administration, while most of the economic benefit of the Interstate Highway System was realized long after the Eisenhower administration.