Buybacks increase the share price because you have a company that is worth (for sake of argument) the same as it was worth before, except now there are fewer shares available.
A fixed market cap divided by fewer shares equals a higher share price.
In the limit case imagine buying back all but 1 share. Now that 1 share represents the entire value of the company, so the share price would equal the market cap.
Markets tend to reward companies that use buybacks as there is a belief the buybacks are a demonstration of discipline by the management team. Conceptually COMPANIES SHOULD BUY BACK STOCK IF THEY DO NOT HAVE BETTER ROI PROJECTS IN THE PIPELINE. This frequently happens in mature industries.
As noted above, buybacks are another means to return cash to investors. Today, in the US, the tax rate on qualified dividends and long-term capital gains are equivalent for most shareholders. This has not always been the case. When tax rates for capital gains are lower than dividends, buybacks are a more efficient means to return capital to investors.
Buybacks also allow for more tax planning. When dividends are issued, the investors have to pay taxes on them at that time. Stock buybacks allow investors to choose when they want to pay taxes. They can sell into the buyback and pay taxes now or hold the stock and pay taxes at a later time.
Buybacks are can be part of normal corporate capitalization decisions - what is the appropriate debt to equity ratio for the company.
Finally, changing dividend levels has its own impact on stock price. If a company increases its dividend, them market expects it to remain increased. In this case the stock price goes up as investors expect more dividends in the future. When a company cuts its dividend (rare event), the stock price drops dramatically as the market punishes company not only for the reduced expectation of future dividends but also because companies only cut dividends when they are having severe problems. Some companines issue a special dividend related to a one-time event such as selling a division. The stock price does not do much in these events.
All of this is to say that stock buybacks are not why corporations reduced basic research investment. I was at GE watching the famous research centers getting cut. The bottom line was the research coming out of the centers was not creating a meaningful ROI. At one point the researchers went to the various GE businesses looking for projects where their expertise could add value - an internal consulting group. They gave up after a year as there was so little success. Corporate research centers are expensive. They need to earn their keep.
If the buybacks are at a discount to whatever the stock turns out to have been worth at the time, then that benefits all the shareholders. That can be a great use of money for all shareholders.
But buybacks at inflated prices benefit only exiting shareholders. Exiting shareholders tend to include hired management. Of course nobody really knows the valuation that well, so obviously there's a guessing game.
This is pretty hard to argue against for anybody who agrees that valuation is a thing at all.