Shifting the responsibility to stay safe from complex attacks to the user will never work for a tech company, even one with the resources available to Google. If Google Assistant is a reliable way of scamming grandmas it will fail because there'll be constant negative news about it in the press.