At a guess you've seen the police with rifles at high value terrorist targets - airports, major stations, national tourist attractions/monuments. Such locations will have police with SMGs or assault rifles in most other places too, including places like the UK where police are rarely armed otherwise...
but I can imagine seeing everywhere policemen with big guns can be scary, luckily in my not so multicultural location in Europe it's still odd to see policemen carrying those uzi guns, usually you see just policemen with pistol or nothing at all
and while we are discussing guns, my biggest experience with guns presence in Eastern Asia was in Phillipines, where you can see security guards with shotguns at normal shops every few meters (at least in Manila, outside Manila it seemed better), that was quite crazy and also security guards checking me when going to subway obviously not for drugs, but for guns
This is still a thing in Delhi. You can see sand bags or security shields with some security personell hiding behind them all over the city.
Here in Delhi, you will be checked very thoroughly for guns and bombs whenever you enter a luxury hotel, shopping mall or the subway. That checking includes the combination of a doorframe metal detector, hand held metal detector and airport style baggage x ray scanner.
This one amused me, as an American who has been to India (though not Pune, so maybe the area was different in this sense), because while I don't remember what police carried (though I saw a fair number), I remember being shocked by the number of security guards I cam across (mostly in front of office buildings and when entering malls) who were carrying rifles (they looked a bit like AK47s/AKMs, though I don't know for sure if they were), whereas most security I've encountered in the US carries sidearms at most, and often no firearms at all.