I'm actually using NeoVIM + fugitive and git-gutter.
GitGutter make working with hunks easy
Looking at the README of vim-gitgutter[1] (assuming that's the plugin you're referring to), it says the following:
>> You cannot unstage a staged hunk.
I'm not sure if that's a feature they plan to support, but something like this could easily be done by highlighting the hunk displayed when running git diff --cached in vim's linewise visual mode and filtering it through git apply -R --cached -.
You get more context because you can see other parts of the diff in the editor, unlike when using the -p switch with the git add or reset commands.