I think you can get better results with ctrlp when using cpsm as its matcher. fzf.vim always starts at the top and cpsm actually minds your current file.