Probably the most famous words of Jesus, from John 3, say that "whoever believes in [Jesus] will not perish". The word 'belief' here is from the Koine Greek root 'pistis' (1), which is also translated as 'faith' in its noun form. It absolutely carries connotations of trust, relationship, and fidelity. To the point where it is nonsense to have faith without personal consequences such as changed behavior (see James 1). So I believe your last definition fits the ideas of the Bible better.
Anyway, both Christians and non-Christians in the English speaking world tend to use the word 'faith' to fit your second definition, something more like wish/hope/belief/feeling. So I can understand atheist's offense when Christians say atheism and science are each expressions of faith, but I think a lot of it comes from the sort of 'linguistic gap' you talk about.
Anyway, it's a shame we don't more words to distinguish between these overloaded ideas: faith, facts, love, etc.
(1) https://www.blueletterbible.org/lang/lexicon/lexicon.cfm?str...
The word "faith", for example, probably means something completely different to a scientist who has "faith" (confidence) in a particular experimental method, versus a religious person who has "faith" (belief/hope) in God, versus a parent who has "faith" (trust) in a child.
A tool for discussion could help elucidate these differences in usage and bridge the linguistic gap that can form between groups.