Library - a collection of useful things. Framework - a collection of things, and a methodology, used to structure your application.
In general, a lot of new, modern frameworks do 'call you' (Inversion of Control). Exmaple: Most PHP frameworks are setup so that you put all your models in one folder, your routes in another, etc. Then the framework itself accepts requests and determines which parts of your code to pass it through, and in what order.
However, you can also have frameworks that don't do IoC. You setup the framework, tell it what to do, tell it how to do it. But you still structure your code around that specific framework's methodology (which is why it's hard to mix 'n' match).
jQuery is an odd thing, because it's not really either of these. It provides lots of useful functions, but it also does so in a way that it replaces a lot of the 'core language features', and causes you to use it across most of your code.
However, I'd still say it's a library (with a very nice syntax) - since it doesn't tell you 'this is how you structure your application'. You can use jQuery how ever you like, and it imposes almost no constraints. Therefore it's not a framework.
This approach gives the advantage of ceding ultimate control back to the programmer, at the cost of some internal inconsistency.
To me a tool is an extension of one's mind or body that goes into all projects that you work on. I can give away an application or a library. I can't give away a tool, or at least, it wouldn't really make sense to. I want to use all my tools to build a project, all my projects to use the same tools. That's why I stick to Ruby rather than bouncing around. I use Coffeescript/jQuery on web projects but I consider it a kludge, though not as much of a kludge as using Opal would be. I'd want Ruby integrated tightly into the browser before I'd prefer it over Javascript. I don't mind Coffeescript because I find that if you stick to a certain convention, Coffeescript is as easy to maintain as Javascript and I prefer the terser syntax.
My ultimate goal would be to be able to hack on all my tools like I hack applications. For example, I'd love to be able to replace Sublime Text with redcar, an editor written in Ruby, I've taken baby steps in that direction with my git scripts. I haven't decided if it would be better to write my own editor though. I'm probably at least a few years from that being able to integrate that into my workflow.
I find myself refactoring code a lot, pulling pieces out and putting them elsewhere. Sometimes I think, "this piece would be better as a gem", or that "this gem needs to be its own application.
When I think about the overall thrust of what I'm doing, I get the sense that I'm building a framework. I need to be able to work web applications into the framework, and the only framework worth really using in this fashion is Rails. But Rails isn't intended to be included as you would a library. I need to understand it better if I'm going to be able to work it into my own framework.