Introduction to Algorithms by Cormen is a great. It is set up as a sort of reference and isnt necessarily meant to be read cover to cover. However, you'll need to read/understand the first 4 or so chapters (on asymptotic complexity, recursion, log rules, master theorem, ect) to really be able to grasp anything else in the book. The proofs and explanations can be rather terse, as it assumes you have some working knowledge with algorithmic theory and math. I wouldn't recommend it for starting out _at all_ as it can be very intimidating at first.

I started out with Algorithms by Tardos (you can pretty easily find a copy of the book and solutions online somewhere). It is less terse and goes through all the basics in a much less formal format. In this book I would advise starting at chapter 1 and going straight through to 8 or 9, doing the exercises and programming up most of the algorithms on your way. At this point you'll have a really nice working knowledge and can dive into Cormen and tackle stuff that interests you.

Check these books:

http://www.amazon.com/The-Algorithm-Design-Manual-ebook/dp/B...

http://www.amazon.com/Programming-Pearls-2nd-Jon-Bentley/dp/...

Prefer Python ?

http://www.amazon.com/Python-Algorithms-Mastering-Language-E...

http://www.amazon.com/Structures-Algorithms-Python-Michael-G...

I've been taking an algorithms course on Coursera and I'm really enjoying it. It's taught by two professors from Princeton, Kevin Wayne and Robert Sedgewick. Heres the link:

https://www.coursera.org/course/algs4partI

I'm sure there are plenty of other online algorithms courses online too.

Art of Computer Programming by Knuth

Introduction to Algorithms by Cormen

I like http://rosettacode.org/wiki/Category:Programming_Tasks

What kind of algorithms?

Whatever the answer, check out Numerical Recipes (http://www.nr.com/).

I was thinking of something that will explain, maybe step-by-step, the implementation of various algorithms (like these, for example, http://en.wikipedia.org/wiki/Sorting_algorithm), so that I could understand the logic behind them.

How about taking a look at some sample pages? I'd be interested in knowing if it's along the lines of what you're looking for.

http://apps.nrbook.com/rollover/index.html

wikipedia has pages for all the popular sorting algorithms, most with pseudo-code and graphics too.

http://en.wikipedia.org/wiki/Merge_sort

http://en.wikipedia.org/wiki/Heapsort

http://en.wikipedia.org/wiki/Quicksort

Something like this http://bigocheatsheet.com , but more detailed.

How do people view Udacity's algorithms subject?