TLS/HTTPS is slow and takes a lot of time to restore connection on fragile networks.
MTProto was born as a result of an experiment: whether it would be possible to create something that is both secure and fast.
Disclaimer: Unfortunately, I'm not Nikolay. But I do work for Telegram.
Sorry, you have to do better than "TLS/HTTPS is slow". Anything over TCP is slow on congested and lossy networks. That's given, but that's hardly a reason to reinvent the whole crypto stack.
How is your protocol superior / faster / better than DTLS with session resuming? Is it faster than IKE in Quick Mode or IKE v2, which are both datagram protocols designed to work over lossy networks? What are the inherent problems with adopting DTLS or IKE to your purposes? This is dead easy to do without touching original crypto design and it instantly removes all questions pertaining to the quality of your crypto design. So why not?
I'm not a professional cryptographer, but I know applied crypto well and I've seen my share of custom crypto designs. Virtually ALL of them are is a result of thinking that it's better and easier to invent something new than to diligently learn what exists and understand how it works. It's fine for some areas, but it is decidedly not a way to go in the cryptography domain.
I'd like to see some papers/surveys/case studies on the matter. Yes, it's an "experiment", doesn't mean it's a good idea to roll out your own implementation before performing (and publishing) extensive tests.
Not saying you're doing something wrong, just that security-oriented people (usually who'd be interested in your product) may want to know more than just "yes guys it's secure, trust us".
http://core.telegram.org/mtproto/TL-dependent
look like a great parody to me? can't tell if they are serious with these 'dependent types'
> TL Language (Type Language or Time Limit)
So it is a partially applied acronym with a branch in it!
There is something seriously messed up in this project.
The replay protection is overly complicated and doesn't kick in after the message is decrypted. This makes it possible to DoS the server with forged messages.
Key derivation uses a custom scheme. Typically there's no reason NOT to piggy-back on existing schemes and there's plenty to choose from - from TLS to IKE.
Also, as already mentioned, there's again NO reason not to use TLS in Anonymous DH mode with an app-level authentication of the session handshake.
Designing your own crypto protocols is a very interesting challenge, but for practical purposes you just have to recycle existing designs. There's really no other way about it. A custom crypto doesn't make any difference for those who doesn't know/care about it, but it certainly will not make you any friends between those who does. Unless, of course, you can explain and prove why your design is better than those that exist already, and these guys don't do this.
[1] http://core.telegram.org/mtproto/description