Developers may implement other protocols in the future, but we are targeting HTTP as an accessible starting point.
We are definitely anticipating heavy mobile use, both through mobile web apps as well as native. There will eventually be iOS and Android frameworks to handle all of the communication with Tent servers.
As a mobile/desktop/server engineer, I would love the opportunity to work with other server-side teams that aren't wedded to the web/HTTP via historical accident and thus don't force us to use HTTP.
Do you have any suggestion that provides the same features, or should we forgo them because HTTP is "hefty"?
Message passing.
That's all HTTP really is, but it's dressed up in a bunch of historical complexity and inefficiency centered around supporting web browsers.
Load balancers know how to load balance straight TCP. HTTP caching servers are an HTTP-centric idea.
The 'libraries' you'll need can be much, much smaller when all you need is a bit of framing and serialization, instead of a complete complex RFC compliant HTTP client stack.
It's not writing the protocol that I find the most difficult. It's reimplementing everything the uses the protocol.
Load balancers know how to load balance straight TCP.
Which is only useful if all the nodes are exactly the same, but that prevents you from distributing the data across them based on the user profiles, and then load balance according to the user id, as (if I'm not mistaken) Netflix does. Since they're using subdomains as user identifiers, you'd get that for free using an existing, well-tested HTTP load balancer.
HTTP caching servers are an HTTP-centric idea.
That's a tautology. The question is: are they a useful idea? Is being able to take advantage of existing and deployed solutions like CDNs useful? Seems to me like it would be.
The 'libraries' you'll need can be much, much smaller when all you need is a bit of framing and serialization, instead of a complete complex RFC compliant HTTP client stack.
I think you underestimate the advantages that some of the core HTTP concepts provide.
I'm not sure what you think makes that complicated to implement without HTTP, or why you consider it 'free'. Netflix had to write custom code to support that, and could have just as easily done so on top of a message passing architecture ala ZeroMQ or even AMQP.
> That's a tautology. The question is: are they a useful idea? Is being able to take advantage of existing and deployed solutions like CDNs useful? Seems to me like it would be.
Not really, no -- neither a tautology nor are they particularly useful for API implementation. Their primary value is in caching resources for HTTP requests in a way that meshes well with the complexity of HTTP.
If you need geographically distributed resource distribution than HTTP may be a good idea simply because:
- There's widespread standardized support for HTTP resource distribution.
- Its inefficiencies are easily outweighed by the simple transit costs of a large file transfer.
We're largely talking about server "API", however.
> I think you underestimate the advantages that some of the core HTTP concepts provide.
No, the core concepts are more-or-less fine. It's the stack that's inefficient and grossly complex, largely due to browser constraints and historical limitations.
It's free because it already exists. Load balancers for hypothetical protocols don't.
Not really, no -- neither a tautology nor are they particularly useful for API implementation. Their primary value is in caching resources for HTTP requests in a way that meshes well with the complexity of HTTP.
If you need geographically distributed resource distribution than HTTP may be a good idea simply because:
- There's widespread standardized support for HTTP resource distribution.
- Its inefficiencies are easily outweighed by the simple transit costs of a large file transfer.
We're largely talking about server "API", however.
Isn't the whole point of this system to transfer people's content - posts, pictures, videos, etc - between servers? I would think pure API "calls" would be a small part of the whole traffic.
No, the core concepts are more-or-less fine. It's the stack that's inefficient and grossly complex, largely due to browser constraints and historical limitations.
But to implement them, you need more than "a bit of framing and serialization".
I posit you're still grossly overestimating complexity based on your own experience with HTTP, coupled with grossly underestimating the complexity, time costs, and efficiency costs of the stack HTTP weds you to.
A TCP stream is simple. It's as simple as it gets. Load balancing it requires a few hundred lines of code, at a maximum. It only gets complicated when you start layering on a protocol stack that is targeted at web browsers, grew over the past 20 years, requires all sorts of hoop-jumping for efficiency (keep-alive, websockets, long-polling), requires a slew of text parsing and escaping (percent-escapes, URL encoding, base64 HTTP basic auth, OAuth, ...), cookies, MIME parsing/encoding, so on and so forth.
All this complexity is targeted at web browsers, introduces significant inefficiencies, and requires huge libraries to make it accessible to application/server engineers.
What's the gain? Nothing other than familiarity, as evidenced by your belief that the core of what HTTP provides is so incredibly complicated, and you couldn't possibly replace it.
No -- it's the complexity of HTTP that's complicated, not the concepts that underly it. Drop the HTTP legacy and things get a heckuvalot simpler.
Furthermore, I think that even if the developers of this project could replace the required tools and forgo the rest, I doubt it'd make sense.
Frankly, you'd need a working prototype to convince me of the contrary, so I guess we'll have to leave it at that. I'm a stubborn man ;)
By the way, the AMQP spec is roughly the same size as the HTTP spec, and the latter spends a lot of pages listing just status codes.
And of course, AMQP uses a model based on Sessions, which is great if the components of the system are static, but not that great if you're talking to a lot of nodes who come and go, since you'll end up with uneven load distribution on your servers. Regardless of HTTP as a particular implementation, I think statelessness makes perfect sense in a unreliable network of nodes.
To test this, we implemented fallback-to-HTTPS behavior in a very widely used previously non-HTTP client. We then observed the number of clients that failed to connect via our custom protocol, but succeeded in falling back to HTTPS.
The numbers were negligible.
It's ridiculous that we'd seriously believe that we can't trust that TCP works on the internet. We joke about it being the "interweb", but I see no reason to sow fear, uncertainty, and doubt, and thus and actually turn the interweb into reality.
I don't see that we should model the internet architecture on bad technical choices made on a limited number of open wifi networks.
Or, we just frame our standard protocol over websockets as an (unfortunate) fallback, if it ever is revealed to be a real problem.
Given how many other things are broken by networks that foolishly only open port 80 and 443, and their (in my experience) relative rarity, I'd suggest that it's not worth bothering with, except possibly as a fall-back to measure the actual number of people trying to use your service behind such a network.
It's starting to sound like you've never used CORBA.
You really should also think about two separate protocols... one server-to-server, the other server-to-client.
At least, that's how I've been working on it :)
Our protocol has two distinct parts, one server-server, the other server-app. It's just easier to manage a single "API" than having some support server-server but not server-app--we're hoping to avoid ecosystem fragmentation into those who don't want to support apps or vice versa.
I also don't understand the concern over "inventing" a new binary protocol. It's not like it's any more complicated or difficult than "inventing" a non-binary protocol. The framing rules just don't use ASCII/UTF-8 field delimiters. There are plenty of existing encodings to use, no more difficult than JSON.
I'm not sure exactly what you mean by maintaining state, but if you mean long-lived connections: this already exists in HTTP with chunked-encoding and the new web socket extensions. Stateful protocols tend to be more difficult to write programs for and load balance.
Relying on HTTP, SSL, Ruby, JSON does not inspire confidence . When I see "Ruby 100%" on the github page for something that aims to move us forward out of centralisation, it makes me shudder. You need to think beyond the web, and get lower level (Tor hidden services, being outside the centralised web, are the right idea, but Tor has a certain stigma). You do need to get below the level of partisan languages- for a number of reasons, one of which is to make it open and easy for anyone to build on, not just web developers and people who know certain languages. That's a given. Get people connected (stateful) and all that will follow.
Something more like cjdns. It's the connection that's important, not the web or whatever else you chose to use the connection for. A webserver is one of many things that can be offered over a connection. I don't want to make the connection through an all-purpose application like a web browser, written by some company (Chrome in all its complex glory is trying to supplant your OS's DNS fer chrissakes, wake up- in the end, it's all about control). I want a very small and simple open source app that handles the connection to my peers and which works via the OS, not "the web". It makes the connection, keeps it alive and otherwise stays out of the way.
I think these kids need to go back to the drawing board. But anyone working on stuff like this should not give up!
Whatever the successor to centralised social networking is, it will largely be a matter of timing. The best solution might not prevail. Instead it might be the one that hits the news at exactly the right time and catches on for some unexplainable reason.
There will be more to come. That's a promise.
Tor hidden services, being outside the centralised web, are the right idea
But most Tor hidden services are as much as the web as normal web services! They talk in a server-client model using HTTP - that's the web, right there.
I don't want to make the connection through an all-purpose application like a web browser
Using the web doesn't mean using a web browser.
written by some company (Chrome in all its complex glory is trying to supplant your OS's DNS fer chrissakes, wake up- in the end, it's all about control).
...and it certainly doesn't mean using Chrome or other browser written by some big company; there's Amaya, Arora, Camino, Dillo, Dooble, ELinks, Flock, Galeon, GNU IceCat, K-Meleon, Konqueror, Links, Lynx, Midori, Firefox, SeaMonkey, Shiira, Uzbl, Luakit and more.
I want a very small and simple open source app that handles the connection to my peers and which works via the OS, not "the web". It makes the connection, keeps it alive and otherwise stays out of the way.
Which is completely feasible using HTTP - the web - as a core protocol.
But the point is the same - the control is concentrated in middlemen (e.g. DNS, Hosting - why do I need them?[1]) - and you articulated this correctly: client-server. Calf-cow. Not peer-to-peer. That's what I'm keen to get past.
1. That's my point about Tor's hidden services. You need Tor's help of course but those services are free from the need for DNS and Hosting - free from the middlemen that control "the web" (not to mention controlling email- does anyone send email using IP numbers anymore? the spam fighting fanatics think every revolves aroudn DNS and domain names- your mail might well get rejected because you lack a "domain name"). I must confess I've never actually used the hidden services. I only read the docs and source code.
"Using the web doesn't mean using a web browser"
Not sure what you mean here. You need to use HTTP. So you need an http client. Chances are you'll be fed heaps of html and other garbage. Parsing it is a PITA. And eventually, if you want to view tables and such, you'll be using a "browser".
Then there's the matter of state. So you're saying HTTP but not in a RESTful way? You just like HTTP headers, chunking and what not? HTTP is popular but it is not exactly unique. There are hundreds of other protocols in the RFC's, all of which would probably work just as well. HTTP is aimed at the client-server concept. That's fine. But it's a limited use for a network with so much potential. Something like a telephone conversation is not "client-server".
I'm talking about making connections that are application agnostic. Like Ethernet. If you're suggesting tunneling everything in HTTP I think that's unnecessary. There are ways to deal with firewalls. HTTP tunneling is a last resort.
So why HTTP? Why does it have to be at that layer? Why something that tied to specific applications that presume so much about what I want to do?
I want freedom from applications. I can write my own apps, thanks.
I want freedom to create new protocols, just for me and my friends. I want my own network, that we control. [This is possible using stuff that's been around for many years, and I have a working prototype. You folks are a bit too cynical to be beta testers, it's command line driven. Maybe someday.]
I've tried nearly all those "browser" options you mentioned, believe it or not. All except for one suck.
And if I had my way I'd extract the html table parser from it and have that as a standalone filter. I'd make it a UNIX utility.
The whole "browser" concept is outdated. People want to watch video, listen to audio, look at photos and read plain TEXT (with great fonts of course). I don't need html to do any of those things. And I don't need html or other HTTP junk to do search to find video, audio or images. I need a tcp (or other) connection and a video player/audio player/image viewer/typesetter, as the task requires.
All that said, hypertext is neat. But it's not world-changing. I can still do great research without "hyperlinks". Hypertext is the great benefit of HTTP. But at this stage, it is so weighed down with cruft and used in so many silly ways, making everything dependent on a monstrous abomination of a program called a "browser" (Firefox is freakin HUGE), it has become more of a burden than a benefit. It is a limitation, not a path to the future.
But the client-server is only for a single request, it's not a static property of a node. You can be both a client and a server simultaneously, making and accepting requests at all times.
Not sure what you mean here. You need to use HTTP. So you need an http client.
Yes.
Chances are you'll be fed heaps of html and other garbage.
No, HTML is definitively not necessary. Think of what it's commonly called "Web Services"; they're often only available using structured data encoding formats like JSON and they serve a whole lot of applications that are not web browsers: native mobile apps, for example.
HTML is just one of the many formats that can be transported by HTTP, nothing forces anyone to use it in order to use the web.
Then there's the matter of state. So you're saying HTTP but not in a RESTful way?
REST doesn't prevent state. It just prevents session/context state from being stored on the server. You can still store it on the client (which, again, would just be one of the roles of a node) and permanent state on both.
If you're suggesting tunneling everything in HTTP I think that's unnecessary.
If you mean tunneling as in SOAP, definitively not. I suggesting using HTTP as it's supposed to be used. Something similar would have to be reimplemented anyway, and HTTP is already there and supported by plenty of tools, services, etc.
Why does it have to be at that layer? Why applications? (...) I want freedom from applications.
I don't get what you're saying.
And if I had my way I'd extract the html table parser from it and have that as a standalone filter.
The whole "browser" concept is outdated. People want to watch video, listen to audio, look at photos and read TEXT. I don't need html to do any of those things. And I don't need html or other HTTP junk to do search to find video. I need a tcp (or other) connection and a video player.
Again, no HTML is needed. And no, a TCP connection and a video player is definitively not enough. You need some way to identify the video you want to watch, to decide what part of the video you want (e.g. if you already watched the first half), of knowing if that video still exists on the server, possibly some way to authenticate yourself (I don't want to share my personal videos with the world), and it would be nice if the video player could tell what formats it supports so that the server could send it the right version or say that none exists without having to download it.
HTTP is well known, ubiquitous, provides all that and it certainly doesn't need HTML for it.
For the same reason cars all want to drive on the same roads.
If firewalls were actually an issue, you could always run the service on port 443. I don't know how you'd tell anyone else that you're running on a non-standard port, but it would be possible. That's actually another potential problem... this is a server that is masquerading as an HTTPS server... I hope that it plays nice with non-Tent HTTP clients. It isn't nice to hijack ports like that.
They are seem to be assuming a web-based client, so that traffic (which does need to worry about firewalls), should pass though a firewall without issue.
Edit: parent got deleted, it was
One word: Firewalls
Another issue is that this assumes that the web will be the client of choice in the future... with mobile apps being as big as they are in the social space, this seems a bit shortsighted.
Don't get me wrong, I like the idea behind having a "social server", but I don't necessarily think that starting with HTTP is the way to go.
I don't have any particular argument with using JSON for data transfer though... I think that is probably a good choice. Also using SSL for all connections is probably a good call too.