News for hackers. Stuff that matters.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by walrus01 on Wednesday May 02, 2018 @04:19PM (#16980800)
Actual ISP network engineer here: I'm sure these people are having fun and all, but it's insignificant in the larger scheme of things.

(correction to original post: I did not at first see an ASN for them, or announcements of their own IP space. They have an ASN and announce two /24s, and peer at one physical location in metro NYC).

Real five nines network infrastructure takes significant capital investment.

This part is so much unadulerated bullshit:

""The internet doesn't really cost you anything, it's just the connection [that has a fee]. So however you can get plugged in — then you're on the internet. Nobody owns the internet, there's no one to pay.""

Actual, reliable internet costs real money, both in the salaries of people to engineer and architect it, salaries for 24x7x365 clued-in NOC staff, equipment, salaries and expenses for field technicians to build it. And that's before you get into things like establishing colo at major IX points, serious core routers that cost $15k each (do you really want to deploy something in the year 2018 to take several full BGP tables that doesn't have a 4 million FIB capacity?), etc. The Internet is a significant construction project at OSI layer 1, whether you're putting PTP radios on rooftops, running aerial fiber, or underground fiber. Otherwise you're just piggybacking on something that another, larger entity has already built.

I give them an A for enthusiasm and effort. It just needs to be channeled the right way so that they can figure out what it actually costs to run a reliable ISP. I'm all in favor of new startup ISPs.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by nycmesh on Wednesday May 02, 2018 @04:30PM (#16980879)
Our ASN is AS395853. Our peering info is here- https://nycmesh.net/peering/ We have v4 and v6 space.

Our uptime when connecting to supernode1 is significantly better than Time Warner Cable as many of our members have reported.

We are already established. This is something we have already done! It isn't impossible.

Our IXP is DE-CIX, the worlds biggest IXP. We are currently in one data-center/IXP and talking to a few more colocs.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by bogomipz on Wednesday May 02, 2018 @11:29PM (#16983149)
>"Our IXP is DE-CIX, the worlds biggest IXP."

Yep, DE-CIX is great and 32 Ave of the Americas is one of the most wired buildings in NYC. Great choice. Kudos on you folks, this is a great initiative. Reading this article made me happy. And I'm glad you are getting some press!

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by quiet_obrserver on Wednesday May 02, 2018 @05:44PM (#16981419)
So what's the Wifi NAME in Manhattan?
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by nycmesh on Wednesday May 02, 2018 @06:04PM (#16981552)
The public access point SSIDs are usually "-NYC Mesh-" or "nycmesh" followed by location.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by pietroglyph on Wednesday May 02, 2018 @08:31PM (#16982364)
How do you deal with malicious networks broadcasting under the same name? Do you use WPA2, or is data sent naked over the air?
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by acct1771 on Thursday May 03, 2018 @02:35AM (#16983813)
How do you deal with someone making a Facebook with your first and last name?

Same answer.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by oliv__ on Wednesday May 02, 2018 @09:28PM (#16982648)
Inb4 everyone renames their networks to nycmesh-something and MITM unknowing crowds
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by gcb0 on Wednesday May 02, 2018 @10:19PM (#16982883)
why would you trust random people more or less than other random people? That's crazy talk.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by samstave on Wednesday May 02, 2018 @06:25PM (#16981698)
Are you using Ubiquity HW? If not, why?

They are cheap as hell and awesome.

Actually - what you should do, is have a customer pay a fee of $100 to connect to the system - and take that $100 and buy another mesh was with each customer on boarding.

https://unifi-mesh.ubnt.com/#products

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by mplewis on Wednesday May 02, 2018 @06:59PM (#16981894)
I think you misunderstand the goals of this community-run mesh. The FAQ has some info on why they don't charge, who runs routers, and what they're trying to accomplish.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by walrus01 on Wednesday May 02, 2018 @04:35PM (#16980912)
Digi Desert LLC, which existed in 2010? Looks like that's the AS of some entity that predates the existence of your project and current public relations effort.

You announce a whopping total of two v4 /24.

Kudos for having enough clue to know that you needed to establish a presence at a major IX, and actually doing it, because it looks like you're adding peers. But your actual network presence is minuscule.

How do you intend to compete with the six NYC based companies I can think of off the top of my head that are putting fiber fed, $9,000 to $20,000 5 to 10Gbps 71-86 GHz PTP links on rooftops to build their own backbones, when you're playing around with 1000BaseT to the roof and AF24s?

You have high uptime? Do you have any of the following, because some of your much larger competitors sure do:

sites with parallel A and B side power systems

-48VDC rectifier + battery systems sized for 24 hour runtime at load

diesel generators

propane generators

generator resupply contracts

chassis-based routers with hotswap fan, N+1 power supplies, dual redundant routing engines

pair of identical core routers

singlemode fiber to the roof

-48vdc power to the roof

ironclad rooftop lease agreements with building owners, drawn up by professional telecom/real estate lawyers that run for 5+ year terms

a 24x7x365 NOC staffed by live humans

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by Someone1234 on Wednesday May 02, 2018 @04:45PM (#16980992)
Your original post was interesting because it raised legitimate questions about their operation. They answered those questions.

This follow up comes across as though you've drawn a conclusion and are now arguing towards that predetermined conclusion while ignoring the additional information provided. Plus your argument has kind of devolved from talking about specific concerns to throwing criticisms at the wall to see what sticks.

You don't seem to be arguing in good faith.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by walrus01 on Wednesday May 02, 2018 @04:48PM (#16981017)
I'm sharing some harsh reality with them: The ISP market in NYC is highly competitive. If they want to be serious about it, it's not going to work as a nonprofit. I'm trying to tell them bluntly about what sort of infrastructure their competitors operate so that they can get an idea of the actual capital expenditure requirements involved in architecting/engineering a MAN-scale, five nines ISP composed of point to point wireless links.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by ixiebosoeb on Wednesday May 02, 2018 @05:33PM (#16981340)
Once upon a time there was a company called Microsoft. They hated free software! It costs millions to write an OS! Why is it free?? They hated the free guys so much. Then the free software got better. It got so much better than what you would have to pay for from Microsoft! Eventually Microsoft said oh well, and built the free software into their product. They even contributed back to that free software and everyone learned that you can have free and paid and as long as we all contribute the world is a better place. Thank you.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by neuromantik8086 on Wednesday May 02, 2018 @06:11PM (#16981604)
Once upon a time there was a chipmaker named Intel. Sun decided to open source its SPARC architecture. Other folks tried to design their own FOSS CPU architectures. Intel still dominates.

FOSS zealots like to point to the success of Linux over Windows (which is restricted to the server market, I might add), but there's little evidence that the FOSS philosophy is effective when it comes to physical infrastructure. Software requires bytes and labor. Bytes are cheap and labor can be donated or paid for by companies. Hardware requires fabs and factories, which are expensive.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by master-litty on Wednesday May 02, 2018 @05:33PM (#16981343)
The reality is harsh but the discussion doesn't have to be.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by stale2002 on Wednesday May 02, 2018 @08:08PM (#16982245)
Well it seems to be working fine for them right now.

So I guess the points you brought up don't matter.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by nycmesh on Wednesday May 02, 2018 @04:44PM (#16980981)
I don't think you get what we are doing at all! Anyway we are in 375 Pearl with a lease and it has most of your long list, diesel generators etc.

We compete on price (donation only)

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by vermontdevil on Wednesday May 02, 2018 @05:29PM (#16981306)
Keep up the good work. Love these kind of projects!
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by Dowwie on Wednesday May 02, 2018 @07:17PM (#16981993)
Hello from just across the Hudson River! Keep up the great work. You are finding a way, despite what doubters and naysayers think! You will know what is possible by trying.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by nycmesh on Wednesday May 02, 2018 @04:37PM (#16980928)
https://www.peeringdb.com/asn/395853 NYC Mesh

What is your problem??

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by walrus01 on Wednesday May 02, 2018 @04:44PM (#16980985)
You're wasting your own time and those of your "customers" unless you have at least $750,000 to establish a serious presence, in my opinion. Amateur-hour WISP stuff is fine in a rural area. You're pretending to be a highly reliable capable ISP, and will eventually either overextend yourselves, run out of rubes to fund you as a nonprofit, or once you reach a size much bigger than you are now, get stomped on and obliterated by a much larger competitor that overbuilds your entire network with carrier-grade infrastructure and takes all your revenue.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by nycmesh on Wednesday May 02, 2018 @04:47PM (#16981013)
We are already self funded by individual donations. Every question I answer seems to bring up new objections.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by JustSomeNobody on Wednesday May 02, 2018 @04:50PM (#16981036)
This thread has to be one of the most overtly hostile threads I've read on HN in a while. What is your concern here? Why do you seem to feel they shouldn't even be trying?

What ISP are you affiliated with?

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by nycmesh on Wednesday May 02, 2018 @06:35PM (#16981751)
We bypass traditional ISPs by connecting directly at an internet exchange point (DE-CIX IXP) and peering with other networks. This is basically how the internet is formed, by networks peering with each other. We don't need this ISP layer.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by untog on Wednesday May 02, 2018 @06:47PM (#16981827)
I think the OP was asking what ISP walrus1 works for, given they've already identified themselves as a network engineer.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by JustSomeNobody on Wednesday May 02, 2018 @08:16PM (#16982285)
I was referring to walrus1. Sorry for the confusion. Y'all keep doing what you're doing. I have big hopes for efforts like yours.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by nycmesh on Wednesday May 02, 2018 @09:34PM (#16982672)
Thanks!
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by quxbar on Wednesday May 02, 2018 @04:50PM (#16981028)
I'm pretty sure the real secret is ensuring your customers have no alternatives. Spectrum may or may not have all of that gear and they still have given me outage and throttling issues for years. I don't want somebody to compete with the terrible ISPs out there today - they're fundamentally the wrong kind of organization. I'd rather have honestly crappy service than an opaque monopoly dictating my access.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by jimktrains2 on Wednesday May 02, 2018 @04:54PM (#16981078)
> Actual ISP network engineer her

So then you must intimately understand the depth of the lies and anti-consumer business practices that spur such networks.

> Real five nines network infrastructure takes significant capital investment.

5-9s is 5 minutes of downtime a year. I've never had a commercial, let alone residential ISP hit that mark.

> Actual, reliable internet costs real money, both in the salaries of people to engineer and architect it, salaries for 24x7x365 clued-in NOC staff, equipment, salaries and expenses for field technicians to build it. ...

Going back to my first point, almost no one is unwilling to pay for their internet connection.

They're unwilling to pay less to get TV service they have no use for and internet than they are internet alone.

They're unwilling to look at the cost of network access in other countries and be happy when comparing that to the cost of internet in a major, densely populated area in the US.

They're unwilling to accept transfer caps and accept the lies they they help the network by making people share when they know that transfer is irrelevant and bandwidth is the limited commodity.

They're unwilling to accept lies of bandwidth speed that isn't attainable even under the best of conditions.

They're unwilling to accept shady and underhanded business practices such as random, "accidental" charges.

> It just needs to be channeled the right way so that they can figure out what it actually costs to run a reliable ISP. I'm all in favor of new startup ISPs.

However, as an "actual ISP network engineer", you must understand the (very arguably unfair) regulatory issues that the big players have put in place to make this impossible to do. Google failed at starting an ISP, and it's not like they are tight on either cash or lawyers.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by Spooky23 on Wednesday May 02, 2018 @05:24PM (#16981278)
It’s pretty hilarious to hear a cable network engineer get on a soapbox about reliability.

As an enterprise customer, I only get 99% commitment, and frequently get credits. They’ve improved it in the last 5 years, but our whole region had an outage when the single fiber connecting to their core routers was cut by a house fire.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by adrianN on Thursday May 03, 2018 @12:47AM (#16983416)
I'm no network engineer, but when you have a large network with lots of equipment, you want the individual components to have a couple more nines than the resulting network. Maybe that's what they meant.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by walrus01 on Wednesday May 02, 2018 @05:00PM (#16981118)
Google did not fail at starting an ISP. They decided that the ROI for doing individual residential home FTTH was not worth it, considering the cost per house passed. Then they went and bought Webpass, which has a totally different condo/apartment based business model, which has been wildly successful.

Starting a small ISP based out of extreme dissatisfaction with huge, evil incumbents is something that I encourage any plucky upstart to do. If you don't like Comcast, Charter, RCN, etc, if you have sufficient clue, go do it yourself. Just don't try to do it with $20,000 and some good wishes, because something with much deeper pockets like Webpass will come along and obliterate you.

The point that I am rather bluntly trying to get across to these folks is that if they're going to invest significant amounts of their personal time/engineering resources into it (unpaid labor?), that they should treat it like a serious business and not a nonprofit. Raise sufficient funds, somewhere in the $500,000+ range to start, to build a network that actually does meet five nines through sufficient diversity in topology and routing.

Compared to random YC companies that seem to raise $1m to $5m to do some vaguely defined web2.0 SAAS thing, the amount of money that a group of people need to raise to build the core of a serious ISP is not huge. Elsewhere in this thread I threw out a figure like $750,000 in beginning CapEx if they're serious. For some good reasons, however, it's easier to get $5m of series A funding from a VC for a pure software development business than a wholly-owned-facilities-based startup ISP.

Even if you completely ignore things like colocation costs, IP transit costs, IX/peering costs, office rental, office utilities and overhead, running a serious ISP with live, clued in humans watching the network takes a lot of money per year for salaries.

3 to 5 NOC persons for 24x7x365 NOC phone coverage. At a certain point if you peer with the big boy ASNs they will expect a live human to answer the phone at 3am when something unusual happens in traffic between your mutual networks. That live human needs to be somebody who has the equivalent of 'enable' on your peering router and sufficient clue to wield it.

2 network engineering/architecture positions, multiply by expected fully loaded yearly salary and benefits package for each person.

at least 2 field technicians

vehicles, vehicle insurance, tools for technicians

At NYC salaries? In a greenfield scenario, I could easily spend $900,000 a year in just payroll and benefits, just to begin to attempt to duplicate what some other serious ISPs already have in place for manpower.

And again that is before you spend one dollar on equipment.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by danShumway on Wednesday May 02, 2018 @05:43PM (#16981412)
> At NYC salaries? In a greenfield scenario, I could easily spend $900,000 a year in just payroll and benefits

I kind of don't understand what you're getting at here. They're a nonprofit. The majority of their work is coming from volunteers. They aren't paying New York salaries.

This is like going to Linus and saying "You can't possibly expect Linux to be sustainable - do you have any idea how much money Microsoft pays its developers?"

Nonprofit/open models aren't less work than commercial ventures; the difference is that people and companies donate their time/money because they want the project to succeed. If they end up with a competitive enough product, then businesses will start sponsoring more development, because it'll be a cheap alternative to dealing with Comcast.

At that point, large businesses operating in the city will pay/sponsor New York salaries to maintain the parts of the project they find useful. Exactly like every other Open Source or community run project on the planet.

I have plenty of doubts about volunteer mesh networks, but I'm not so skeptical to say they literally can't work. I think you're looking at one particular model of the market and assuming that anything deviating from that model is broken, rather than just targeting different segments or trying a different strategy or even just accomplishing different goals.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by smallnamespace on Wednesday May 02, 2018 @06:32PM (#16981734)
> The majority of their work is coming from volunteers.

> the difference is that people and companies donate their time/money because they want the project to succeed.

And that volunteering time and energy are probably valuable then, especially if they can do the work of a real ISP.

I think one real question to ask though is what is actually more efficient—donating volunteer time and energy to a nonprofit, or donating money and having that nonprofit hire professionals instead?

GP doesn't sound like he is saying a volunteer mesh network literally cannot work, just pointing out that a network is inherently expensive to run, whether it is 'fully accounted for' on a payroll, or merely hidden in the cost of 'free' volunteering effort. It's an open question of whether asking for people's time or their money is better in the long run. Capitalism tends to suppose the latter.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by danShumway on Wednesday May 02, 2018 @07:25PM (#16982040)
> I think one real question to ask though is what is actually more efficient—donating volunteer time and energy to a nonprofit, or donating money and having that nonprofit hire professionals instead?

That's a good question. Judging from software development (which may or may not map perfectly to hardware infrastructure) both seem to work pretty well. Depending on who you ask you'll get different answers about whether closed (purely paid) or open (either volunteer or mixed) is "winning".

But certainly they seem to both work, on the whole. Neither strategy is so radically inefficient that it's safe to dismiss a company that relies on one over the other.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by pasbesoin on Wednesday May 02, 2018 @08:43PM (#16982410)
The incumbents are really good at suppressing competition.

One response/workaround: Guerrilla networking.

Why, in part? Because there's nothing much to target, financially. In some ways, legally. Instead, the incumbents have to elicit suppression through regulators and law enforcement, if they can.

Google has deep pockets. But it could spend years trying to get access to poles. Incumbents don't just pay to oppose; they pay to slow things down as much as possible. Good luck deploying, when you have to wait 6 months for a court case, that's then going to get appealed.

Stay under the radar. That's a whole different approach.

And communities are getting desperate enough to try anything that might actually work.

While I cheer them on, I simultaneously hope the press coverage doesn't bring the corporate mercenaries down on their heads.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by paulific on Thursday May 03, 2018 @12:37AM (#16983380)
Reminds me of the Corey Doctorow/Charles Stross short story Unwirer: https://craphound.com/unwirer/
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by Retric on Wednesday May 02, 2018 @07:22PM (#16982023)
FTE can work a few extra hours a week at a non profit more easily than they can make more money at their job and donate that. They may also enjoy their time and learn valuable skills which make the time vs money argument misleading, because for most tech workers they are very different pools.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by nycmesh on Thursday May 03, 2018 @01:16AM (#16983516)
Volunteers donate their time because they enjoy it. We have so many volunteers it is hard to keep track of them all. Obviously it is more efficient to use volunteers if there are enough. We are considering using contractors for some huge projects we are planning.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by pdkl95 on Wednesday May 02, 2018 @05:56PM (#16981502)
> they should treat it like a serious business and not a nonprofit.

A non-profit organization can be as "serious" as any for-profit business. The only difference is the non-profit might, for example, be required by their charter to re-invest income back into the business instead of paying it out as profit to the owners/shareholders/etc.

The level of "seriousness" of an organization should be determined by their business plan (is it profitable? And sustainable?), not their profit motive or financial footprint.

> if they're going to invest significant amounts of their personal time/engineering resources

First, it's their time and resources to spend as they wish.

Additionally, a suggestion I've heard is that pursuing anything that requires a serious investment of your time and resources (e.g. a job/internship, school, non-paid opportunities for "exposure") is worth pursuing if and only if it at lest two of these are true: [pays well/profitable, will improve future opportunities, you really enjoy it]. (If you love doing something and you're learning a lot or gaining experience that looks great on a CV, it can be worth a serious pay cut)

> build a network that actually does meet five nines through sufficient diversity in topology and routing

That is a good goal, but it isn't strictly necessary at first. Limit growth to what is possible at the time, and grow into a larger organization if it's successful.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by Retric on Wednesday May 02, 2018 @05:21PM (#16981252)
Google fiber had over 450,000 customers and was thus several times Webpass size when they acquired them. They have continually grown over time and by any reasonable measure very successful.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by jimktrains2 on Wednesday May 02, 2018 @05:27PM (#16981296)
> Google Fiber to cut jobs and halt expansion of US internet service

> Ambitious – and expensive – high-speed internet program will stop plans to expand to new cities as business reportedly under pressure to cut costs

https://www.theguardian.com/technology/2016/oct/26/google-fi...

Although it looks like they've started expanding again as per https://www.techrepublic.com/article/the-rise-and-fall-and-r.... So perhaps I counted them out too early. However, even with the backing of Google their success isn't guaranteed and they've had many non-technical, not-purely-financial problems along the way.

I still think it's a good example of why a "plucky upstart [with VC funding]" isn't able to waltz in and build a large, traditional ISP in the current environment.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by Retric on Wednesday May 02, 2018 @07:12PM (#16981969)
There are zero nationwide high speed ISP's. So, it's kind of a question of how do you measure success.

Yea, google paused expansion for ~3 months but building a nationwide network is a ~100 billion dollar investment. The only option to get there quickly is a LEO satellite constellation, otherwise it's going to be a really long slog with lot's of local governments getting involved.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by jimktrains2 on Wednesday May 02, 2018 @07:52PM (#16982177)
I hadn't heard they started expanding again, hence my not changed original stance.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by jimktrains2 on Wednesday May 02, 2018 @05:19PM (#16981241)
> Google did not fail at starting an ISP. They decided that the ROI for doing individual residential home FTTH was not worth it, considering the cost per house passed.

~Kind of sounds like it failed.~ Apparently Google Fiber is starting to expand again. I still think it serves as a good case against why the current climate makes it near-impossible for a "plucky upstart [with vc funding]" to start a large, traditional ISP.

> Starting a small ISP based out of extreme dissatisfaction with huge, evil incumbents is something that I encourage any plucky upstart to do. ... Just don't try to do it with $20,000 and some good wishes, because something with much deeper pockets like Webpass will come along and obliterate you.

You can't be a plucky upstart while simultaneously having pockets deep enough to fend against the current major ISP players.

I'm not going to sit here and reply line-by-line to your post. The issue isn't that people want a free network. The issue is that people are fed up with the large ISPs and that the large ISPs have made it all-but-impossible to start an ISP.

You seem to think that VC-sized money is the only thing people need to enter the market. You're entirely wrong. Entrenched players have made it next-to-impossible to compete in terms of capital and regulatory hurdles.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by rayiner on Thursday May 03, 2018 @12:38AM (#16983386)
Google is just a terrible example. On net, Google got incredibly preferential regulatory treatment. https://arstechnica.com/tech-policy/2012/09/how-kansas-city-...

They just realized it makes no sense for them to be in a low ROI industry: https://www.mckinsey.com/~/media/McKinsey/Business%20Functio...

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by walrus01 on Wednesday May 02, 2018 @05:23PM (#16981274)
> The issue is that people are fed up with the large ISPs and that the large ISPs have made it all-but-impossible to start an ISP.

that's the thing, they haven't. Not in the service territory this new nonprofit is operating in. There's something like 14 different ISPs which overlap service area with them in their part of New York City, none of which are big nationwide Comcast or Centurylink sized entities.

What they're trying to do on a shoestring is on super hard mode, they've picked the single most competitive market in the entire USA to try to compete with independent, non-LEC, non-major-telco companies that do have a few million dollars to throw at their own facilities-based network infrastructure.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by jimktrains2 on Wednesday May 02, 2018 @05:35PM (#16981357)
NYC, and potentially a few other urban areas, have the capability to run fiber easily.

I'm also just not sure why you're acting like this. You're being simply dismissive of the project. There are many successful meshnets.

You've yet to give a specific complaint about this project and have simply ranted about how they're just small, underfunded, and not a real ISP without ever actually substantiating any of that or specifically what they're doing wrong.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by walrus01 on Wednesday May 02, 2018 @09:33PM (#16982664)
NYC is actually the textbook example of most expensive places to run new fiber, due to needs for nearly 100% underground, and extensive pre existing utilities. I'm going to guess you've never been involved with an outside plant fiber project.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by stephengillie on Wednesday May 02, 2018 @11:54PM (#16983244)
With your knowledge of the subject, how would you suggest they overcome this challenge?

Do they need to run their own fiber - is there a way they could rent fiber space from an existing line? At least for a little while?

Don't be worried that a new player has entered the space; be worried that the existing ones haven't improved the space. The total market space can be increased by creating new connections, and someone has to do it.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by nycmesh on Thursday May 03, 2018 @12:47AM (#16983415)
There are about 10 fiber providers in the city. We've spoken to most of them. It's pretty much impossible for us to lay fiber, so we have to lease it. We'd like to eventually lease a fiber ring across the city so we can cut costs when connecting multiple buildings. If you don't have a ring then you have to pay for the run all the way back to a coloc.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by stephengillie on Thursday May 03, 2018 @11:23AM (#16986815)
That's a good start to having "your own" network, and then you can choose to have packets traverse your network or the peering networks. You could shift egress points around, based on cost and other factors as well. Each link you add can multiply your routing options.

Get large enough, and you can start carrying other traffic too, and eventually start buying/laying fiber. But you know this, it's probably part of your business plan.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by dzonga on Thursday May 03, 2018 @01:19AM (#16983525)
African Internet is really slow. Being a noob and knowing we can access the undersea cable. How would you go about building such a mesh network for an medium densely populate african city i.e 3+ million people ?
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by rayiner on Thursday May 03, 2018 @12:34AM (#16983373)
Do you have any experience with either ISP level network engineering or regulation? Because you speak awfully authoritatively.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by rdiddly on Wednesday May 02, 2018 @08:49PM (#16982441)
I think maybe the whole point is that not everybody wants all that shit, or they want to experiment with doing without some of it. That last 5% of reliability is on the order of 50% of the cost, or more, isn't it? Is it worth paying double for the small 5% improvement? Not for everyone, obviously.

And that's true even assuming just bare costs without markups. Obviously when you take into account how much profit & graft is attached to every single level of that stack, it really starts to add up. Throw in the absolutely, inexcusably bad, awful experience nearly everyone has with the big telecom companies and their employees, and that's enough ill will to propel many an alternative DIY network.

Metaphorically speaking, you just told a bicycle owner what a terrible car he has. Or rather, warned him how much the car is going to cost that he doesn't want in the first place. Is it possible the bicyclist can actually tell the difference between a bike and a car? Maybe he even sees some advantage you don't? (As opposed to the implausible but apparently favored hypothesis by car-owners that no, he just likes punishing himself while unsuccessfully pretending the bike is a car. When we don't know a stranger's rationale we glibly tend to assume there is none, and they're just cray-cray.)

To make yet another metaphor, when all you know is hammers, the whole world looks like it's full of 1) nails, 2) hammers, and 3) things that suck at being hammers.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by koolba on Wednesday May 02, 2018 @04:48PM (#16981019)
> Actual, reliable internet costs real money, both in the salaries of people to engineer and architect it, salaries for 24x7x365 clued-in NOC staff, equipment, salaries and expenses for field technicians to build it.

A phone network that objectively sucked compared to AT&T and Verizon but was boatloads cheaper seems to have worked out just fine for T-Mobile. I can see the same thing working for this. $20/mo for an internet connection that works most of the time would have plenty of customers. I'm not familiar with the wireless tech involved so no clue how well this scales outside of high density cities but within some place like NYC there's plenty of people that would pay for this over $50+/mo (min!) to the local cable companies.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by bogomipz on Wednesday May 02, 2018 @11:24PM (#16983125)
I have to wonder at the level of condescension in your comments. I'm wondering if it's because you actually feel threatened by this.

>"Actual ISP network engineer here: I'm sure these people are having fun and all, but it's insignificant in the larger scheme of things"

No it is actually very significant. In a past life I was also an "Actual ISP network engineer" at a large ISP. And what I can say is that large ISPs aren't exactly wellsprings of innovation. However these folks are hacking on things, solving an actual problem and seem to be having fun in the process. It's exactly this ethos that usually ends up moving the needle. Sometimes by accident. It's curious that you are so keen to poo-pooh them.

>"Real five nines network infrastructure takes significant capital investment"

There is no ISP that provides customers 5 9's of uptime. Not even for business customers. With a Time Warner or Comcast you are lucky if you get 3 9's from their network. This is what you get from "the duopoly."

>"And that's before you get into things like establishing colo at major IX points, serious core routers that cost $15k each (do you really want to deploy something in the year 2018 to take several full BGP tables that doesn't have a 4 million FIB capacity?), etc."

There a clue in the article about which IXP they're - 32 Ave of the Americas(DE-CIX.) It's one of the 3 of the most "wired" buildings in NYC. You can peer with everyone there. These people seem to have done their homework. And you don't need to buy $15K Cisco ASR any more. You can take full tables on commodity hardware running Linux and ExaBGP or BIRD. There are many CDNs and IXP's doing just that.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by Roritharr on Wednesday May 02, 2018 @05:38PM (#16981381)
The whole 5 nines aspect is interesting. I wonder why there isn't more cheaper more unreliable service available for customers built on cheaper hardware using less reliable infrastructure. Laser Links to cross distances etc.

I imagine consumers would be fine with 95% uptime of a gigabit connection compared to 99,99999% uptime of 16mbit.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by juliangoldsmith on Wednesday May 02, 2018 @07:06PM (#16981930)
>I imagine consumers would be fine with 95% uptime of a gigabit connection compared to 99,99999% uptime of 16mbit.

I can guarantee that nobody would be okay with 95% uptime. That means that your Internet access is down for over an hour every day. Given that people go berserk when their Internet is down for even a few minutes, 95% uptime will be unacceptable to consumers.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by ForHackernews on Wednesday May 02, 2018 @07:33PM (#16982082)
Depends which hour it's down. Plenty of people would be happy to have their internet down from 3-4 am every morning if it meant they paid $10/month instead of $70/month.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by bigiain on Wednesday May 02, 2018 @09:17PM (#16982596)
FWIW, many of the people selling five nines would just claim those 3-4am windows as "scheduled maintenance" and go down for an hour every night anyway whether they need to or not...

(Actually, if I were an evil ISP lawyer/marketer, I'd try to claim a "Maintenance window of 1hr downtime sometime between 4am and 3am", then ignore any downtime of less than an hour during the day and if you complained about not getting yur expensive five nines I'd laugh and say "published schedule maintenance not included!"...)

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by adamsea on Thursday May 03, 2018 @12:00AM (#16983263)
First guess is it would be most likely to be down when traffic is highest.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by zimablue on Thursday May 03, 2018 @05:43AM (#16984535)
You could switch to mobile data on downtime? And make that switch automatic/unobtrusive?
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by juliangoldsmith on Thursday May 03, 2018 @10:12AM (#16986191)
That means that you either have to provide an LTE connection for every customer, which would entirely defeat the purpose of your main connection, or your customers would need to have their own LTE connection, which would entirely defeat the purpose of your service in general.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by rsync on Saturday May 05, 2018 @06:30PM (#17004232)
God, this reply couldn't be worse:

"Real five nines network infrastructure takes significant capital investment."

No, it doesn't. Overly complex, corporatized infrastructures designed by committees in meetings - that takes significant capital investment. Simple infrastructure designed by hackers can achieve bullshit metrics like "five nines" without breaking the bank.

"And that's before you get into things like establishing colo at major IX points, serious core routers that cost $15k each"

Establishing colo at a "major IX point" is a few emails and phonecalls away. Your salesperson will fall all over him or herself and waive the NRCs to get you in there. It doesn't cost much, even for a full rack with 20 or 30 amps. If you're just doing networking, you won't even need that. Then you get a fiber cross connect to he.net for 10 or 20 or 40 gbps and you're in business.

You can buy very, very nice used routers and since you're a hacker and not Mr. Enterprisey IT guy, you can run it and support it yourself.

We had many iterations of this exact critique in our early days - explaining how we couldn't possibly deploy this volume of storage without fancy bullshit enterprise solutions from EMC or Compaq and how we couldn't possibly run a global storage network without fancy routers (we currently own zero routers) and fantastically complex clustering and failover ... and 17 years later we're still making it all work just fine.

Fuck those people. Fuck those school trained, passionless, non-hacker, no-talent-pussies.

Good luck with your mesh network!

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by chriswarbo on Thursday May 03, 2018 @08:02AM (#16985141)
This comment sounds very much like the AT&T engineers mentioned in http://www.shirky.com/weblog/2010/04/the-collapse-of-complex... (a post which appears on HN quite regularly).

---

> In the mid-90s, I got a call from some friends at ATT, asking me to help them research the nascent web-hosting business. They thought ATT’s famous “five 9’s” reliability (services that work 99.999% of the time) would be valuable, but they couldn’t figure out how $20 a month, then the going rate, could cover the costs for good web hosting, much less leave a profit.

...

> The ATT guys had correctly understood that the income from $20-a-month customers wouldn’t pay for good web hosting. What they hadn’t understood, were in fact professionally incapable of understanding, was that the industry solution, circa 1996, was to offer hosting that wasn’t very good.

...

> For a century, ATT’s culture had prized—insisted on—quality of service; they ran their own power grid to keep the dial-tone humming during blackouts. ATT, like most organizations, could not be good at the thing it was good at and good at the opposite thing at the same time. The web hosting business, because it followed the “Simplicity first, quality later” model, didn’t just present a new market, it required new cultural imperatives.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by dboreham on Wednesday May 02, 2018 @07:24PM (#16982030)
Agree the article is BS. Mesh networking is incompatible with QoS. So they can surely deliver crappy Internet cheaply. More than that? Nope. That'd need serious infrastructure such as licensed P2P, 60GHz short haul links, leased fiber. $$$$$

Oh, and they'd need peering: more $$$$$

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by nycmesh on Thursday May 03, 2018 @01:22AM (#16983536)
Did you read any of our comments?
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by mentos on Wednesday May 02, 2018 @09:04PM (#16982536)
You have any opinion on if internet should be treated as a utility?
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by niroze on Wednesday May 02, 2018 @04:28PM (#16980867)
I wish I could upvote this many times. Great reply. Agree 100% and thanks for speaking for us that work behind the scenes to make things happen.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by nycmesh on Wednesday May 02, 2018 @04:31PM (#16980884)
you are upvoting [things that are not true about us]
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by mediocrejoker on Wednesday May 02, 2018 @04:43PM (#16980977)
> you are upvoting nonsense

I don't think this is a helpful reply.

Since it sounds like you're associated with the project, can you address some of the specific concerns mentioned, like reliability? Do you disagree that hundreds of thousands of dollars are required for five nines connectivity? Do you disagree that that level of reliability is required for your customers etc.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by nycmesh on Wednesday May 02, 2018 @04:52PM (#16981060)
No you don't need hundreds of thousands. You do need tens of thousands. We've had no downtime this year at supernode1. Yes reliability is a big priority, but it is not a big priority for the big ISPs in NYC. Time Warner Cable (now Spectrum) has barely two nines. In Manhattan it would go down in the East Village after every large storm. We are already providing faster more reliable connections than this.
Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by mediocrejoker on Wednesday May 02, 2018 @05:34PM (#16981350)
Thanks. This seems like a cool project.

Do you know what the cost per user is currently?

What frequency do the point-to-point links run at? Have you considered using some sort of lower frequency packet radio for longer distance links, perhaps to other nearby nodes?

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by nycmesh on Wednesday May 02, 2018 @06:01PM (#16981535)
Our recurring costs are only for the lease at supernode1, which is around $1000/month. Divide that by our member node count (158) multiplied by users of each node ~4, 1000/(158*4)gives you under $2/month per user. Members usually pay for their own routers so that isn't included. To set up the supernode was about a $10K one-off charge for install fees, servers and antennas.

DE-CIX, our IXP, donated bandwidth to us, and we have transit also donated from Packet Host and WebAir. We actually pay nothing for bandwidth. Probably one day we will pay but it is not that expensive if you do this at an IXP and use peering.

We have an AirFiber pair that is 24Ghz, but all of our sectors are 5Ghz wifi.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by eric_h on Wednesday May 02, 2018 @07:46PM (#16982140)
> DE-CIX, our IXP, donated bandwidth to us, and we have transit also donated from Packet Host and WebAir.

What is the incentive for them to donate bandwidth/transit to you? Your non-profit status? Do they get something else in return?

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by nycmesh on Wednesday May 02, 2018 @08:05PM (#16982225)
We get a lot of support from the NYC networking community, and we have a lot of friends at the NYNOG meetups. Everyone is very helpful and they also want the big ISPs to have some competition, so we get donations from quite a few people. Some also join our network and get nodes on their own roofs and help with other installs!

DE-CIX is the biggest IXP and they measure their network in terabits/sec. It is not a big deal for them to donate us a 1 gig connection. (it's a big deal for us!) Eventually we may upgrade it to 10 gig.

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by bigiain on Wednesday May 02, 2018 @09:33PM (#16982668)
So you're sharing a 1G connection between 600-ish users - all presumably on 802.11ac 450Mbps (or 1.3Gbps if they've got three-stream?) connections?

How often do you see full saturation on that link? I'm guessing if all 600 of your users all tried to stream one stream of $TV-show-de-jour at once they'd barely get 16Mbps each? Can one user's home office full of Apple gear flood the bandwidth downloading a dozen Apple software updates simultaneously?

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by StudentStuff on Wednesday May 02, 2018 @10:35PM (#16982952)
That is a pretty low ratio in terms of over-subscription, on many cable networks you'd have 24 downstream Docsis channels @ 38Mbps usable per channel, for 912Mbps usable across all houses on said node. A single node often supports 500 homes, and that 912Mbps of capacity carries switched digital video, voice & data traffic.

Cable companies consider these high ratios to be quite embarrassing actually, as it shows how crummy their networks are: https://www.dslreports.com/forum/r31258251-Speed-Issues-Char...

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by nycmesh on Thursday May 03, 2018 @01:01AM (#16983459)
Netflix standard streaming is 3Mbps, so even with your math (which isn't how bandwidth works) everyone is happy.

We've never come anywhere near saturating our gig connection. We monitor it. Each building is limited to their own rooftop connection which is an average 100Mbps, so they can't do more than that if they tried. The thing is not everyone is downloading a file at the same time, and streaming uses much less bandwidth than downloading. From Netflix site-

0.5 Megabits per second - Required broadband connection speed

1.5 Megabits per second - Recommended broadband connection speed

3.0 Megabits per second - Recommended for SD quality

5.0 Megabits per second - Recommended for HD quality

25 Megabits per second - Recommended for Ultra HD quality

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by ybloviator on Thursday May 03, 2018 @01:21PM (#16987839)
Ah, so you get free bandwidth from people that want to balance their traffic for better peering contracts. That's clever. Probably not a helpful model for other areas that aren't bandwidth centers like NYC, and should probably be more out there when you're evangelizing so people understand you're getting a large portion of your operating expenses donated...

I have to assume another cost that's not noted is roof rights. That's not free - most people either have to pay or give the property management a kickback.

NYC is an interesting place for this - if you live in Manhattan, you clearly can afford the $80 for 1Gb/s FiOS or $40 for 100Mb/s FiOS or whatever Spectrum is charging. It would be way more interesting to plop this in a rust-belt city where people are on a paycheck-to-paycheck salary...

Re:'Anti-authority' tech rebels take on ISPs, connect NYC wi... by bigiain on Wednesday May 02, 2018 @09:26PM (#16982635)
FWIW, I have many many companies as customers for whom I've offered pricing for three, four, and five nines of reliability (for mobile app backends, not residential ISP service, but lets run with it any way).

Not a single one of them has ever signed up for anything more expensive than three nines. Only a few have even discussed the differences between 3 and 4 nines solutions.

I'm reasonably sure I'd choose the same for my home internet connection - if offered representative pricing based on costs of providing 3, 4, or 5 nines, or possibly even 2 nines - I'd choose the least expensive because losing a few minutes or even occasionally 15mins a day of connectivity at home (or maybe more realistically an ~8 hour outage per month) really isn't going to bug me greatly - not if it's two or more orders of magnitude cheaper than a four of five nines connection.