News for hackers. Stuff that matters.
Re:Man who built ISP instead of paying Comcast $50K expands ... by samwhiteUK on Wednesday August 10, 2022 @09:45AM (#32411821)
I'm going to put my hand up and say I have absolutely no idea how an ISP works. He runs cables to each house in the area... now where does the other end go?
Re:Man who built ISP instead of paying Comcast $50K expands ... by beezlebroxxxxxx on Wednesday August 10, 2022 @10:22AM (#32412468)
There is a very good Ars Technica article on how an ISP works. It traces the whole network, from submarine cable through to last mile into a house. It was written in 2016, but I imagine it's still relevant:

https://arstechnica.com/information-technology/2016/05/how-t...

Re:Man who built ISP instead of paying Comcast $50K expands ... by digdugdirk on Wednesday August 10, 2022 @12:00PM (#32414197)
Thank you, that was a great link for us uninitiated folks.

Also another great plug for ArsTechnica (even though the main article is them as well, and I'm sure most of this audience is well aware of them) and the excellent technical writing they do.

Re:Man who built ISP instead of paying Comcast $50K expands ... by Bloating on Wednesday August 10, 2022 @10:24AM (#32412493)
There are wholesalers that provide "dark fiber", then you buy data services from another "wholesaler". When I looked into it, dark fiber was available through some utilities and through a government funded non-profit. Data to light-up the fiber was available through several different data centers that connected to that dark fiber.

You still had to build-out the last mile though, and thats what will get you. You either need private easements, or be a registered telecom utility to use public utility easements. That last mile is $20k +/-, depending on your circumstances. If your semi-rural or less, there's ROI sucks. Hence, many smaller ISPs are wireless.

At least in area, there are already a number of wISPs, 5G is rolling out, Starlink eventually. and lots of gov't funding going to the big players to expand their networks (and drive the start-ups out of business.)

There some other business models out there too that look interesting. Underline in Co Springs, for example. They provide a basic tier of service, in order to qualify as a telecom, install the fiber and then allow multiple competing ISPs to use their network.

IMHO, any utility that has the benefit of government privilege should be required to allow competors to use the infrastructure that the taxpayers funded.

I'm waiting on one of you brilliant folks to defy the laws of physics to create a decentralized, wireless mesh internet.

Re:Man who built ISP instead of paying Comcast $50K expands ... by thedougd on Wednesday August 10, 2022 @01:10PM (#32415350)
https://www.segra.com/

These guys have dark fiber right in front of my neighborhood. They service cell sites for Dish Network near me as well. It's interesting to look through their services. For example, you can get fiber service with layer 2, where you're responsible for adding your IP stack over top of it. Or you can buy at layer 3, where Segra is already running a stack, and establish mesh connectivity. So if a fiber is cut, you'll get another working path. Build your network over the top.

Pretty interesting to understand what's available.

Re:Man who built ISP instead of paying Comcast $50K expands ... by wyager on Wednesday August 10, 2022 @11:59AM (#32414186)
Last mile subsidies are super weird. I was looking at a property in montana in the middle of nowhere that had no electricity nearby, but had gigabit fiber. I called the ISP and it was cheaper to get phone+Gb than just Gb due to subsidy rules.

Basically everyone out there (including me) is on starlink now. Turns out the subsidies were not only inefficient, but pretty pointless.

Re:Man who built ISP instead of paying Comcast $50K expands ... by aftbit on Wednesday August 10, 2022 @12:04PM (#32414257)
Why would you be using Starlink if you have gigabit fiber available? Or was it still quite expensive to install even with the subsidy?
Re:Man who built ISP instead of paying Comcast $50K expands ... by wyager on Thursday August 11, 2022 @10:43AM (#32426466)
I didn't get that property sadly.
Re:Man who built ISP instead of paying Comcast $50K expands ... by daniel-cussen on Wednesday August 10, 2022 @01:42PM (#32415804)
They helped a bit, for a while. Gigabit fiber is less maintenance than electrical power, and it's easier to roll-your-own electrical power than it is to get a Gb connection like how would you do that before Starlink, buy an insane amount of radio spectrum? I heard one HN user who did exactly that in Brazil, got a 20-meter tower to connect by radio to the internet some distance away, and it was a very solid high bandwidth connection. But still much harder than a generator and solar panels, or a tiny little hydropower generator on a stream (a great option in places like Southern Chile, not a joke by any means). Or wind.
Re:Man who built ISP instead of paying Comcast $50K expands ... by the_only_law on Wednesday August 10, 2022 @09:47AM (#32411847)
Not sure if it’s what the person in question did, but there’s a whole guide that pops up on here occasionally regarding building a wireless ISP.

https://startyourownisp.com/

Re:Man who built ISP instead of paying Comcast $50K expands ... by dataflow on Wednesday August 10, 2022 @09:57AM (#32412013)
I can't find any section of that guide that talks about peering or whatever ISPs are supposed to do to connect to the broader internet. Do you see any step that explains this?
Re:Man who built ISP instead of paying Comcast $50K expands ... by bombcar on Wednesday August 10, 2022 @10:06AM (#32412178)
As a small ISP you don't peer - you just buy transit from a bigger ISP. So the basic steps are:

1. Buy a 1G/1G or 10G/10G whatever link to a building you own.

2. Resell that link in parts to customers.

Or you can get yourself into a POP (point of presence) somewhere that multiple providers are also in, and get transit that way. Depends on where you are and what you can get access to.

Re:Man who built ISP instead of paying Comcast $50K expands ... by nixgeek on Wednesday August 10, 2022 @10:25AM (#32412515)
As a small ISP you definitely can peer and many do, you just aren’t going to get settlement-free peering with any of the big eyeball networks like Comcast.

Something like Seattle IX is a good example of where lots of peering sessions could be established (although I haven’t looked at Jared’s ASN in any detail to see where it’s present).

https://www.seattleix.net/home

Any traffic you’re able to offload via peering you wouldn’t be paying an IP transit to haul, so it’s worth seeing if networks like Netflix are on the Route Servers (https://www.ams-ix.net/ams/documentation/ams-ix-route-server...) at any IX nearby your network, seeing if you can negotiate a session over the IX even if they don’t participate in the RS, or seeing if you can do PNI (sling a cable between your networks in a facility you’re both located in).

Edit: Jared’s on Detroit IX. https://www.peeringdb.com/net/20268

Re:Man who built ISP instead of paying Comcast $50K expands ... by 2Gkashmiri on Wednesday August 10, 2022 @12:34PM (#32414736)
Wait. The poster above said in point 1 to buy a line,1G, 10g depending on your upstream seller. Why do you need peering then?

If I have 1Gbps line for example and 10 users each are using equal amount 100% of time, it shouldn't matter they send the data to Alaska or Russia or Australia ? Or does it?

Do you buy the pipe and the data itself also?

Re:Man who built ISP instead of paying Comcast $50K expands ... by icedchai on Wednesday August 10, 2022 @06:06PM (#32419021)
You don't "need" peering but it offloads your upstream (transit) links, which are generally much more expensive. In the old days, I worked for couple ISPs and we typically had 3 or 4 upstreams (generally UUNet, Sprint, MCI...) This was back when a T1 was still considered fast.
Re:Man who built ISP instead of paying Comcast $50K expands ... by toast0 on Wednesday August 10, 2022 @12:39PM (#32414827)
> Any traffic you’re able to offload via peering you wouldn’t be paying an IP transit to haul

When you're small enough, the difference in price between transit and what it takes to get you to an IX is likely to be pretty small. But, you probably want to be at an IX sooner or later anyway (easier to get multiple transit offers at an IX than on the side of the road), so might as well peer while you're there.

Re:Man who built ISP instead of paying Comcast $50K expands ... by twothamendment on Wednesday August 10, 2022 @10:30AM (#32412609)
Yes, it can be pretty simple. Back in the day when DSL and comcast were the options and all of the connections were things like UP TO 5 or even 20 Mbps, but speeds were rarely that - I paid for a dedicated 2Mbs up and down ($180/month) with no restrictions on use and started sharing/reselling it to others in my apartment building, not with wireless, but with cat5 out the window, up the gutter, back inside, etc. Across the parking lot another guy was sharing his comcast with another building - but comcast was starting to be so slow they couldn't use it. We merged our empires by stringing some cat 5 across the parking lot, around a pole and to his place. Later we added more nearby buildings, all wired until we had 5 buildings and about 20 "subscribers". Even with 2Mbps, everyone on the network was happier with a guaranteed speed than their flaky "up to" speeds they used to have. Did I run an ISP? I had subscribers, had to maintain a network, had a proxy server to reduce requests out of the network, had to deal with abuse and collect money - so I'd say yes, a small one, but yes.
Re:Man who built ISP instead of paying Comcast $50K expands ... by smeyer on Wednesday August 10, 2022 @11:56AM (#32414130)
Out of curiosity did you do things above board from a business standpoint (taxes etc.) or was this more of a blackmarket setup?
Re:Man who built ISP instead of paying Comcast $50K expands ... by twothamendment on Thursday August 11, 2022 @12:09AM (#32421495)
Taxes? I was a college student, so I didn't make enough to owe any taxes, but it was mostly cash. I wouldn't have done it if I had to hop through the business hoops.
Re:Man who built ISP instead of paying Comcast $50K expands ... by spmurrayzzz on Wednesday August 10, 2022 @12:20PM (#32414505)
You definitely can (and should) peer as a small ISP, even if you are buying transit from other providers. This is especially true if you're running an MPLS headend as you'll still have choke points at L2 circuits in your own network. Owning your own peering can be a great way to offload traffic to other circuits that share destinations, most commonly traffic destined for VOD/streaming CDNs.

(N.B. — This is what has worked well for the WISP I cofounded, but YMMV depending on headend infra).

Re:Man who built ISP instead of paying Comcast $50K expands ... by haunter on Wednesday August 10, 2022 @09:59AM (#32412058)
Re:Man who built ISP instead of paying Comcast $50K expands ... by dataflow on Wednesday August 10, 2022 @10:01AM (#32412093)
So they're leasing ("buying"?) fiber from the same ISPs they're trying to displace and relying on that payment to provide them with continued internet access? This doesn't sound like a real first-class ISP, but something akin to an MVNO where they're at the mercy of the same companies they're competing with. I get the initial sale might seem fine, and the established ISPs might be fine with this as long as the company is small, but why wouldn't these companies shut them off (or raise the prices, etc.) when they grow too big to become dangerous?
Re:Man who built ISP instead of paying Comcast $50K expands ... by dboreham on Wednesday August 10, 2022 @12:05PM (#32414285)
You're misunderstanding this market. There's a wholesale market, which he is buying from. There's a retail market which he is selling into. Some providers service both the wholesale and retail markets, but typically with different divisions, people, tech, resources. It's like saying that if you build a gas station and buy your gas from Exxon then that's bad because Exxon also operates gas stations. It's not like an MVNO where all you're doing is sending the customer a bill, and provisioning API requests to Verizon.
Re:Man who built ISP instead of paying Comcast $50K expands ... by dataflow on Wednesday August 10, 2022 @12:12PM (#32414383)
> You're misunderstanding this market. There's a wholesale market, which he is buying from. There's a retail market which he is selling into. Some providers service both the wholesale and retail markets, but typically with different divisions, people, tech, resources.

The difference in divisions/people/tech/resources doesn't explain anything for me. They're both the same company with the same CEO, whether it's one business unit or a dozen. It's not like the executives are oblivious to how much money each unit is making and whether another unit could make more in place of it. If you're the CEO and see you could charge twice as much by doing retail instead of wholesale then you'd obviously try to do that.

Rather, the explanations I'm getting from the other comments seems to be that (a) regulators require some kind of reasonable wholesale to exist to third parties, (b) the big ISPs aren't planning to serve those markets anyway, so they're not missing out on any income by taking money from the last-mile ISPs. And as long as those last-mile ISPs don't try to compete for the same customers then they're fine.

Re:Man who built ISP instead of paying Comcast $50K expands ... by fragmede on Wednesday August 10, 2022 @02:44PM (#32416660)
> The difference in divisions/people/tech/resources doesn't explain anything for me. They're both the same company with the same CEO, whether it's one business unit or a dozen.

Then you've not worked in large B2B companies before. Eg Apple pays Google money and Google pays money to Apple, any perceived public rivalry goes out the window as far as business between the two is concerned.

If you're the CEO of Comcast, you've never even heard of this small time ISP, you have far bigger things to spend your time on, and the "upstream" business unit of Comcast really doesn't care what you're doing, so long as your money's green. It's all business. See also: Netflix using AWS despite Amazon having a streaming video service of their own.

Re:Man who built ISP instead of paying Comcast $50K expands ... by bombcar on Wednesday August 10, 2022 @10:08AM (#32412211)
Because it's all business to them, and if they did it overtly they could get sued.

But also because once you're in a single location, you can pretty easily get multiple providers to that location for a Price, so there's really no point. Even small rural towns usually have multiple internet connections from different companies, and if they don't you can pay to run fiber if you really wanted to.

People find it hard to believe, but Comcast et al are actually businesses, not Satan's marketing department; and they happily take money even from "competitors".

Re:Man who built ISP instead of paying Comcast $50K expands ... by themoonisachees on Wednesday August 10, 2022 @10:33AM (#32412658)
To expand on that:

Comcast would much rather sell a dedicated fiber to a business with capital and guarantees.

Selling to the individual consumer doesn't make a lot of sense business-wise, because of the deployment costs and continued support costs.

Comcast is also abusing their status as oligopoly to gouge costumers financially and qos-wise, but if they're selling to a business that buys large quantities and has staff who's job it is to handle network problems, they actually like that (right up until that business threatens to compete with them in areas where said oligopoly is in place, of course)

Re:Man who built ISP instead of paying Comcast $50K expands ... by haunter on Wednesday August 10, 2022 @10:10AM (#32412242)
>This doesn't sound like a real first-class ISP

I'm not an expert but afaik you can't just be a Tier 1 network member https://en.wikipedia.org/wiki/Tier_1_network

Even Tier 2 very limited https://en.wikipedia.org/wiki/Tier_2_network

In this guide's case yes you will be akin to an MVNO, you won't peer but just buying transit traffic. That's why most of these guides are also focusing on making the network wireless only (easier to build infrastructure)

Re:Man who built ISP instead of paying Comcast $50K expands ... by q3k on Wednesday August 10, 2022 @10:06AM (#32412183)
You can lease fibre/lambda/L2 transport to an IXP (and there peer with other local ISPs and get global transit from Tier 1 providers) from many companies that don't even have any residential offering.

Or if (technically/financially/legally) possible, even run your own fibre to a PoP housing an IXP on your own.

Once you're in multiple PoPs and on multiple IXPs and with multiple upstreams/peers you're pretty much independent from the whims of a single ISP.

Re:Man who built ISP instead of paying Comcast $50K expands ... by mbreese on Wednesday August 10, 2022 @10:14AM (#32412322)
Because there are different business units in the upstream company handling the dedicated access vs consumer sides. The dedicated business side have their own sales goals and if you compete with the consumer side, that’s not a problem for them. I’m sure there are some regulatory/anti competitive measures at play here too, but economically, the two sides of the business will act more or less independently.
Re:Man who built ISP instead of paying Comcast $50K expands ... by wins32767 on Wednesday August 10, 2022 @11:21AM (#32413459)
He's not trying to displace the majors. In rural areas, owning and maintaining a bunch of fiber to service less than a thousand customers isn't a business Comcast really wants to be in unless they get paid a ton for it.
Re:Man who built ISP instead of paying Comcast $50K expands ... by mananaysiempre on Wednesday August 10, 2022 @06:05PM (#32419009)
If you want to try your hand in a playground for the software (routing) parts, DN42[1] is essentially one.

[1] https://dn42.us/, https://dn42.eu/, https://dn42.dev/

Re:Man who built ISP instead of paying Comcast $50K expands ... by kevmo314 on Wednesday August 10, 2022 @09:59AM (#32412060)
From https://startyourownisp.com/posts/fiber-provider/, doesn't this site basically say connect to another ISP?
Re:Man who built ISP instead of paying Comcast $50K expands ... by moffkalast on Wednesday August 10, 2022 @10:08AM (#32412223)
Well there three tiers of ISPs, each one buying service from the one above them. It's ISPs all the way down, and the higher up you go the more expensive the hardware to run it gets.
Re:Man who built ISP instead of paying Comcast $50K expands ... by Macha on Wednesday August 10, 2022 @10:19AM (#32412417)
At the T1 level it's more completely a mesh type setup, but even lower tier ISPs might set up peering agreements to bypass their main higher tier ISP where it makes sense for cost or service quality reasons. Or refuse to to extract more money as in the comcast vs level1 disputes over netflix traffic a while back
Re:Man who built ISP instead of paying Comcast $50K expands ... by carlhjerpe on Wednesday August 10, 2022 @07:06PM (#32419565)
Who is T1, T2 and T3? I really don't see it. Seems like that tiering has aged about as well as the OSI model.
Re:Man who built ISP instead of paying Comcast $50K expands ... by MerelyMortal on Wednesday August 10, 2022 @10:26AM (#32412531)
I feel like this could be made into an ISP Tycoon game.
Re:Man who built ISP instead of paying Comcast $50K expands ... by haunter on Wednesday August 10, 2022 @11:04AM (#32413211)
It actually does exist lol! Made by Cisco as an e-learning tool

Gameplay https://www.youtube.com/watch?v=Foa34qoRzjs

https://web.archive.org/web/20150317144142/https://cisco.edu...

Re:Man who built ISP instead of paying Comcast $50K expands ... by RockRobotRock on Wednesday August 10, 2022 @12:26PM (#32414599)
Oh god, it seems to essentially be packet tracer under the hood. It's a great tool, but I HATED using it in school.
Re:Man who built ISP instead of paying Comcast $50K expands ... by jethro_tell on Wednesday August 10, 2022 @05:07PM (#32418473)
gns3 perhaps? I haven't had that setup for a while but I loved it. I had my whole small ISP in it at some point to work as a test/lab env for testing things out. It's a trick to get going but was kinda fun. I took a copy of that when I left and every now and then I fire it up and mess around with my old dsl/dialup ISP from back in the day.
Re:Man who built ISP instead of paying Comcast $50K expands ... by cptcobalt on Wednesday August 10, 2022 @11:37AM (#32413770)
Dang, as someone that enjoys Tycoons, Tactics, and Management Simulators, this really sounds fun.
Re:Man who built ISP instead of paying Comcast $50K expands ... by southerntofu on Wednesday August 10, 2022 @11:12AM (#32413311)
As the other commenters have pointed out, a possibility is simply to "resell" transit from other providers. However, on the Internet all peering networks are somewhat equal and it's entirely possible to extend the "other end" over time to establish dedicated peering with other networks, so that for example traffic from your network to Youtube doesn't have to go through (paid-for) 3rd parties.

There's good chances there are Internet eXchange Points around where you live where for a small maintenance fee anyone can come and place their router and cables to interconnect with others.

So the likely steps are:

1) Find a transit provider, that will serve your trafic to any other network, and where to connect with this provider 2) (Optional) If you don't have the necessary infrastructure, find another provider to get from your last-mile network to your transit provider 3) (Optional) Find other networks to peer with so that you can significantly reduce your transit bill and provide better routes (therefore better service)

Some non-profit ISPs take the problem from the other side, and build a core network without necessarily owning any last-mile infrastructure, which is leased from other operators (opérateurs de collecte) with whom they interconnect at some datacenter/IXP. The most famous example of that in France is FDN.fr which has been operating since early 90s. That approach is more cost-effective in high-density area where the local infrastructure is already quite good, and construction jobs to lay new cables is very costly, but will still set you back 10-30€/month/line.

Re:Man who built ISP instead of paying Comcast $50K expands ... by andix on Wednesday August 10, 2022 @12:21PM (#32414510)
I think you more or less just buy connections from bigger ISPs, so for example you get a 100 Gbps connection to one location and distribute it to your end users from there.

Most of the equipment you can buy, you can even get a lot of the needed things as a service. You just need to organize all those hardware and software things, and get the economic and legal part right too. And in the end it needs to tie together in a way, that your earnings are bigger then your expenses.

I think it’s not so different to opening a car repair shop for example. Just more nerdy.

Re:Man who built ISP instead of paying Comcast $50K expands ... by wil421 on Wednesday August 10, 2022 @10:19AM (#32412399)
Depending on how close they are he could run cables (ethernet) or fiber. Single mode fiber can go 10km according to some Ubiquiti spec sheets I found on google. Ubiquiti also sells AirMax products that can do PTP or PTMP over the air, although some will be affected by rain. They could even rent space from a radio/cell tower. There are probably a decent amount of other products out there I am only familiar with Ubiquiti.
Re:Man who built ISP instead of paying Comcast $50K expands ... by nixgeek on Wednesday August 10, 2022 @10:29AM (#32412590)
You can shoot light over SM at distances up to 200km (several important caveats at this distance) and it’s very usual to see spans of between 50-80km.
Re:Man who built ISP instead of paying Comcast $50K expands ... by wil421 on Wednesday August 10, 2022 @10:41AM (#32412785)
Looking further you can get a UFiber OLT Terminal for $1,799 that can run 20km and support 1024 clients or 128 ONU CPEs per port.

How much would a 200km switch run?

Re:Man who built ISP instead of paying Comcast $50K expands ... by Nikbul on Wednesday August 10, 2022 @11:39AM (#32413814)
At this distance you would want a good repeater at about half point instead. Don't forget that data has to travel back and other side might not have such a strong signal
Re:Man who built ISP instead of paying Comcast $50K expands ... by iptrans on Wednesday August 10, 2022 @02:48PM (#32416721)
The switch does not care what kind of optics you use. You can use a $50 switch is you like.

The 200 km optics, however, cost about a grand each.

Re:Man who built ISP instead of paying Comcast $50K expands ... by notRobot on Wednesday August 10, 2022 @12:38PM (#32414816)
Re:Man who built ISP instead of paying Comcast $50K expands ... by dbelson on Wednesday August 10, 2022 @08:57PM (#32420345)
Also very much worth reading on this topic: Tubes by Andrew Blum (https://www.andrewblum.net/tubes-2)