I got lucky and hit all of these problems in my first few hours of trying (much to Fly's detriment, unfortunately). I wanted to love it, but I encountered nothing but bugs when I played with it over a weekend a month or so ago. Internal DNS worked for one but not the other of my nodes; Dashboard accepted my credit card but didn't actually upgrade my plan; SSH console would hang, requiring killing the client process with SIGKILL; my builder hung with a Dockerfile that was literally just "FROM ubuntu" and "apt update"; this failure resulted in `flyctl machines list` hanging and `flyctl machines kill` hanging. Maybe it was just a bad weekend at Fly, but their status page was all green, too.

So many problems between me and even giving fly a serious try. Unfortunate, but maybe in a year it will be better.

It'll definitely be better in a year. It sounds like you hit some of our capacity issues. In a year we'll either be ahead of those, or not growing anymore due to ongoing capacity issues. I'm hoping for the former.

The SSH and Builder issues are interesting. Those sound like something related to our wireguard stack.

Yeah, that would certainly have slowed me down too if I'd experienced that. I haven't had any of these issues. Fingers crossed! And hopefully they're working on stability.

My account got marked as "high possibility of fraud", even though I validated my email and added a credit card they even charged. The error message said that I had to contact [email protected] but that address only accepts email from paid subscriptions.

With no way to evaluate the service, I opted to delete my account.

Exactly what happened to me as well. In my case, I was (fortunately?) messaged by someone from Fly.io after asking around the community forum how to remedy the issue. I think the person I talked to was inferring that the system got tripped because I was using my own domain for my email address.

It's since been corrected, and I still have a handful of services running with them, but getting started with the platform definitely was a rougher experience than I was expecting it to be.

So they will regard any large company using their service as fraud? Insane

Large company? Literally every company I ever worked for, the smallest one were 5 people, had their own domain for emails. And they were not all in IT.

So we agree its insane to cut off every company with a domain? Good.

That would be insane, yes. It isn’t the case, though. I signed up with my own domain just fine.

I don't know what merchant system they are using but I will say it is very common for them to have their own anti fraud detection and rejection on charges. Your specific case could be related to a huge list of possibilities (even many of them a rollup of many other interactions on other sites that happen to use the same merchant or foundational fraud data).

Thinking this is just because of a domain name is silly.

Also Fly.io -- You may want to clearly accept and manage trial issues (or at least a subset) via a support path. It seems silly to effectively bounce users with the impression that you have no support because they are not YET paying customers while in the trial phase.

as a counter-point, i use custom domain for email, and haven't had a problem with fly.

It may not be custom domain but rather self-hosted mail server.

That's worse. A self-hosted mail server implies a significant cost investment that probably wouldn't be required for ye olde hotmail account.

No, we don't flag accounts as high risk based on email domain.

We don't charge credit cards to try and remove the fraud flag. We do a preauth, which is immediately canceled. Sometimes this looks like a charge.

> We do a preauth

Probably that was it, but my point is, there is no process you (as a customer) can follow to remove the fraud flag.

Also the flag was still there long after the preauth.

My main gripe is the wild variation in response time.

My server response time seemed to vary from <100ms to >300ms for the same operation.

Is this due to the network i.e. you get routed to a different server or due to CPU sharing?

I thought it could have been because of a shared CPU at first as well, but I upgraded to dedicated CPUs and it didn't make a difference.

I don't know whether this is a platform issue with Fly or something else, but it was kind of annoying to have such a high variation with seemingly no way to fix it.

isn't the whole selling point of fly is edge servers which supposed to reduce latency

Sure, but what's edge? The world is big. Can't have servers everywhere (ok, maybe Cloudflare is close). It's possible due to routing decisions that you get sent to a "different edge" at different times.

So what's the point then? 250ms is enough to literally send traffic to the other side of the world. So why bother with edges if you can just have a single server where most of your customers are and get the same latencies?

* * *

Venture funded stuff like this should be fast, good, cheap, pick 3! Until they need to make a profit or shut down.

After I have experienced several fly.io render railway northflank platforms, I think the problem with fly.io is that the web console is not fully functional, too dependent on CLI and the response time of the application is unstable. The problem with render is that it is too slow to build, north is similar to railway, and the experience is good. But although the north console is very comprehensive, it is too complicated and dazzling. By contrast, the interface of railway is very simple and comprehensive, and UX is great. (found by wappalyzer analysis that it is based on nextjs,very nice)

I tried railway and northflank. Really like the UI northflank has; I think anyone familiar with Kubernetes should feel right at home. I also agree with the sentiment that railway ui is simpler.

I get up to 20secs of downtime when redeploying on railway... I was using the sample golang repo from fly : https://github.com/fly-apps/go-example

That's just unacceptible imo.

More context: The UI shows that the service is available but when I visit the live url, I get a 404.

What I think is happening is that the UI shows the status of the underlying container and there is some fault with the reverse proxy they are using to expose the container to the internet.

Fly.io doesn't have this issue but I am leery of all the complaints here.

Hey there fyzix! Angelo here, Support Engineer from Railway. I was here procrastinating from work when I saw this comment.

The downtime you see is our proxy taking time to cutover- however, we have https://docs.railway.app/deploy/healthchecks that will only cutover once we have a 200 from your API. This way we can keep your old deploy live and serving requests if and only if it's live.

Theres more we can do to make it magical, but this should help in the meantime.

Thanks for the reply, Issue resolved:)

This could be done automatically by pinging the root and searching for a header set by railway's default page. If it doesn't exist then the service is live.

We used to do that by default, but it led to issues with load on our proxy. (Imagine n * m pings against your host for a shit-tonne (official measurement) of builds + deploys a day.)

We are hiring Network Engineers for exactly this reason :')

>The problem with render is that it is too slow to build

This is only true for the free tier, and (hopefully) won't be true for much longer.

Yes, but I think the difference between free layer and paid users should be distinguished in terms of capacity and scalability. This crude entry experience may lead to a reduction in user conversion.

There’s an edge case in which you can lose access to your account as well when switching from github to email login with the same email. It’s an edge case but quite a bad one.

Yep. Fly is half-baked. Deployed a service earlier this month and got bitten in a few ways:

- It took several hours to get a 1 GB disk provisioned (their API would time out and I'd get left with an allocated but unusable disk that couldn't be deleted without deleting the entire app it was attached to.) The Fly forums have numerous reports of the same issue, going back months, across multiple data centers. All resolved with the moral equivalent of "oops, try again later?"

- The fly.toml reference is incomplete and internally inconsistent, documenting keys which are not valid, while omitting valid keys that are mentioned elsewhere in the docs.

- Volume snapshots are retained for either 5 or 7 days, depending on which of the multiple conflicting docs pages you believe.

- I wanted to let them know about that last one, so I emailed the address listed at https://fly.io/docs/about/support/, only to get an autoresponse saying that they don't read that mailbox and that I should pound sand unless I'm already paying them to read my emails.

Reading between the lines, it seems like a lot of the infrastructure pains are pinned on Nomad, with the hope that moving to Fly Machines / Apps V2 will obviate a lot of those issues.

I hope they're right, because what they're trying to deliver is exactly what I want.

Well Nomad certainly isn't at fault for our docs.

Volume provisioning issues have been a game of whackamole, largely as the result of scaling pains. We've grown like 4x in the last 3 months (thanks, Heroku) and it's put a strain on pretty much everything.

Echo this, and I _did_ notice it in my first two weeks.

3 separate issues prevented me deploying. One was my fault, but hard to diagnose because error messages contain no useful information.

The other two were the platform falling over, with the support forum having no resolution but plenty of other people in the same boat.

Lots to love about fly. The latency is noticeably better than anything I’ve used except cloudflare. Prices are good. Hopefully they’ll get the growing pains sorted and we’ll have another contender.

These deploy issues are all kinds of ulcer inducing. They're not strictly Nomad's fault (we're holding it wrong), but the complexity of flyctl -> nomad -> global hosts makes it difficult to bubble errors back up to you.

If you end up trying again, give the machines based apps a shot. It's much simpler infrastructure: https://community.fly.io/t/fly-apps-on-machine-prerelease/10...

The TLS cert renewal was a big failure on our part. I know that's, like, a one strike and you're out problem, but we've improved that part of our stack substantially. It should be much more resilient.

Redis/Postgres timeouts sound like they might be a problem we can help with, though. Both those services work through a load balancer with a TCP idle timeout. If you get timeouts after a period of inactivity, it's usually a simple config tweak to have a driver handle that seamlessly.

The timeout issues surprised me, more drivers than we expected can't handle a DB connection timing out when it's idle.

I had (a couple years ago) some weird websocket throttle or debouncing characteristic. I tried to construct an evidence, but it's quite hassle so I just ignored it.

I think they are too engineering culture. Target is devs, sure, but they could use some product guys.

I'd spend some time reading community.fly.io before committing to it. I recently helped a startup that has been on fly.io and there was a lot of frustration about things not working.

it was very frustrating to see un-answered problems reported, fly.io down for what seemed to be many users, the same time seeing a fly.io co-founder arguing about politics all day long here on HN...

In fact, when you enter the fly.io console, you will feel that this is an unfinished construction site, especially when you start to deploy a new app, and the frame text icons in the options are simply truncated, which feels rough.