Don't developers need to handle deliverability and retries to Diahook?
User endpoints on the other hand, fail all the time, and often require a few retries.
With that being said, we plan on having client side (in our libraries) redundancy (try another endpoint if one fails), and ways for you to gracefully handle errors locally for users who need this.
Edit: I realise this comment was a bit sloppy, added some context in another comment https://news.ycombinator.com/item?id=26400970
My suggestion would be not only a library, but a local queue that can be managed with the library. Also, having some experience with working at a no code provider previously, ensure that it's trivial to query your service for what webhooks you can confirm were received by your infra and then delivered, perhaps by UUID.
Last, make sure it's dreadfully difficult to make changes to your systems where data loss could occur (messages dropped while returning 200 to counterparty systems, for example).
I agree about all the issues you outlined though. One thing we plan on doing in terms of internet being down and cloud provider not having routes: have a lot of endpoints spread geographically in every region, and maybe in the same/nearby data centers.
Thanks again for the feedback!
We just send the same requests over and over until they're successful. It's relatively simple to use.
Edit: ahh, I already mentioned it, it's buried in my comment.
That doesn't sound realistic to me. Sendgrid, Twilio, and all other APIs have downtimes and other problems, just check their status page: https://status.sendgrid.com/history.
I always had to implement some retry logic in API clients at some point or another. That doesn't mean your service doesn't bring value but it's not enough to just expect it to work because it is focused on uptime. When implementing a service that requires external dependencies I would always expect them to fail and try to design with that fact in mind.
We already have users that have their own queue system for sending us messages because that's how they deal with similar APIs too.
So I definitely see the problem, what I meant in my comment is that it's similar to other API companies, and it's not unique to us!
I work in healthcare tech, and in the last couple years I've worked with legacy medical systems that use FTP to implement "pseudo-webhooks". For each integration with a third-party technology provider, there is a dedicated FTP server that reads/writes files to an FTP server owned by the third party. When this service is interrupted there is no retry logic - someone has to spend time manually checking if files made it across.
If someone could make this legacy process easier on developers and HIPAA compliant they would be rewarded handsomely, as these organizations (hospitals, medical device makers, health-tech companies, etc) are comfortable paying enterprise-level prices for any technology they use.
Is this what it would look like? https://docs.mulesoft.com/ftp-connector/1.5/
If there’s any links or vendors you could share so I could learn about this I’d greatly appreciate it.
I will say there are EMRs like Athena Health which actually build for the developer experience. Integrating with Athena is relatively easy, and you can scale up the integration across customers easily. Epic (the market leader) is much harder and more expensive to do this for, since every distribution usually has some customization.
Here's the only post I wrote for it that focuses on security, which is pretty critical for a webhook system https://www.easywebhooks.com/how-to-secure-a-webhooks-api. You might want to consider adding protection against Replay Attacks and support for Challenge-Response Checks if you do not already!
Thanks for the link, will read it tomorrow!
Basically you periodically send a GET request to the target API with a token, and have them respond with the token encrypted with the same secret they'd use to decrpyt your webhook signature.
You could also consider sending dummy ping messages that may or may not have a valid signature (of course make sure this behavior is documented) that you would expect the target API to return a 4xx error if the signature is incorrect.
These extra steps are definitely not table stakes for a webhooks system, but could be enough to make sure the webhook event providers are being the best possible stewards of their user's data that they can =D. A lot of this complexity can also be wrapped by a client library you provide, which is a big win for everyone on its own.
Twitter's webhook API has an example of CRC btw! https://developer.twitter.com/en/docs/twitter-api/enterprise....
My employer (Segment) has something similar to Diahook (as I understand it) at its core: https://segment.com/blog/introducing-centrifuge/
Thanks for sharing this though, a cool read for tomorrow!
Diahook makes it easy for developers to send webhooks. Developers make one API call and we take care of deliverability, retries, and offer a great developer experience for their users. Essentially, we make it possible for everyone to offer a Stripe-like webhooks experience.
At my previous company, our users were constantly asking us for webhooks, both for consuming in their own services and for integrations with no-code solutions like Zapier. However, we kept on deferring building them because we weren't willing to commit the engineering time, resources and ongoing maintenance required of a webhook delivery system.
There are a variety of challenges when it comes to sending webhooks. For example customer endpoints fail or hang much more often than you would think, so you need to implement retries, but also make sure that such failures don't slow down or block your send queue or the rest of your system. Additionally, because of how webhooks work, anyone can send fake webhooks to your customers, so you need to make sure to cryptographically sign the payload, and make it easy for your users to verify it. You also want to avoid overloading your users' endpoints, so you want to automatically rate-limit webhook sending, as well as disabling failing ones, and notifying your users when you do.
I love webhooks, and I think everyone should be offering them! Our goal with Diahook is to make it faster for developers to add webhooks to their service, take care of the above challenges (and more), relieve them of having to worry about maintenance and scaling, and offer their users a UI for inspecting, debugging and replaying of past webhooks out of the box.
I'd love to hear about your experience building (or using) webhooks systems. What's a must have? Any war stories to share? Got any questions? Suggestions? Please let me know!
Docs: https://www.diahook.com/docs/
API viewer (and OpenAPI specs): https://api.diahook.com/docs/