pbbakkum

This is great, thank you for sharing. I work on these APIs at OpenAI, it's a surprise to me that it still works reasonably well at 2/3x speed, but on the other hand for phone channels we get 8khz audio that is upsampled to 24khz for the model and it still works well. Note there's probably a measurable decrease in transcription accuracy that worsens as you deviate from 1x speed. Also we really need to support bigger/longer file uploads :)

Heyo, I work on the realtime api, this is a very cool app!

With transcription I would recommend trying out "gpt-4o-transcribe" or "gpt-4o-mini-transcribe" models, which will be more accurate than "whisper-1". On any model you can set the language parameter, see docs here: https://platform.openai.com/docs/api-reference/realtime-clie.... This doesn't guarantee ordering relative to the rest of the response, but the idea is to optimize for conversational-feeling latency. Hope this is helpful.

If you're interested in GPT-powered shell autocomplete, check out https://butterfi.sh

This also enables shell-aware LLM prompting!

Do you work from the command line? Butterfish is a project I wrote for myself to use AI prompting seamlessly directly from the shell. I hope it's useful to others, give it a try and send feedback!

I've experimented with it, the reason I haven't yet added it is that I want deployment to be seamless, and it's not trivial to ship a binary that would (without extra fuss or configuration) efficiently support Metal and CUDA, plus download the models in a graceful way. This is of course possible, but still hard, and not clear if it's the right place to spend energy. I'm curious how you think about it - is your primary desire to work offline or avoid sending data to OpenAI? Or both?

Plugging a project of mine: I've been working on a similar idea for the era of LLMs: https://butterfi.sh.

It's much more bare-bones than Fig but perhaps useful if you're looking for an alternative! Send me feedback!

If you're looking for a lighter weight TUI for Top information check out a recent project of mine here: https://github.com/bakks/poptop

Really like the design of these tools so that you can easily pipe between them, this a good way to make things composable. Also really cool to see all of the other CLI tools folks have posted here, lots that I wasn't aware of.

I've been experimenting with CLI/LLM tools and found my favorite approach is to make the LLM constantly accessible in my shell. The way I do this is to add a transparent wrapper around whatever your shell is (bash,zsh,etc), send commands that start with capital letters to ChatGPT, and manage a history of local commands and GPT responses. This means you can ask questions about a command's output and autocomplete based on ChatGPT suggestions, etc.

You can see this approach here, I hope it proves useful to other folks! https://github.com/bakks/butterfish

A few notes here:

- An unmentioned alternative to this pricing is that GCP has a deal with Cloudflare that gives you a 50% discount to what is now called Premium pricing for traffic that egresses GCP through Cloudflare. This is cheaper for Google because GCP and Cloudflare have a peering arrangement. Of course, you also have to pay Cloudflare for bandwidth.

- This announcement is actually a small price cut compared to existing network egress prices for the 1-10 TiB/month and 150+ TiB/month buckets.

- The biggest advantage of using private networks is often client latency, since packets avoid points of congestion on the open internet. They don't really highlight this, instead showing a chart of throughput to a single client, which only matters for a subset of GCP customers. The throughput chart is also a little bit deceptive because of the y-axis they've chosen.

- Other important things to consider if you're optimizing a website for latency are CDN and where SSL negotiation takes place. For a single small HTTPS request doing SSL negotiation on the network edge can make a pretty big latency difference.

- Interesting number: Google capex (excluding other Alphabet capex) in both 2015 and 2016 was around $10B, at least part of that going to the networking tech discussed in the post. I expect they're continuing to invest in this space.

- A common trend with GCP products is moving away from flat-rate pricing models to models which incentivize users in ways that reflect underlying costs. For example, BigQuery users are priced per-query, which is uncommon for analytical databases. It's possible that network pricing could reflect that in the future. For example, there is probably more slack network capacity at 3am than 8am.

We've been running it for about a year to power Quizlet - overall things have been good and we're happy. AWS and GCP are complicated enough that they're tough to compare holistically, but on most of the things we care about we find GCP to be equivalent or better (sometimes significantly) than AWS. It really does have better networking and disk technology, and the pricing is much better. Here's the analysis we did: https://quizlet.com/blog/whats-the-best-cloud-probably-gcp.

Rough patches are:

- Live migration is sometimes not seamless.

- Pub/sub is missing some core features.

We thought a lot about this question to pick a cloud for Quizlet (~200 cloud machines) and ended up going with GCP. Here's a summary of our analysis, hope its helpful: https://quizlet.com/blog/whats-the-best-cloud-probably-gcp

To be more explicit on this point - I think Azure is a good product and the growth that Microsoft is seeing speaks for itself. However, for our use case, which is at least somewhat representative of a rapidly growing Linux-based startup, we didn't see any compelling advantage in using the Azure compute product (we may use one of their ML products in the future). Hence, it made sense to narrow the focus somewhat to what we thought were the best options. We eliminated Azure from our list based on the fact that our preliminary analysis didn't uncover any big advantages to use it over the other clouds, we wanted a cloud focused more on linux, and we don't currently use any products in the Microsoft ecosystem.

Yes I know Azure runs Linux, let me unpack that point: We had run previously on a cloud that wasn't focused on Linux hosting as their flagship OS. The effect we observed was that Linux was a second-class citizen in terms of features and performance. Perhaps its unfair to project that onto Azure, but I think its true that AWS and GCP think about Linux first, and Azure doesn't. Running a company on the cloud means relying on the compute product (GCE/EC2) as the foundation for your infrastructure, so we think this makes a difference.

It would be valuable for a lot of people to see more comprehensive stats across all clouds - I would love to see this personally and I think it would help people make better decisions about cloud infrastructure.

Sure - this isn't really a dimension of comparison, just something that I found interesting / surprising. It seems like SDN is probably the future, and this is an illustration of how its different.

Hey davisr,

I work at an ed-tech company in the Bay Area - We're a for-profit company but we've found success in building study tools targeted at students and teachers rather than top-down school district procurement. If you're interested in working in this space, or just chatting, feel free to message me. Same to others!

Not unique to Mongo, other systems that shard based on a user-generated key (HBase) have issues with this too. We developed this UUID generator as an attempt at a more thoughtful shard key: https://github.com/groupon/locality-uuid.java

We encountered some of these same issues and wrote this library to mitigate them: https://github.com/groupon/locality-uuid.java. I think UUIDs make good unique ids overall, particularly in distributed environments where id generation can't be coordinated, but should be used carefully, as the article notes.

Here's my theory on MongoDB, having spent a lot of time thinking about it:

There is going to be some huge number of new developers every year for the foreseeable future, and Mongo is the lowest barrier to entry database available. These people don't know SQL and don't care that "Postgres is better". Mongo does a few useful things really well, and everything else (like sharding) is a moat around that core value proposition, which is why Mongo gets away with being bad at a lot of stuff.

Mongo is good at:

* Starting up (when you download it, you literally just get a mongod binary).

* Inserting JSON documents.

* Indexing those JSON documents.

* Composing queries programmatically (because they're JSON documents).

* Replication (oddly).

Mongo is bad at:

* Analytical queries. I've literally written a book about Mongo and still can't remember most query syntax.

* Efficient data storage - documents take a huge amount of space.

* Scaling.

* Sharding - this feature has always been half-baked.

* A lot of other stuff.

Your immediate inclination is probably to say something like "but look at (Couchbase|MySQL|Cassandra|Rethink|Postgres), its so much better written and it does (document storage|relational querying|scaling|sharding|json documents now in version 9.4!!!!!) so much better". Again, thats not the point, the lesson here is that MongoDB is a thing because barrier to entry is a better feature for many users.

While its intentions are noble, in my experience the GPL is Free as in "you probably need a lawyer to ensure you're not violating it". Doubly so for AGPL.

The irony of RMS spreading his own vision of "Freedom" is that the GPL restricts things which he doesn't agree with, which seems to violate the most basic principle there. I've been involved with several organizations which try to avoid GPL code, by policy, in favor of MIT/BSD because it raises too many issues. Not saying GPL hasn't done good things (my understanding is that much of Apple's compiler work is public bc they started with GPL code), but the wind seems to be blowing towards less restrictive OSS licenses.

This is a problem close to my heart, thanks for pointing it out. I have a project up at http://commonwealth.io attempting the reduce the barrier to using data like this. Even when the data is accessible, it could be in any format and you have to load it into a DB to get any value.

The idea is to enable real SQL queries directly on data sets, so you don't need to worry about accessibility or how you'll query it. Not much data in it now, and it doesn't solve your immediate problem, but perhaps a better model for storing and representing this data.

Since the article is a resubmission, I hope you all will excuse a comment resubmission:

This is the 3rd or 4th time I've seen this article in the past few days so I've decided to post my take. I work with Mongo in a production scenario but I'm hesitant to post because these things tend to turn into a pointless argument. So let me stress: this is not an attack on the blog post, I'm hoping to improve the discussion here. Mongo has some real problems, these mentioned are not the problems. Here goes, written along with the article's segments.

- "It lies." Mongo used to have a default where a driver would fire off a write and not check that it succeeded with the server. This was very obviously a decision made to improve benchmark performance, though I imagine a benchmark with only default settings would be rather naive. Regardless, yes, the default was a stupid corporate decision but it is well known and should be apparent if you're deploying a Mongo cluster. Additionally, as the author notes, this default has changed and this entire point is no longer a concern.

- "Its Slow." A real point he raises is that its a little wonky you need to send a separate message for getLastError. I suspect this is an artifact of Mongo's historical internal structure. http://docs.mongodb.org/meta-driver/latest/legacy/mongodb-wi.... . If you look at the wire protocol, I think it is designed such that only the OP_QUERY and OP_GET_MORE message types get an OP_REPLY back. getLastError is a command, which are run through an OP_QUERY message. He notes that using this check affects performance. It does, but lets dig into this: "Using this call requires doubling the cost of every write operation." If the author benchmarked this, I suspect he would find that it vastly more than doubles the latency of a single write operation from the client's perspective. My understanding is that when performing this kind of safe write, the driver sends an OP_INSERT, for which it doesn't have to wait for a reply, then immediately sends an OP_QUERY message (getLastError), on which it hangs waiting for the OP_REPLY. In other words, this is now slower because we've created a synchronous operation immediately after just firing off the insert command. Again, its a little wonky that we send Mongo two messages, but one is immediately after the other and that is vastly overshadowed by the fact that we now have to wait for a reply. I believe 1 synchronous send and receive is unavoidable to ensure a safe write in ANY system, and the argument about sending 2 messages really boils down to sending about 200 bytes over a socket vs 400 bytes, I personally don't worry about it.

- "It doesn't work pipelined / it doesn't work multithreaded." He is also missing the real complexity here, and this is where I have a problem with this blog post because this is not a theoretical discussion, if its content is true then it should just be proven rather than FUD launched into the world. As noted in the docs ( http://docs.mongodb.org/manual/reference/command/getLastErro.... ), getLastError applies for the socket over which the command was run, so its up to the driver to execute the getLastError on the same socket as the write, which is an implementation detail and a solved problem. The way the drivers do this in practice is you set the write concern and the driver takes care of the rest. If you run getLastError manually then it depends on the driver, but for Java the correct procedure is addressed at http://docs.mongodb.org/ecosystem/drivers/java-concurrency/ . So for the fastest possible safe performance, you multithread (which is effectively pipelining from the server's perspective) and run operations with the driver's thread-safe connection pool. Suffice to say people actually use these drivers in a multithreaded context in the real world, and they work.

- "WriteConcerns are broken." There are several relevant settings here, including write concern acknowledgement and fsync, the author is confused about how these map to the Java driver WriteConcern enum values. The acknowledgement setting (elegantly stored in a variable named "w", thanks 10gen) is the number of replicas that must confirm the write before the driver believes it has succeeded. I personally set this to 1, but you could potentially wait for the entire replica set to acknowledge. The fsync setting is whether or not this acknowledgement means that the server on these machines has completed the write in memory, or actually synced the data to disk. I set this to false for performance. There is an excellent StackOverflow answer on Java Mongo driver configuration at http://stackoverflow.com/questions/6520439/how-to-configure-.... . The author also spends time noting that if you only ensure the write succeeds on a single machine, and you irreparably lose that machine before replication, then the data is lost. This is obviously true for every distributed system.

I've used a 4-shard Mongo in a production multithreaded environment that handled several hundred million writes. For part of this period our cluster was extremely unstable because of a serious data corrupting Mongo bug (more on this in a second). I haven't done a full audit but based on our logs, in about 6 months I've seen exactly 1 write go missing (which I believe was in the rollback log), so I'm personally not concerned about the things mentioned in the blog post. I've also been happy with performance as long as the data size comes under the memory limit. If your data exceeds memory, Mongo essentially falls on its face, though this is hard to avoid when a query requires disk accesses.

Mongo is not without its problems, however. As I mentioned, QA is a real concern, we hit a subtle bug when we upgraded to v2.2 that caused data corruption when profiling was turned on and the cluster was under high load. It was very difficult to debug, and basically should have been caught by 10gen before their release.

Another serious problem is that sharding configuration is still somewhat immature, and it seems like every new release is described as "well it used to be bad, we finally fixed it". Here is an example: you pick a shard key that can't be split into small enough chunks, and now shard balancing silently fails. Ok, so you pick a better shard key, but you can't migrate shard keys, so you have to drop the collection and start again. Except dropping a collection distributed across a cluster is buggy, so you can't recreate the collection with the same name and a different shard key. So you just pick a new name for your collection, and you have this weird broken state for the original one sitting around forever unless you completely blast your cluster and start from scratch. This sort of thing is not fun!

Mongo has many pros and cons, personally I think its real advantage is simplicity for developers which makes it worth putting up with the other stuff. Sorry for being long winded, hopefully this has been useful.

I've been using this for about a year, really good interface