thomastay

This is really cool! It's like Excel's goal seek but can also handle the case of arbitrary input cells. Goal seeek can only handle one input and one output cell.

But how do you handle the case where multiple variables can be changed? If multiple input cells is the key difference from Goal seek, i think some more rigor should be placed into the algorithm here

e.g. setting A1 + B1 and wanting the result to be 5. Currently it bumps both A1 and B1 equally. What's the thought process behind this?

> However, diving into a new caching approach without a deep understanding of our current system seemed premature

Love love love this - I really enjoy reading articles where people analyze existing high performance systems instead of just going for the new and shiny thing

I think it's pretty cool that the source code is all in a single file called app.js, and it's just doing simple DOM manipulations, no React, no minification, no libraries. I like to think it's just written like that too, a gigantic file that the author just iterates on.

Dhall has imports from URLs, much like Javascript. From their tutorial:

  {- Need to generate a lot of users?
  
     Use the `generate` function from the Dhall Prelude
  -}
  
  let generate = https://prelude.dhall-lang.org/List/generate
  
  {- You can import Dhall expressions from URLs that support
     CORS
  
     The command-line tools also let you import from files,
     environment variables, and URLs without CORS support.
  
     Browse https://prelude.dhall-lang.org for more utilities
  -}
  
  let makeUser = \(user : Text) ->
        let home       = "/home/${user}"
        let privateKey = "${home}/.ssh/id_ed25519"
        let publicKey  = "${privateKey}.pub"
        in  { home, privateKey, publicKey }
  
  let buildUser = \(index : Natural) ->
        {- `Natural/show` is a "built-in", meaning that
           you can use `Natural/show` without an import
        -}
        makeUser "build${Natural/show index}"
  
  let Config =
        { home : Text
        , privateKey : Text
        , publicKey : Text
        }
  
  in  {- Try generating 20 users instead of 10 -}
      generate 10 Config buildUser

icic, yeah that definitely shouldn't be allowed

Obv it depends, but one way to "solve" this it is to show an edit history, or at least the latest edit timestamp along with some visual indicator that the message was edited recently

yeah, having eventual consistency for messages across homeservers makes the work on the client harder. I guess they just have to accept that messages will "appear in the past" as you said.

But at least for messages sent within the same homeserver, I would think that those two apis should return the same data

Having dealt with this problem at work for several years now, I feel the pain of keeping different clients in sync - it's extremely difficult. Not sure if it's possible in Matrix, but consider having a message ID that increments by one on every message in a room. That lets the client know pretty quickly if there's a gap or a misordering.

Not really getting this point though:

  The /sync API returns events in an order "according to the arrival time of the event on the homeserver".

  The spec for /messages says it returns events "in chronological order. (The exact definition of chronological is dependent on the server implementation.)".

Why would those two return different results? When does the chronological order of two messages differ from the arrival time of the event on the homeserver?

Whenever iterator design comes up, I always have to refer to munificent's excellent article: https://journal.stuffwithstuff.com/2013/01/13/iteration-insi...

tldr: The gold standard is if your iterator syntax / syntaxes can support these two types of iteration:

1. Interleaving two sequences

2. In-order tree traversal

Thanks! just read a few, was a real blast from the past

I really wish there was another platform than the app formerly known as Twitter, where AI discourse is thriving. Because right now, it seems that AI papers, discussions and articles are all being shared there first. Threads might be a competitor for that but I don't see that happening anytime soon.

It's $8/24 per M input/output tokens. For reference, GPT4-Turbo is 10/30, and GPT4 is 30/60

https://docs.mistral.ai/platform/pricing/

Hi mtlynch, I just wanna say that your post brought back a lot of memories for me. I distinctly remember reading your original post back in 2018. I remember feeling that I really hoped you would succeed since your complaints were so authentic. When I was reading this post, I was like - wait, i think i remember this post - and it's you! Happy to hear that you've made it with tinypilot

Can't you just ask GPT to give you a summary? Here's my prompt for 3.5, it's not very sophisticated.

  System: You are a helpful AI assistant who follows instructions to the letter. You will generate a summary of the article.
  Article: {ARTICLE}
  User: Summarize the previous text in one paragraph. Include as many topics as possible, make every word count. Create only one single summary and stop once you are done.

A bit tongue in cheek, but I summarized your article using artificial intelligence ;)

P.S. GPT 3.5 has a 16k context window now

--- AI Generated summary:

Marco's newsletter introduces a tutorial on developing a system in Node.js to generate summaries from YouTube videos using OpenAI's completions API. The architecture involves extracting text from videos and then creating text summaries. The process includes scraping subtitles, chunking text, and recursively summarizing using OpenAI. The system is built using Yarn, Node.js, Axios, and other libraries. The CLI allows users to summarize videos by providing the YouTube link and language code. Node.js is favored due to its flexibility, efficiency, and compatibility with the ChatGPT API. The project code and details are available on Marco's GitHub repository.

1: What prerequisites do I need to know before using ChatGPT API and Node.js for summarizing YouTube videos?

A: Before using ChatGPT API and Node.js for summarizing YouTube videos, you need to have Yarn and Node.js installed, familiarity with ES6 syntax, knowledge of HTTP calls using Axios, understanding of handling queues for promises, and the ability to work with interactive command line tools using yargs.

2: How can the ChatGPT API be applied to summarize the content of a YouTube video?

A: The ChatGPT API can be applied to summarize the content of a YouTube video by first extracting text from the video, chunking the text into manageable parts, and then using the recursive summarization technique to generate a summary by submitting prompts to the completions API. This process involves managing multiple promises concurrently to avoid rate-limiting issues and slower response times.

3: Why is Node.js a preferred choice for implementing a project that summarizes YouTube videos with the ChatGPT API?

A: Node.js is a preferred choice for implementing a project that summarizes YouTube videos with the ChatGPT API due to its efficiency in handling asynchronous operations, ease of building scalable applications, compatibility with various libraries and tools required for the project, and support for modern JavaScript features like ES6 syntax.

I'd recommend anyone who's interested in testing Chatbots to checkout https://chat.lmsys.org/

It lets you test out two random different chatbots with the same prompt and compare them. Best thing is, your votes are used to rank LLMs on a public leaderboard, which helps AI researchers.

Here's my prompt I was playing with, which basically only Claude 2 and GPT4 answers well:

  How many legs do ten platypuses have, if eleven of them are legless? Platypuses have 3 legs. Walk it through step by step

Like the idea and the vision but I'll admit I was initially turned off by the name which sounds really cringe. Went spelunking through the blog and found this line which explains the vision (which I like!):

  The kind of professional developer I’m talking about will probably fall into one of two categories.
  a: Someone who works on massive codebases
  b: Someone who writes a lot of code.

  I am targeting the small % of developers with very specific needs.

So he's targeting professional developers and not novice developers, in fact he explicitly says so later in the post.

Landing page is ok but why the animated swirling gif? Doesn't add much and draws my attention away.

Wow that's amazing! The best part is that you managed to get their entire database, that must have taken a lot of work. How did that burner account thing work?

My favorite experience with "hacking" in school involves wifi. My school had free wifi, but you had to log in with your student password. Well, the login step involved a GET request in which the password was sent in plain text as a URL parameter... so if you had your friend's laptop, it was a simple matter of looking at his browser history to see his password!

Never did anything with it, but always wondered what someone seriously motivated could have done with it

Sorry, but since the about page of sslping is down, what is it? From what i can tell from the scant information on the internet, it's a monitoring tool that tells you when your websites have a SSL vulnerability?

As others have mentioned, this is very well studied in the field of distributed computing, where servers can acquire a lock and then immediately disconnect due to a power failure.

Other people have recommended CRDTs and Optimistic Concurrency control, which is great if you are designing an app from the ground up. If you have an existing large app that just needs locks, though, I recommend following the Chubby paper: https://research.google.com/archive/chubby-osdi06.pdf

Section 2.4 Locks addresses how Chubby thinks about locking in a distributed system:

It is costly to introduce sequence numbers into all the interactions in an existing complex system. Instead, Chubby provides a means by which sequence numbers can be introduced into only those interactions that make use of locks. At any time, a lock holder may request a sequencer, an opaque byte-string that describes the state of the lock immediately after acquisition. It contains the name of the lock, the mode in which it was acquired (exclusive or shared), and the lock generation number. The client passes the sequencer to servers (such as file servers) if it expects the operation to be protected by the lock. The recipient server is expected to test whether the sequencer is still valid and has the appropriate mode; if not, it should reject the request. The validity of a sequencer can be checked against the server’s Chubby cache or, if the server does not wish to maintain a session with Chubby, against the most recent sequencer that the server has observed. The sequencer mechanism requires only the addition of a string to affected messages, and is easily explained to our developers.

Although we find sequencers simple to use, important protocols evolve slowly. Chubby therefore provides an imperfect but easier mechanism to reduce the risk of delayed or re-ordered requests to servers that do not sup- port sequencers. If a client releases a lock in the normal way, it is immediately available for other clients to claim, as one would expect. However, if a lock becomes free because the holder has failed or become inaccessible, the lock server will prevent other clients from claiming the lock for a period called the lock-delay. Clients may specify any lock-delay up to some bound, currently one minute; this limit prevents a faulty client from making a lock (and thus some resource) unavailable for an arbitrarily long time. While imperfect, the lock-delay protects unmodified servers and clients from everyday problems caused by message delays and restarts

Consider Captura: https://mathewsachin.github.io/Captura/ and if that doesn't work for you, you can always fork it.

Warning: there are a lot of Captura clones in the Microsoft store, download it only from Github.

Wow! This program is really cool and I'm honestly not surprised it's written by Mark Adler, the guy who wrote zlib, that guy is an amazing programmer.

Thanks, this would have saved me a bunch of time and I wish I'd known this before writing this article. I'll add it in as a footnote later on.

For anyone who's curious, here's the output of the infgen program on the gzip file in the article (I remade the file so the timestamp is different)

  $ cat test.out.gz | ./infgen -ddis
  ! infgen 2.6 output
  !
  time 1637813470 ! UTC Thu Nov 25 04:11:10 2021
  name 'test.out
  gzip
  !
  last                    ! 1
  fixed                   ! 01
  literal 'a              ! 10010001
  literal 'a              ! 10010001
  match 6 1               ! 00000 0000100
  literal 10              ! 00111010
  end                     ! 0000000
  ! stats literals 8.0 bits each (24/3)
  ! stats matches 66.7% (1 x 6.0)
  ! stats inout 5:6 (4) 9 0
                          ! 00
  ! stats total inout 5:6 (4) 9
  ! stats total block average 9.0 uncompressed
  ! stats total block average 4.0 symbols
  ! stats total literals 8.0 bits each
  ! stats total matches 66.7% (1 x 6.0)
  !
  crc
  length

Thanks, appreciate it. Honestly, the switch from little endian to big endian within the same compressed blocks is incredibly confusing and is probably one of the things that is so weird about the Deflate specification. I had to read that part of the spec over and over again, and compare it to other people's blog posts (made harder because some people write bits from Right to Left???)

I personally think it's easiest if you write bits Left to right, from LSB to MSB, which fits how humans write words, but i also get that when you're programming, you tend to think of MSB being on the left and LSB on the right (for bit twiddling operations)

So I can see why people would write bits right to left too, to fit how you program, but frankly I don't think that is any easier to read on paper.

Hey, I'm really curious about how that works. Could you go into a bit more detail into that?

From what I can gather on your comment, it seems that if you are given a splay tree, and a batch of transactions i.e.

  GET 1
  SET 10 "hello"
  GET 15

You would write all the visited nodes, in order? But splaying involves a lot of tree operations and rotations too. I don't get how that works.