cccybernetic

An incredibly written film that's infinitely quotable.

"Balls. We want the finest wines available to humanity. We want them here, and we want them now!"

"Here Hare Here"

"I feel like a pig shat in my head."

"I don't advise a haircut, man. All hairdressers are in the employment of the government. Hair are your aerials. They pick up signals from the cosmos and transmit them directly into the brain. This is the reason bald-headed men are uptight."

Most PDF parsers give you coordinate data (bounding boxes) for extracted text. Use these to draw highlights over your PDF viewer - users can then click the highlights to verify if the extraction was correct.

The tricky part is maintaining a mapping between your LLM extractions and these coordinates.

One way to do it would be with two LLM passes:

  1. First pass: Extract all important information from the PDF
  2. Second pass: "Hey LLM, find where each extraction appears in these bounded text chunks"

Not the cheapest approach since you're hitting the API twice, but it's straightforward!

Shameless plug: I'm working on a startup in this space.

But the bounding box problem hits close to home. We've found Unstructured's API gives pretty accurate box coordinates, and with some tweaks you can make them even better. The tricky part is implementing those tweaks without burning a hole in your wallet.

Sure, you can try the demo at:

https://www.subsystem.ai/demo

I don't feed documents directly to an LLM. First, extract and process the data in a structured way that maintains the hierarchy and metadata of the content (this is important!). Then convert this into a scheme that you can control — it doesn’t really matter what it is (JSON, XML, markdown). From there, feed this to the LLM in chunks. This will get you most of the way there.

There's different ways to validate, but that's why maintaining hierarchy and metadata is so important. If you track this information properly, you can cross-check responses across different LLMs!

I built a web app that extracts data from documents, like PDFs, Word, etc. I've seen people say "GPT wrapper", but it consistently outperforms similar tools in the space. My main customer is a private equity fund that randomly reached out. I didn't know much at all about fintech, but it works and gets the job done.

I don't have a proper marketing site yet since I've been focused on building the app, but it's coming soon (hopefully...)

I built a drag-and-drop document converter that extracts text into custom columns (for CSV) or keys (for JSON). You can schedule it to run at certain times and update a database as well.

I haven't had issues with hallucinations. If you're interested, my email is in my bio.

I haven't seen it framed this way, but yeah - well put.

Absolutely, email is in my profile. Please reach out.

This is a problem I’m working on.

I’m a software engineer at major US research university developing AI-powered software to improve critical reading and writing skills in higher ed. The idea is to provide immediate, high-quality feedback to students, closing the “latency” of submitting something and waiting to hear back from you professor.

I do genuinely think AI can reshape teaching and learning, but it will be a slow iterative process. We can use it scale what works (personalized learning and tutoring, helping students develop mastery/automaticity on topics, targeting areas where they struggle). It can also automate time-consuming tasks that bog teachers down.

If you're interested in pedagogy, AI, and tech, please reach out.

Awesome, thanks for sharing.

To add to this, John Perry Barlow, one of the Dead's two main lyricists, co-founded the Electronic Frontier Foundation.

This presumes a theory of justice where "social benefit" is relevant at all — not everyone accepts this.

this is the hardware version of the (in)famous hackernews comment on Dropbox:

> you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem. From Windows or Mac, this FTP account could be accessed through built-in software.

There's something to this.

I’m a senior dev in NYC with ~7 years experience working across the stack (NextJS/ReactJS, Node, Python, Postgres, SQL Server, etc). I’m also not half bad at design.

I haven’t been able to get a response much less an interview.

Insulin is available at Walmart for about $25. The newer analog versions cost around $80. It's just private label Novo Nordisk, so all high quality.

I see your point, but honestly, the only way I can respond to your question is with more questions.

Why do we expect deep psychological or philosophical insights to make things better for us, or lead to “improvements”? Why do we think that a profound revelation will bring clarity and resolution to our lives? It could just as well make us more crazy.

There’s the famous case of the “Rat Man”. Whether Freud cured him is up for debate, though he most likely didn’t. But it’s a deep mediation on neurosis, obsession, cognitive dissonance, love, hate. It raises more questions than it answers but it's incredibly insightful. It gets at the heart of human fragility and I think that's something to take seriously.

This is a very limited perspective, reducing Freud and the development of psychotherapy theory to, as you put it, feeling “a bit better”.

Freud was as much a mystic, philosopher, and literary genius as he was a psychiatrist. I've found his ideas to be extraordinarily deep and they have influenced my life and thinking in profound ways.

Similar thing just happened to me.

Click on link > Get API Key > "We are sorry, but you do not have access to Early Access Apps"

Since I'm the admin, I checked and confirmed that I do have access. It's enabled for both my account and the entire organization.

Whatever.

Thanks for sharing, you're right - that does improve performance!

Obviously not. Perfectly reasonable to share anecdotes though.

Also, I ran a few different tests, and every GPT-4 response was superior, but I didn't want to clutter my comment with queries and code.

I'm glad there's growing competition, but GPT4 still outperforms. Here's a recent question I asked:

Could you please provide me with a query for my PostgreSQL table? It consists of three columns: 'id,' 'teaser,' and 'full_text,' all of which are of type 'text.' Most 'full_text' entries are articles around 1000 words in length. I'd like to update the 'teaser' column for each row with the first approximately 200 words from the 'full_text' column. How can I achieve this?

Phind:

  CREATE OR REPLACE FUNCTION get_first_n_words(text, integer)
  RETURNS text AS $$
  DECLARE
     word_count integer := 0;
     word_start integer := 1;
     word_end integer;
  BEGIN
     WHILE word_count < \$2 AND word_start <= length(\$1) LOOP
         word_end := position(' ' in \$1 from word_start) + 1;
         IF word_end = 0 THEN
             word_end := length(\$1) + 1;
         END IF;
         word_count := word_count + 1;
         word_start := word_end;
     END LOOP;
     RETURN substring(\$1 from 1 for word_start - 1);
  END;
  $$ LANGUAGE plpgsql;

GPT4:

  UPDATE your_table_name
  SET teaser = (
      SELECT STRING_AGG(word, ' ') 
      FROM (
          SELECT SPLIT_PART(full_text, ' ', i) AS word
          FROM generate_series(1, 200) AS i
      ) AS words
      WHERE word <> ''
  )
  WHERE full_text IS NOT NULL;

A blog on using using LLMs to clean, process, and enrich data. It includes prompts and code snippets. The post draws on my experiences and two really interesting papers:

- Can Foundation Models Wrangle Your Data? (https://arxiv.org/abs/2205.09911)

- Large Language Models as Data Preprocessors (https://arxiv.org/abs/2308.16361)

I cover:

- Error and Anomaly Detection - Enriching Data with LLMs - Matching Data Labels - Identifying Matching Records

Thank you and I'd appreciate your feedback.

The API isn't redundant, it's convenient. You don't have to use it. Just delete the /api folder and make your requests either from the client side or fetch the data inside the handler that you're complaining about. With the new app router, you can fetch it from the server component itself without the handler.

I'm the founder of a startup (www.getsubsystem.com) that builds solutions on top of Airtable, so I'm familiar with their systems. While it's true that their REST API has some sharp-edges, particularly around linked fields (columns), rate-limiting, and filtering records, I think it’s mostly solid and user-friendly.

Getting records from a table is as straightforward as making an API call to: `/{baseId}/{tableId}`. For getting the actual ids, there's a metadata API. The docs are also quite comprehensive. I’m curious as to where you have run into issues?

It takes < 10 seconds to go from no account to database w/ bit.io

Consumers have always been willing to pay a premium for simplicity & convenience. No difference here: devs will pay for DX.

There’s a lot of magic with Vercel, but I think the true “wedge” was clear to many people: DX. The trick was really the execution, which they nailed. They realized early that there was a whole class of developers out there, many of them quite talented, who couldn’t stand SSH’ing into Linux boxes, configuring htaccess files, installing SSL certificates, etc. Many of these developers also worked at big companies, and held sway.

They distilled the entire deployment process into a few clicks and boom, you have a real, functioning, fast website. Then they added templates, good marketing, and doubled down on React. The rest is history. It’s a well executed vision and I give them a lot of props for that.

yeah I agree with a most of this, but useEffect is still a shit show. It's nasty, and it sits in your code like an anvil, heavy and terse, it even looks like one, with it's lopsided dependency array. The semantics are so strange, and it's not even really for side-effects, more so for synchronizing state. And while I love React, it's not uncommon to see entire codebases where every function is wrapped in useCallback and useMemo, you know, "just in case".