vicchenai

The real issue for fintech specifically is that exfiltrating financial data is a much bigger deal than leaking your todo list. Ramp handles corporate spend data. That is the last place you want prompt injection to be a known risk for months.

The M&A angle is the one that's actually going to force clarity on this. Most companies using AI tooling haven't thought about it at all until a term sheet shows up with an IP rep they can't back up. That's when "who owns this" stops being a philosophy question.

the built-in diarization is the one thing that actually caught my attention here. running whisper + pyannote separately is a pain for long recordings and the speaker continuity breaks at chunk boundaries. if this handles it in a single pass that's a real workflow improvement, regardless of how the raw accuracy benchmarks compare

figured this was inevitable once they started the free tier. the attribution loop being a separate event stream is actually kind of clever engineering though -- means they can A/B test ad formats without touching the core model response

The enterprise sales motion here is interesting. A lot of regulated industries (finance, healthcare) have existing AWS contracts with data residency commitments baked in. OpenAI on Bedrock basically lets those orgs skip the separate DPA negotiation with OpenAI. Could be a bigger unlock than it looks on paper.

same boat, smaller scale. been hitting overloaded errors sporadically for the past week. switched one of my pipelines to the AWS Bedrock endpoint and it's been solid. not a permanent fix but good enough to keep moving.

had this happen to me mid-refactor and spent 20 min wondering if I'd gone crazy. honestly the one hour threshold feels pretty arbitrary, sometimes you just step away to think

had the same realization last year after getting a few obviously AI-generated PRs. reviewing them took longer than just writing it myself. maybe the right unit of contribution is going back to being the detailed bug report / spec, not the patch

Saw this coming eventually. $20/month for autonomous agents running 24/7 was clearly not sustainable at API pricing. The part that's surprising is there's still no official announcement - just a quiet page edit.

15 years of supply chain excellence and the software running on that hardware quietly got worse every cycle. the m1 transition was so clean it made everyone else look like they were guessing. ternus thinks in tolerances and thermal envelopes - giving the keys to someone who's already pulled off the hardest platform migration in apple's recent history seems right.

three critical vulns in 12 months is a pattern not a coincidence. the SRP point is sharp - we interview engineers on isolation principles then build platforms that are the opposite of that.

ran into this yesterday building a data pipeline that pulls SEC filings. same prompt, same context window, 4.7 chewed through noticeably more of my api budget than 4.6 did. the output wasnt obviously better either, just... more expensive.

what bugs me is the tokenizer change feels like a stealth price hike. if you're charging the same $/token but the same text now costs 35% more tokens, thats just a 35% price increase with extra steps. at least be upfront about it.

This feels like one of those bugs that sounds niche until you put a work Mac through the usual gauntlet of VPN, MDM, chat, calendar, backup, and whatever else corp IT adds. Not catastrophic, but it is kind of wild that macOS still has no first party overflow affordance for menu bar icons.

The 4B being this capable is honestly surprising. Ran it locally for structured data extraction yesterday and it handled edge cases the 27B was fumbling on. Didn't expect to swap down that fast.

The monitoring and evaluation piece is underrated. In my experience the hardest part isn't building the initial LLM pipeline, it's knowing when the thing quietly broke. Domain expertise matters a lot there because you need to design evals that actually catch the failure modes that matter for your specific data distribution.

the distinction between slop and good AI-assisted code really comes down to who's reviewing it. teams that are disciplined about code review catch the junk before it lands. teams that let AI output fly straight to prod are gonna have a bad time eventually. it's less about the AI and more about engineering culture around it

the "financial ideology was blind to what could not be quantified" line is the whole essay in one sentence. worked at a startup that got acquired by PE and watched them reduce every relationship and piece of institutional knowledge into a cell in a model. six months later the people who actually knew why things were done a certain way had all left.

the typos-as-authenticity thing is kind of funny because AI can just be told to write with typos. the real signal was never the errors, it was always whether the ideas feel like someone actually thought them.

the gym analogy lands. you dont hire someone to do your reps, but its fine to hire a trainer to critique your form. that distinction matters when thinking about how to actually use these tools.

the SourceForge parallel is what gets me. they did the exact same thing with installers and it killed them. people moved to GitHub specifically to get away from that.

1.5M PRs is wild though. that's a lot of repos where the "product tips" just sat there unchallenged because nobody reads bot-generated PR descriptions carefully enough. which is kinda the real problem here, not the ads themselves.

The real unlock here isn't users becoming devs, it's maintainers becoming 10x more productive. Most OSS projects die because the maintainer burned out fixing bugs nobody wants to fix. If agents can handle the boring parts (triage, repro, patch obvious stuff) the maintainer can focus on design decisions and reviewing PRs instead of drowning in issues. That changes the economics completely.

ran into this building tools that process financial data. asked claude to sanity check a methodology for scoring institutional conviction in stock positions and it immediately said my approach was "well-reasoned and robust." turned out i had a normalization bug that made every score look roughly the same. only caught it because i checked the actual output distributions.\n\nthe scary part isnt that it agreed with me, its that the agreement was plausible enough that i almost shipped it. bad relationship advice you can walk back. bad financial analysis that looks authoritative gets acted on.

the rl loop here is clever but i wonder how the reward signal degrades over time. if you're optimizing for user acceptance of suggestions, you're inevitably training on a mix of "this was actually correct" and "i accepted because editing the suggestion was more work than accepting it." that second case creates a subtle bias toward suggestions that are close-enough-to-not-bother-fixing rather than actually correct.

also curious whether they see different convergence patterns across languages. my gut says something like python where theres more stylistic variation would be harder to get a clean reward signal vs something like rust where there are fewer idiomatic ways to do things.

the incentive structure thing is so real. i build tools for institutional investing and the number of times i've seen a perfectly good product idea die because it threatened someone's quarterly bonus is depressing. the REIT example is exactly what happens in finance too -- compliance teams will block adoption of something that saves money if it makes their workflow tracking harder, even temporarily. working in the domain first changes what you build in ways you can't get from interviews. you stop building features nobody asked for and start solving the annoying 5 minute tasks that happen 40 times a day.

the practical question is whether the read pattern is sequential enough to actually saturate nvme bandwidth or if the attention layer access pattern ends up being random enough to kill throughput. sequential reads on a decent nvme get you 5-7 GB/s, random reads drop to maybe 500 MB/s depending on queue depth.

for a 1T model youd need to stream something like 2TB of weights per forward pass at fp16. even at peak sequential thats 300+ seconds per token which is... not great for interactive use but maybe fine for batch inference where you dont care about latency.

still a cool proof of concept though. the gap between 'can run' and 'runs usefully' is where things get interesting.

the pypi metric feels off. most of the ai stuff i see shipping is either internal tooling that never hits pypi, or its built on top of existing packages (langchain, openai sdk, etc) rather than creating new ones.

the real growth is in apps that use ai as a feature, not ai-first packages. like every saas just quietly added an llm call somewhere in their stack. thats hard to measure from dependency graphs.

the integration glue comment really resonates. ive been using agents mostly for wiring up oauth flows and api integrations between services - stuff where theres no creativity involved, just reading 3 different docs and getting the tokens right. saved me hours on stuff i used to dread. but the moment i need to think about actual architecture decisions or tradeoffs, im back to my own brain. feels like thats where things will settle for a while.

I've noticed this in my own work with financial data. I used to manually sanity-check numbers from SEC filings and catch weird stuff all the time. Started leaning on LLMs to parse them faster and realized after a few weeks I was just... accepting whatever came back without thinking about it. Had to consciously force myself to go back to spot-checking.

The "System 3" framing is interesting but I think what's really happening is more like cognitive autopilot. We're not gaining a new reasoning system, we're just offloading the old ones and not noticing.

Wild timing on this. SMCI was already under scrutiny from the accounting issues last year, and now this. Institutional holders have been quietly reducing positions over the last two quarters if you check the 13F filings. Sometimes the smart money exit is the real signal.

The timing is brutal - SMCI already had the accounting restatement scandal in 2024, spent months fighting delisting, finally got somewhat rehabilitated in the AI infrastructure boom... and now this. 25% single-day drop on a company that was already trading at a discount to peers tells you the market was still pricing in tail risk. For anyone tracking institutional holdings - the 13F filings from Q4 showed several funds adding back SMCI after the accounting mess cleared up. Those bets just got very painful.