It's encrypted (on top of Bitlocker) and local. There's all this competition who makes the best, most articulate LLM. But the truth is that off-the-shelf 7B models can put sentences together with no problem. It's the context they're missing.
For example, consider the classic situation of accidentally giving someone the same Christmas that you did a few years back. A sufficiently powerful personal LLM that 'remembers everything' could absolutely help with that (maybe even give you a nice table of the gifts you've purchased online, who they were for, and what categories of items would complement a previous gift), but only if it can practically store that memory for a multi-year time period.
Just did the second phase of using ocrmac (vision kit cli on GitHub) that extracts text and dumps it in a SQLite with FTS5.
It’s simplistic but does the job for now.
I looked at reducing storage requirements by using image magik to only store the difference between images - some 5 min sequence are essentially the same screen - but let that one go.
Well, that's basically how video codecs work... So might as well just find some codec params which work well with screen capture, and use an existing encoder.
And if you want to use the data for LLM only, then you don't need to store the screenshots at all. Then it's ~ 15MB a month
Cries in MacBook Pro
Shipped to the UK for me added a bit to the overall price with shipping and import duty but it was still better value for money and hugely reliable brand than anything I could have bought domestically.
“Purchased socks from Amazon for $10 on 12/4/2024 at 5:04PM, shipped to Mom, 1600 Pennsylvania Av NW, Washington DC 20500, order number 1463355337
Probably stored in a vector DB for RAG.
No mention of using any LLMs in there at all which is how you are presenting it in your comment here.
Any idea what would make you feel safe? 3rd party verification? I had it verified and published by the Microsoft Store. I feel eventually it all comes down to me being a decent person.
because I'm sure you and people like you will succeed in your endeavors, naively thinking you're doing good. and you or someone like you will sell out, the most ruthless investor will take what you've built and use it as one more cludgel of power to beat the rest of us with.
For a reality check, name one instance of policy that has stopped the amoral march of tech being a tool of power to the hands of the few? Last one I can name is when they broke up Ma Bell. Now of course you can pick Verizon or AT&T, so that worked. /s
And that was 42 years ago.
* In-depth technical explanation with architecture diagrams
* Open-source and self-hosted version
Also I didn't understand if it talks to a remote server or not. Because that's a big blocker for me.
This is my application, it does not have AI running on top.
I didn't see the benefit when the OCR content is fully searchable, in addition to not wanting to pay OpenAI to spy on me.
And then announcing "I can do your job now. You're fired."
It can also be local to my own computer. People do write software while they're away from work.
You humans think that the AI will have someone in charge of it. Look, that's a thin layer that can be eliminated quickly. It's like when you build a tool that automates the work of, say, law firms but you don't want law firms getting mad that you're giving it away to their clients, so you give it to the law firms and now they secretly use the automating software. But it's only a matter of time before the humans are eliminated from the loop:
https://www.youtube.com/watch?v=SrIf0oYTtaI
The employee will be eliminated. But also the employer. The whole thing can be run by AI agents, which then build and train other AI agents. Then swarms of agents can carry out tasks over long periods of time, distributed, while earning reputation points etc.
This movie btw is highly recommended, I just can't find it anywhere anymore due to copyright. If you think about it, it's just a bunch of guys talking in rooms for most of the movie, but it's a lot more suspenseful than Terminator: https://www.youtube.com/watch?v=kyOEwiQhzMI
And since that has to be the case, we might as well find fun and profit wherever we can -- while we still can.
If that means that my desktop robot is keeping tabs on me while I write this, then so be it as long as I get some short-term gain. (There can be no long-term gain.)
It probably breaks the spirit of the employment contract just as hard, but it's essentially undetectable for the work computer.
Kinda like making a 3D CAD model from a few images at different angles, but for documents?
As in human marriage, there should be a law that said your AI-companion cannot be compelled to testify against you :-)
https://philosophy.williams.edu/files/Egan-Learning-to-Be-Me...
And what is the likelihood of that "of course" portion actually happening? What is the business model that makes that route more profitable compared to the current model all the leaders in this tech are using in which they control everything?
I understand the actual screen recordings don't leave your machine, but that just creates a catch-22 of what does. Either the text based summaries of those recordings are thorough enough to still be worthy of privacy or the actual answers you get won't actually include many details from those recordings.
[1] - https://help.rewind.ai/en/articles/7791703-ask-rewind-s-priv...
Back then, I used on device OCR and then sent the text to gpt. I’ve been wanting to re-do this with local LLMs
Only for people who'd pay for that.
Free users would become the product.
The x230 is a popular and interesting thinkpad with a powerful i5 processor suitable for today’s needs.
The T60 can also suit your needs and is one of the last IBM thinkpads. It featured the latest Intel mobile processor at the time of its release.
If you want the most powerful thinkpad the T440p is sure to suit you perfectly without leaving your morals behind.
...accompanied by the wrath of countless others discouraging you from trying to fork if you even so much as give slight indications of wanting to do so, and then when you do, they continue to spread FUD about how your fork is inferior.
I've seen plenty of discussions here and elsewhere where the one who suggests forking got a virtual beating for it.
Machine learning is fun and ultimately it doesn't require a lot of code. If people have the compute, open source maintainers will have the interest to exploit it due to the high coolness-to-work-required ratio.
It can do as much as it wants in the background turning that into timely and non-intrusive actionable behaviours is extremely challenging.
I spent a long time thinking about a global notification consumption system that would parse all desktop, mobile, email, slack, web app, etc notifications into a single stream and then intelligently organizes it with adaptive timing and focus streams.
The cross platform nature made it infeasible but it was a fun thought experiment because we often get repeated notifications on every different device/interface and most of the time we just zone it out cuz it’s overload.
Adding a new nanny to your desktop is just going to pile it on even more so you have to be careful.
I don't know if it's public yet, but they sent me this video with the invite: https://youtu.be/dXvhGwj4yGo
Clippy + high IQ: red flag, right here
Clippy + high IQ + intimate knowledge of you: do you seriously want that? Why?
When you consider it, we aren't very far away from that at all.
Could you elaborate on what was useless about the screen capture timeline?
1. I use a limited set of tools (Slack, GitHub, Linear, email), each providing good search capabilities.
2. I can remember things people said, and I said, in a fairly detailed way, and accessing my memory is faster than using a UI.
Other minor factors include: I take screenshots judiciously (around 2500-3000 per year) and bookmark URLs (13K URLs on Pinboard). Rewind did not convince me that it was doing all of this twice as well.
A use that seems scientifically possible but technically difficult would be to have an LLM help you engage in essentially immersion learning. Set up something like a pihole, but instead of cutting out ads it intercepts all the content you're consuming (webpages, text, video, images) and translates it to the language you're learning. The idea would be that you don't have to go out and find whole new sources of language to set yourself with a different language's information ecosystem, you can just press a button and convert your current information ecosystem to the language you want to learn. If something like that could be implemented it would be incredibly valuable.
I'll probably get pushback on the quality of things like auto-generated subtitles, but I did the above to watch and understand a long interview I was interested in but don't possess skill in the language they were using. That was to turn the content into something I already know, but I could do the reverse and turn English content into French or whatever I'm trying to learn.
Imagine if it was wrong about something. But every time you tried to submit the bug report it disables your arms via Nueralink.
When it can gently nag you via a phone app to study and have a fake zoom call with you to be more engaging it feels like that could get much better results than the current online courses.
He was talking about eventually having an agent that watches your screen and remembers what you do across all apps, and can store it and share it with you team.
So you could say “how does my teammate run staging builds?” or “what happened to the documentation on feature x that we never finished building”, and it’ll just know.
Obviously that’s far away, and it was just the ramblings of excited founder, but it’s fun to think about. Not sure if I hate it or love it lol
Speech-to-text works. OCR works. LLMs are quite good at getting the semantics of the extracted text. Image understanding is pretty good too already. Just with the things that already exist right now, you can go most of the way.
And the CCTV cameras will also all be processed through something like it.
Saw this yesterday, 1M context window but haven't had any time to look into it, just an example new developments happening every week:
https://www.reddit.com/r/LocalLLaMA/comments/1as36v9/anyone_...
But yeah, dystopia is right down the same road we're all going right now.
The Four, especially with its AI, is going to control the market in ways that will have a deep impact on government and society.
As technological society advances the threshold to enter the market with anything not completely laughable becomes exponentially harder, only consolidating old money or the already established right?
What i found so amazing about the early internet, or even just the internet 2.0 was the possibility to create a platform/marketplace/magazine or whatever, and actually have it take off and get a little of the shared growth.
But now it seems all growth has become centralised to a few apps and marketplaces and the barrier to entry is getting harder by the hour.
Ie. being an entrepreneur is harder now because of tech and market consolidation. But potentially mirrored in previous eras like the industrialisation - i'm just not sure we'll get another "reset" like that to allow new players.
Please someone explain how this is wrong and there's still hope for the tech entrepreneurs / sideprojects!
https://www.reddit.com/r/memes/comments/bb1jq9/clippy_is_qui...
I bet meta is thinking of doing this with quest once the battery life improves.
>Our service, Ask Rewind, integrates OpenAI’s ChatGPT, allowing for the extraction of key information from your device’s audio and video files to produce relevant and personalized outputs in response to your inputs and questions.
The rest of the company has info on their other free/paid offerings and the split is pretty closely "what do we need to pay for an API to do vs do locally".
Again, I'm not associated with them, but that was my expectation after looking at it.
Of course taking notes and bookmarking things is possible, but you can't include everything and it takes a lot of discipline to keep things neatly organized.
So we take it for granted that every once in a while we forget things, and can't find them again with web searching.
But with the new LLMs and multimodal models, in principle this can be solved. Just describe the thing you want to recall in vague natural language and the model will find it.
And this kind of retrieval is just one thing. But if it works well, we may also grow to rely on it a lot. Just as many who use GPS in the car never really learn the mental map of the city layout and can't drive around without it. Yeah, I know that some ancient philosopher derided the invention of books the same way (will make our memory lazy). But it can make us less capable by ourselves, but much more capable when augmented with this kind of near-perfect memory.
What they fill their homes with are definitely microphones, with the google assistant and amazon echos.
By the way, Meta is already working to realize such a device. Like Alexa on steroids, but it also sees what you see and remembers it all. It's not speculation, it is being built.
https://twitter.com/_akhaliq/status/1760502294016036986
Privacy concerns will not stop it, just like it didn't stop social media (and other) tracking. People have been taught the mantra that "if you have nothing to hide, ...", and everyone accepts it.
At that point, why pay you at all?
This is already possible to implement today, so it's very likely that we'll all have our own personal AIs that know us better than we do.
Where agents will potentially become extremely useful/dystopian is when they just silently watch your entire screen at all times. Isolated, encrypted and local preferably.
Imagine it just watching you coding for months, planning stuff, researching things, it could potentially give you personal and professional advice from deep knowledge about you. "I noticed you code this way, may i recommend this pattern" or "i noticed you have signs of this diagnosis from the way you move your mouse and consume content, may i recommend this lifestyle change".
I wonder how long before something like that is feasible, ie a model you install that is constantly updated, but also constantly merged with world data so it becomes more intelligent on two fronts, and can follow as hardware and software advances over the years.
Such a model would be dangerously valuable to corporations / bad actors as it would mirror your psyche and remember so much about you - so it would have to be running with a degree of safety i can't even imagine, or you'd be cloneable or loose all privacy.