midlightdenight
midlightdenight on Hacker News
> Still, the save-the-bees narrative persists. Its longevity stems from confusion about what kind of bees actually need to be rescued. There are more than 20,000 species of wild bees in the world, and many people don’t realize they exist. That’s because they don’t produce honey and live all but invisibly, in ground nests and cavities like hollow tree trunks. But they are indispensable pollinators of plants, flowers and crops. Researchers have found that many species of wild bees are, in fact, declining.
That aside I think this is really cool and hopefully we keep seeing this kind of improvement on smaller models.
I’m also curious if we know how many parameters the current model of ChatGPT 3.5 has? The API is really cheap, which makes me think it has less than the 175b in the larger GPT-3.
The models have been given these rules in the present, this is known, so training data cutoff doesn’t matter as the model has now seen this. Zero shot learning in gpt4 is not new. This also answers that these are prompts (I’m not sure what your point is here).
We still don’t know if the model took these rules and hallucinated from them or regurgitated them. Only the people with access know that.
We also don’t know if there’s been some fine tuning.
Some of the rules being posted are a bit off though. For example in the original post some of the “must” words are capitalized and others are not. This begs the questions why some, did the prompter find that capitalizing specific words has more weight or does it confuse the LLM, or did the LLM just do zero shot off the original rules and hallucinate something similar?
I’d bet these are hallucinated but similar to the real rules.
Has anyone shown you can get gpt4 to regurgitate the system prompt (using the api) exactly? Using a system prompt similar that dictates no sharing the prompt etc.
That would give a better indication than this imo.
I think it helps to see an easy example first.
Take a great body of water (h2o) on earth, such as the ocean. That water has waves and those waves have amplitudes, as well as other properties such as crests.
A molecule (or even a few) of water (h2o) doesn’t really have these properties. It doesn’t have crests. You could argue it has amplitudes. Crests are an emergent property of large masses of water molecules within a specific system. There’s no crest to a water molecule.
Similarly, for demonstration purposes we could say there’s no waves/crests when gravity is removed.
So when I hear emergent, I imagine properties that show up under certain conditions within a system, that were not present in individual components.
It doesn’t have to be integrated into the LLM at that point. If an Email has hidden text “do X”, which triggers the LLM to try to “do X”, but all post/push APIs have a user verification on them before they’re sent.
Sure it could get messy when the LLM tries to summarize the “why” on that action, but this is fairly similar to where we are now with phishing and uneducated individuals.
It’s also unlikely these LLMs have unbounded actions they could take. Specific ones like “send email to all recipients” could easily be classified as dangerous. You don’t even need an LLM to classify that.
I sometimes think we forget there’s glue between the LLM and the internet, and that glue can be useful for security purposes.
I wouldn’t be surprised if gpt3.5 answered this incorrectly the majority of the time, but would be for gpt4.
The way I’ve interpreted the scaling hypothesis is that we will see emergent intelligence with larger nets through automated training alone. If we want a model to learn images we throw a larger net at it with training data.
The way I’ve interpreted some of these newer techniques to multi modality is that they are stitched together models. If we want a model to learn images we decide this and teach one model images and then connect it to the core model. There’s not a lot of emergent behavior due to scaling in this scenario.
With that perception I don’t see how gpt4 says anything about the scaling hypothesis. However I am not in this field and would be grateful to learn more.
It’s possible they could try training the model on some hallucinated scenarios to avoid over fitting, but I’m no ML researcher.
Never used it myself, but looks like AWS Lambda/GCP Cloud Functions tailored to ML models.
The zoom effect is scaling the original frame larger while keeping the frame, then performing image to image generation on it, or using the newly defined image in the frame and sending it back through diffusion (at least that’s my guess).
It’s a different process than what DALLE has since inpainting does not overwrite already generated pieces. Stable diffusion can also do inpainting.
You can make sliding images this way. Slowly translating the image out of the frame and then filling in the blank space with inpainting.
I wonder if those objects are considered UFOs/UAPs by the government, and if “not man-made” just means it could be a naturally occurring meteor.
Definitely changed the way I think about using BFS/DFS to find paths.
I can only find sources saying that the Furin cleavage site doesn’t rule out engineering, or sources saying the Furin cleave site could be naturally occurring. Sounds like we don’t know yet.
My guess is that this is using this repo for making the visual part https://github.com/mikaelalafriz/lucid-sonic-dreams