You’re not wrong, but AI being passably good at bullshitting (while mostly keeping a rhyme/meter) is still mindblowing to me.
Subjectively incredible; objectively flawed.
Edit:
I guess in this description dijkstras is more specific, in that dijkstras is the specific instance of A* with a zero heuristic.
But I think what HAL over here was saying is that you can use dijkstras in a superset of scenarios in which you can use (non-trivial) A*, so in that sense dijkstras is more general than A*, so it's not wrong.
For A*, everything depends on the heuristic, as you said. If h(v) = 0, A* is equivalent to Dijkstra's algorithm, so potentially, it can be as general. But the wrong heuristic (inadmissible, inconsistent) will lead to wrong results, and so calling it a "specialized" tool is correct. The heuristic gives you a specialized version of Dijkstra's algorithm which is faster on specific graphs.
> Dinkstras is a-star with a "min weight on shortest path first" heuristic.
I am not quite sure what you mean here. Dijkstra's algorithm is A* with no heuristic. I am not sure how a "shortest path first" heuristic would look like - do you mean that Dijkstra's algorithm chooses the node to expand next based on its shortest-path cost to the target t? Even if you would construct such a heuristic h(v) = c(v, t) by explicitly calculating c(v, t) each time, this is not what Dijkstra does, as it would basically be a perfect heuristic - you would then only visit nodes on the shortest path.
If you meant that Dijkstra's algorithm is A* with a "neighboring node with shortest path from start node first" heuristic, that's also not strictly true, because expanding the nearest node first is already built-in to A*. A* does not chose the next node v with the smallest h(v), but with the smallest g(v) + h(v), where g(v) is the shortest path cost from the source node to v.
I take issue with this statement. "any case" implies it's more general than A star, which is not true, it's a specialization of A star with (ok you win) h(v) set to nothing. It necessarily cannot solve as many problems as a star.
And Dijkstra's won't work in "any case" at all, esp graphs with negative edge weights.
The explanation struck me as glossing over too much, to the point of being misleading. Perhaps I'm being too pedantic, fine, but a more enlightened comparison would read more like our discussion than what was given.
Could you give an example of a positive-weight shortest path problem with A* can solve, but Dijkstra's algorithm cannot? I do not believe there is such a thing. Or did you mean something else?
> And Dijkstra's won't work in "any case" at all, esp graphs with negative edge weights.
A* won't solve those either.
they both solve the single-pair shortest-path problem in digraphs with nonnegative arc weights
a* which explores nodes in a different, better order thank dijkstra's algorithm, but can only be applied in cases where you can compute an admissible heuristic
that makes it less general than dijkstra's algorithm, which works to find shortest paths in any digraph with nonnegative arc weights, not just the ones where an admissible heuristic can be computed
any problem you can solve with a* can be solved with dijkstra's algorithm (usually more slowly) but the converse is not true
now, in a sense, a* with an inadmissible heuristic such as h(v)≡0 is 'more general' in the sense that it can emulate dijkstra's algorithm and also do other things; you could say that a* is a class of algorithms of which dijkstra's algorithm is one
It's not in the style of a psalm.
So I wonder if there was substantial fine-tuning for chatgpt specifically to reward it to generate poems, in a particular style.
And here, it's "over indexing" on that and still generating poems in that familiar style
That's chatgpt right now. To get it to confidently right will take years and years more
In less than a decade, I believe front line support chat support jobs and even graphic artist jobs will be made obselete
We can finally get the eighty hour long book-accurate Lord of the Rings adaptation we deserve.
But seriously, I can't wait to feed a novel into one of these things and get a comic book version in return.
It's already at the point where you start expecting any knowledge worker to be significantly more productive by leveraging these tools.
It's hard to imagine that it will be more than five years before AI tools are available that can handle almost all tasks in these types of jobs.
For example, on my website aidev.codes I just added preliminary knowledgebase support. It can reference the knowledgebase this to write code. I would say that with the code-davinci-002 model at least it seems about at the level of a junior software engineer already since it's pretty effective with close supervision by a senior peogrammer, except for the fact that it cannot interpret visual information.
Knowledgebases/embedding search can also be used right now with these models for answering support questions. The only thing holding it back from very very wide scale adoption is the problem of making up information. There are already solutions in progress for this. It's unlikely that will take more than a few years to roll out and replace the current generation of models. Google and Microsoft will probably roll out their internet-scale chat search interfaces this year even if they can't fully mitigate the hallucination problem immediately.
I would guess more like 2-3 years for many knowledge-based jobs. If you want employment/contracts you will need to be very good at leveraging AI, or people will just use the AIs instead.
Very human I must say.
https://www.theregister.com/2022/01/31/machine_learning_the_...
Even if you are right, you still acknowledge that ChatGPT is on the level of a non-specialist bullshitting human you can have a meaningful conversation with. This alone would be enough to freak me out. If you would've told me that this is possible 10 years ago I would've called you crazy.
While I wont deny that it has impressive summarization abilities in terms of making excellent Q&A if you're willing to vet the information, I wouldn't exactly say that chatGPT is capable of meaningful conversation. It has great powers of recollection but its capacitive powers to produce new interesting information feels highly formulaic.
The sun will not harm you by day,
nor the moon by night.
My life is consumed by anguish
and my years by groaning;
my strength fails because of my affliction,[b]
and my bones grow weak.https://paperswithcode.com/paper/most-language-models-can-be...
Gwern cited it too!
It's not a crystal ball, it's a mirror.
There is obviously more to human intelligence than text based conversation but it is pretty humbling that such an aspect of ourselves can be replicated and be so convincing at such an early stage and perform better than some humans even when it makes stuff up: toddlers can't talk, kids are smarter but don't have the technical knowledge, most adults only have a few specific areas of expertise, etc.
but not always; And that's what makes it scary. You know people will use it and trusting it.
as far as i can tell it seems perfectly correct but admittedly my understanding of a* search is limited to messing with amit's visualizations of it
This is just generic poetry with a sprinkling of "ye's."
Notice that it sometimes struggles to rhyme?
Does it really matter when it comes to real life? Kids take medical advice from TikTok influencers. Half the country believes absurd news they see on Facebook. Half the country believes anything their president says.
Maybe it will just peter out at this level like self driving car technology did, but I really doubt it, and it seems to me that the building blocks are already there to commercialize this and use it in a real way. The future is going to be interesting...
"Do X" is great. But "Now change X to Y in the style of Z" is what blows my mind.
I initially put in a prompt like "tell me a story about the hunting dog and the guard dog." and the results were pretty meh. I was interested to see if it was actually pulling in data from FB so I put her name in instead (she had an FB page yes). It did not seem to do much. So I put in some things like "the little hunting dog is always hungry." "the hunting dog is always looking to hunt" and then did a fuller prompt of "the hunting dog and the guard dog are going to the park. tell me a story about it" and finally got a very good fully fleshed out story about how the little dog was on the prowl for a squirrel while the guard dog watched out and it was really well written and made about as much sense as a fictional tale could. There was no awkwardness or other tells that would have made you think that this was written by anything other than a human.
Anyway, that is huge that it can do this. I do remember when I used to play with other chatbots that remembering past context was something they were unable to do, at least not well, but in most cases it seems they didn't even attempt to. ChatGPT has blown me away in many ways, but this is a very specific leap forward.
I was showing it off to my wife. I asked it to write a lecture for an intro to Latin American poetry with a focus on Pablo Neruda, including a background on Chilean history.
The first result was just 5-6 paragraphs with the most rudimentary 101 on all the above topics.
Then I told it to rewrite it as if the lecture was going to be given to grad students. Next, I told it to include examples of Neruda's poetry along with some analysis. I also asked it to flesh out the segment on Chilean history, with a focus on the independence movement.
The final lecture was good enough that you could walk into a classroom of college seniors and deliver it without any edits. The first one might have been good enough for 9th graders at most.
In fact, I'd submit that phenomenon as its own entry in the Weirdest or Most Surreal Tech Trend contest. People who, when confronted with a talking dog, say things like, "This talking dog is an idiot. It thinks 429 is prime, its painting looks like something from Bob Ross on acid, and the C++ code it wrote for me is full of bugs. I don't see what all the hype is about."
Some of this dismissiveness is just whistling past the graveyard, but much of it comes from people who genuinely think the human brain is something that operates by means of ethereal, ineffable, and unreproducible mystic forces.
Its just a mindset I don't understand. And on places like reddit there is definitely some astroturfing going on or at least some definite "camps" that don't like SDC technology, but on HN AFAICT its been genuine disdain. So weird.
Apparently Cruise and Waymo are doing much more impressive things, but they only get a fraction of the press.
Tesla refuses to publish this data.
Tesla operates nearly everywhere. The scale of what's happening isn't even comparable.
https://sites.google.com/view/fsdbetacommunitytracker/home?p...
Even the best self reported areas are less than 100 miles between disengagement.
Waymo and Cruise are at around 30,000 miles between disengagement.
Tesla has released nothing.
No one serious in the self driving space is using disengagements to pick winners or leaders on the path to commercialization. There is no standard about what qualifies as a disengagement and companies will interpret and report their disengagements differently. Those interpretations also change over time, making it even more difficult to actually understand how a company has progressed in its technology.
And again you're missing the point. Waymo/Cruise are operating in an extremely different and much more narrow context than Tesla's FSD. Let's go put Waymo on a random road in a random city and see how it fares compared to Tesla FSD. I guarantee it will be tremendously worse and no one who knows anything about the space would argue otherwise.
Also, why is Tesla so adamant about how many accidents occured simply with FSD disabled? Shouldn't we know how long prior to the accident FSD was engaged? I mean, hitting the brakes in panic only technically makes it "not FSD".
I'd be happy if you showed me any reported data, self or Tesla, of 30k miles before disengagement with FSD. Even within one order of magnitude instead of two.
You pretty clearly don't know what you're talking about.
> Apparently Cruise and Waymo are doing much more impressive things, but they only get a fraction of the press.
They are doing objectively much less impressive things by relying on sensors that require much less intelligent analysis and will lead them to dead ends at the higher end of abilities that Tesla's approach won't.
This is one I take the side of the skeptic, and I can explain why. In college ~1994 a friend entered an autonomous ground vehicle contest. It was a big deal and our university hosted it each year - this was the 2nd year. He had cobbled together an unusual electric wheelchair base, a 486 PC and a tripod with an old camcorder looking down and forward with a frame-grabber card. The course was about one lane wide, bounded by 2 white lines painted in a grass field. There were some other obstacles and stuff (later years they had sections of dashed white lines and other complexity). We had talked about line following algorithms and such, and he had tried some things with limited success.
One day I said I had an idea, and he said OK but I need the hardware back in an hour. I don't recall if it was C or Pascal, but it was familiar enough that I coded something in an hour - with a visual display showing the 15 horizontal lines of pixels my code analyzed (at about 10 fps). We added some stuff later to incorporate sonar data, but that hour of code got us second place. I'm not here to brag about my fabulous code, I'm here to tell you that "faking it" with lane following and simple driving skills is really easy. There was zero "understanding" in what it was doing, zero ability to recover when it went wrong, zero of just anything but the simplest of code to enable moving forward without crossing the lines. I could write pseudo-code for it here in a comment in 10 minutes. When you think about it, driving safely on real roads requires semantic understanding of a lot of things and it's impossible to observe a vehicle driving and have any idea what level of comprehension it has. This combined with personal experience knowing how easy it is to do rudimentary lane following causes me to set the bar very high indeed. Parlor tricks are not safe, and whatever they have on the road today surely falls somewhere between that and fully sentient AI. But where is it exactly? What is good enough for your spouse and kids? And what is good enough for someone else to be letting drive on the road with them? Or while they cross the street on foot?
When a car can't "see" a semi crossing the road, or one doesn't identify something as human because it's not in a crosswalk, or another runs into and pushes an airplane, no I'm not impressed in the slightest at what these things CAN do reasonably well. The instances of what they get wrong tell me so much more about how far they have to go.
GPT wasn't hyped much at all, in comparison. It just appeared. The next couple of iterations will be interesting to watch, since right now we don't really know what the rate of improvement is going to be. We might be fighting Skynet next year, or we might be in for another 50-year-long AI winter.
I think it's a good parable. It's something expensive and man made. Even though it's on average dumber than a typical human, people try to ascribe godliness to it. And so many judge whether it has achieved omniscience. For many of us, it's just a better tool, an IDE, but for some, it's an oracle or divination tool that they use to decide whether they should marry someone.
I think the term "artificial intelligence" makes it even more confusing.
1. We've learned that relu is a really good function to use for neural networks.
2. We've learned that regularization is essential to having large networks without overfitting.
3. We've learned how to scale models up and down so that you can train something on a giant cluster then run it on your phone.
4. We've learned how to apply this to lots of specific kinds of problems.
etc, etc, etc.
We do NOT simply do what we did in the 80s then throw more hardware at it. Instead we are getting better at making the hardware do something useful.
ChatGPT3 isn't really the definitive sign of the latter. Its just a stepping stone on the way. As soon as humans discovered differential calculus, that was the genesis to the eventual development of a general AI. Everything else along the way is an issue of optimization and spending raw compute time, whether its done by humans or machines.
I've seen so many pieces of incredible technology struggle to find a use-case. By contrast, ChatGPT almost has too many use-cases to catalog, and while it's far from perfect, it seems obviously, to me, to be a tipping point of sorts.
We're living in exponential times.
I can accept a system such as ChatGPT synthesizing from data it sucked in, making educated guesses and so on. But to see it do such lingustic gymnastics with a very non-concrete request was humbling. It's given me a lot of pause about the way I absorb digital information and the varying degrees to which I have implicitly assumed the reliability of that information; site A slightly more trustworthy than site B, et cetera. To me, that old trust heuristic I relied on, one I have honed thanks to unfettered broadband for two decades, is now completely upended. It probably has been for some time to be fair, but my time with ChatGPT really cemented that feeling.
Every freshly written statement that comes to me through an internet connected device now gets a side-eye by default.
On that note I find it interesting that this has sometimes been an argument to dismiss chatGPT as "non intelligent". What are we if not statistical machines, synthesizing from the data we've sucked in over our lifetime?
I feel I think too much about mundane things and sometimes about wondrously confusing things often to my detriment. And at least a few times a week I would swear I run into a person with the opposite problem. Their behavior, choices, preferences. Their thought processes (at least what they share with me) come off as closer to machine than human.
Something in my gut tells me we are more similar to some of the things we build than we might care to admit.
Reasoning machines who are actually pretty bad at statistics.
For example since the world is uncertain and chaotic we seek either to conserve or expend for future gains. This can explain many stock trading behavior, despite often being a non optimal viewpoint
A scary number. I wonder about a per-subreddit distribution, though. I imagine the primary subreddits have slightly worse human-to-bot ratio, niche subreddit somewhat better, with non-political, non-easily-monetizable subreddits having the best.
Did your analysis also attempt to identify troll farms? Would the content produced by protein bots be grouped in the ~65% of bot content, or the remaining 35%?
you have won cake. There will be a party in your honor
Please lie on your stomach with your arms at your sides. A party associate will arrive shortly to collect you for your party.
I think this has been conventional wisdom for a few years now.
https://en.wikipedia.org/wiki/Correct_the_Record
The methods matter less than that it's happening at all.
I don't know whether to believe people are so wasteful of their own time or whether this is just low-effort bot posting to build consensus. Combined with how harshly and instantly main subreddits like /r/politics and /r/news shadow ban accounts, it's basically impossible to dissent
I grew up using archaic, Jamesian English regularly (in a religious community), so it always sounds wrong when people use them incorrectly while attempting to be old-timey, which is a bit distracting and ruins the effect.
Apparently ChatGPT suffers from hypercorrection as much as any modern speaker.
(The meter is also dreadful, which is another thing that I think many modern English speakers don't often do precisely.)
There was an "ask HN" a week or so ago about knowing how your personal machine was or was not compromised ...
I am imagining training a model on all of the PDFs and email and receipts and contracts that I have spent the last 30 years protecting ... and then exploding my home directory with 2M parallel copies.
Now what ?
Which is the real correspondence ? Which contracts are real ? Which invoices and receipts ?
Could one construct a framework where all data at rest is worthless and only witnessed transactions and traffic have any surveillance value ?
On the other hand, note that you'd also need some way of distinguishing ... wheat from chaff ... for your own ability to track your correspondence, transactions, records, etc.
Otherwise, that is an interesting idea, particularly in a world in which storage is cheap relative to the size of documents. We're rapidly approaching an era in which storage capacities sufficient to hold the entire Library of Congress collection of books (as text) on a residential PC or server will be highly viable.
The larger problem is probably that your copy of that data isn't particularly important in most cases, it's what lies on others servers that's been generated by you / snarfed from your devices / activities. And that's far harder to chaff.
Another issue would be that any generated content that raises / matches suspicions could, in a regime with poor civil liberties protections, simply become more ammunition used against you. The old Cardinal Richelieu "six lines" dictum, though in this case, not necessarily in your hand.
I have a very good understanding of how deeplearning neural network work and language model like "Word2vec" that convert a sentence or a word to a vector space.
But I completely fail to understand how Given an initial text as prompt, ChatGpt will produce text that continues the prompt.
All of this in one session.
[0]