It could still have been possible to work things out based on JATO launch, though the hang-glider approach of low sand-dune testing / training would probably have been safer and a better overall option.
Powered aviation also helped drive and prove materials design enhancements and general aviation theory, controls, avionics, etc., all of which transfer well to gliders, but would very likely have been far more difficult to develop in a a glider-only regime.
I'm saying that having an airworthy craft once you're off the ground takes some doing, and it's probably easier to get there if you're experimenting with powered craft.
The early powered heavier-than-air craft were not especially aeronautically sound. But with the ability to perform design-build-test cycles, and not kill overly many pilots in the process, once engines existed design progressed rapidly.
Getting from the Wright Flyer to a modern sailplane would have been far more challenging without engines.
Or land it.
So you build your best guess at what a working glider might be. You don't have robotics or radio to remotely control it, so someone's got to fly the thing.
You build a kite flinger and/or ramp or rail to toss it off the side of a building or cliff or whatever.
And then you hope, desperately, that you weren't too terribly mistaken about a great many things.
Franz Reichelt wasn't quite building airplanes, but he turned out to have misjudged his own competence in a somewhat similar manner:
https://allthatsinteresting.com/franz-reichelt
Now consider the same situation, except that you've at least got a motor to buy your way out of design and/or piloting errors, at least a little bit.
Under which of those scenarios do you think actual useful design might have progressed faster?
Keep in mind that people have had the notion of flying with wing-like contraptions dating to the ancient Greeks (see the legend of Icarus). There were numerous inventors who threw themselves off hills or cliffs with various attempts. Lack of power, and the frequent short professional career track of such inventors tended to stymie progress.
Successful gliders and sailplanes almost entirely postdated powered heavier-than-air flight. The first designs appeared after WWI, and practical use didn't appear until the 1930s. My understanding is that popularity of recreational gliding didn't emerge until the 1950s or 1960s, again benefiting from aeronautical engineering, materials, radios, much better knowledge of aircraft operation, controls, and instrumentation.
Glide ratio is one measure of aerodynamic efficiency and sophistication. Early 1930s gliders achieved about a 1:17 ratio. Most modern gliders exceed 1:30, and the best 1:50 or more. This expresses altitude loss per unit foreward travel (e.g., 1 meter loss for 30 meters forward flight).
Google's Ngram viewer is a somewhat fickle guide, but suggests an initial spike in mentions in the late 1930s / 1940s, again in the 1950s, then a third in the 1970s:
https://books.google.com/ngrams/graph?content=sailplane&year...
Adding in "glider" (multiplied 10x) still lags "aeroplane" by decades.
https://books.google.com/ngrams/graph?content=%28sailplane%2...
There may have been earlier terminology used, and "glider" has other meanings which might confound matches, but at least for "sailplane", the trend line lags "aeroplane" and "airplane" considerably. (I've multiplied "sailplane" results 100x in this plot):
https://books.google.com/ngrams/graph?content=%28sailplane%2...
Rocket does have the trust ratio needed. But then there is questions of burn time and control. So you could get up there, with some risk. But not stay there for very long time.
Short googling for solid fuel rockets seem that longest burn times are less than 3 minutes... And that is best case scenario with 20th/21st century technology...