Does being easier not influence whether it's better? I mean this in that for many of the ways AI art would be used, MJ already seems to be "good enough" at a lot of it.

Secondarily: doesn't Midjourney's increased user base and increased ratings they get from users help it refine its model, thus meaning that "ease of use" creates a feedback loop with "quality of output" because more users are engaged?

I'm asking real questions, not making a statement I believe in and just adding a question mark.

As mentioned, what is better, MS Paint or photoshop? If MJ ignores your prompt and spits out a half related picture, are you going to continue using it?

If anything MJ is a stepping stone to SD. You get a taste of AI art, but want to do something specific that MJ cannot do. You learn about control-net, alternative models, inpainting, etc... and you decide you need to move on from MS Paint to Photoshop.

I personally used free AI art(cant remember which), it was super cool, but quickly I wanted to use different models and generate thousands of pictures at a time. I wanted to make gifs, img2img, etc... and the only people doing that were on SD.

The question is not "which is easier?", but rather "which is easier to use to produce high-quality output". In your analogy, I'd argue the answer to that question is actually Photoshop. Likewise the answer in the MJ/SD case is MJ.

Metaphor is useful, but this feels overly-reductive. The gap between the amount of effort it takes to make something great or approaching the vision you had is massive between MS Paint and Photoshop. Not so for SD and MJ.

However, I am appreciating that SD seems to be clearly better if you need something more specific / precise. I don't think I'm convinced (yet) that because it can get more precise inherently makes it a better tool.

There are definitely some professionalized paradigms emerging in the use of SD: one video tutorial I saw this morning covering basic photobash + img2img and use of controlnet had a commenter saying that they preferred using the lineart function in controlnet to get more control and leverage their drawing skills.

When you see that kind of thing it's a huge signal for professionalization, because someone suggesting the optimal first step is "learn how to draw" deviates so completely from the original context of prompt-based image generation: "just type some words and the image magically appears".

Midjourney let's you type in a thing and get a result that will look great, which is no small accomplishment. If you want "incidental art" like blog post heros there is no competition. But it's really hard to use if you want to get exactly what you want.

Alternatively, if you don't know what you want, it's really good for inspiration.

In my opinion, it is tedious to discuss whether Midjourney or Stable Diffusion is better.

Both are tools with their own strengths and weaknesses.

Midjourney

- Comparable to a ready-made consumer product.

- Easy to use and you get excellent results quickly.

- The feature set is limited. So you can't influence so well how the result should look like.

- Restrictions on what you can generate with it (gore, nudes).

- Fee required.

Stable Diffusion

- Comparable to a development kit. Technical, Difficult to use, different tools that you have to configure and use correctly.

- For excellent results you have to invest much more time.

- Extensive feature set (inpaining, outpaining, upscaling, ControlNet, many different models, LORAs, textual inversions). You can much better define how the result should look like.

- No restrictions on what you can generate.

- Free if you install it locally.

https://www.runpod.io/console/templates

Look for "RunPod Stable Diffusion". I spent a whole $0.35/hr playing around with my own SD instance that I had running in minutes.

The main upside of running your own SD is that you can completely automate it, but I'm not sure how useful that really is.

No, the main upside of running your own SD web UI is that you can select and deploy your own checkpoints (not just using the base SD models), LoRas, embeddings, upscaling models, and UI plugins supporting additional services/models/features like multidiffusion (bigger gens and controls of which areas within the image different prompts apply to), ControlNet and associated models, video synthesis, combinatorial prompts, prompt shifting during generation to do blending effects, and, well, a million other things.

Also, you can completely automate it.

It's a little inconvenient to use non-base models and plugins this way (you pay extra for more storage), but it's definitely an easy way to use the full power of SD.

https://github.com/AUTOMATIC1111/stable-diffusion-webui

Or are you looking for the cutting edge stuff like control net?

If you want to use colab instead, I used this a month or two ago.

https://colab.research.google.com/github/TheLastBen/fast-sta...

I hope other people can give you further reading.

Even trained their own models using a cloud GPU.

The SD ecosystem is wild.

If you can afford Colab (which is free if you don’t want to use it too much), you can just click one of the existing A1111 colabs and run that, you don’t need to figure out python, git, or A1111 installs.

I was under the impression that midjourney was just running a form of SD and it's real secret sauce are the peripheral prompts it injects on the backend along with your prompts.

I could be totally off the mark here.

I could be entirely wrong, though. Maybe the person making that decision is just an idiot.

I have an IRC bot that has triggers for DALL-E, GPT3, and ChatGPT. I really want to make one for MidJourney, and I would happily pay MJ for the privilege. But I can't. Not without breaking some rules.

They're good enough technically at what they do that their audience is okay with the interface that they have to use, I'd reckon (I've heard similar beefs about the UI stuff though, so it sorta makes sense).

Would you mind explaining what you mean by that? Thanks!

The only way to do it right now would be to create a separate dedicated discord account linked to midjourney, and then have your bot control that user account and that's a good way not only to get your MJ access revoked but also to have your discord account banned.

As far as kneecaping their growth, they have one of the largest discord channels in history and they frequently run up against compute limits, so if this was one of their ostensible goals they failed at it pretty spectacularly.

AFAIK, this isn't possible.

Nothing so… social

By what measure? Midjourney v5 is massively better with every prompt topic I've thrown at it than SD. It's not even close. SD, however, is much better if you want an actual customizable toolchain or to do things like train it on your own face/character.

https://paperswithcode.com/sota/text-to-image-generation-on-...

Parti and Imagen are still on top, followed by Dall-E 2.

If their model is so great, why are they afraid of benchmarks?

For a while I was using an FID variant for evaluation during training, but didn't find it very helpful vs just looking at output images.

A 1000 V-rolls won't get you there. For something like this control net combined with inpainting is indispensable. Not to mention the excessively heavy handed censorship in MJ.

Midjourney excels in overall quality, but it completely falls down if you have an actual complex vision.

Here have another prompt: "Rapunzel has let her hair all the way down a tower. The hero has been tied up by the witch, and annoyed at Rapunzel's continual attempts to escape, the witch throws the bottom of her hair into a paper shredder at the base of the tower."

You could v-roll until the heat death of the universe without even getting close.

Midjourney is great if all you're capable of conceiving is 90s Mad Magazine templatized mad Libs, banal crap like "Darth Vader as a French pantomime street artist".

Unfortunately that also describes the majority of midjourney users.

https://www.bing.com/images/create/a-monk-playing-chess-agai...

https://www.bing.com/images/create/a-blue-book-on-top-of-a-y...

Simple (prompt only) use of generative models is quite good at creating simple artistic pictures you might actually hang on a wall. Trying to create a complex scene with just a prompt seems still a few years off though.

I use SD to augment MJ, like fixing hands with specific LORAs for example, so I definitely appreciate that it exists. But for actually creating a full image in one shot, they're not even comparable.

I've twice spent a couple of hours unsuccessfully trying to generate a simple background image that would be blurred out when rendering 3D models. SD out-of-the-box was far worse, but Midjourney still was not up to the task. It's incredible how well they can generate images of nearly any subject/object and make some changes to the style and placement, but trying to precisely achieve critical broad-stroke things like like perspective, sizing, lighting direction/amount/temperature, etc. was far too cumbersome. Prompt refining is just like having a program with a bunch of nebulous undocumented menu entries that you just have to click on to see what they do rather than just giving you the tools to make what you need to make. Was that the right entry or the wrong entry? Who knows! Maybe just try it again to see if it works better!

There's a fundamental disconnect between professional-level and consumer-level tools. Consumer tools must be approachable, easy to use, quickly yield appealing results, affordable, and require little maintenance. Professional tools need to be precise, reliable, capable of repeated results with the most demanding tasks, and easily serviceable into perfect working order.

These are consumer-level tools. If you merely need a cool picture of a space ship done in such and such style with such and such guns blah blah blah (that for some reason always looks 10%-50% Thomas Kinkaid,) these tools are great, but they abstract away the controls that really matter in professional work. Novices who get overwhelmed by all of those factors love it because they don't understand, and probably don't care about what they're giving up. For serious work, aside from getting inspo images or maybe generating deliberately weird bits of whatever, they're hit-or-miss at best. Without exception, doing a rough mock-up in a modelling program took FAR less time than trying to wrangle exactly what I needed from one of those generators.

I'm sure they'll get there someday but right now they're miles away from being professional-quality image generation tools.

I think a lot of people have unrealistic expectations of the tech-- they think they can get exactly what they want if they are articulate enough in describing it with words.

Feed your rough mock-up to img2img (or use inpaint sketch) and you'll land much closer to where you're trying to go.

It's a power tool. It will do tedious manual work (producing art) very quickly. The difference between professionals and consumers in how they use it is that the professional asks the machine to "finish what I started," whereas the consumer tells the machine to "do all of the work for me."

Canva fits most people's needs but it only fits the needs of a narrow slice of professionals. Trying to use it for professional work takes vastly more time and effort than using a proper professional tool. LaTeX fits academic paper publisher's needs and can pump out formatted and laid-out text far quicker than someone using InDesign but you'd go crazy trying to assemble a modern magazine or high-end book. It doesn't need polish or sheen. It needs something fundamentally structurally different.

I'm holding out for an instruct-based model that will take explicit instructions, or at least layered prompts. Mutating the original prompt along with the picture (or changing the entire thing to only describe certain parts, a la inpainting) is frustrating to me.

I think what most developers don't realize is that creating media, even snazzy, captivating images, is a tiny portion of the work commercial artists and designers do beyond Fiverr or the people working at sign shops. We've got bazillions of stock photos, asset stores, etc. etc. etc. available at our fingertips at a pretty low cost... and we use them, just as we'll use AI generated images in our processes... Just not for anything that actually matters. Reducing art and design to what so many people have is nearly akin to reducing coding to how fast someone can generate code regardless of it's suitability for the purpose. I could go into Adobe Illustrator and make most of the vector art you see on the net in a few minutes. Knowing what needs to go on the screen, exactly how it needs to go on the screen, and the effects of putting it there, what it communicates and to whom, etc. etc. etc. are the hard parts. You could have a tool that would instantly translate millions of pretty images from someone's imagination in a matter of milliseconds but that's not going to make them useful for communication or anything else.

So very much of this hype is people simply assuming that they understand something that they don't. And that's why the UIs for most FOSS applications suck yet the project contributors will defend them like their own children.

Who's fault is this though? The hype is absolutely hysterical.

We already now see deepfakes (e.g. of Trump or the Pope, recently even videos) that a far beyond what we saw in the years before, indicating that the old professional tools weren't so powerful after all. Now if we extrapolate this a few years into the future...

As I mentioned in another comment, so much of this hype is based on developers assuming they understand something that they don't. I've indulged in this hubris as a developer but straddling both sides of this line has been illuminating.

https://www.adobe.com/sensei/generative-ai/firefly.html

This sounds all very similar to the Stable Diffusion tool chain, though probably cloud based and with a more intuitive UI on top.

Being able to quickly generate and iterate on assets is great for inspiration but pretty useless for professional output without fine-tuned, predictable, repeatable controls. These tools will simply integrate with existing professional tools until they can do it better.

Imagine the first person to make an electric saw made some automated thing that could cut the wood to make a cool looking flat pack house somewhere in the neighborhood of your specifications in 5 minutes. The caveats: while it would assemble perfectly, the actual angles of the cuts might be unpredictable... Like 40 and 50 degrees rather than 45 and 45, and the layout was never quite what you expect even if it was OK more often than not. Pros knew those were fundamentally deal breakers for professional work, and remained more professionally useful with their hand saws because they had the required precision, control, and predictability. While enthusiasts were gong crazy exploring all of the different kinds of oh-so-slightly wonky structures they could generate and predicting the end of carpentry, the old school saw companies started making circular saws, chop saws, drills, and the like. The market for handyman-built dog houses, sheds and playhouses would immediately be lost to the automated machine but I guarantee you that all consequential work would still be done by carpenters with power tools.

Midjourney definitely generates really high quality art based on simple prompts, but the inability to really customize the output basically kills its utility.

We heavily use Stable Diffusion with specific models and ControlNet to get customizable and consistent results. Our artists also need to extensively tweak and post-process the output, and re-run it again in Stable Diffusion.

This entire workflow is definitely beyond a Discord-based interface to say the least.

https://twitter.com/P_Galbraith/status/1649317290926825473?c...

This is using Stable Diffusion and the Control Net Lineart Model. The coloured version is pretty rough but it was a quick test.

In my opinion Stable Diffusion is vastly superior to Midjourney if you have the skill to provide input to img2img/ControlNet.

I have some other earlier workflow experiments on Youtube if you're interested in this kind of thing https://www.youtube.com/pjgalbraith

I'd be interested to know where the art ends up in the game? Do you mean 2D backgrounds and billboards in-game? Or are we talking cut-scenes and menu screen art?

Excited by the potential for artists to accelerate their creative process with advances like ControlNet, but makes sense there's a lot frustrating about the process today.

Exploring tools that might help. If you might be interested in chatting, email me at [email protected]. Thanks!

None of this stuff is copyrightable so I dont care that its not private