Flux the future of AI image generation

[Legolas18]

For those interested in AI generated imagery (text2img), the team that originally created Stable Diffusion have created a new company called Black Forest Labs, and just released Flux.

I am posting this because I am really impressed.

FLUX DEV open source (can run locally, I am waiting a guide for how to do so)
https://replicate.com/black-forest-labs/flux-dev

FLUX PRO This is close source version (API version only)
https://replicate.com/black-forest-labs/flux-pro

A dramatic and epic scene showing a lone wizard standing in brightly lit grass on top of a mostly stone mountain with his arms raised and four fingers outstretched, silhouetted against a vivid, starry night sky with dynamic clouds. A leather-bound book with the words 'Open source magic' in gold foil lays on the ground. Glowing grass at the wizard's feet is illuminated by the first rays of the rising sun. The sky is filled with glowing, swirling energy patterns, creating a magical and powerful atmosphere. The word 'ZONEGFX' is prominently displayed in the sky in bold, glowing letters, with bright, electric blue and pink hues, surrounded by the swirling energy that appears to faintly originate from the wizard's hands. The wizard appears to be casting magic or controlling the energy, adding to the sense of grandeur and fantasy. The wizard is wearing his pointed hat, and his cape flows backward by the force of the energy.

meme image with two men in it. On the left side the man is taller and is wearing a shirt that says Black Forest Labs. On the right side the other smaller scrawny man is wearing a shirt that says Stability AI and is sad. The taller man is hitting the back of the head of the small man. A caption coming from the tall man reads "That's how you do a next-gen model!"

Young woman playing a violin, she is dressed in a beautiful turqoise dress with frills in an opera house.

photo of a sith young woman in a dark robe, holding a red lightsaber in a defensive pose. She is on a dusty planet during a mild sandstorm

[Frank21]

This looks very interesting and follows prompts very well. Hands are really good. Thanks Legolas.

[Frank21]

The ability to display accurate text is quite amazing.
How do you think it compares to SDXL?

[Legolas18]

@frank22 From what I have seen so far, this one is better than sdxl, even after all of the community training.

Things out of the box just look better, the details are superior and I was able to run it locally with comfyui already (although, I don't know what are the minimum requirements).

I am certainly excited, especially imagining what this will become after all the community training, ControlNet, and all the other fluff 🙂

Model size....22GB though

[Frank21]

22GB... ouch, but I do have plenty of space on my SDD. Is that the Dev or schnell model?
What's your system specs?

[Legolas18]

@frank22 Both have the same size, unfortunately.

From what I am reading on reddit, it might not be super hungry?

You can run Flux on 12gb vram
byu/Far_Insurance4191 inStableDiffusion

although you should use

t5xxl_fp8_e4m3fn.safetensor

since the t5xxl_fp16.safetensors.safetensors requires 32 GB RAM (I do have it, but it is giving me an error).

For all the required files (it currently only runs in ComfyUI though, AFAIK):
https://comfyanonymous.github.io/ComfyUI_examples/flux/

[Frank21]

So you're on a 3090/4090? A 4090 is >$3000 where I live.
My sys could handle it with a better PSU but I can't justify spending that much 🙁 .

[Legolas18]

@frank22 Yes, here it sells for half that amount. But I was talking about 32 RAM, not VRAM thankfully.

A 3090|4090 only has 24 GB VRAM. For 12GB...you could go as low as a 3060? How much VRAM do you currently have?

[Frank21]

I've got a 4070 with 12GB VRAM and 32GB of DDR5 6000.

[Legolas18]

@frank22 then you can definitely run it locally. Just use the fp8 version.

[eelgoo]

Not perfect, but that certainly looks a step change better.

What is it like for character & clothing repeatability?

🙂

[Legolas18]

@eelgoo a bit too early to tell, since that it's typically a bit more advanced stuff that requires some extensions to work. Maybe, in a few weeks I'll be able to know.

[eelgoo]

Yes, that makes perfect sense.

I thought it was worth asking the question though! 🙂

[Frank21]

I did some consistency tests for you with the same prompt. The clothing and character looks quite consistent, the text not so much. There's no way to use any loras ATM with Flux so character consistency will be down to whatever you prompt.
If I tried this with a1111 it would be a dog's breakfast.

[eelgoo]

That does look promising, although the clothing is more consistent than the character. 🙂

[Author]

Posts