Expressive Text-to-Image Generation with Rich Text

Re:Expressive Text-to-Image Generation with Rich Text by 90-00-09 on Wednesday October 04, 2023 @06:27PM (#37772662)

I like this idea. It could be handy to be able to focus on individual descriptions in complex prompts. Is this then mostly a "UI" feature that is being translated to a traditional prompt?

(As a side note: using decorative typefaces was an unconvincing example.)

[ Parent | on HN ]

Re:Expressive Text-to-Image Generation with Rich Text by jbhuang0604 on Saturday October 07, 2023 @12:38AM (#37799051)

The UI part is basically a way to organize the user's intention. In the backend, we develop method for extracting "token maps" (i.e., which spatial regions correspond to specific words) and use region-based diffusion to achieve these localized editing results.

The second half of the video provides an overview of the method. https://www.youtube.com/watch?v=ihDbAUh0LXk

[ Parent | on HN ]

Re:Expressive Text-to-Image Generation with Rich Text by 90-00-09 on Saturday October 07, 2023 @02:46PM (#37804399)

TIL about region-based diffusion. Thanks for the context!

[ Parent | on HN ]