OMGnotThatGuy

I got it installed and tested with a SD 1.5 model. SDXL models did not work. I have a 4090 and in Stable Diffusion Webui. I generated the same test prompt with and without their TensorRT acceleration. The generated images are nearly identical, with very minor, almost imperceptible differences, which is pretty standard for techniques that optimize the attention layers.

Prompt: A cat wearing pajamas Model: deliberate_v2 Size: 768x512 Batch Size: 4 Clip Skip: 2 Samler: DPM++ 2S a Karras Steps: 70 Seed: 2024828515

Generation times avg over 5 runs

================================

without TensorRT: 19.2 seconds

with TensorRT: 12.3 seconds

Benchmarks

Batch Size 1 / 2 / 4

=======================================

without TensorRT: 31.11 / 36.23 / 42.34

with TensorRT: 55.32 / 58.06 / 62.27

It appears that it's already in there. The trick is that you have to install an extension in some apps to make it work. For instance, there's a TensorRT accelerator extension for Stable Diffusion WebUI that takes advantage of it. I'm installing it right now, though the docs leave a bit to be desired.

https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT