firefly2000

Do you think it can be done by adjusting GC aggressiveness (or even disabling it for short periods of time) and correlating it with execution time?

Are there plans to elucidate implicit GC costs as well?

I have a question about what "validation" means exactly. Does this process work by having the main model compute the "probability" that it would generate the draft sequence, then probabilistically accepting the draft? Wondering if there is a better method that preserves the distribution of the main model.

If the workload were perfectly parallelizable, your claim would be true. However, if it has serial dependency chains, it is absolutely worth it to compute it quickly and unreliably and verify in parallel

Does the lack of forward progress guarantees (ITS) on other architectures pose challenges for async/await?

Is this Nvidia-only or does it work on other architectures?