I think about this a bit in the context of Virgil. Virgil's compiler is a whole-program optimizing compiler that does a lot of devirtualization and constant-folding. In the higher optimizations it does a bit of inlining, but I haven't found the huge 10X speedups that you get in, e.g. Java. More like 10-40% performance improvements from inlining.

I think Virgil could benefit a little from runtime information. For example, it could make better inlining and register allocation decisions, as well as code layout. I have a feeling that Virgil code would benefit a little from guarded inlining, but I don't think full-on speculation would help. In general, a lot of polymorphism can melt away if you can look at the whole program. Couple that also with Virgil's compiler doing monomorphization, which means that using parametric polymorphism costs only code space, and I think the gap is pretty small. I'd expect you could maybe get another 10-20% from these things all together--that's a lot of work to get a small amount.