I was actually involved in a similar discussion about a week ago (although instead of VLC we were talking about LAV), and I think a lot of what was discussed applies here as well. To summarize the other discussion, the OP wanted to change the decoder that WMC uses because he observed better deinterlacing performance with LAV vs the Microsoft DTV decoder. He was convinced that Microsoft's decoder was at fault. It was suggested that he try a different GPU...which led to him arguing that there was no problem with his GPU because it worked just fine with LAV. Unfortunately, he was wrong...the reason he saw a difference was because when he was using Microsoft's decoder, deinterlacing was being offloaded to the GPU via DXVA, whereas when he switched to LAV, deinterlacing was being offloaded to the GPU via CUDA. He was comparing apples to oranges, and wasn't able to make an apples-to-apples comparison because LAV is unable to offload deinterlacing to the GPU via DXVA (even when LAV is running in DXVA mode). Ultimately, copy protection forced him to leave WMC set to use the Microsoft decoder, so he upgraded his GT 610 to a GT 640. After doing so, here is what he posted:
tzr916 wrote:After searching for HQV results, I went with a GT640 fanless. WMC with MS decoder does look much better. Almost 95% of the video defects I was seeing with the GT610 are gone. It's enough to be satisfied (for now). Just very disappointing that WMC decoder is DXVA / locked down and requires so much GPU just to get a decent picture.
You can read the full thread here: www.thegreenbutton.tv/forums/viewtopic.php?f=5&t=5563
I've been actively involved in several discussions about the progressive_frame flag for a few years now, and every time someone tries to compare WMC performance to something else (e.g., other software, set-top-boxes, etc.), it always ends up being an apples-to-oranges comparison, and I suspect the same is true in your case. What you are seeing in VLC is likely due to VLC not offloading deinterlacing to the GPU via DXVA. It's likely either offloading deinterlacing to the GPU via some other API, or it's not offloading deinterlacing to the GPU at all. Even if VLC is offloading deinterlacing to the GPU via DXVA, it may not be honoring the progressive_frame flags.
As far as the GPU power goes, the overall power of the GPU is irrelevant...what matters is how powerful the GPU is at performing this specific task. Unfortunately, most GPUs are better suited for things like gaming instead of playing video that contains fluctuating progressive_frame flags.