VAR and the ref association should be completely different entities. Also, VAR should be on a contract basis so if company with the contract fucks up the contract can be given to someone else.
IMO there should be three bodies.
Oversight - Not comprised of current/ex officials. They should be responsible for adjusting the rules as required, judging the performances of the groups below them and generally managing everything related to rule enforcement. This group tells the separate VAR/official groups how to execute rules, rather than the VAR's and officials working together.
VAR - A different group with different training for a different skillset. If they see a mistake they tell the referee to review it on the monitor, or just overturn it if it's egregious enough. They report directly to the oversight group. If a referee makes a mistake and the VAR doesn't catch it, the VAR takes the blame. If the VAR recommends the referee review it and the referee sticks with the original mistake, the referee takes the blame.
Officials - Their only roles are referee, linesman and 4th official. They're judged both on the frequency that VAR needs to overrule them and the frequency with which they incorrectly stick with their original decisions. Frequent enough mistakes lead to increasing punishments, such as demotion from the big games, demotion to the EFL and just outright being fired.
Additionally, all audio between officials should be released after every round of fixtures and all per-match performance reviews should be public. Officials should not be allowed to officiate matches for other countries/organisations on the side, and officials shouldn't get to officiate local teams even if they don't claim to support them.