I think that trust needs to be 'pushed deeper' than that so to speak. While this would be an improvement, what happens if there is a malicious actor at Github? This may be unlikely, but would be even harder to detect since so much of the pipeline would be proprietary.
Ideally, we would have a mechanism to verify that a given build _matches_ the source for a release. Then it wouldn't matter where it was built, we would be able to independently verify nothing funky happened.