Playback does not take time signature into account
If I have two scores, the first is 8/4, the second one is 8/8, the measures will be played twice as fast in the second one. That's not how it was supposed to work, as both of them have 8 beats per measure. It seems that the quarter note is always the time of a beat, regardless of the time signature.
Comments
No, one has 8 beats, the other has 4 of them. and at the default setting of 120BPM this is what you hear. MuseScore (and MIDI) Counts beats as quarter notes.
Use tempo texts to get the tempo you want, see https://musescore.org/en/handbook/tempo
In the next version we'll even have relative tempo textx, for cases where the time sig changes mid-score.
If you switch on the metronome, you'll find that playback does take the sime sig into account, just not like you expect, for the default playback speed