How do I delete a "rest" or a "note"?

Ctrl/Cmd + Del
Knowing that this will modify/reduce the duration of the measure (eg 3/4 vs. 4/4) Is this what you really want?
If not, to simply remove a note, press Del (in normal mode, eg). The same applies to rests, provided they are in Voices 2, 3 or 4. Rests in Voice 1 cannot be deleted (only hidden).
It would have been helpful if your question had been more precise (by providing a concrete example or, if necessary, an image)

Try reading thus part of the handbook https://musescore.org/en/handbook/4/editing-notes-and-rests