how do i remove this + sign

See: https://musescore.org/en/handbook/3/measure-operations#duration
A small - or + sign appears above the measure when its duration is different than indicated by the time signature.

"... when I add any note to the measure, I get this + icon, which is not what i want it to do"

It sounds to me as if you might be using Insert mode for your note entry? Normally you should always use the default mode which is Step-Time entry:

With Step-Time (Default) entry, each note will replace an existing rest. So the measure remains the standard duration, and you will never see this + sign appear.

With Insert mode, each note entered will add the note's length to the measure's duration. And then your measure is no longer the standard duration (4/4 in your example), and you will of course see the + sign.

Try not to think about adding notes to a measure. Instead, think about replacing the existing rest(s) which already fill an empty measure.

You will find the Handbook section on "Note Input" very helpful:
https://musescore.org/en/handbook/3/note-input