Volume in Fluidsynth

Hello,
in this blog post I want to write about how the volume is calculated in Fluidsynth and describe why I think this solution is problematic.

Lets start at Fluidsynth's heart - the write method of a voice. That is the essential part that generates the actual audio. It generates n samples and adds them to the output. This method gets called in a real time context by the main synth process. Lets have a look at the part that deals with volume. (There is actually a quite similar statement to this right next to this one - only difference is that in the envelope volume of the attack phase is treated somewhat different (begin a linear ramp))
fluid/voice.cpp Voice::wirte(unsigned n, float* out, float* reverb, float* chorus)

            target_amp = fluid_atten2amp (attenuation)
               * fluid_cb2amp (960.0f * (1.0f - volenv_val)
               + modlfo_val * -modlfo_to_vol);

As you can see the volume of the voice depends on attenuation, a value of the amplitude envelope and the modlfo. We're interessted in attenuation since that is what gets set by velocity, CC11 and CC7. But first - what does that call to atten2amp do?

fluid/conv.cpp

/*
 * fluid_atten2amp
 *
 * in: a value between 0 and 1440, 0 is no attenuation
 * out: a value between 1 and 0
 *
 * Note: Volume attenuation is supposed to be centibels but EMU8k/10k don't
 * follow this.  Thats the reason for separate fluid_cb2amp and fluid_atten2amp.
 */
float fluid_atten2amp(float atten)
      {
      if (atten < 0)
            return 1.0;
      else if (atten >= FLUID_ATTEN_AMP_SIZE)
            return 0.0;
      else
            return fluid_atten2amp_tab[(int) atten];
      }

As you can see it basically looks up a value from a table. This table transforms a cdB value into a factor that can be used as a gain value. As you might know db is calculated like this: v_cdB = 200 * log(v/v_ref) with v_ref in digital systems being 0dBFs or the maximum value can be represented by the system or just 1.0 if you're expecting values between -1.0 and 1.0 for your floating point audio system. Fluidsynth works with values between -1.0 and 1.0.

Lets look up how that table is constructed in Fluidsynth:
fluid/conv.cpp fluid_conversion_config()

      for (int i = 0; i < FLUID_ATTEN_AMP_SIZE; i++)
            fluid_atten2amp_tab[i] = (float) pow(10.0, (double) i / FLUID_ATTEN_POWER_FACTOR);

FLUID_ATTEN_AMP_SIZE is 1441 and FLUID_ATTEN_POWER_FACTOR is -200.0. So it is basically just the equation above solved for v/v_ref. The power factor is negative because we expect the cdB values to be actually negative because we want to generate attenuation (values smaller than 1.0).

But how is the attenuation value determined?
fluid/voice.cpp Voice::update_param(int _gen)

            case GEN_ATTENUATION:
                  attenuation = gen[GEN_ATTENUATION].val * ALT_ATTENUATION_SCALE + gen[GEN_ATTENUATION].mod + gen[GEN_ATTENUATION].nrpn;
 
                  /* Range: SF2.01 section 8.1.3 # 48
                   * Motivation for range checking:
                   * OHPiano.SF2 sets initial attenuation to a whooping -96 dB
                   */
                  attenuation = qBound(0.0f, attenuation, 1440.0f);
                  break;

So it is the sum of the generator value (weighted with some factor), generator modulation and generator nrpn modulation. What does that mean? In the SoundFont specification we have parameters of a voice or channel, that are controlled by generators which are then changed by modulators. The generator value is the one that is described in the SoundFont (the file) itself. The modulation is changed by parameters like velocity or midi CCs. NRPN are special Midi messages that can change generators as well.

Lets have a look how these modulators work!
fluid/voice.cpp Voice::modulate(bool _cc, int _ctrl)

void Voice::modulate(bool _cc, int _ctrl)
      {
      for (int i = 0; i < mod_count; i++) {
            Mod* m = &mod[i];
 
            /* step 1: find all the modulators that have the changed controller
             * as input source.
             */
            if (m->has_source(_cc, _ctrl)) {
                  int g = m->get_dest();
                  float modval = 0.0;
 
                  /* step 2: for every changed modulator, calculate the modulation
                   * value of its associated generator
                   */
                  for (int k = 0; k < mod_count; k++) {
                        if (fluid_mod_has_dest(&mod[k], g)) {
                              modval += mod[k].get_value(channel, this);
                              }
                        }
                  gen[g].set_mod(modval);
 
                  /* step 3: now that we have the new value of the generator,
                   * recalculate the parameter values that are derived from the
                   * generator
                   */
                  update_param(g);
                  }
            }
      }

So it loops through all defined modulators, checks if the event that was received (like a CC message or velocity) shall change the modulator, looks for the corresponding generator, loops through all modulators again to sum up its values and changes the generator. After the generator was changed the parameters get updated.

Lets have a look at the defined modulators:
fluid/fluid.cpp struct attributes are: dest, src1, flags1, src2, flags2, amount

static const Mod defaultMod[] = {
      { GEN_ATTENUATION, FLUID_MOD_VELOCITY, FLUID_MOD_GC | FLUID_MOD_CONCAVE | FLUID_MOD_UNIPOLAR | FLUID_MOD_NEGATIVE, 0, 0, 960.0 },
      { GEN_FILTERFC,
         FLUID_MOD_VELOCITY, FLUID_MOD_GC | FLUID_MOD_LINEAR | FLUID_MOD_UNIPOLAR | FLUID_MOD_NEGATIVE,
         FLUID_MOD_VELOCITY, FLUID_MOD_GC | FLUID_MOD_SWITCH | FLUID_MOD_UNIPOLAR | FLUID_MOD_POSITIVE,
         -2400 },
      { GEN_VIBLFOTOPITCH, FLUID_MOD_CHANNELPRESSURE, FLUID_MOD_GC | FLUID_MOD_LINEAR | FLUID_MOD_UNIPOLAR | FLUID_MOD_POSITIVE, 0, 0, 50 },
      { GEN_VIBLFOTOPITCH, 1, FLUID_MOD_CC | FLUID_MOD_LINEAR | FLUID_MOD_UNIPOLAR | FLUID_MOD_POSITIVE, 0, 0, 50 },
      { GEN_ATTENUATION, 7, FLUID_MOD_CC | FLUID_MOD_CONCAVE | FLUID_MOD_UNIPOLAR | FLUID_MOD_NEGATIVE, 0, 0, 960.0 },
      { GEN_PAN, 10, FLUID_MOD_CC | FLUID_MOD_LINEAR | FLUID_MOD_BIPOLAR | FLUID_MOD_POSITIVE, 0, 0, 500.0 },
      { GEN_ATTENUATION, 11, FLUID_MOD_CC | FLUID_MOD_CONCAVE | FLUID_MOD_UNIPOLAR | FLUID_MOD_NEGATIVE, 0, 0, 960.0 },
      { GEN_REVERBSEND, 91, FLUID_MOD_CC | FLUID_MOD_LINEAR | FLUID_MOD_UNIPOLAR | FLUID_MOD_POSITIVE, 0, 0, 200 },
      { GEN_CHORUSSEND, 93, FLUID_MOD_CC | FLUID_MOD_LINEAR | FLUID_MOD_UNIPOLAR | FLUID_MOD_POSITIVE, 0, 0, 200 },
      { GEN_PITCH,
           FLUID_MOD_PITCHWHEEL,     FLUID_MOD_GC | FLUID_MOD_LINEAR | FLUID_MOD_BIPOLAR  | FLUID_MOD_POSITIVE,
           FLUID_MOD_PITCHWHEELSENS, FLUID_MOD_GC | FLUID_MOD_LINEAR | FLUID_MOD_UNIPOLAR | FLUID_MOD_POSITIVE,
        12700.0 },
      };

I highlighted the interesting ones but have a closer and aligned look.

      { GEN_ATTENUATION, FLUID_MOD_VELOCITY, FLUID_MOD_GC | FLUID_MOD_CONCAVE | FLUID_MOD_UNIPOLAR | FLUID_MOD_NEGATIVE, 0, 0, 960.0 },
      { GEN_ATTENUATION, 7,                  FLUID_MOD_CC | FLUID_MOD_CONCAVE | FLUID_MOD_UNIPOLAR | FLUID_MOD_NEGATIVE, 0, 0, 960.0 },
      { GEN_ATTENUATION, 11,                 FLUID_MOD_CC | FLUID_MOD_CONCAVE | FLUID_MOD_UNIPOLAR | FLUID_MOD_NEGATIVE, 0, 0, 960.0 },

So the attenuation gets influenced by the velocity, CC7 and CC11 - as expected by the SF spec. The first constant in the flag tells whether it is a Midi CC or an internal parameter. The other three say something about in which shape or curve the parameters change. The last one is the amount in which the parameters change. As you can see none of the defined modulators actually uses the second source.

To complete our overview lets have a look how the values of a modulator are gathered and calculated. Lets start with gathering:
fluid/mod.cpp Mod::get_value(Channel* chan, Voice* voice)

      if (mod->src1 > 0) {
            if (mod->flags1 & FLUID_MOD_CC) {
                  v1 = chan->getCC(mod->src1);
                  }
            else {  /* source 1 is one of the direct controllers */
                  switch (mod->src1) {
                        case FLUID_MOD_NONE:         /* SF 2.01 8.2.1 item 0: src enum=0 => value is 1 */
                              v1 = range1;
                              break;
                        case FLUID_MOD_VELOCITY:
                              v1 = voice->vel;
                              break;

I just copied the lines that are important for attenuation. If it is a Midi CC it is quite simple, it just gets the last CC value that was send to the channel. If it is an internal parameter it gets it from the internal data. Now how do these Midi values get translated to the actually parameters?
fluid/mod.cpp Mod::get_value(Channel* chan, Voice* voice)

                  case 5: /* concave, unipolar, negative */
                        v1 = fluid_concave(127 - v1);
                        break;

It all depends on which flags are set. In the attenuation case it is always a negative (negative actually means reversed in this case) concave shape. The function above just makes sure the given value is in a sane bound (0 to 127) and looks up the value from a table. (very similar to the atten2amp() above)

At the end this method just returns the product of the amount and the two sources (if there is no first source it gets 0 if there is no second source the second parameter becomes one).
fluid/mod.cpp Mod::get_value(Channel* chan, Voice* voice)

      return mod->amount * v1 * v2;

I wrote a very simple python script which constructs tables in the same way as in fluid/conv.cpp and plotted these values with matplotlib - so we can get an idea what these shapes actually look like!

So here are the basic shapes:

And here is how they would add up:

As you can see reversed concave and convex are actually made up of the same formula - one is just one minus the value of the other. Have a look at the piece of code that construct them:

fluid/conv.cpp fluid_conversion_config()

      /* There seems to be an error in the specs. The equations are
         implemented according to the pictures on SF2.01 page 73. */
 
      for (int i = 1; i < 127; i++) {
            double x = -20.0 / 96.0 * log((i * i) / (127.0 * 127.0)) / log(10.0);
            fluid_convex_tab[i] = (float) (1.0 - x);
            fluid_concave_tab[127 - i] = (float) x;
            }

I wonder what kind of error in the spec this codes refers to? The main problem is - even if we would use one of the bipolar shapes for CC11 (to realize both crescendos and decrescendos without modifying the velocity) it would either that the crescendo or decrescendo shape wouldn't be a concave one (because in the bipolar ones one half is always actually concave and the other half is convex - you may have a look at the bipolar images again). You could think of a concave-convex shape that mixes both of them and is always concave like this one:

But even this would give us headaches if we want to reach the same volume level that we would reach with a certain velocity, because you actually would need to look up the values in these tables, add them up and find the closet one to the value in the velocity table.

I would propose dropping that particular part about CC11 in the SF spec and instead have CC11 as a second source for the attenuation changed by the velocity and a new flag that would modify the velocity value before being translated via its shape. This way you could simply add up Midi values.

Calculation would be like: v1 = v1+v2 with v2 = (v2-63) * 2

So 63 would not change the current velocity and 127 would change a velocity from 0 to 127.

What do you think of this idea?

EDIT: Other variants I could think of is just dropping volume change via velocity and just change it via CC11 or having linear shapes which would make the math much simpler! I think all of these three variants should work ok with external synths as well.