I think you will find that you can implement this functionality using Hijack Ir translation rules.
In that you can say "when I press the remote Source button and I am in Player mode send 3 source button presses to the player', which will switch it from Player to Aux, then AM then finally FM.
And you leave the existing behaviour unaltered otherwise.
That way when you are in Tuner mode, you will advance to the next source, when in Player mode, source will skip straight to FM.
If by some reason you ended up in Aux or AM mode, then source will advance to the next mode (either AM or FM respectively for those two buttons).
Have a look in the rioar FAQ section on Hijack - there are some good instructions on how to do exactly this sort of thing.