That's the price you pay for learning a Fighting Style. Sometimes you win, sometimes you lose.
This is not an issue if your non-whip is the light weapon.
Quote:
Two-Weapon Fighting [Combat]
You suffer a -2 to hit with both attacks (with a further -1 if one of the weapons is not light). Once both attacks are resolved, increase your Clock by the slower weapon’s Speed +2.
Having the talent, it does not matter what order or weapon you attack with; just that one of the weapons is not light. Depending upon the combination, the whip can be designated as the off-hand weapon without any further penalties.
Steel Whip + Sidesword:
1. Steel Dragon + TWF/Spinning Strike = -2 Whip, -2 Balanced (Stance until next action)
2. Balanced Blades + TWF/Spinning Strike = -1 Whip, -1 Balanced (Stance until next action)
There is nothing stopping you swapping between Stances each time your action comes up.