It is normal for your Star Score to fluctuate even if you win.

Winning scores that are close to your average may not raise the Star Score. Lower winning scores can slightly drop your Star Score.