The mast will dynamically bend backwards, not just to the side. This backwards bend is just as important to the dynamics of the sail handling IMHO.
The shape of the luff curve pre-tensions the sail. It is designed to complement the bend curve characteristics of the mast so when tensioned/set the sail has the desired shape and dynamic flex.
In race/slalom/speed sails, where a lot of the shape of the sail is determined by seam-shaping, using a different bend curve mast can still make quite a difference to the way the sail sets and handles.
eg: Use a more flex top mast - deeper draught down low, more twist up top for the same down-haul tension. Use a less flex top - Less depth low, tighter upper leech with same down-haul tension. In the first instance the sail will feel more locked and stable and may not pump as nicely. In the second, it will have a great pumping feel but may have less low speed grunt and high speed stability.
I am sure the same applies to non cam sails.
It can be very useful when tuning a speed sail to have the option of trying different characteristic masts. In my case I can also switch tops and bottoms from different sizes as well to change the flex and bend of the mast. It can be a very interesting exercise and can sometime yield great benefits for a particular purpose.

One of the difficulties some of us face is that some race sails are often designed and tested by gorillas that resemble humans. Since I am more like a chimp, I find a softer, or different flexing mast can often make it work for me in a way that closer resembles the intent of the designer.
But as to why some designers start with hard top, and some start with flex top, I am as in the dark as the OP.