Hi Addikt
I ride without fins all the time now. What I noticed when I first took them off were, the board felt a lot more skaty, but after about 10 minutes I changed my stance slightly and it then felt solid. The board felt faster as less drag. I could get more pop, but for boosting I think the fins helped hold it in a bit longer so could not get quite as much hight with sent jumps. I also noticed I had to land more accurately with my weight rite over the board.
Taking the fins off made no difference to upwind ability or light wind performance, In fact I think it forces you engage more of the rail. Upwind ability is about using the rail, fins only help with tracking and a lighter back foot will help here. I recon fins make a board easier to sail but almost detune it a bit if that makes sense.
So go for it take them off, but give it a couple of sessions to get used to it will feel different.