Someone much wiser than me one said:
Don't go out in any surf you are comfortable swimming out through , and then back in
Don't go out further than you are comfortable to swim back in unaided
Don't go out where your downwind and self rescue options are going to hurt
Whilst I am sure most of us have broken those rules at one time or another (hell, I know I have managed to break all 3 at the same time

) it is important to consider that any self rescue will be 45 degrees down wind at best, not including the potential tea bagging and/or dead downwind drag you will get while packing up.
Packing up in the water is not a 5 exercise either, even if you manage not to get yourself tangled in your own lines...