4 days in Marrakech is enough to see the sights that the tour guides recommend ie Bahia and Badi palaces. Ben Youssef madrasa, Marrakech museum, souks, tanneries etc. You'll be able to get a reasonable taster of the city but it isn't lng enough to actually relax into the pace and rhythm of the place (which is why a lot of the less positive reports seems to come from people who are only there for 2/3 days).
My advice is not to worry too much about first impressions upon arriving. You will probably be a bit overwhelmed and shocked - there is a reason why they says it is an "assault on the senses"!
My first day there, we arrived about 4pm and left the riad for a quick reccy at about 6pm. By 8pm I was ready to book the next flight home! I didn't dare tell my husband (he never wanted to go there in the first place) and he said nothing negative to me either so we both went to bed that night a little unhappy but without discussing it.
Next morning, after a good sleep and a lovely breakfast, we headed out again into the chaos of the souks. By the end of that day we were both hopelessly, passionately in love with the city. It is all about attitude and perception......................and having a great sense of humour at all times!
I think if we had talked about it the first night, we may well have talked ourselves into hating it before we really gave the place a chance. Instead, we had each silently determined to "make the best of it" and the result was that we had the best time imaginable.
So yes, 4 days is enough but I'd probably say that the first day you'll need just to acclimatise!
Edited: 11:12 am, January 10, 2013