Most people will not hear 'Jewish quarter' and decide that it is something they want to go see. I would encourage everyone to make the effort.
When we started trying to find the cemetery it was difficult to say the least. The area is very maze like. Expectedly someone walking past us introduced themselves and started casually leading us. His name was Moustaffa and it turned out the be one of the best help we got all holiday. He guided us to the cemetery first which is a sight to behold - much larger then you ever would have expected and with a couple of giant mausoleums, its hard to believe the such a huge space has been kept as a Jewish cemetery for so many years and not built over. Suggested donation to the cemetery guard was 20 Dirham
He next took us back to his house for a tea. Though it can seem quite daunting when locals do this I really hope that you go with the flow. Most of the time they are just being nice and want to show you where they live.
Finally he took us to the Synagogue after spending around an hour with him. Of course he did ask for money - i gave him 45 Dirham which seemed like a fair amount (though he did try for 100!)
Definitely worth making a visit to the Jewish quarter of the city if you have a spare few hours