The history of Rye can be traced back to the first millennium CE, when the town was a small fishing village on the water. Since then, the ocean has retreated and the town is some two miles from the sea. The town became a part of Normandy when the area was bestowed to the Norman Abbey of Fecamp in the 11th-century. The town grew into a prominent trading port, and in 1205 was placed under French rule when Normandy was returned to the French crown. The English took back control of the region in 1247 under the rule of King Henry III. The English heavily fortified the city at this time, surrounding it with walls.

    In the 13th-century, Rye became a member of the Charter of Cinque Ports, being granted exemption from taxes and custom duties in exchange for providing safe harbor and a steady supply of war ready ships. Under these relaxed economic conditions, Rye thrived as a port town. During the 14th-century, Rye was subject to a series of attacks by the French and the Spanish, including a particularly damaging one by the French in 1377 that left the town in ruins. This attack prompted a retaliation by the citizens of Rye, who sacked the French coast, retaking many of the spoils that had been stolen from them, including a bell that had been taken from St. Mary's Church.

    In the 18th-century, smugglers were an important part of the city's life, and a series of tunnels still remains beneath the city through which the smugglers secreted their wares.

    Today, Rye is known for its bohemian character and its rich literary tradition.