The current hofje was built in 1787 from the legacy of Pieter Teyler van der Hulst, just like the Teylers Museum close to it. Pieter Teyler van der Hulst founded a hofje when his wife died in 1752 and he then purchased the Kolder hofje. That hofje was considerably renovated during his lifetime, but he found it still needed improvements for the residents. In his will he stipulated that a new hofje should be built in his name and the old hofje premises sold (and the premises were then sold to the oldest surviving hofje foundation in Haarlem, the 'Vrouwe- en Antonie Gasthuys'). The "new" hofje is by far the most impressive and imposing hofje in Haarlem.

The Hofje was founded by the Haarlem Guild of Beer brewers, one of the most powerful guilds in Haarlem. It was initially named the St. Maartenshofje after the patron saint of the brewers.[1] It was intended as a charitable home for 22 poor people ("rechte arme huyszitten") who worked in the brewing industry. The Brewer's guild used the money for the hofje from a donation it received in 1472 by Jacob Huyge Roepersz and his sister Katharina, who donated a series of small houses (kameren) in the Gasthuisstraat (the Tuchthuisstraat used to be a northern extension of the Gasthuisstraat) for housing, and some land in Akersloot for rents. The donation made the requirement that the houses needed to be rebuilt if they were damaged by fire or any other reason.

The hofje was founded from the legacy of Dirck van Bakenes (or Dirc van Bakenesse) in 1395. This makes it the oldest hofje in the Netherlands. The earliest mention of it in town records is from the History of Haarlem by Samuel Ampzing in 1628, who claims to have copied the stichtingsbrief or letter of foundation (since lost).