Building with a Memory is a subtle responsive intervention that aims to provide cohesion and community awareness through the use of light and color. The installation delivers thought-provoking information by capturing, analyzing and rendering real-time and archived human activity in a workplace setting. The installation senses movement in the space through an IR camera and computer vision techniques. Two custom lighting fixtures and a video monitor render the aggregated movements. The visually simple aesthetic of the piece aims to balance active engagement and passive contribution, providing a rewarding experience for both occasional passersby and regular users of the space. This paper describes the motivations and contributions of the installation, together with insights gained from an informal evaluation and directions for future explorations.