We present a decentralized, scalable approach to designing a reconfigurable manufacturing system in which a swarm of robots assembles heterogeneous parts into target amounts of products. The sequence of part assemblies is determined by interactions between robots in a decentralized manner in real time. Our methodology is based on deriving a continuous abstraction of the system from chemical reaction models and formulating the strategy as a problem of selecting rates of assembly and disassembly. The rates are mapped onto probabilities that define stochastic control policies for individual robots, which then produce the desired aggregate performance. We illustrate our approach using a physics-based simulator with examples involving 15 robots and two types of final products.