Hot carrier solar cells depend critically on the energy relaxation dynamics of photo-generated carriers in an absorber material, where hot carriers are extracted through energy selective contacts. Here we combine ensemble Monte Carlo (EMC) simulation with an energy balance equation approach, to simulate the microscopic carrier relaxation processes and corresponding electron and hole temperatures in semiconductor quantum well (QW) hot carrier solar cell structures, both under transient and steady state illumination. We include nonequilibrium optical phonons, in which a detailed balance of emission and absorption events is used to simulate the phonon population in time, with the anharmonic decay of the optical phonon population to acoustic phonons described using a phenomenological phonon lifetime. Simulation of femtosecond laser excitation in GaAs QWs show reduced cooling, depending on the optical phonon lifetime and excitation intensity. Steady state simulation under AM0 solar illumination shows a build-up of hot phonons over long times depending on the phonon lifetime, although they are not readily re-absorbed due to momentum and energy conservation considerations.