Governments around the globe use non-pharmaceutical interventions (NPIs) to curb the spread of coronavirus disease 2019 (COVID-19) cases. Making decisions under uncertainty, they all face the same temporal paradox: estimating the impact of NPIs before they have been implemented. Due to the limited variance of empirical cases, researchers could so far not disentangle effects of individual NPIs or their impact on different demographic groups. In this paper, we utilize large-scale agent-based simulations in combination with Susceptible-Exposed-Infectious-Recovered (SEIR) models to investigate the spread of COVID-19 for some of the most affected federal states in Germany. In contrast to other studies, we sample agents from a representative survey. Including more realistic demographic attributes that influence agents' behavior yields accurate predictions of COVID-19 transmissions and allows us to investigate counterfactual what-if scenarios. Results show that quarantining infected people and exploiting industry-specific home office capacities are the most effective NPIs. Disentangling education-related NPIs reveals that each considered institution (kindergarten, school, university) has rather small effects on its own, yet, that combined openings would result in large increases in COVID-19 cases. Representative survey-characteristics of agents also allow us to estimate NPIs' effects on different age groups. For instance, re-opening schools would cause comparatively few infections among the risk-group of people older than 60 years.