The family remained at the house until 1934 when Wong's father returned to his hometown in China with Anna May's younger brothers and sister.