Concern over attractiveness to the opposite sex affects the masculine image of physics and mathematics only in England, while having a negative view of intellectual women is correlated with a masculine image of mathematics as a field only in Japan, according to a survey conducted in the two countries by a Japanese research group. This comparative study shows that programs to increase women's representation in these fields must take into account each country's social context surrounding gender roles.