"new math" is an old or dialect word meaning a dramatic and short-lived change in the teaching of mathematics in grade schools in the 1960s, focusing on abstract topics such as symbolic logic, to the detriment of simple arithmetic etc. If your teen uses it, they have most likely picked it up from a book, video game, period drama, or a regional grandparent rather than a modern peer group. There is no cause for concern here; the word is largely out of contemporary teen usage. If it appears in a school text, it is being studied for vocabulary or historical context. You can treat it like any other archaic vocabulary item.