I haven't taken a math class in 30 years, but 30 yrs ago, I took a lot of them. Even so, the algebra problems your kid is doing hasn't changed much since the 17th century. Find the earliest reference in the book, or perhaps even inside the cover. I strongly suspect the author has minted his own notation. In context, it looks like you're supposed to replace the squiggly with another standard symbol, such as < etc.