Cutie NT | Math Problem

Click here to read the complete problem statement.

I need a big hint. I can’t solve it.