Recent evidence has shown that noncognitive skills matter for success in life and can be shaped through interventions. Because of this evidence, policy makers and researchers have increasingly become interested in measuring noncognitive skills and typically rely on self-reported measures in which respondents rate their own skills. Such self-reports have been applied in program evaluations, as well as school accountability and improvement systems. We demonstrate that self-reports are sensitive to survey administration conditions, including whether a survey administrator describes the skills being assessed and whether respondents receive incentives tied to performance on other tasks. These findings have implications for the interpretation of self-reported measures. Social policies or interventions might affect responses on self-reported noncognitive skills without affecting the skills themselves.