The increasing popularity of distance education poses exciting new challenges. In particular, current pedagogical paradigms, such as competency-based education, require students' continuous evaluation. That is, to master skills, students need to receive constant feedback to guide their experimentation processes. However, teaching teams are usually under-dimensioned to support the large number of students that online courses usually have. This paper presents the approach we have adopted at the National University of Distance Education to overcome this problem for the case of computer programming practices, which complements human evaluation with an automatic assessment system. The paper describes our system and reports its benefits with an empirical study from 2011 to 2018 that involved 14,944 students.