Coding for...the innately able
Coding for…those who score highly on a test
One of the claims I talked about in my previous blog was that the skills of coding were somehow so clearly defined, and particular, that one could separate those who could code, from those who could not. The claim is that this separation is so reliable, that there really is no point in teaching the ‘non-coders’, because they will never have the aptitude. There is some research in this area, but the one I’ve heard cited opens with the uncomfortable (and, later moderated) line: “We (Saeed Dehnadi, Richard Bornat) have discovered a test which divides programming sheep from non-programming goats. This test predicts ability to program with very high accuracy before the subjects have ever seen a program or a programming language.”
The test setup gave a few questions each with a few lines of code, and was analysed to look for students using a consistent mental model of assignment (of values to variables) across those questions. They didn't have to be the correct assignment (i.e. not understanding the syntax of the code wasn't a problem), but it did need to be consistent. (For a basic explanation see this blog post, which also has some interesting comments on it http://www.codinghorror.com/blog/2006/07/separating-programming-sheep-from-non-programming-goats.html)
The importance of robust research
There are a few problems with this study, including a relatively select-sample, and a concern that there might sometimes be good reasons to change one’s mental model (i.e. not to have a consistent answer) – for example, if you’re aware there are other possible sorts of answer, you might select from among them in multiple choice questions to maximise your chances of getting >0 correct. And indeed, the group who conducted the research later moderated their claims in light of subsequent work (Bornat, Dehnadi, & Simon, 2008).
Furthermore, one paper in particular suggests that evidence for natural "good v bad" coders is poor (and not supported in literature) but that there are good explanations for why this finding may be characteristic of the data. One possibility here is that learning a particular concept – x – might support a whole set of other skills – abc, while not learning x might have a negative impact on the learning of abc. This effect would increase the likelihood of the ‘typical’ binomial distribution in results on introductory programming courses (Robins, 2010).
Moreover, the difficulties students faced in this particular case may not reflect the teaching elsewhere. For example, some argue that Computer Science courses don’t need to forefront coding in their teaching (e.g. (Beynon, 2009), others that cognitive aspects (Ma, 2007) and psychological elements of learning programming (see http://www.ppig.org/keywords.html or/and (Sajaniemi, 2008)) should be focussed on. Others have argued that Problem Based Learning and similar inquiry pedagogy strategies may be effective for learning sequencing and model holding (Ambrosio, Costa, Almeida, Franco, & Macedo, 2011).
The data says it, so it must be a law
Fundamentally though, papers such as this can easily fall into the trap of suggesting that because something is the case empirically, it must be the case (in a law like way). – that is, because fail rates are high, that indicates programming is necessarily difficult to learn. Now while there may be some truth to that, it is not a foregone conclusion on at least three fronts –
- different teaching and learning methods may yield different results
- students with different backgrounds may have different success rates – as ‘coding’ becomes more common, success rates may rise
- effects of new technology, and visual methods of coding are unknown in this study. Tools such as Scratch may well improve results – particularly for novice programmers (after they are given some training in using such tools).
The important thing in any case is that it isn’t always those we’d expect to excel who do, and the idea of natural aptitude for various skills even if it’s true, fails to acknowledge the role that tuition plays even for those who are naturally gifted. Indeed, I was recently told about a student who got the top marks in a ComSci course having converted from a law degree in 2nd year (and the student was female!). So the important things are:
- If the skills of coding are important, we can’t restrict them to a select few
- It is highly unlikely that we can predict – particularly prior to exposure to coding – aptitude for coding in reliable ways
- More research should be done on ways to teach coding, visual coding languages, and the psychological/cognitive aspects of such tools
Ambrosio, A. P., Costa, F. M., Almeida, L., Franco, A., & Macedo, J. (2011). Identifying cognitive abilities to improve CS1 outcome. Frontiers in Education Conference (FIE), 2011 (pp. F3G–1 –F3G–7). Presented at the Frontiers in Education Conference (FIE), 2011. doi:10.1109/FIE.2011.6142824
Beynon, M. (2009). constructivist computer science education reconstructed. ITALICS Volume 8 (2) June 2009, 73. Retrieved from http://www.heacademy.ac.uk/assets/documents/subjects/ics/jun2009_vol.8_2_constructivist_computerscience.pdf
Bornat, R., Dehnadi, S., & Simon. (2008). Mental models, consistency and programming aptitude. Proceedings of the tenth conference on Australasian computing education - Volume 78, ACE ’08 (pp. 53–61). Darlinghurst, Australia, Australia: Australian Computer Society, Inc. Retrieved from http://dl.acm.org/citation.cfm?id=1379249.1379253
Ma, L. (2007). Investigating and Improving Novice Programmers’ Mental Models of Programming Concepts. PhD thesis. University of Strathclyde.
Robins, A. (2010). Learning edge momentum: a new account of outcomes in CS1. Computer Science Education, 20(1), 37–71. doi:10.1080/08993401003612167
Sajaniemi, J. (2008). PSYCHOLOGY OF PROGRAMMING: LOOKING INTO PROGRAMMERS’HEADS. THE PROBLEMS OF PROFESSIONALS, 3. Retrieved from http://www.humantechnology.jyu.fi/articles/volume4/2008/ge-sajaniemi.pdf