A separable neural code in monkey IT enables perfect CAPTCHA decoding

Abstract

Reading distorted letters is easy for us but so challenging for the machine vision that it is used on websites as CAPTCHA (Completely Automated Public Turing Test to tell Computers and Humans Apart). How does our brain solve this problem? One solution is to have neurons selective for letter combinations but invariant to distortions. Another is for neurons to encode letter distortions and longer strings to enable separable decoding. Here, we provide evidence for the latter possibility using neural recordings in the monkey inferior temporal (IT) cortex. Neural responses to distorted strings were explained better as a product (but not sum) of shape and distortion tuning, whereas by contrast, responses to letter combinations were explained better as a sum (but not product) of letters. These two rules were sufficient for perfect CAPTCHA decoding and were also emergent in neural networks trained for word recognition. Thus, a separable neural code enables efficient letter recognition.