The classical debate between empiricism and rationalism (now voiced in terms of representationalism) has seen a new impetus from an unexpected corner. Contemporary defenders of empiricism argue that deep networks finally vindicate their stance. We will start by introducing the very concept of deep networks, distinguishing them from classical connectionism and explaining how they manifest “transformational abstraction” (Buckner 2018). Then we will introduce the phenomenon of adversarial attacks that significantly complicates the interpretation of deep networks’ successes. Based on this phenomenon and its various implications, we will close by voicing skepticism about the helping hand deep networks offer to empiricists.