The idea of classifier construction via 'Iterative Denoising' trees --- that is, by successively partitioning (at the internal nodes of the tree) a class-labeled training data set into ever-more homogeneous subsets without consideration of class labels, and only subsequently (at the leaves of the tree) using the available class-label information, while at each node (internal or leaf) choosing a dimensionality reduction appropriate to and specific to (the data falling in) that partition cell --- may seem counter-intuitive, but is, in fact, in (rough) accordance with Fisher's conditionality principle and can, in fact, provide performance superior to that of competing approaches. We describe the theory and application of these 'Iterative Denoising' trees and illustrate their performance, and relate the ideas to 'Integrated Sensing and Processing' and theorized thalamocortical brain circuit computation.