Reject the premise that a NN algorithm must find the NN for every instance.
The first NN density estimator that has ulatext stixSupport mathImg" data-mathURL="/science?_ob=MathURL&_method=retrieve&_eid=1-s2.0-S0031320314000314&_mathId=si0092.gif&_user=111111111&_pii=S0031320314000314&_rdoc=1&_issn=00313203&md5=a504b6af0a3d8ca15fa58892d44a366b" title="Click to view the MathML source">O(n) time complexity and ulatext stixSupport mathImg" data-mathURL="/science?_ob=MathURL&_method=retrieve&_eid=1-s2.0-S0031320314000314&_mathId=si0093.gif&_user=111111111&_pii=S0031320314000314&_rdoc=1&_issn=00313203&md5=0ef5333d49a499172794188fc88b6550" title="Click to view the MathML source">O(1) space complexity.
These complexities are achieved without using any indexing scheme.
Our asymptotic analysis reveals that it trades off between bias and variance.
Easily scales up to large data sets in anomaly detection and clustering tasks.