In modern terms, what the Buddha meant by not-the-atman (S., anātman) is the existential self which is taken to be localized to the corporeal body as in: “this is mine”, “I am this”, “this is my self”. All theories and views of self relate to the existential self which is not the real atman. The existential self is an aggregation of five aggregates (S., skandha) of experience: (1) shape or objectness, (2) feeling, (3) perceiving (also in the sense of ’thinking’), (4) volitional function, (5) sensory consciousness/awareness.
The existential self is a conceptual self—certainly, unreal. These five aggregates or domains of experience are aspects of the existential self which is not to be confused with the atman, the latter being closer to an observer which cannot be observed or measured. Nor should the atman or observer be confused with awareness which is the fifth domain of experience this being consciousness/awareness.
The atman in its observing role is outside of the five aggregates of experience which is why it is transcendent since it cannot, itself, be observed by any of the aggregates such as awareness. The atman’s fundamental nature becomes evident for us only when we have thoroughly abandoned desire for the existential self. In this regard, it is also unconditioned which is why the Buddha says the very atman attains highest nirvana, nirvana being unconditioned (the unconditioned comes together with the unconditioned). Only in perfect negation or the same, the absolute setting aside of the existential self is nirvana realized.
In the midst of the finite, the existential self and its world have limits and boundaries. Try as we might to go beyond these limits and boundaries in every case we fail. As long as we ignore the transcendent atman, which I hasten to add is not a self or an individual, we ever remain bound down to a kind of biological vicious circle which Buddhists call samsara.