"AAU membership is by invitation only, which requires an affirmative vote of three-fourths of current members. Invitations are considered periodically, based in part on an assessment of the breadth and quality of university programs of research and graduate education, as well as undergraduate education."
While BYU has some great graduate level programs, BYU has gone out of its way to emphasize that undergraduate education is it's core. Undergraduate students are more likely to be taught in the classroom by a professor than at many other universities. BYU's research emphasizes involving undergrads in the research. AAU schools tend to be more graduate level heavy and involve a lot more research than BYU.