Many scholars would agree with you that the Library was part of the Mouseion! The earliest scholarship I have found making this point is Parsons 1952: 5; the most recent is Cavalieri 2017: 29.
Parsons, E. 1952. The Alexandrian Library, Glory of the Hellenic World: Its Rise, Antiquities, and Destructions. Amsterdam.
Cavalieri, M. 2017. “βιβλιοθήκη/bibliotheca: le mot et la chose en Grèce et à Rome.” In Amoroso et al.: 23–125.