This is the result of a quick search of HPLovecraft.com for the word “museum.” Very exhaustive, and fool-proof, I’m sure. The following are museums (but not libraries), real and fictional, mentioned by name (or implied specifically enough to ID) in the written fiction of H. P. Lovecraft:
The Australian Museum, Sydney, Australia. Holds the “Alert” Cthulhu idol from The Call of Cthulhu. Real.
The British Museum, London, UK. Holds a copy of the Necronomicon, as stated in The History of the Necronomicon. This is curious, as The British Library would seem to be a better custodian of such a rare book. Real.
The Cabot Museum of Archaeology, Boston, Massachusetts, USA. Greatest collection of in mummies in America, as mentioned in Out of the Aeons. Fictional.
Captain Orne’s ‘marine museum,’ Martin’s Beach, Massachusetts, USA. Short-lived museum built on a boat, containing the remains of a sea monster, from The Horror at Martin’s Beach. Fictional.
The Miskatonic University Museum, Arkham, Massachusetts, USA. Contains some Innsmouth jewelry, as mentioned in The Shadow Over Innsmouth. Fictional.
The Newburyport Historical Society, Newburyport, Massachusetts, USA. Innsmouth jewelry, The Shadow Over Innsmouth. Fictionalized version of The Museum of Old Newbury.
The Paterson Museum, Paterson, New Jersey, USA. Local and natural history, mentioned in The Call of Cthulhu. Real.
The Rhode Island School of Design Museum, Providence, Rhode Island, USA. Art modern & historical. Mentioned as containing materials related to the local witch trials in The Shunned House. Real.
Rogers’ Museum, London, UK. Wax figures / monsters, from The Horror in the Museum. Fictional.