Yup, that's it! Check out the entrance here:
http://www.marylandromanticgetaways.com/man.html

Strange it isn't being advertised as a commercial property. Supposed to be a replica of Thomas Point having become a sanctioned light by the Coast Guard in 1993.