For activites for children, the harbor area has a great aquarium and science center. I went to both numerous times as a kid on school trips. There is also a zoo in Baltimore not too far away from the harbor. If memory serves me correctly there is a play ground area not too far away from the science center as well.
If the weather is nice and you kids are old enough, try the paddle boats out for a half hour or so. That was always the best treat my parents could give my sisters and I while we were in Baltimore.
If you need more ideas let me know. I grew up in the area and am always going home for the weekend.

Enjoy your trip.
shirin