Most Tokyo attractions will remain open during the "Golden Week" holiday period, which runs until May 6th.
Places that are normally closed on Monday will be open on May 4th, including spots like the Tokyo National Museum or the East Gardens of the Imperial Palace.
Some attractions will remain open during Golden Week, then have irregular closing dates following it. For example, both the Edo-Tokyo Museum and Shinjuku Gyoen National Garden will be closed on May 7th (Thursday).
If your visit is during or immediately after the Golden Week, it is a good idea to check the official website to avoid any surprises upon arrival.