Passwords just aren’t secure anymore, as evidenced by the many security breaches over the last few years. Nowadays, social engineering attacks make it easy for practically anyone to easily acquire access to your accounts. If you are going to use the same password for everything regardless of how many people tell you it’s a bad idea then at the very least make your email password unique.
When someone has your email password they can often get their hands on other passwords that may be in old emails or may be easily retrieved through lost password retrieval systems. Email passwords are also commonly stored some where on computers in spots often targeted by viruses.