Its not really illegal to download them, but its illegal to posses an ISO if you didnt rightfully own it. The act of download is no crime. Who is to say you even knew what you were downloading? Watch for the little print on a web site or an agreement check box. This also depend on your country.
Here is some detail general info.
No its not illegal simply be cause you have no idea what you are download. I could name a file readme.txt and it could be a rom. How is that the downloaders fault? The only way it could be illegal is if you check a box agreeing to it. Its the same grey area as leaving a store with an unpaid item in your cart. Point is its beatable in court and no one is going to prosecute you over it. Its what we refer to as ethics.
Possession is illegal receiving is not. If you "receive" a stolen item, you can ethically report this to the police. Your not going to get sited for doing so? Because there is no crime in that. However if you keep it and get caught with it, its considered possession of stole property.
If you watch the warning on most rom sites, it says right there on the page. You may not download this file unless you have the original version. This pretty much covers there buts and give you the responsibility of knowing what you are downloading. Though, what if you got a download linked form another site without the warning? In that case its the middle mans issue.
In light of the question at hand, and as jbone1337 pointed out, yeah your ok downloading. No internet provider is going to tangle with you. I here that people with large amounts of stolen software can get stung, but you have a better shot winning the lottery.