I don’t think listening breaks any rules, but I might be wrong. If you look at this summary of the Berne Convention, which is the framework used for most international copyright law, there are certain rights that are exclusive for a rights holder. However, none of these rights are ‘listening’. There might be some nuance I’m not getting, but my understanding is that if you listen to a copyrighted song on archive.org then archive.org would be the potential rights infringer in this situation.
That being said, if you check out the source for my second quote or the Wikipedia page on the rule of the shorter term for more info you will find that whether or not a material counts as copyright depends on the country where that material is being used.
Not that I’m telling you not to use copyrighted material if you want. As you said, you can just use it and deal with the consequences if they come up.
I just wanted to give some resource for anyone who was concerned about it.