The site, which is based in Russia, offers albums to download for as little as £1 and claims to be licensed.
The BPI plan to sue allofmp3.com in the UK courts but say that users would not be prosecuted.
Allofmp3.com is the second most popular download site in the UK, accounting for 14% of downloads, a survey has said.
BPI spokesman Matt Phillips said: "There is no doubt it is totally illegal.
"It is illegal for them to sell the recordings in Russia or anywhere else because they are not licensed, and UK consumers are infringing copyright law because they are making illegal copies from an unlicensed source."
And he added: "Because it is a professionally put together site it does look legitimate, although it should be obvious from the price that it isn't."
allofmp3.com says it is licensed by the Russian Multimedia and Internet Society (ROMS) and the Rightholders Federation for Collective Copyright Management of Works Used Interactively (FAIR), however, the International Federation of the Phonographic Industry (IFPI) said the ROMS licence was not legitimate and would not cover consumers in other countries anyway.
The IFPI say that criminal proceedings were ongoing in Russia in relation to allofmp3.com.
