How to scrape many files of the website without getting blocked

• Jun 2, 2019 - 16:23

I am working on a project that consists of a neural network trying to recreate music. To do this, I need a lot of data. I wrote a web scraper to download about 200,000 solo violin files from MuseScore. After downloading about 200, I got blocked from the site and only get a http 403 access denied error. My problems are these: how do I get unblocked? (Or more likely, how long until I get unblocked). Second, how can I download 200,000 music files without getting blocked from the website?


Comments

I suggest that you use the contact link at the bottom of this page and explain what you want to do. They can advise you and possibly even give you a special temporary permission for doing this. Keep in mind that today is Sunday, even in Russia and they may need to do some internal talking before they give you an answer.

Do you still have an unanswered question? Please log in first to post your question.