I have just implemented a new process to download larger amount of files from the same source. Before we used to simply loop through single download processes. This was not ideal in terms of performance and also I would get spammed with tons of errors if something went wrong.
We use these for the user icons and for the block icons in the shop manager. Now we start one process, pipe all the the urls into it and then process them in one go.
- It’s closer to best-practice coding standards
- it’s friendlier to the server where we pull the data from
- we know get 100% of our user icons from our own server once the user sees them, so it’s faster and more reliable. We used to pull them partly from external server
- I found some other bugs along the way and fixed those
- I get less but more meaningful errors from the system in case the external source of the data fails.
By the way, I won’t be online for the coming 3 days a lot, so if you need something, please submit a ticket or send an email.