Great question, we have done many many AB-tests, it's often the case that the control group are manually curated recommendations. In general, the more SKU:s, the better result our method yields, since they have to deal with sparse datasets & cold start problems if they use classic collaborative filtering based methods. And naturally, if you have more products it becomes a huge pain to do them all manually, especially if you have a reasonable product turnover.