Generally, designing with accessibility in mind improves the product for everyone and adds a little to the cost. Retrofitting accessibility produces an expensive dog's breakfast of a product.
In other words, if you don't design to include, you are planning to exclude.
Unfortunately I simply don't agree that it's a better product for everyone, for a few reasons, but the most simple is because there are costs involved.
A hypothetical zero additional effort inclusion at design stage, still wouldn't remove the issue where I now have an additional requirement limiting how I can approach things. I might also end up with higher overhead to modify a system as change requests come.
This is time and money I have to spend not working on the rest of the product.
Then you get into deeper issues, where if you actually have accessibility as a first class goal, much of your design should change. Consider an ideal UI for attracting new unguided users vs one with the expectation that users will have training and become experts. Then add in design for specific impairments. It's not easy and should be a big driver of the overall design, usually only done if the customer explicitly has a reason for it.
Again, it's not about deliberate exclusion; it's that I am trying to spend my energy in a way that gives me the most financial benefit (the selfish reasons as per the article), and accessibility does not do that.
In other words, if you don't design to include, you are planning to exclude.