> The cost of failure is low: Most domains (physical, compliance, etc) don't have this luxury where the cost of failure is high and so the validator has more value.
This is not entirely sensible, some code touches the physical / compliance world. Airports, airplanes, hospitals, cranes, water systems, military they all use code to different degrees. It's true that they can perhaps afford to run experiments over landing pages, but I don't think they can simply disrupt their workers and clients on a regular basis.
I did say "not all software is like this, but many domains are". So I agree with you.
Also note unlike say for physical domains where it's expensive to "tear down" until you commit and deploy (i.e. while the code is being worked on) you can try/iterate/refine via your IDE, shell, whatever. Its just text files after all; in the end you are accountable for the final verification step before it is published. I never said we don't need a verification step; or a gate before it goes to production systems. I'm saying its easier to throw away "hallucinations" that don't work and you can work around gaps in the model with iterations/retries/multiple versions until the user is happy with it.
Conversely I couldn't have an AI build a house, I don't like it, it demolishes it and builds a slightly different one, etc etc until I say "I'm happy with this product, please proceed". The sheer amount of resource waste and time spent in doing so would be enormous. I can simulate, generate plans, etc maybe with AI but nothing beats seeing the "physical thing" for some products especially when there isn't the budget/resources to "retry/change".
TL;DR the greater the cost of iteration/failure; the less likely you can use iteration to cover up gaps in your statistical model (i.e. tail risks are more likely to bite/harder to mitigate).
This is not entirely sensible, some code touches the physical / compliance world. Airports, airplanes, hospitals, cranes, water systems, military they all use code to different degrees. It's true that they can perhaps afford to run experiments over landing pages, but I don't think they can simply disrupt their workers and clients on a regular basis.