> What improves (and guarantees) software quality is rigorous testing. To deliver high quality software, there is no other solution.
This is 100% false. You can’t inject quality into a bad design via testing. You can’t guarantee quality or correctness through testing. Testing is the second worst place to find errors.
You get a MUCH higher roi by producing thoughtful written designs and getting them peer reviewed. And again with code reviews. (Peer review is best but self review can also be very effective.) I’m not saying don’t test — testing is valuable, especially because it forces you to design for testability, and a suite of automated tests is valuable when making changes later.
I wonder how much experience you have in the real world. All of us developers had this thought at some point.
I've seen terrible code with plenty of quick hacks on top of that, which was running very stable, because it was battle tested for years in production.
What do you think will happen when you refactor such code to a better design? All juniors would think this is the best way to get a stable codebase.
Reality is that you will introduce bugs by refactoring, not get rid of it. Writing code means writing bugs, no matter how pretty your code is.
This reminds me of some quote that "Junior developers write 1 bug per 10 lines of code, experienced coders know they write 1 bug per 10 lines of code"
You might be one of those experienced developers who has gone through their entire career without ever meeting a well designed, beautifully architectured application, and therefore concluded, sensibly, that all real-world applications look like shit if you look inside, and you seem to have come to believe that this is the only possible way.
I have a different experience. Code that is well designed makes mistakes look obvious and the logic feel natural. It's very hard to write code like that, but I've seen it and I've been able to write it myself, so when I see a bunch of complicated code that only tests can tell whether or not it works, I know that the problem is really that the best design simply has not been found yet. If I find code that looks like shit and I realize there's a better design for it that will make the code clear and natural, I won't think twice to refactor it ruthlessly if such code has been a source of problems and time wasted (and fixing difficult to read code is huge costly to morale as well, which when taken into consideration, increases the cost of technical debt by a lot).
I've written code in all sorts of scenarios: government, web application back and front ends, auth, simulations, testing software, statical analysis, long-maintained products, throwaway but complex scripts for one-use-only, and multiple popular libraries I maintain.
Difficult-to-read code always causes problems, either because it directly causes bugs due to its logic being obfuscated, or because no one can approach it to make necessary modifications when change is invevitably required.
You claim that "The reason something was stable was never how the code looked, but how battle tested the product was". I am claiming that you're wrong and your counter-argument is that I will learn to be like you one day. This shows that you're the one who has to grow up.
I am constantly looking for ways to write code that works in all scenarios, provably. I don't need to battle test code if I can do that. I only need battle tested code in the absence of that property. I will concede that in many occasions, this is all you can get, but that's due to a lack of time and skills, not an inherent property of a system or the process of writing software.
It's sad to see people who gave up on creating good software (because if you only can create reliable software after it's battle tested, that means it was buggy all along before that - so for the majority of the life of your product, your software was buggy - that is not acceptable).
If you believe I am a dreamer, an unexperienced developer, let me prove you wrong. Show me some code you believe it's impossible to make reliable without "battle-testing" it over a decade, and I will show you how.
The original argument is what I consider one of the biggest beginner mistakes. So the inexperience argument is very much relevant.
I was young and inexperienced once, and I had the same belief. But after bumping my head against the wall, I learned. In the mean time, I saw plenty of people bumping their head against the same wall. They all learned.
So therefore I assume this person is indeed inexperienced. If I'm talking to some senior developer, then it's up to me to reassess my way of thinking, and looking where I'm missing something. But if not, then it's just the phase we all went through.
> I would love to see highly optimized code that is easy to read.
That that's a trade-off is [just] a deficiency in our programming languages and tooling around them. As we evolve are young field (and if you've had a long career, then it is even younger to you!) I trust we will find more and more highly-optimized code that is easy to read.
Computer languages are a means of communication between a human and a computer. They clearly evolved closer to human readability and understanding, and away from computer language itself.
When you optimize, your code needs to be closer to computer language again, and therefore further away from human readability.
The only way to have both, is to have a system that can optimize the code automatically. We are getting closer to that, but still sometimes this is not enough.
As a 20+ year vet, what worries me is when someone starts talking in terms of good or bad.
It's a spectrum, not a binary. There is code that's just egregiously bad. There's code that's just egregiously good (and it's not clear which is more expensive to produce and maintain). But the vast majority falls into the grey area in between those extremes.
The other poster (koonsolo) has just seen enough different codebases to have come to the conclusion that it's not nearly the most important aspect of a projects success. And I agree with them on that point (wholeheartedly in fact).
Developers concentrate on it because it's what they're in all day, the same way that a DBA is going to concentrate on the data model because that's where they're at all day.
But in the grand scheme of things, developers way overblow the value of great code. I also have reservations about the claim that most developers even know what great code _LOOKS_ like, I've seen too many terrible codebases that were claimed to be well designed by their creators.
My point is that if the code isn't actively causing you problems then leave it be, and stop calling it good or bad. Call it what it is, working code that doesn't need to be bikeshedded over.
> I've seen terrible code with plenty of quick hacks on top of that, which was running very stable, because it was battle tested for years in production.
"Battle tested" is not what is meant by testing here, though. And I've seen software that was "battle tested" and rock-solid despite barely being _actively_ tested at all. Mind you, it was at least part of the economics of the software itself - "battle" (i.e. "in production") testing was perfectly viable in its case.
I value testing, but primarily because (in years of experience, etc.) it makes code easier to refactor or reuse, not easier to make _correct_. Actually, this also applies to static types, which I also value.
But suitability-to-purpose (the best proxy for correctness that I've yet found) is far more determined by peer-reviewed design than by tests.
I agree that testing is not a panacea, but I also don't think it is completely divorced from design.
I see testing and design as cooperative processes, I agree that design is almost always the highest yielding process in any non-trivial program, but I think of testing almost as a design phase that tries to invert the design to look for weaknesses.
I suspect what he’s referring to is the fact that so many people think that, because they have 100% code coverage in their tests, they have quality software. I’ve seen so many times a team claim that with pride when it’s brutally obvious to anyone who looks that the product is broken. They just don’t seem to be able to reconcile that problem when asked about it.
What you test matters every bit as much as how much you test. Tests require thought and design as well. Too many developers don’t seem to understand that.
That's part of what I was talking about, but the original statement in the article was that rigorous testing guarantees software quality, and it also says "there is no other solution", and that is demonstrably false.
One example: A telecom product that processed phone calls. I stepped into a role on the team building the call progression handling system, which was based on an ad hoc state machine and some inter-task communication that included a combination of mutexes and message queues. It was a mess. Our testers succeeded in churning out a lot of bug reports. Not a single one of those bug reports did anything to improve the quality of the system -- even if we're not being pedantic and saying that only the patches in response to the bug reports were what might have improved the quality. Because none of the patching we could do really made any net increase in quality. The only thing that improved system quality was stepping back and re-examining the design. We changed the existing undocumented design to get rid of deadlocks by just using one inter-task mechanism (just queues, no mutexes) and by formalizing the state machines so that none of them got stranded in an unexpected state. Once the design was formalized this way we found -- by inspection, not testing -- a number of fundamental bugs in the states and transitions. After fixing those, more system testing showed some more bugs, but we were able to assess these in the context of the design and fix them at the root cause rather than just hacking on patches. The design changes also enabled automated integration testing that wasn't possible with the old ball of mud. This helped save time in allowing the [mostly manual] system testers to focus on certain areas of the system.
I've worked in other situations where lots of testing just resulted in lots of bug reports, which ended up mostly getting deferred and thus did nothing whatsoever to improve product quality.
And I've worked on software teams that were in fact producing rather high quality software, but we were building the wrong thing. It's really hard to fix bad requirements through testing.
I don't want to sound like I'm anti-testing, but I think it should be understood that testing isn't the source of quality, it's just the last phase of a process that hopefully has (non-test) mechanisms to ensure a quality product in the upstream process phases.
While I agree with you in general, it should be noted that OP was talking about unit tests, not manual or integration tests. Unit tests are always touted by dynamic typing people as their equivalent of the static typing fast feedback loop.
Right, spending all your effort on the one true design gets you in architect astronaut territory. Similarly for existing systems, conjuring up the grand rewrite in the sky (as Robert Martin calls it, I think?) is equally poor. Instead, recognize that design is important but will constantly evolve. Tests give the ability to incrementally redesign quickly and confidently.
Testing tends to validate design from the point of testability. If it feels like the code is resisting testing, that usually ends up being an indicator of bad design (though not always vice versa).
This is where we are now. We know how to make testable designs, since effort has been put towards it.
What we lost is being able to distinguish good vs bad designs in other key aspects. One indicator I use is the number of test cases that are needed for covering. A poorly separated design will need to test many combinations having not isolated the factors.
I have noticed another indicator of good design is how easy it is to add specific fixes/features without having to think too much about how many places you'll need to make changes. Interesting point about no. of test cases. I tend to end up with several combinations because I prefer to focus on integration tests more for crud-type applications (a test configuration type/class comes handy in such scenarios).
As an example, if testing a 'send' function that takes a sender, content/media type, and recipients where recipients can be a single contact or a group, one shouldn't have to check that all combinations of content/media-type works with single or group recipients and other variable option variables. If decomposed in orthogonal ways K+L+N+M tests and perhaps a handful overall edge cases should do and not KxLxNxM if each variable dimension has K, L, N, M possibilities. If not cleanly separated in the implementation structure, the cross product of test cases are needed.
This is 100% false. You can’t inject quality into a bad design via testing. You can’t guarantee quality or correctness through testing. Testing is the second worst place to find errors.
You get a MUCH higher roi by producing thoughtful written designs and getting them peer reviewed. And again with code reviews. (Peer review is best but self review can also be very effective.) I’m not saying don’t test — testing is valuable, especially because it forces you to design for testability, and a suite of automated tests is valuable when making changes later.