It depends on the contract. Some contracts do go to the lowest bidder (the technical term that the contracting folks use is something like "lowest bid meeting technical requirements"). For those sorts of bids, it's extra important that the requirements that the government puts out are well written and thorough.

Some procurements use the "best value" criteria. I think these contracts are more likely to be protested and have lawsuits filed by the losers. It's easy to say "I deserve this contract because $2 billion < $3 billion". Best value can be a bit more subjective.