Fun with expressing software systems, generally, as periodic functions oscillating between limits defined in terms of Exploration and Maturity.
Most software systems are products. One way to prove these systems is to ensure that people are paying money for them. People do not normally pay money for axiomatically unprovable software systems.
But if your only mechanism of provability is that people are giving you money, you could easily lose sight of whether or not you have a provable system, which would render the system immediately unprovable and unprofitable.
There are limits to the provability of all systems — so it goes for things in the universe.3 But you should still arrive at a certain level of comfort in the provability of your systems, because the unprovable things are actually just really ridiculously uncommon.
Practically speaking, no system contains features that are truly unprovable. Clasically unprovable systems are so far outside of (or deeply inside of, straddling the atom level) the boundaries of our day-to-day reality that they are generally not worth speaking about, except in the presence of integration with another system that, itself, is on the edges of provability.4
It's way more likely that you come up against provability-versus-time-investment. "Do we invest in proving this piece of the system or not? And when should we?"
Inflexibility can creep in if the developer is overzealous about proof about what the system is. Teams of contributors, within and across disciplines, should share agreement about what the system actually is.↩
This inflexibility, co-orbital with Software Death, is Software Paralysis.↩