It's a contradiction what you wrote.
If scarcity leads to progress then why Neanderthals did not land on the Moon? They should have just continue what they did - struggling for make a living - and could have discover singularity way before Rome existed, no?!
"Advances" happen just about any time, but real leaps and wonders are created when:
1. there is abundance
2. the society have a mindset that rewards thinking, initiative, inventions, etc.
The "cave man" come up with quite important and ingenious things, for example - the often overlooked - leather tanning was quite a feat on his behalf.
But writing, mathematics, astronomy (astrology), literature, etc was created when the society produced more than subsistence, and there weren't need for the work of a considerable amount of people, they could do nothing else all day just munch the food peasants provided, observe the world and think about it. Sumer, Egypt, India, China, they all were these.
I flip open Wikipedia at "Steam Engine". I see names: Beaumont, Savery, Newcomen, Watt. Were these serfs, or at least hunger gatherers? No. Neither of them had to be busy by providing himself to barely stay alive.
I see England mentioned, rightfully so, but I see missing one part. In England after they killed off her nobility in the War of Roses a new gentry emerged with an entrepreneurial mindset, and a new zeitgeist formed which rewarded those who had the courage to take initiative.
Rome had abundance, but they never had similar mindset. Not in the useless pleb, and not in the even more useless patricians, not in their zeitgeist. If they had a millennia more they wouldn't have invented anything.
I think you guys mistake competition with scarcity. Cold War produced much in technological advancement. But it's a time of abundance. Both US and SU produced way more than subsistence.