Designing for a small screen first is not about shrinking a desktop layout. It is about deciding what matters most when there is only room for one thing.
Priorities first
On a phone you cannot hide a weak hierarchy behind whitespace. The content that earns the top of the screen is the content that matters.
Then add, never subtract
As the screen grows we add columns and context. We never have to claw features back out of a layout that was too crowded to begin with.