Most of us spend our entire lives chasing a bigger number. Paying down the mortgage, stashing money into retirement accounts, building up that emergency cushion. The goal always feels like it's somewhere ahead of us - just a little more saving, a little more time.

But here's something most people don't talk about: there actually comes a point when that number starts going the other way. And the reasons why might surprise you.

There's also a smaller group of people who never really experience this. Their wealth keeps climbing even after they stop working. And once you understand what they're doing differently, you realize it's not some secret and it's something you can actually plan for.

So let's get into it.

According to the Federal Reserve, here's how the typical American household net worth looks by age. People between 55 and 64 sit at around $365,000. By 65 to 74, it ticks up a bit to about $410,000. Then for anyone 75 and older, it drops back down to around $335,000. Decades of work, saving, and sacrifice and then it starts shrinking.

Not everyone experiences this the same way. There are really two camps: people whose wealth keeps growing in retirement, and people who watch it fall pretty fast. The difference between those two groups isn't what you'd probably guess. Most of the wealth people accumulate over their lifetime isn't from brilliant investing or picking the right stocks. The biggest driver, for most households, is just owning a home. Every mortgage payment chips away at what you owe, property values tend to climb over time, and slowly but surely your net worth goes up - even if your investment portfolio is basically just sitting there.

It's better than nothing, obviously. But home equity and retirement income are very different things.

Take a household in their late 30s or early 40s. Median net worth is around $135,000 at that point. Life is expensive and there's probably a mortgage, maybe some student loan debt still hanging around, kids to raise. But as the years go by, the mortgage shrinks, the house goes up in value, and the net worth follows. By the late 50s and early 60s, that number has typically climbed to somewhere between $300,000 and $400,000 - with home equity making up a huge chunk of it.

Then it peaks, somewhere in the late 60s to early 70s, around that $410,000 mark. And after that? For a lot of people, it starts to slide. The main reason isn't that people are out there living it up and spending their savings intentionally. It's more that they just didn't have enough coming in to cover everything going out. Social Security helps, maybe a pension if you're lucky, but for a lot of retirees that's still not quite enough. So every month, they're pulling money out of their savings to cover groceries, utilities, medical bills, whatever it is. No real plan, just keeping the lights on.

That steady drip of withdrawals adds up fast, and what feels like a solid financial cushion starts to feel a lot thinner. The problem is no one knows how long retirement is actually going to last. It could be 15 years. It could be 30. You'd need to know exactly how long you'll live, how markets will perform, what inflation will do and obviously none of us have that information.

The people getting hit the hardest aren't wealthy retirees with a detailed drawdown strategy. They're people with modest savings and maybe a few hundred thousand total, with only a fraction of that actually liquid. Every unexpected expense pushes them to pull from their portfolio a little more. Over time, it snowballs.

A lot of people assume that if you're withdrawing from your portfolio every year, your net worth has to be shrinking. But that's not necessarily how it works. The 4% rule when withdrawing around 4% of your portfolio annually is specifically designed to be sustainable across a long retirement. When investments are growing faster than you're pulling out, your net worth can stay flat or even rise. It's based on decades of historical data that includes crashes, recessions, and high inflation.

People who enter retirement with a plan and don't have the bad luck of retiring at an absolute market low tend to come out just fine. There's also a natural instinct most people have when markets turn rough: they pull back, spend a little less, get cautious. That instinct, it turns out, is actually protective. It gives a portfolio room to recover without being drained at the worst possible time.

Flexibility matters too. A small amount of part-time or freelance income, even irregular, takes meaningful pressure off savings and buys you options when you need them most. People whose wealth keeps growing through retirement aren't doing something magical. They entered retirement with a real withdrawal strategy, stayed flexible when conditions changed, and resisted the urge to raid their portfolio every time something unexpected came up. Over time, that discipline is worth more than any single investment decision they ever made.