Psychology says the reason walking away from disrespectful people feels like guilt instead of freedom is because you were raised in an environment where your comfort was never a valid reason to make someone else uncomfortable — and unlearning that equation is the hardest boundary work there is

Ever feel like you’re the bad guy for walking away from someone who treats you poorly? Like somehow you’re being unreasonable or selfish for not tolerating disrespect?

I get it. I spent years feeling that exact same guilt, wondering why protecting my own peace felt so wrong.

Here’s what I’ve learned: that guilt isn’t about what you’re doing now. It’s about what you learned growing up.

Think about it. If you were raised in an environment where keeping others comfortable was more important than your own wellbeing, of course setting boundaries feels like betrayal. You were essentially programmed to believe that your discomfort was an acceptable price for someone else’s happiness.

And here’s the kicker – unlearning that programming might be the hardest personal work you’ll ever do.

Why walking away feels so wrong

Let me share something that took me way too long to understand. During my mid-20s, I was constantly anxious about disappointing people. I’d stay in conversations that drained me, maintain friendships that felt one-sided, and tolerate behavior that made my stomach churn.

Why? Because somewhere along the line, I’d learned that making someone else uncomfortable – even someone who was actively disrespecting me – made me the villain.

Ilene S. Cohen, Ph.D., a psychotherapist, puts it perfectly: “People-pleasers tend to focus on what others need.” And when you’re focused on everyone else’s needs, your own comfort becomes invisible, even to yourself.

The guilt you feel when walking away? That’s your old programming kicking in. It’s the voice that says you’re responsible for managing other people’s emotions, even at your own expense.

The discomfort trap

Here’s something that blew my mind when I first heard it. Jennifer Guttman, Psy.D., a clinical psychologist, says: “Setting a boundary will absolutely make you uncomfortable initially. But discomfort doesn’t mean you harmed someone or did something wrong.”

Read that again.

Just because something feels uncomfortable doesn’t mean it’s wrong. But when you’ve been conditioned to avoid making others uncomfortable at all costs, your internal alarm system goes haywire the moment you prioritize yourself.

I remember the first time I told a chronically late friend that I’d leave if they weren’t there within 15 minutes of our agreed time. My hands were shaking. My heart was racing. You’d think I was committing a crime, not setting a reasonable expectation.

That physical response? That’s years of conditioning telling you that you’re doing something terrible. But you’re not. You’re simply saying that your time and comfort matter too.

When boundaries backfire

Have you ever set a boundary only to cave immediately when someone pushed back? Yeah, me too.

Julie Radico, Psy.D. ABPP, a board-certified clinical psychologist, points out something crucial: “You may be unknowingly reinforcing others ignoring your needs.”

When we set boundaries but don’t maintain them, we’re essentially teaching people that our boundaries are suggestions, not requirements. And guess what happens next? People learn to push harder, knowing we’ll eventually give in.

The thing is, people who truly respect you won’t make you feel guilty for having boundaries. As Quinelle Hickman, a licensed therapist, notes: “A main sign that someone doesn’t respect your boundaries is if they don’t stop their actions after you’ve expressed discomfort.”

If someone makes you feel guilty for protecting your peace, that’s information about them, not about you.

The family factor

Let’s talk about where this all starts – family.

Research found that 72% of Americans struggle to set healthy family boundaries due to guilt or obligation. Nearly half avoid boundaries to support family members, and over a third fear disappointing others.

That’s not a coincidence. Family is often where we first learn that our comfort doesn’t matter as much as keeping the peace. Maybe you were the one expected to be “understanding” when a sibling acted out. Or you learned to tiptoe around a parent’s moods to avoid conflict.

These early lessons become our blueprint for all relationships. Research indicates that individuals exposed to toxic family environments face higher risks for anxiety, depression, and chronic stress, with guilt often used as a manipulation tactic to maintain control and discourage independence.

In my book, “Hidden Secrets of Buddhism: How To Live With Maximum Impact and Minimum Ego,” I explore how attachment to others’ expectations creates suffering. When we’re attached to the idea that we must keep everyone comfortable, we suffer. When we’re attached to avoiding conflict at all costs, we suffer. The path to peace requires letting go of these attachments.

The action versus the announcement

Here’s where most of us get stuck. We think setting a boundary is about making a declaration. But that’s only half the equation.

Samantha Foti, a counselor, explains it brilliantly: “Setting a boundary involves saying something out loud. Enforcing that boundary or maintaining that boundary involves action – changing your behavior.”

You can tell someone you won’t tolerate disrespect, but if you continue engaging when they’re disrespectful, your words mean nothing. The real boundary work happens in the action – in actually walking away, hanging up the phone, or leaving the situation.

This is where that guilt hits hardest. Because now you’re not just saying something that makes someone uncomfortable, you’re doing something about it. And if you were raised to prioritize others’ comfort, this feels like breaking a fundamental rule.

Rewriting the equation

So how do you unlearn decades of programming? How do you stop feeling like the villain when you’re simply protecting your peace?

First, recognize that the guilt is a learned response, not a moral compass. Dr. Charles Browning, a psychologist, offers this perspective: “The harder it is for people to offend you, the fewer battles you’ll have to fight through.”

Start small. Practice with low-stakes situations. Maybe it’s saying no to plans you don’t want to attend, or speaking up when someone interrupts you. Each time you honor your own comfort, you’re teaching your nervous system that it’s safe to have boundaries.

Remember too that people who truly care about you want you to be comfortable. Leah Katz, Ph.D., a clinical psychologist, reminds us: “We all know a person we deeply respect and admire for their ability to skillfully set boundaries.”

Think about it – do you respect people less when they protect their wellbeing? Or do you actually admire their self-respect?

Final words

The guilt you feel when walking away from disrespect isn’t a sign you’re doing something wrong. It’s a sign you’re doing something different.

You’re breaking a pattern that was established long before you had a say in it. You’re choosing your comfort over someone else’s convenience. And yes, that feels uncomfortable because it goes against everything you were taught.

But here’s what I’ve learned through years of anxiety and overthinking: the people who make you feel guilty for having boundaries are usually the ones who benefited from you having none.

Your comfort is a valid reason for someone else to adjust their behavior. Your peace is worth protecting. And walking away from disrespect isn’t selfish – it’s necessary.

The equation you learned – that your discomfort is an acceptable price for others’ comfort – was always broken. Unlearning it isn’t just boundary work. It’s freedom work. And while it might be the hardest thing you do, it’s also the most important.