"There's black art all around my house," she tells PEOPLE with a laugh. "I literally am like, 'There can't be any identity crises in this house!' "
"I remember that was a big thing for me as a kid," recalls the star. "I didn't see enough people around on the walls or anything like that looked like me. So everywhere you turn in our house, there is definitely black art; you see a representation of yourself."
Recently, Rowland and Weatherspoon, 38, gave their son a sweet pep talk about self-love. In fact, the couple have even begun practicing daily affirmations with their little one.
"We had a conversation about it maybe a week ago and he was like, 'Yeah, you know, because I'm brown, Mom.' I said, 'No, you're black, son,' " the star tells PEOPLE. "He said, 'No, Mommy, I'm brown. I'm brown, look at my skin.' Then he goes, 'Actually, I'm lighter than brown.' "
Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.
Stefanie Keenan/Getty Tim Weatherspoon (L) and Kelly Rowland with son Titan
That's when Rowland decided to give Titan a lesson on what it means to be a part of the all-encompassing black diaspora.
"I was like, 'Son, we are black. We are beautiful. We are rich in love and history' and this and that. I broke it down for him in a solid maybe 3 minutes, and I didn't shut up," she says — but her words paid off. "That night, I heard my husband giving him a bath and he goes, 'Daddy, I am black and handsome and great.' "
"We wrote it on his mirror. It's actually written on his mirror: 'I am smart. I am brave. I am kind. I am wonderful. I am black. I am excited,' " Rowland proudly reveals.
"We read different stories to him as far as African American and African history is concerned. It really means a lot to us for him to know his roots and who he is," adds the singer.
Jesse Grant/Getty Kelly Rowland and son Titan
RELATED VIDEO: Kelly Rowland Enjoys "Role-Play" with Husband, Reveals Sexy Ways She Keeps Her Marriage Strong
Kelly Rowland Enjoys 'Role-Play' with Husband — Reveals Sexy Ways She Keeps Her Marriage Strong
PLUS: Kelly Rowland On How She Learned to Love Herself—and Going Semi-Nude in Her Steamy New Video
The Destiny's Child alum is aiming to give Titan a different experience than what she had growing up.
"I remember seeing the cover of a magazine say, 'This Is What Beauty Looks Like,' and I didn’t see any minorities on the cover at all," recalls Rowland. "We weren't represented at all and I remember that made me feel a certain way. It made me question my beauty. It made me question my contribution of beauty and who I was because I didn't see it."
"I remember the first time a fan said, 'I'm you when we play Destiny's Child because we look the same.' This girl was the same complexion as me and that made me so happy because there's nothing like feeling seen and being heard," she continues.
For Rowland, superstars like Janet Jackson and the late Whitney Houston made her feel represented early on "because their beauty was so taken in worldwide, it made me feel like there is a space for me."
Kelly Rowland Instagram Kelly Rowland and son Titan
As Titan continues to learn about his identity, he's also receiving plenty of love from his parents and Rowland's former Destiny's Child groupmates, Beyoncé and Michelle Williams. Years after going solo, the trio still share an unbreakable bond.
"We love each other. I'd do anything for my girls and I know that it's reciprocated so we're not going nowhere," Rowland tells PEOPLE. "By the way, Titan didn't know anything about Destiny's Child before Coachella. All he knew was that's Auntie Bebe and that's Auntie Meme — not joking, not a clue."
"We talk to each other about everything. They don't even have to say anything and I'll know if they're sad or if they're going through something," she adds. "We have this weird thing with each other ... like, Michelle knew when I was pregnant. We really, truly are like sisters. The greatest blessing we could have for sure is definitely Destiny's Child."
"We go over to each other's houses. Our children are cousins," she says with a laugh. "It's like any other family, honestly. The only thing that separates us is blood, that's it."
For more of Rowland's hard-won lessons about marriage, motherhood and her unbreakable bond with fellow Destiny's Child alums Beyoncé and Michelle Williams, pick up this week's issue of PEOPLE, on newsstands now.