Ina Garten reveals why Oprah Winfrey hit her twice after a speech

Ina Garten sheds light on a “shocking” moment she shared with Oprah Winfrey that she still carries with her.

Radio Cherry Bomb Host Kerry Diamond asked Garten, 76, about the moment – which she reveals in her upcoming memoir. Be ready when luck happens – in the latest episode of the podcast.

“It was very nice,” he said Barefoot Contessa Stern added before diving into the exchange starting in 2010.

This year, Garten received one of New York's Women In Communications' Matrix Awards, an annual honor the organization honors “exceptional women in advertising, books, broadcasting, film, magazines, new media, newspapers and public relations.” ” according to its website.

While accepting the honor alongside Sheryl Crow, Tina Fey and others, Garten said she sat on stage next to Winfrey, 70, who was there to introduce another honoree: longtime friend Gayle King.

Garten and her fellow honorees were each allotted a 90-second time slot for their acceptance speech, which, Garten said, “is pretty much all I can handle without causing a total meltdown.”

“And I basically said I've been really lucky in my life and sat down,” the chef explained. “And [Winfrey] slapped me on the arm in front of about 2,000 people and said, 'You make your own luck.'”

Garten wasn't convinced and told Diamond that she had aligned herself with Winfrey, emphasizing that she owed much of her career to luck.

“I said, 'Well, I've actually been incredibly lucky.' And then she hit me again,” Garten recalled with a laugh. “And I thought, 'Okay'.”

More than a decade after Winfrey trained her, Garten has seen her point. “Many years later, when I was writing this memoir, I thought, 'She was kind of right.' “I had the job done,” Garten said.

“I have to give her credit,” she added. “She was right.”

Garten reflected so much on her happiness that it inspired the title of her memoir: Be ready when luck happens.

“I always felt like I was really lucky, and when I looked back and put the pieces together when I was writing my memoir, I realized that I had actually done a lot of work to be ready,” she told PEOPLE for hers September cover story.

Never miss a story again — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from celebrity news to compelling human interest stories.

Garten's story about Winfrey is just one of many untold moments the chef reveals in the book – which she admittedly never thought she'd even write.

“I just didn’t think anyone would find my life so interesting,” she told PEOPLE.

But after some persuasion from a friend and co-worker, she took the plunge and it ended up being “really fun,” Garten said. “I wasn’t scared at all, but the thought of sending it out into the world is a whole different story.”

Be ready when luck happens is available now.