Annegret Kramp-Karrenbauer, known as AKK, urged critics to support her vision for Germany at the party's annual congress in Leipzig.
She has faced mounting criticism since replacing Mrs Merkel as CDU leader.
However, her threat to resign prompted a lengthy ovation and a pledge of loyalty from her main critic.
Poor election results and her low popularity have led to discontent within the CDU. But in a speech on Friday, Ms Kramp-Karrenbauer, 57, issued an ultimatum to her opponents.
"If you are of the opinion that the Germany I want is not the one you want... then let's end it. Here, now and today," Ms Kramp-Karrenbauer told delegates.
"But, dear friends, if you want this Germany, if you want to take this path together... then let's roll up our sleeves here and now and make a start."
At the end of the speech, delegates gave the party leader a standing ovation that lasted at least three minutes.
Mrs Merkel, who also gave a speech at the congress, struck a more conciliatory tone, calling for party unity.
As leader of the Christian Democratic Union - Germany's largest party - Ms Kramp-Karrenbauer is favourite to succeed Mrs Merkel when she stands down in 2021.
She was also given the role of defence minister during the summer, but shocked colleagues when she suggested sending German soldiers to northern Syria as part of a an international security zone.
Friedrich Merz, a millionaire lawyer who narrowly lost out to Ms Kramp-Karrenbauer in last year's leadership vote, recently described the CDU-led government as "abysmal".
He remains one of Ms Kramp-Karrenbauer's most vociferous critics but applauded her speech and promised loyalty. "We are loyal, to our chairwoman, and to the government," he said.
More about: Merkel