Church of Uganda bishops are insist they stand by their position from last week's workshop at Lweza Training Centre, demanding that Uganda quit the Organisation of Islamic Conference.
The bishops also asked the president to explain his strong ties with Libyan leader Col. Muamar Ghadafi.
Several bishops yesterday said they had no apologies to make for raising the issues and asking the relevant government bodies to explain.
Northern Uganda bishop, Rt. Rev. Nelson Onono, was instead surprised Museveni had reacted so angrily.
"The question still stands," Bishop Onono said.
"Uganda is a secular state. The bishops are concerned about how the country can be a member of an Islamic organisation," he added.
Museveni had responded to the bishops as he laid the completion stone for the National Mosque at Old Kampala, Sunday.
He criticised the bishops for questioning his ties with Ghadafi and demanding that Uganda leave the OIC.
Museveni described the bishops as Pharisees, who were no caring, while he saw Ghadafi the biblical "Good Samaritan."
Onono said the bishops would meet again to discuss the issues.
"Thank God we are meeting again soon. I am sure it will be one of the issues to be discussed since it has become contentious. There is no way Uganda tax payers who are not Muslims should be paying to sustain the OIC," he said.
The Organisation of Islamic Conference was established in 1969 in Rabat, Morocco to "consolidate and strengthen Islamic solidarity and to defend the ideals of mankind all springing from the immutable principles of the noble precepts of the Holy Koran."
"Safeguarding holy places and supporting the struggle of the Palestinian people" is also part of the OIC's original agenda.
Uganda joined it in 1974.
Church of Uganda officials say the issue shall also be raised at the next Uganda Joint Christian Council meeting.
Bishop Michael Ssenyimba of West Buganda however says he has no opinion on the issue.
Other bishops were shy to speak on record, claiming they could be victmised.
The bishops have now asked the Provincial Secretary, Dr. George Tibeesigwa, to write to the President and the Members of Parliament over their concerns.
One Bishop gave The Monitor a copy of a letter he claims was written by Malawi president Bakili Muluzi to Ghadafi in 1999 asking the Libyan leader to fund his campaigns.
Muluzi said he would use the money to "donate to mosques and silence the noise on the arms en-route to the DRC."
"In general, the islamisation evangelism process is going on well save for the financial handicap. For the first time in the history of my country, Arabic has been introduced in the University of Malawi," reads the letter allegedly written by Muluzi.
The Monitor could however not independently verify the authenticity of the letter.
But complained the bishop: "How sure are we that we might not see the same things like is happening in Malawi take place here? We must know that Uganda is predominantly Christian and therefore we expect wisdom to prevail when government is handling this issue."