A deputy speaker of the British House of Commons, Nigel Evans, has described allegations of rape and sexual assault made against him by two men as "completely false."

In a statement delivered outside his constituency home in Pendleton, Lancashire, Evans, an openly gay Conservative MP, described his "sense of incredulity at these events," and said he did not understand why the claims have been made.

Evans, 55, who is out on bail until June 19, said: "Yesterday I was interviewed by police concerning two complaints - one of which dates back four years - made by two people who are well known to each other and who, until yesterday, I regarded as friends.

"The complaints are completely false and I cannot understand why they have been made, especially as I have continued to socialize with one as recently as last week."

He thanked police for their sensitive handling of the matter, and friends, colleagues and members of the public for their expressions of support.

Later Sunday, Evans was excused from his ceremonial duties at the Queen's Speech on Wednesday. He asked to be excused from chairing the event, but insisted he would not resign from his position.

A number of MPs have expressed their support for Evans, but Defense Secretary Phillip Hammond said that it would be "difficult" for him to continue in his role.

The offenses are alleged to have taken place against two men in their 20s in his home constituency between July 2009 and March 2013.

Evans, 55, was arrested early Saturday morning at his Pendleton home. The house and his car were searched.

In a statement, Lancashire Police said: "We take all allegations of a sexual nature extremely seriously and understand how difficult it can be for victims to have the confidence to come forward.

"As a constabulary, we are committed to investigating sexual offences sensitively but robustly recognizing the impact that these types of crimes have on victims."

Evans came out as gay in 2010 after the launch of a parliamentary group set up to help MPs and staff to be open about their sexuality.

Prime Minister David Cameron and Speaker John Bercow have been informed of the arrest.

Swansea-born Evans has served as MP for Ribble Valley in Lancashire for more than two decades since first being elected in 1992.

He has held a number of senior positions in the Conservative Party. Between 1999 and 2001 he served as vice-chairman of the party and was appointed shadow Welsh secretary under then-Tory leader Iain Duncan Smith in 2001.

Three years ago, he was appointed one of three deputy speakers of the House of Commons.