The men were shot in the back of the head as they walked home from Saturday afternoon prayers in Islamic dress, according to police.
No motive for the killings has yet been established by investigators, but worshippers and community members in Queens said the killings were hate crimes.
More than 100 people joined a protest on Saturday night outside the mosque where they chanted: “We want Justice.”
Mosque leader, Malama Akonjee, 55, a married father of seven from Bangladesh, was pronounced dead at the scene, one block away from the mosque. Thura Uddin, 65, also from Bangaldesh, was rushed to Jamaica Hospital but later died.
“I don’t think it’s a coincidence that it is the two main officials of the mosque, it just doesn’t connect,” said Mahfauza Hague, a friend of the imam.
Muslims performing the daily prayers A leader of another mosque in the area, Kobir Chowdhury, said religious intolerance was to blame for the attack. “Read my lips: this is a hate crime. We are peace loving”, he said at a news conference.
But police on Sunday morning had not yet determined the motive for the attack. “There’s nothing in the preliminary investigation to indicate that they were targeted because of their faith,” Deputy Inspector Henry Sautner, from the New York Police Department, said.
Police were still hunting on Sunday for the man who shot the pair dead shortly before 2pm the previous day. Witnesses described a tall Hispanic gunman, who was wearing a dark blue shirt and shorts.
The imam’s daughter, Naima Akonjee, said her father “didn’t have any problems with anyone”.
Ms Akonjee said her father and Mr Uddin were close friends who lived on the same street and always walked together to the mosque.
“These were two very beloved people,” Afaf Nasher, of the New York chapter of the Council on American-Islamic Relations, said. “These were community leaders. There is a deep sense of mourning and an overwhelming cry for justice to be served.”
City Hall issued a statement: “While it is too early to tell what led to these murders, it is certain that the NYPD will stop at nothing to ensure justice is served.”
A report by Georgetown University in May found that hate violence has spiked in the US, in accordance with the US Presidential election.
After the Mr Trump called for shutting down mosques in the wake of the Paris terrorist attacks and the San Bernardino mass killings, anti-Muslim crimes initially tripled. Almost half the attacks targeted mosques.