Witnesses said worshippers had spotted the policeman, Mohammed Pandit, at the mosque in plainclothes, taking photographs of people leaving shortly after midnight.
When they confronted him, a brawl broke out and Ayub fired his gun, wounding three people before the crowd began attacking him, the witnesses said.
Two people suspected of involvement in the attack have been arrested, Jammu and Kashmir Director General of Police S. P. Vaid told reporters.
The attack occurred while a separatist leader was speaking inside the mosque amid tight security on Laylat al Qadr, one of the holiest nights of the Muslim holy month of Ramadan.
The Himalayan region of Kashmir has been divided between India and Pakistan since 1947, and a separatist movement against Indian rule has raged for decades in the Indian part of it.
Hostility toward the Indian state, which Kashmiris accuse of widespread rights abuses, often boils over into violence.
Year 2016 has seen frequent clashes between security forces and civilian protesters.
Police said however that the beating to death of an officer was unusual.
Jammu and Kashmir Chief Minister Mehbooba Mufti said the policeman was at the mosque doing his duty.
“This was a shameless act,” Mufti said.