Austrian Chancellor Sebastian Kurz called it a "repulsive terror attack" and said one gunman was also killed.
A large area has been cordoned off as police search for the other gunmen.
The city's mayor said 15 people had been wounded, seven of them seriously. Among the injured is a police officer, according to reports.
The shootings took place near the central synagogue but it is not yet clear if that was the target.
The attack happened just hours before Austria imposed new national restrictions to try to stem rising cases of coronavirus. Many people were enjoying bars and restaurants which must close until the end of November.
European leaders strongly condemned the shooting. British Prime Minister Boris Johnson said he was "deeply shocked by the terrible attacks".
Police said the incident began near the Seitenstettengasse synagogue, which is the main temple in Vienna.
Jewish community leader Oskar Deutsch tweeted that the synagogue was closed at the time the attack began at 20:00 (19:00 GMT).
An officer guarding the synagogue was among the wounded, newspaper Kronen Zeitung reported.
Footage posted on social media showed people running through the streets as gunshots rang out.
Witness Chris Zhao was in a nearby restaurant when the shooting started.
He told the BBC: "We heard noises that sounded like firecrackers. We heard about 20 to 30 and we thought that to be actually gunfire. We saw the ambulances... lining up. There were victims. Sadly, we also saw a body lying down the street next to us."
As a major anti-terror operation swung into action, police urged people to avoid the area and not to use public transport. Roadblocks were set up around the city centre.
Police in the neighbouring Czech Republic said they had launched random checks on the border with Austria amid fears that the gunmen might head in that direction.
On Twitter, Mr Kurz said "we are experiencing difficult hours in our republic".
"Our police will act decisively against the perpetrators of this hideous terrorist attack. We will never allow ourselves to be intimidated by terrorism," he said.
European leaders condemned the attack, with French President Emmanuel Macron saying that Europe must not "give up" in the face of attacks.
"We the French people share the shock and grief of the Austrian people, struck this evening by an attack in the heart of their capital, Vienna. After France, a friend of ours is attacked. This is our Europe. Our enemies must know who they are dealing with," he said.