Their shock win has set up a clash with defending champions and hosts Qatar, who later on Saturday (Sunday AEDT) beat Uzbekistan 3-2 on penalties after a last-eight encounter that ended 1-1 following extra time at Al Bayt Stadium.
Japan took the lead through Hidemasa Morita in the 28th minute after the midfielder went on a wonderful solo run then watched as Iran goalkeeper Alireza Beiranvand deflected his shot into his own net.
But three-time champions Iran, without top scorer Mehdi Taremi (three goals) as he served a suspension for a red card in the last-16 win over Syria, wouldn't lie down in Saturday's (Sunday AEDT) clash.
Iran equalised in the 55th minute when Sardar Azmoun received possession and brilliantly turned to slip through Mohammad Mohebi, who coolly finished first-time.
The winner arrived in the 96th minute after Japan failed to clear the ball defensively and Ko Itakura recklessly slid in on Hossein Kanaanizadegan, drawing an immediate penalty.
Alireza Jahanbakhsh buried the spot-kick into the top corner at the death, sending Education City Stadium into raptures and causing his teammates to burst into tears after he sealed a deserved win, Iran's first over Japan in 19 years.
"Iran should be proud of their sons, they gave everything they had for their people," coach Amir Ghalenoei, on a 16-game winning streak since taking the reins in March, told reporters.
"I think the result can be a turning point for Iranian football."
Despite being taken to penalties against Syria, Iran clearly finished the game stronger than Japan, who were heavily beaten in second-balls, outmuscled and unable to go with the world No.21's fierce counter-attacks.
World No.21 Iran will face either Qatar or Uzbekistan, who face off later on Saturday, in the semi-final, eyeing their first championship since 1976.
South Korea and Jordan play off in the other semi-final.
The defeat finished off a tumultuous campaign for four-time champions Japan and piles pressure on coach Hajime Moriyasu.
"I feel sorry for our fans and everyone involved in Japanese football," Moriyasu said.
"I regret I couldn't give the earthquake victims a happier moment. I feel responsible for the result."
In the other quarter-final, Qatar goalkeeper Meshaal Barsham proved the hosts' hero as he saved three penalties in the shootout before teammate Pedro Miguel converted the decisive spot kick.
Qatar had taken the lead shortly before the half-hour mark when captain Hassan Al Haydos dribbled into the box and his attempted cross took a deflection which went through the gloves of Utkir Yusupov into his own net to go down as an own goal.
Uzbekistan equalised in the 59th minute through Odiljon Hamrobekov.
Uzbekistan survived a late rally in extra time to force penalties.
Qatar had their hearts in their mouths when Al-Mahdi Ali Mukhtar skied his effort but Barsham turned saviour to seal their progress and give the hosts a memorable night in the desert.