Japan took the lead when Socceroos keeper Mark Schwarzer appeared to be impeded as Shunsuke Nakamura's intended cross sailed over his head.
But Cahill drilled in a loose ball from close range on 84 minutes and lashed home from 18 yards soon after.
John Aloisi then sealed a first World Cup finals victory for Australia with a well-placed shot in injury time.
Japan's Yoshikatsu Kawaguchi was the first goalkeeper called into action, forced into a double save from Mark Viduka.
The Middlesbrough striker broke clear to first fire in a right-foot shot from a narrow angle, and then a left-foot follow-up effort which was parried for a corner.
Australia fell behind in circumstances which were at best soft, and at worst controversial.
Nakamura drifted in a hopeful ball and Schwarzer seemed to be impeded by Atsushi Yanagisawa, but Egyptian referee Essam Abdel Fatah waved away Australian protests.
Kewell almost found a quick response with a curling shot which grazed the top of the bar.
Viduka was proving a handful for the Japanese defence and a foul on the Middlesbrough striker set up a chance for Marco Bresciano, who almost embarrassed Kawaguchi as he went for goal with the keeper expecting a cross.
Aloisi scored Australia's third goal in injury time
Australia coach Guus Hiddink threw on Cahill and Josh Kennedy, and the big striker's height gave the Japanese defence a different set of problems as the Socceroos took a more direct route.
Kennedy was fouled on the edge of the box by Teruyuki Moniwa to give Viduka the chance to power in a free-kick, drawing a great save from Kawaguchi.
But the keeper was at fault as Cahill grabbed a lifeline for Australia.
Kawaguchi failed to claim Lucas Neill's long throw and Cahill ended a scramble in emphatic fashion, firing home Australia's first World Cup finals goal from close range.
The Everton midfielder was at the heart of the action and escaped conceding a penalty when he clearly clipped Yuichi Komano as he headed for goal.
But five minutes after levelling, Cahill found space on the edge of the box to put his side ahead with a thumping shot off the inside of the post.
Australia were fired up and they iced the cake when another substitute, Aloisi, finished coolly after bursting through the middle of the Japanese defence.
Australia: Schwarzer, Neill, Moore (Kennedy 61), Culina, Wilkshire (Aloisi 75), Emerton, Grella, Bresciano (Cahill 52), Chipperfield, Kewell, Viduka. Subs Not Used: Beauchamp, Covic, Kalac, Lazaridis, Milligan, Popovic, Skoko, Sterjovski, Thompson.
Booked: Grella, Moore, Cahill, Aloisi.
Goals: Cahill 84, 89, Aloisi 90.
Japan: Kawaguchi, Komano, Miyamoto, Santos, Tsuboi (Moniwa 56), Nakazawa, Fukunishi, Hidetoshi Nakata, Nakamura, Takahara, Yanagisawa (Ono 79), Moniwa (Oguro 90). Subs Not Used: Doi, Endo, Inamoto, Kaji, Maki, Koji Nakata, Narazaki, Ogasawara, Tamada.
Booked: Miyamoto, Takahara, Moniwa.
Goals: Nakamura 26.
Fifa man of the match: Tim Cahill.