Trailing 1-0 to Andy Keogh's 14th minute efforts, Royals had work their work cut out to secure a sixth straight Championship victory against a battling Millwall side.
But Robson-Kanu, only in the starting line-up because of an injury sustained by Jimmy Kebe in training, gave his side the lift they needed when he lashed home a superb volley from David Forde's palmed clearance on the stroke of half-time.
Not only did the goal galavanise the visitors just before the break, it also halted the home side's growing momentum, giving Royals' the perfect platform to go on and win the match in the second period.
They did so thanks a controversial goal similar to that scored by Hull City at the Madejski back in January - the last time Brian McDermott's tasted defeat.
With Adam Le Fondre clearly offside from Mikele Leigertwood's pass, the Milwall defence paused for a split second, seemingly waiting for the linesman's flag to be raised.
But, with Jason Roberts running from an onside position, the whistle never arrived.
Instead, Roberts collected the pass and took a touch before drilling the ball across goal for Le Fondre to slide in and score at the back post.
The effort was just Royals' second shot on target in the entire match and, after a scrappy encounter, McDermott was understandably delighted with his side's ruthlessness.
"It might not have been the best looking game to watch but, for me, it was a fantastic performance on a tough pitch to play on," he said.
"We kept going and showed some guts. And that is the word I'd use - complete and utter guts.
"How many times have I said it? It's not about 11, or even 16 players; it's about every single person in that dressing room. I keep saying it and I mean it.
"They're all going in the same direction. I said to Alfie (Le Fondre) the other day 'keep going - you're training really, really well, and you're going to be important to us'. And that was proven on Saturday."
Aside from the enforced change on the left flank, McDermott was able to name an unchanged side from the one that won at Middlesbrough, as was his opposite number Kenny Jackett, who kept faith with the same starting line-up that were impressive 3-1 winners at Burnley.
And it was the hosts who looked the livelier team in the early exchanges, with ex-Reading midfielder James Henry firing a couple of long-range shots wide of Adam Federici's goal.
The Australian had not conceded for over seven hours of football prior to kick-off, but it took Millwall just 14 minutes to break through the Royals' rearguard.
A poor clearance from Kaspars Gorkss was neatly flicked on by Tottenham Hotspur loanee Harry Kane to Keogh, who calmly rolled a shot past the goalkeeper and into the net.
The hosts came close to doubling their lead moments later as first Josh Wright whipped in a cross that was just too high for Hamer Bouazza and then the Algerian forced Federici to palm out an effort from on the line.
Royals continued to look off the pace and they showed few signs of fighting back against a Millwall side battling for Championship survival.
Henry again went close with a free-kick that sailed past the post while Keogh almost broke through after Alex Pearce's sliced clearance.
Not until late in the half did the visitors eventually come to life and mount a series of attacks.
Ian Harte had their first real effort on goal with a free-kick that whistled over the bar before Jobi McAnuff threatened with a couple of probing crosses, the second of which produced a corner deep into injury-time.
Forde rose well to punch the delivery out amongst a crowd of players but could only clear to the edge of the box, where Robson-Kanu was perfectly placed to drill in a volley into the far corner.
The referee blew for half-time just moments after the restart, with the momentum suddenly swung in Reading's favour.
But the visitors were forced into a change at the break as the injured Matt Connolly was replaced by Jay Tabb, with Jem Karacan shifting to right-back.
And the second half started much the same as the first, with neither side able to play the ball around on a tricky pitch and both teams resorting to long balls to the front men.
Henry remained Millwall's biggest threat in midfield but he was guilty of going for goal from distance with promising options in support to his left and right.
Reading, although improved from the first period, struggled to keep hold of the ball, until they took the lead through Le Fondre in the 75th minute.
With the large travelling contingent chanting 'we're going up with West Ham', Royals began to frustrate Millwall, who were unable to break down a now dogged defence.
The home side's best chance fell to captain Paul Robinson, but he could not direct his header far enough either side of Federici to trouble the goalkeeper.
Reading (4-4-2): Federic - Connolly (Tabb 45), Pearce, Gorkss, Harte - McAnuff (c), Karacan, Leigertwood, Robson-Kanu - Hunt (Le Fondre 65), Roberts. Subs not used: Andersen, Cywka, Church. Booked: Pearce.
Millwall (4-4-2): Forde - Dunne, Robinson (c), Lowry, Craig - Henry ('83 Mason), Abdou, Wright, Bouazza ('71 Feeney) - Kane ('79 N'Guessan), Keogh. Subs not used: Allsop, Ward. Booked: Henry.
Attendance: 11,271 (1,489 Reading)
Referee: Chris Sarginson (Staffordshire).