Albion were back to full strength following Tuesday's golden goal exit from the LDV Vans Trophy for a makeshift line-up at Cambridge United.
Taylor named the team which took the Seagulls three points clear at the top of the Second Division with a impressive 2-0 victory at Bury last Saturday.
Paul Brooker made his first home start since October 30 in an adventurous 4-3-3 formation.
Pitcher's energetic performance at Cambridge earned him a place on the bench alongside Steele, at the expense of Dirk Lehmann and Steve Melton.
Rushden, beaten once in their last 13 outings in the Third Division, made a couple of changes to the side which saw off Bristol Rovers 3-1 at home last Saturday.
Mark Peters and Barry Hunter were back from one-match bans in the centre of defence.
Albion, knocked out at the second round stage in the previous two seasons under Micky Adams, were facing a team with a much better recent pedigree in the FA Cup.
Diamonds reached round three as a Conference club in 1998-99 and 1999-2000 before losing replays against Sheffield United and Leeds.
Albion: Michel Kuipers, Paul Watson, Kerry Mayo, Danny Cullip, Simon Morgan, Gary Hart, Charlie Oatway, Paul Brooker, Richard Carpenter, Nathan Jones, Bobby Zamora. Subs: Pitcher, Rogers, Steele, Virgo, Pethick.
Rushden and Diamonds: Billy Turley; Paul Underwood, Mark Peters, Garry Butterworth, Jon Brady, Scott Partridge, Paul Hall, Ritchie Hanlon, Andy Sambrook, Onadi Lowe, Barry Hunter. Subs: Burgess, Rodwell, Pennock, Setchell, Thomson.
Referee: Mr S. Bennett (Orpington).
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules hereComments are closed on this article