Federer had a match point at 5-4 in the third set, but struck a backhand into the net before the eighth-seeded Anderson broke in the following game and held serve.
The last time Federer played a Wimbledon match away from Centre Court it had no negative impact as he trounced Gilles Simon at the same stage of the 2015 tournament before beating home favourite Andy Murray to reach the final where he was undone by a dominant Novak Djokovic.
This was Federer's earliest departure from Wimbledon since his shock second round defeat against Sergiy Stakhovsky in 2013.
Anderson, a 2017 U.S. Open runner-up, is now looking good going into the semifinal against either Canada's Milos Raonic or U.S. competitor John Isner.
It was his first win against Federer at the fifth attempt and means he is the first player representing South Africa to reach the semi-final here since Kevin Curren in 1983. That's the mindset you need against someone like Roger.
He said: "I just asked the chair umpire how did I deserve that warning?"
But Anderson, playing in his first Wimbledon quarter-final, finally made the 20-time Grand Slam champion look human.
"Maybe you should ask the questions over in Russian Federation, how they're going to feel about Wimbledon being played at the same time", Federer was quoted as saying by AFP.
Anderson had failed to take a single set off Federer in their previous four meetings.
France vs Belgium predictions, stats and betting tips
The game will feature plenty of talented attackers and has the potential to be one of the most exciting games of the competition. France and Belgium are set to square off Tuesday with the victor reaching the 2018 World Cup final game.
But this time Federer cracked, serving his first double fault at 11-11 to give Anderson the crucial break that ushered the Swiss legend to the exit door.
Federer, aiming for a record-extending ninth Wimbledon title, settled for a tie-break which he took 7-5 - his 300th set overall at the Championships.
The last time he won as many consecutive sets was in 2005 and 2006 at the height of his dominance.
Novak Djokovic of Serbia returns the ball to Karen Khachanov of Russian Federation during their men's singles match on the seventh day at the Wimbledon Tennis Championships in London, Monday July 9, 2018.
Federer is a ideal 12-0 in Sets thus far and a flawless 4-0 in Matches at Wimbledon 2018.
Tennis legend Roger Federer has been hailed for a flawless batting technique after he showed off his cricket skills at Wimbledon.
Djokovic has had a hard two years but is beginning to look and feel a lot more like his old self again.
Novak Djokovic advanced to the last four at a Grand Slam for the first time since 2016 with a 6-3, 3-6, 6-2, 6-2 Wimbledon quarterfinal victory over Kei Nishikori on Wednesday. When I'm doing that, I think I'm a very risky player.