The controversy began in 2013 when an IRS official admitted the agency had been aggressively scrutinizing groups with names such as "Tea Party" and "Patriots." It later emerged that liberal groups had been targeted, too, although in smaller numbers.
The fact is conservative groups are not an incumbent president's opponent. Now if you said Mitt Romney, etc then you have a point. But it was both sides that were scrutinized more.
Link: https://www.npr.org/2017/10/27/560308997/irs-apologizes-for-aggressive-scrutiny-of-conservative-groups