If you uninstall the app you used to change the appearance of the apps, it should revert back to normal. I just tested it using x Icon Changer and also an icon pack.
Thats the obvious thing to do, but it doesnt really work like you say it would.
I've uninstalled the icon pack app but they were still there, even after a system reboot. However, every day or so i see one or more icons reverting back to normal (stock) appearance, as if its a disease slowly being cured. Weird af i know.
Its been 10 days since i've posted this question and from ~20 icons getting the makeup treatment by that app, its now down to 4 icons with that icon pack theme. My theory is that every few days, a few random apps are getting an update in the background which pushes a new (stock) icon into the homescreen.
This is the icon pack app btw (cant post full links apparently, just put dots instead of spaces) -
play google com/store/apps/details?id=ddt.destroyer.icon.pack