I think it's still best to try to analyse the problem, find the source and then find the solution.
FC, I'm gonna strip you of your medals and light your tail on fire! This IS analyzing the problem.
JT, I think you were typing when mine posted 45 minutes ago - please try that. This is the location where YOUR choices are stored but there is another location where taskbar shortcuts can be stored: the defaults for all users. Your personal settings should override them but I believe maybe your person settings are not telling the system that you do not want Windows Explorer in the task bar. I would like to try to fix your settings (which I am hoping deleting desktop.ini will do) but if that doesn't work then it may be worth actually removing it from the default location.
I will turn on the light, stab FC in the rear, and rephrase, It's not a stab in the dark, I am looking for something...
If Clifford's idea of creating a new account allows you to unpin and it doesn't return then it is definitely your settings but the trick is to find them. desktop.ini as far as I know maintains an ordering but I don't know it can suppress entries, this may be a registry thing, but as I suggested it is worth a try. On the other hand, if FireCat's idea to boot in safe mode makes the icon go away under your ID then it would seem to be a program/application in your start-up.