Musar על קידושין 79:13

We must not question this exegesis by saying that the "path" the Torah speaks about some people forsaking is the adoption of idolatry instead of Judaism, because of a rule that the only time G–d considers the mere contemplation of a sin as already culpable is when the sin in question is idol worship. This is based on the expression (11,28) ללכת אחרי אלהים אחרים, "To prepare to go after other deities." Clearly, the Torah, by not writing "when you have followed other deities," warns you that we are culpable for the mere intention. In all other instances the Torah does not threaten punishment for the mere contemplation of a sin.