פירוש על עבודה זרה 93:4

The particular issue arises if the tree was not planted for the sake of idolatry and then was worshipped. If it was planted for the sake of idolatry, it is forbidden even for secular use. Furthermore, it is only a question for those rabbis who hold that in such cases the tree is permitted for secular use. While the branches and fruit could be used for non-religious purposes can the lulav be used for the mitzvah on Sukkot?