פירוש על עבודה זרה 83:21

An Asherah is a tree that is worshipped. One may not derive any benefit from such a tree. R. Yose says that one may not plant under such a tree, because that would be taking advantage of the falling leaves as fertilizer. But again, why not consider the leaves to be like an idol that breaks on its own. After all, that is essentially what happened to the tree.
In this case we cannot consider the leaves to be annulled because the tree is still standing and nothing has happened to it. Unlike an idol that breaks, no one would say that the tree has broken just because some leaves fell off.