פירוש על ברכות 72:5

With the fruit to make up the minimum bulk which can contract impurity.

And the blossom bears the same relationship to the pomegranate that the caper-bud does to the berry.

If, then, the pomegranate-blossom is subject to 'Orlah, so should the caper-bud be.