פירוש על ברכות 72:4

The caper-bud falls from the bush when ripe; hence it cannot be regarded as a protection so that it should be subject to 'Orlah.