פירוש על ברכות 71:18

And for that reason may not be eaten when in a state of 'Orlah.

Hence the buds are regarded as fruit.