Whisk the egg yolks, sugar, cornstarch, and salt together. Heat the oat milk until steaming, gradually whisk it into the yolks, and return everything to the saucepan.
Cook the pastry cream over medium heat, whisking continuously, until thick and bubbling. Cook for 1 more minute, stir in the plant butter and vanilla, then cover the surface and refrigerate until cold.
Preheat the oven to 425 F and line two baking sheets with parchment paper. Bring the water, plant butter, sugar, and salt to a full boil in a saucepan.
Add the flour all at once and stir vigorously over medium heat for 2 minutes, until the dough forms a smooth ball and leaves a light film on the pan. Transfer the dough to a mixing bowl and cool for 5 minutes.
Beat in the eggs one at a time, making sure each is fully incorporated before adding the next. The finished dough should be glossy and slowly fall from the beater in a thick ribbon.
Pipe 12 evenly sized mounds onto the prepared sheets. Bake for 15 minutes, reduce the temperature to 350 F without opening the oven, and bake for another 20 to 24 minutes until deeply golden and firm.
Turn off the oven, make a small opening in the bottom of each puff with a piping tip, and leave the puffs in the open oven for 10 minutes. Transfer them to a rack and cool completely.
Pipe the chilled vanilla pastry cream through the opening in each puff. Dust with powdered sugar and serve within 2 hours for the best crisp and creamy contrast.