If you backtrack a bit at any point, you might be able to catch a glimpse of Flowey popping into the ground, signaling that he was following you.

For example, after encountering Papyrus and Sans when Papyrus was going to use the Gauntlet, if you walk back (to the left) across the bridge, Flowey will be there, then pop into the ground once on screen.