Keeping that big of a secret is hinging on a deal breaker, if it ain't one. That's just me, though. I mean if we're starting off keeping secrets, then we don't have a good foundation on which to build anything worthwhile.
I understand how it is, though. You decide not to tell because you don't even know if the relationship is going anywhere. By the time you realize it may be going somewhere you feel you've waited to long, and every day it just gets harder and harder to reveal.
From his POV, though, it looks like you may have had other motives for hiding it. (This "you" is the hypothetical one, just in case someone misreads that) I mean he may get to thinking that you had other reasons for not telling him, like you were trying to run some kind of game and pin the child on him. Of course, after a few months that ploy is no longer feasible, lol.