Single-child families and population decline

There's worry in certain circles right now about population decline in much of Europe. Families are becoming smaller as investment in individual children (supposedly) continues to rise. And if average family size is below, what is it, 2.1, then voila, it's below replacement level.

I wonder if there's another factor that's contributing, making the effect larger and more self-reinforcing. With very few exceptions, all the only-children I know are single in their adult lives, and all those who had at least one brother or sister are now partnered, again with very few exceptions.

If this is true, then as average family size decreases, logically, the number of single-child families increases as a proportion of the whole. If most of those children are not going to reproduce, then you have a runaway decline.