From what I've read, the overall population of the earth is actually going down, even in Africa, Asia, and Latin America. Many countries do not have sustainable reproduction rates, with some like Japan and much of Europe trying to figure out how on earth they are going to care for their elderly. Abortion has something to do with this, even in the US, since 1/3 of babies are killed before birth, but even in places where this is discouraged or outlawed, there are still fewer babies born than there were a few decades ago. Even in the most fertile areas, numbers per family have generally gone down from 8 or more to 5 or 6, and seem on a further downward trajectory. Few places in the US, Europe, or Oriental countries currently have replacement numbers, unless there are many immigrants, and even then, they are barely holding even.
I do think there have been significant increases in nutrition and agriculture apart from GMOs and other problems to sustain what we currently have reasonably well. I also think there are serious problems that come about when a population decreases dramatically. (For an example, look at the political and social upheaval that resulted from the Black Plague across most of the world in the 1300s--1/3 to 1/2 of the population died, more in some areas, and it led to the destruction of the feudal system.) OTOH, I do think for each person or family, it's worth thinking about what environmental consequences our decisions will have in a few generations, as well. I don't think there are easy answers for these things, but it seems to me that a knee-jerk reaction in either direction is likely to not work well, in the end.