Good question and one I have thought about quite a bit. I think the answer is in two parts. The first part has to do with changes over the years in our microbiota. With rampant use of antibiotics and preservatives, there is no doubt the diversity of our gut microbes has been reduced. This change could make it more difficult to process huge amounts of fermentable and complex carbs as we may have once been able.
The second part has to do with how we count fermentable carbs. The strict definition focuses on dietary fiber in it’s various forms. But I think the definition should be expanded to include all carbohydrates that routinely escape digestion and absorption and are available for fermentation. Thus I created the FP formula to measure this. The number of fermentable carbs is also higher for people with various carbohydrate intolerances – lactose, fructose intolerance, etc. You can add fiber and resistant starch intolerance as well. In addition to fiber, other foods containing fructose, sucrose, lactose, resistant starch and sugar alcohols have many of the attributes of dietary fiber.
When you look at things in this way, we are not starving our gut microbes but more likely overfeeding them. The typical Western diet contains well over 100 grams of fermentable carbs per day. This is much higher than the 12 grams of dietary fiber per day cited by Sonnenburg and others. Over feeding our gut microbes can result in blooms of bacterial growth but a less diverse microbe population overall.