I've been following the LLMs and I don't think it's 'running out of road', except in the sense that it's expensive. The model perplexity keeps decreasing when you scale up the parameters and exaflops and dataset sizes in the right proportions, and this decreasing perplexity is what has been unlocking so many amazing cognitive capabilities. I think the limitations are about regulatory deals not about any kind of technical roadblocks.