I'm not sure that there is more to it than continuous learning. If an LLM of top-tier strength could learn from its experiences even the way a junior developer could, I'm not sure I can place an upper bound on how capable it would be at software engineering. But from what I understand this will require a completely different architecture.