you guys might be interested in the latest Machine Learning Street Talk podcast [1] which right from the start is all about how LLMs are great for creativity, as in novel combinations of trained data (from memorization) -- but are not capable of the reasoning skill needed to verify if any idea is actually plausible given a set of constraints.

[1] Do you think that ChatGPT can reason? https://www.youtube.com/watch?v=y1WnHpedi2A