ChatGPT, your new Agile coach? We tested it for you! 

Can this innovative Artificial Intelligence, described as totally bluffing, be your future Agile coach? Katia Bradkte, consultant and agile coach has tested on a customer case: the answer of ChatGPT. 

First of all, what is ChatGPT?

ChatGPT is a conversational robot ("chat") capable of interacting in natural language with humans. It is equipped with a Generative Pretrained Transformer (GPT) tool, based on artificial neural networks. These networks are pre-trained on a large corpus of texts to learn to understand and generate language.  

This Artificial Intelligence (AI) progresses from documented knowledge, and by reinforcement from user feedback. This conversational robot also integrates a control module that moderates and tempers its responses. 

During a chat session with ChatGPT, you can send it requests on just about any documented topic. This can range from cooking recipes to script writing to generating computer code. In response, ChatGPT generates elaborate and contextualized content based on the data on which the tool has been trained. 

This response evokes fluid human writing. ChatGPT can integrate facts and knowledge. It can also generate structure, emulate an analytical approach, make jokes, write articles, narratives... And all this, in a user-defined style (all multilingual). 

The ChatGPT site displays a holding page because it reached its maximum capacity on January 18, 2023.
Test ChatGPT? The tool is a victim of its own success

How is ChatGPT's innovation impressive?  

The first incredible feat of ChatGPT is to understand the contexts and nuances of the language. The qualitative leap achieved by this version, compared to the initial version of GPT3, is to offer, thanks to reinforcement learning, an unparalleled level of alignment with human intentions.  

The second incredible feat of this tool is its performance in terms of writing. It incorporates relevance, apparent or real, and in all cases, clear and concise answers. Since the answers are very generic, this relevance is present on explicit or factual questions. On the other hand, the more complex the question, the more this relevance will be undermined. 

An old map of Alsace - Lorraine with points of interest: Strasbourg cathedral, the Vosges mountains, the Rhine, Nancy, Metz, etc.
To the question "Should Alsace leave the Grand Est?", ChatGPT cautiously concludes, "It is important to consider the arguments on all sides of this issue before making a decision."  

So can ChatGPT help address the challenges of enterprise agility? 

Full-scale test: ChatGPT, the new Agile coach?

Let's go to a concrete experience by asking ChatGPT about a customer problem: 

ChatGPT's response to a company's problem with its software development team
Here, ChatGPT's answer is right, the state of the art is very well rendered. As such, it is a bluffing rendering, but is it enough to meet the customer's need, and replace the human coach?

The Agile coach's answer to ChatGPT

This description of an organizational situation is quite typical of the difficulties of implementing agility. An agile coach detects implicit elements, gaps, even biases. The unspoken, often unintentional, can be excellent leads to key issues.

For example, here there is an unintentional, but glaring omission: the role of the Product Owner. From experience, even outside a Scrum framework, having a pilot in the plane is a key need. Even if it is often underestimated at the level of a product department.

A table shows agile notions used by an agile development team within a company: bugs, ROI, satisfaction, minimize rework, package, agile process, estimation; in the background, an agile coach is sitting at her desk
SmartView's agile coach and trainer Katia Bradtke during the Agile Product Manager training in November 2022 at Montpellier Business School

Let's continue the test: how far will ChatGPT's advice go?

Since the tool did not suspect the possible critical node, we prod it more explicitly:

ChatGPT answers the agile coach on the absence of Product Owner in the agile software development team
ChatGPT gives an excellent overview of the state of the art, rendered with finesse and simulating an understanding.

However, ChatGPT is relevant within its intrinsic limitations as a hyper-powerful contextualized statistical correlation tool.

It echoes the good words of the Count of Saxe*, which apply to both average consultants and professionals without genius: "Not knowing what to do, they do what they know how to do".

*(General Marshal of the Armies, 1757)

The "+ product" of the human agile coach

What a human coach can bring to the table will be, among other things:

- real understanding, because in the complex situations that agile coaches encounter, only deep understanding is real understanding.

For example, this tool is based on the existing body of knowledge and lacks the capacity to feel and analyze. It will therefore find its limits in solving new problems.

Agile coaching is precisely about meeting new situations and building customized solutions.

- the integration of organizational, cultural and human singularity. For example, a coach will know better than an AI how to take into account human factors. And in particular those that are under the radar, or weakly explicit (the authoritarian style of the leader, the devious strategy of an actor who plays a double game, the stifled but real tensions in the team, etc.).

- a direction based on a will which ChatGPT is devoid of (fortunately). If the tool lists the different good options and remains cautious, the coach will be able to choose and guide the decisions to develop a prioritized and adapted strategy.

And what does the principal say about it?

ChatGPT's answer: can he be an Agile coach?

ChatGPT itself answers the question: Can ChatGPT be an agile coach for a team
ChatGPT can't be an Agile coach, the AI algorithm tells us

The term Artificial Intelligence is misleading here: there is indeed a field of research on Artificial Intelligence. But the current tools are based on technologies such as Machine Learning and more specifically Deep Learning. This is NOT intelligence in the sense of understanding, analysis, reasoning, which requires consciousness.

Based on this principle, children find the right answer in the fill-in-the-blank texts thanks to their knowledge base, but without necessarily having understood. Similarly, ChatGPT is capable of complex productions, at a dizzying scale. The algorithm manipulates concepts that make sense to us. But without relying on the tool itself to understand the concepts.

Here again, the tool itself tells us clearly:

ChatGPT itself answers the question: Is ChatGPT an intelligence?
Our agile coach and trainer even debated philosophy with ChatGPT!

ChatGPT : soon your assistant ?

ChatGPT can already provide good summaries of the state of the art on agility, and in general, be an excellent assistant. This scholarly prowess takes the form of a true understanding. This similarity represents a risk of confusion, carried by the possible fantasy of our imaginations. It remains thus to trust human intelligence not to get carried away, and not to confuse knowledge and intelligence.

Indeed, if intelligence can be defined as the capacity to grasp meaning and to use it to reason or decide, then intelligence requires consciousness. But this tool, however sophisticated it may be, is currently devoid of it.

Our own intelligence will be of great help here, in order not to be misled. This slippage is indeed all the more tempting as the name AI and the quality of the ChatGPT rendering are confusing.

And you, do you dream of a good agile coach or a great "AI" for your organization?

A smiling woman holds an open bag. Inside the bag, the kit of an agile coach and trainer: post-it notes, kit, pens...
AI with or without post-it notes? Katia Bradtke shows us the secret of the agile coach...


"In theory, there is no difference between theory and practice. But in practice, there is." Yogi Berra


An article by



"In theory, there is no difference between theory and practice. But in practice, there is." Yogi Berra

Want to go further?