The AI Consultant — Context Is Key To Unlocking The Benefits of Generative AI Models (like ChatGPT)
Get better answers from AI models by setting the right context
As ChatGPT has exploded in popularity over the last few weeks I’ve been exploring the concept of “The AI Consultant” which is centered around the idea that tools like ChatGPT could eventually replace human consultants across a range of industries. This is fundamentally an intriguing concept to me in the sense that it challenges the status quo of knowledge work being perceived as more protected from the rise in technology innovation than say the classic example of self-driving technologies replacing truck drivers or drones displacing last-mile delivery workers (which may well be my own biases showing). The eventual reality will most likely be a middle ground where these new tools make consultancies more efficient — a reduction in the human capital leverage factor. I’ve nonetheless set out to explore the current possibilities and limits of this technology to test whether it could indeed replace a human consultant.
The Experiment
Naturally I felt compelled to challenge ChatGPT in an area where I have domain knowledge — IT strategy and cloud computing. I wanted to litmus test the technology in the arena that I compete in. The first challenge / experiment was a simple prompt where I presented ChatGPT with a high level question one may ponder while developing a cloud strategy perhaps to craft a cloud business case, “Provide 10 reasons for investing in the adoption of public cloud”. I purposefully prompted ChatGPT without setting any prior context to see what it would generically provide.
ChatGPT’s Response Without Context
The following interaction with ChatGPT frames a question around reasons for and justification of adopting public cloud at a large financial institution without any prior context setting.
The results are not groundbreaking; however, ChatGPT is able to provide a solid list of 10 reasons in the format that was requested (rank order). I would never copy / paste this result directly, but it’s definitely a starting point and I could see how it may accelerate ideation sessions or initial narrative crafting during the early phases of development. I still wondered though, would setting context actually improve ChatGPT’s response?
How impactful is context?
The ability for a consultant in real life to understand context is critical for success especially in situations where ideas require nuance and the customer expects material that’s been fine-tuned per their own value set. Simple things like knowing what the motivating factors are, the level of technical knowledge of the audience, or the style that has historically resonated in the past can be the difference between a solution gaining traction or falling flat. Part of this ability is being able to “ingest” context clues — for a human this is a skill honed through experience and may happen naturally without concious thought, but for an AI model these context clues must be fed to the model deliberately. Given this understanding, I decided to feed ChatGPT a coordinated set of context setting details prior to asking the same prompt I raised earlier. Would additional context improve the response like it would improve a human consultant’s response?
ChatGPT’s Response With Context
The following interaction with ChatGPT frames a question around reasons for and justification of adopting public cloud at a large financial institution with deliberate context setting.
While both answers were impressive and hit on almost all of the common reasons cited for public cloud adoption the response with context was much more aligned to what you’d expect a financial institution to be concerned about. For example, security and compliance are foundational to any regulated industry whereas scalability and flexibility may be slightly lower on the list of importance. Additionally, the tone of the responses with context set were more professional and read cleaner than the ones without context. I even found it interesting that with the context ChatGPT formatted the score using an out of one hundred structure whereas without context it just listed the score — a minor improvement that it implemented only with the context provided.
Although these tools do not currently guarantee factually accurate information one can start to see how feeding in prior context can influence the responses and may be a good skill to learn for interacting with these tools in the future. Even OpenAI itself has stressed the importance of context in their FAQ (source) and noted that the current ChatGPT model can store and reference approximately 3000 words of information (source) which could consistent of context setting information or previous responses.
More about the Approach
Putting the context setting approach into practice isn’t too dissimilar to how you’d mentally walk through any new problem space. The dimensions of important context to set are described below:
- Explain how the model should act e.g., “I’d like you to act as a management consultant”
- Describe the role and the relevant audience members e.g., “I am the head of public cloud platforms and I will be presenting to the CTO and CIO”
- Stress the style of response e.g., “You should look to include specific examples, bullet points, and provide citations for further reading if you think they will be relevant.”
- Ask the model to weight specific sources e.g., “In your responses you should weight information more heavily sourced from reputable management consulting and technology consulting companies, publications, and reports. Examples include research organizations and publications such as the Harvard Business Review, the MIT Sloan Management Review, the McKinsey Global Institute, the BCG Henderson Institute, Garter, Forrester, and IDC.”
- Ask the model to weight specific domain areas e.g., “you should weight academic research and information to be a more well rounded consultant in areas such as systems thinking, sociotechnical systems, organization change, software development, computer science, computer engineering, AI, and similar areas”
In the example above, I asked ChatGPT to act as a technology management consultant. I provided context in specific areas such as my role as the head of public cloud platforms at a large financial institution, the audience’s technical knowledge, and the style of response desired. I also emphasized the importance of considering information from reputable management consulting and technology consulting companies, industry reports, and academic research. By providing this context, the model was able to provide more relevant and meaningful answers to my initial prompt. This could potentially be further supercharged if these prompts were delivered to a custom model trained or tuned on an industry specific model or a consultancies own priors.
The Conclusion
As demonstrated in the experiment, providing context in areas such as the role, audience, style of response, and sources of information can lead to more relevant, professional, and nuanced answers. By taking the time to carefully set context when interacting with these models, you can increase the probability that the answers provided are more in line with your needs and requirements and will require less re-tooling during the ideation phase of any process. The skill of effectively setting context and the ability to obtain more relevant answers quickly may become increasingly important as more organizations adopt these technologies, and those who understand how to effectively use these models will be better positioned to succeed in the evolving technological landscape.
The broader impact of ChatGPT on industries like IT consulting remains to be seen. Do I think they will replace humans? Thankfully — not anytime soon, but there is indeed an opportunity for a multitude of use cases. Whether it’s custom models trained on a consultancies own knowledge bank opened up via paid APIs to potential customers to support a self-service delivery model, a digital assistant behind the scenes to make consultants on the ground more efficient, or an AI-driven onboarding process to reduce the time that a normal discovery process of document ingestion and stakeholder interviews takes, it is clear these new generative AI models will be a forcing function for innovation in the industry.
Addendum
Shoutout to Coté for the inspiration to write this post. His original take on the subject can be found on his post here.
The views expressed in this post are my own and do not necessarily reflect the official policies, positions, or views of the global EY organization or its member firms.
