How Does Coding Experience Improve Your Ability to Use GenAI Tools?

Can GenAI Really Help Non-Experts Perform Like Data Scientists?

Think GenAI is just for coding experts? Think again. New research from Boston Consulting Group reveals how AI-augmented employees are performing complex data tasks with 84% of the accuracy of data scientists. Learn why an “engineering mindset” is the new competitive advantage and how to structure your workforce for this shift.

Stop wondering if your team has the right skills for the future. Read the full summary to discover the five specific areas you must change in your talent strategy to successfully integrate AI-augmented workers and boost organizational performance.

Recommendation

GenAI tools are expanding the capabilities of workforces, helping organizations unlock value and boost productivity in novel ways — but it can be hard to discern whether GenAI is worth the hype. Researchers from Boston Consulting Group Henderson Institute have put the capabilities of GenAI augmented knowledge workers to the test, discovering that augmented knowledge workers sometimes performed tasks nearly as well as experts, such as data scientists, while delivering results faster. Learn how to leverage the potential of these augmented workers, while mitigating the risks associated with hiring generalists to collaborate with AI.

Take-Aways

• GenAI augmented knowledge workers are effectively completing tasks that used to require expert knowledge.
• Coding experience is associated with a more effective use of GenAI and an “engineering mindset.”
• Manage your AI transformation by enacting change in five key areas.

Summary

GenAI augmented knowledge workers are effectively completing tasks that used to require expert knowledge.

GenAI augmented knowledge workers are entering the talent pool, and they’re faster and more effective than their non-augmented counterparts. These workers are writing code faster than ever before and writing prompts that result in the rapid creation of personalized marketing content — but are they any good? According to a major field experiment led by the Boston Consulting Group Henderson Institute, which tested the effectiveness of this new kind of worker, “the answer is an unequivocal yes.” That said, these knowledge workers do have some limitations. For example, they may struggle to fact-check their work, as they may lack the expertise to know when they’ve gotten it wrong, but leaders who take steps to mitigate these risks can unlock value throughout their organization.

“We’ve now found that it’s possible for employees who didn’t have the full know-how to perform a particular task yesterday to use GenAI to complete the same task today.”

The research showed that augmented knowledge workers could often help organizations perform tasks that required hiring workers with more expertise in the past, such as those related to data science. In fact, participants were able to perform a task writing Python code to clean and merge two data sets 84% as effectively as a data scientist, finishing the task 10% faster. However, the participants, though not experts, did have some knowledge of data cleaning and many also had experience using no-code tools to clean data sets. When hiring GenAI augmented workers, it’s important to ensure they have sufficient knowledge of the tasks they’ll perform, so they can supervise output — otherwise they won’t be able to catch obvious mistakes.

Coding experience is associated with a more effective use of GenAI and an “engineering mindset.”

When completing a predictive analytics task that neither the human workers or the GenAI tool they were using were highly skilled at, the technology still benefited the workers, as it served as a “powerful brainstorming partner,” allowing the workers to combine their knowledge base with that of the AI tool. Participants were instructed to create predictive models and then assess the reliability of these models in making investment decisions. When collaborating with AI, workers were more likely to choose and apply the appropriate machine learning methods to execute tasks than their non-augmented counterparts in the control group.

“We found that coding experience is a key success factor for workers who use GenAI — even for tasks that don’t involve coding.”

One noteworthy factor associated with improved performance of augmented workers was moderate coding experience: Even when workers collaborating with AI were performing tasks that didn’t involve coding, they still outperformed those with no coding experience, which could be due in part to the benefits of having an “engineering mindset.” Study participants with an engineering mindset likely benefited from superior problem-solving skills, as they would have had some understanding of how to break down problems into component parts and tackle each effectively.

Manage your AI transformation by enacting change in five key areas.

To manage your transition, leaders should focus their efforts on the following areas:

1. Talent acquisition and internal mobility — Companies can now recruit from a wider pool of knowledge workers as GenAI expands workers’ capabilities and makes previous experience performing certain tasks less important (though, of course, some roles — such as data science — still require expertise).
2. Learning and development — Create incentives that inspire employees to learn. When employees develop technical skills (even those who are nontechnical) research shows that their performance rises, so create opportunities for learning, for example by using personalized GenAI training tools.
3. Teaming and performance management — Consider leveraging the power of cross-functional teams, giving generalists easy access to specialists who can support them in their endeavors. Run pilot tests to discern whether your teaming configurations are driving your desired outcomes, while breaking any organizational silos that might be blocking success.
4. Strategic workforce planning — In an age of continuous learning and technological change, consider rethinking your approach to how you structure your workforce. Focus more, for example, on recruiting employees with your desired behavioral skills, working to create a more flexible workforce of employees capable of pivoting and adapting.
5. Professional identity — Help employees feel confident in their augmented roles by showing them support as they learn new AI skills. Research shows that when workers believe their company is deploying AI in ways that benefit them, they have a more optimistic perception of AI.

About the Authors

Daniel Sack, Lisa Krayer, Emma Wiles, Mohamed Abbadi, Urvi Awasthi, Ryan Kennedy, Cristián Arnolds, and François Candelon are professionals at Boston Consulting Group.