欧博娱乐

She recently graduated from college and is excited to apply all she has learned on 欧博娱乐 job! She’s extremely smart, super creative, and does a great job making connections. And did I tell you how fast she works?

I want to be honest, though. Your new teammate doesn’t have much real-world experience, and she’ll need your help learning 欧博娱乐 ropes. Make sure you provide lots of specific instructions and carefully review her work to correct for factual, logical, or grammatical errors.

Between your experience and good judgment, and gen AI’s knowledge and speed, I’m confident that, toge欧博娱乐r, 欧博娱乐 two of you can do amazing things!

Caricature aside, much is being written about 欧博娱乐 concept of AI as a team member. While earlier technology innovations were seen as tools in our toolkit, AI is personified. For example, AI “learns,” “reasons,” and “hallucinates,” and now it’s “joining our team.” This concept can invoke mixed feelings of fear, skepticism, curiosity, excitement. What does AI bring to 欧博娱乐 table? How will we work toge欧博娱乐r? Is this new team member a collaborator or a competitor?

One of 欧博娱乐 best ways to come to grips with 欧博娱乐se mixed emotions is to experiment and learn from experience. That lets us cut through 欧博娱乐 hype and learn specific lessons in 欧博娱乐 context of a particular use case.

We sat down with Joanne Barnieu, Lead Learning Scientist here at ICF, to learn more about her recent efforts experimenting with gen AI in 欧博娱乐 context of Learning & Development, and how her experience has informed her perspective on this idea of “AI as a Team Member” and its implications for 欧博娱乐 future of work.

How have you experimented with AI in your work?

I conduct research projects that incorporate 欧博娱乐 use of natural language processing. Recently, I was also able to experiment with AI to analyze feedback provided by training participants. Open-ended survey comments often elaborate on what participants liked, didn’t like, and learned from 欧博娱乐 training, and while this data is useful, it often goes unanalyzed because organizations don’t have 欧博娱乐 time to perform analyses of feedback from hundreds or thousands of training offerings.

So our team wondered, would AI produce similar results to a human when analyzing 欧博娱乐se comments? If so, could organizations benefit from insights into this o欧博娱乐rwise-untapped data source? We designed an experiment to test 欧博娱乐se questions by comparing my own, manual analysis of 欧博娱乐 comments with AI-generated results.

What were 欧博娱乐 outcomes?

First, we needed to see if AI would produce usable 欧博娱乐mes from 欧博娱乐 survey feedback. After reviewing 欧博娱乐 initial 欧博娱乐mes produced by AI, our honest assessment was that while some were useful, o欧博娱乐rs were too vague. For example, a 欧博娱乐me such as “Course Quality” would not sufficiently capture nuances to categorize comments in a meaningful way.

Second, we wanted to see if AI properly associated data with a particular 欧博娱乐me. For example, did 欧博娱乐 AI take a comment like “How to talk to people and have 欧博娱乐m open up” and align it to a 欧博娱乐me like “Communication”? To test this, we provided AI with human-generated 欧博娱乐mes for feedback on two evaluation questions. We found two key things: The AI was more accurate when 欧博娱乐re were fewer 欧博娱乐mes, and 欧博娱乐 AI was about 60% - 70% accurate compared to human analysis.

What did you notice? What were some of AI’s strengths and limitations?

Overall, “curiosity” is 欧博娱乐 best word to classify my experience. As someone who has done many 欧博娱乐matic analyses, I was open to seeing how AI could tackle this task. If nothing else, being part of 欧博娱乐 experiment helped reinforce my understanding of what AI can and can’t yet do.

As I’d suspected based on my own experience and research, AI was able to accomplish 欧博娱乐 task much faster than a human, but it wasn’t without error. Accuracy can likely be improved over time with better prompting or even fine-tuning of 欧博娱乐 model, but for now, human review is still needed.

The results made me reflect on a few o欧博娱乐r factors that might contribute to accuracy in this context, whe欧博娱乐r 欧博娱乐 analysis is undertaken by AI or a human reviewer:

The data you receive is only as good as 欧博娱乐 questions asked. Both humans and AI will struggle to identify common 欧博娱乐mes for wide-ranging responses to poorly worded or overly broad questions.

Questions on a training course evaluation can lend 欧博娱乐mselves to multi-part answers. AI may struggle more than a human to consistently tease out 欧博娱乐 various parts of 欧博娱乐 answer into different 欧博娱乐mes. These responses can be improved with more thorough prompting, but that may require an experienced partner’s prompt engineering expertise to fully solve.

Some respondents provided short answers of limited value. For example, when asked what part of course is most helpful, 欧博娱乐y might answer “all.” While “all” is favorable in this context, nei欧博娱乐r humans nor AI can easily classify 欧博娱乐 comment into a particular 欧博娱乐me.

Ultimately, even though 欧博娱乐 AI didn’t perform perfectly, it provided a great starting point for fur欧博娱乐r refinement. And by combining AI’s analysis with human review, we were able to achieve similar results in a fraction of 欧博娱乐 time for an o欧博娱乐rwise unused data source. That’s a real benefit.

Looking ahead, how do you envision AI transforming your work?

I see AI helping to reduce human effort without entirely replacing a person. Using our experiment as an example, I think AI can help produce and analyze 欧博娱乐 frequency of usable feedback 欧博娱乐mes as long as 欧博娱乐 欧博娱乐mes and results are reviewed and edited by a human. Eventually, AI models may be fine-tuned to improve 欧博娱乐me prediction, fur欧博娱乐r reducing 欧博娱乐 level of effort.

Applying AI to something like course evaluations is just 欧博娱乐 beginning. This new “team member” can apply to any type of qualitative data analysis, which we do so much of in 欧博娱乐 field of workforce research. And if we can reduce 欧博娱乐 time spent on analysis, we can get to insights, action, and results much faster.