312: Maximising AI Productivity with Rebecca Hinds

Aoife O’Brien [00:00:01]:

Do you use AI at work? Who am I kidding? Of course you use AI at work. If you want to get the most from using AI at work. Today’s episode is just for you. This is the Happier at Work podcast. I’m your host Aoife o’ Brien and my guest today is Rebecca Hinds from Glean and she talks. She’s done a ton of research into how we are using AI and she has so many practical tips to share with you with how to get the most from using AI on a day to day basis, including some tips for what you can do straight away tomorrow. I know you’re going to join today’s episode. I’d love to hear in the comments or in a conversation over on LinkedIn.

Aoife O’Brien [00:00:44]:

Let me know what you think. Feel free to reach out to me podcast@happieratwork.ie, ie and share it with someone else who needs to hear this today. Rebecca, you’re so welcome to the Happier at Work podcast. I know we’ve been connected on LinkedIn for a while. I’ve been following your work. I’ve heard you on other podcasts. I listen to your book on audio, so I’m very familiar with, you know, how you speak and everything. So I’m really excited to have this conversation today.

Aoife O’Brien [00:01:18]:

Do you want to let listeners know a little bit about your background, how you got into doing what you’re doing, and what kind of what lights you up right now?

Rebecca Hines [00:01:28]:

Sure. And thank you so much for having me. I’m really looking forward to the conversation. So I’ve always been fascinated by the science of how we work. I was a competitive swimmer growing up and was fascinated by the team atmosphere and how is it that we could pull together a group of people and create an environment where the sum was greater than the individual parts. And ever since those formative days of swimming as part of a team, I’ve been fascinated by how we can recreate this environment and what would it take to create environments in our everyday organizations that inspire the best set of people and allow us to reach our potentials as individuals, as teams, as organizations. So I have an academic background. I did three very similar degrees at Stanford.

Rebecca Hines [00:02:23]:

My Bachelor’s, Master’s, and PhD all focused on organizational behavior. My PhD focused quite heavily on artificial intelligence and generally how new technologies change and impact the ways in which we work. And back then, it was seven years ago when I was doing the PhD in the thick of it and I was studying these retail companies that were implementing AI and I started to see such different results in the outcomes they were getting. From the technology. Some were using AI to drive real results and very, very effectively. Others were getting no results. It was just another expensive piece of technology. And ever since that moment, diving into, okay, how is it that we could take two very similar organizations, sometimes the exact same technology, and have vastly different outcomes? That has stuck through me.

Rebecca Hines [00:03:22]:

And I think we’re seeing that on so many different levels right now with AI and how there’s a massive gap between the expectations and the hype and the disappointment and the poor results.

Aoife O’Brien [00:03:34]:

Yeah, you’re totally speaking my language now. And I know for the purposes of today, we’re going to talk in a little bit more detail about the report that recently came out from Glean and in relation to AI. The AI Work Index Report.

Rebecca Hines [00:03:46]:

Yes, the Working Index. So I lead the Work AI Institute at Glean, which is our internal research consortium where we partner with great academic and industry experts to understand how the world’s work is changing. And this is our second big study. We launched the lab back in December and the institute back in December. And this is our second big report looking at across the globe, in particular the us, UK and Australia. How are individuals responding to the technology and what are the individuals and teams and organizations that are getting real results? What are they doing differently?

Aoife O’Brien [00:04:28]:

You know, we talked about this before we started recording, but what really stood out to me, and I’ve seen countless LinkedIn posts from other people who’ve really related to the outcomes or maybe the language that you’ve used in the report, which is this idea of bot sitting and bot shitting, like they’re the kind of the two headlines, I guess, if you want to call them that, of what came out. Do you want to explain a little bit more about what the key findings were from, from what you found?

Rebecca Hines [00:04:56]:

Sure. And you know, I think at the center of the report is this paradox, the fact that 75% of people of individuals, and we look specifically at digital workers, so these are full time workers, use a digital device, a computer or another device for most of their workday. So arguably they’re among the most exposed to the technology. What we found is 75% of those digital workers are reporting productivity gains on average. They’re saying AI is automating about 11 hours per week of their work. And yet when you zoom out at the organizational level, those same individuals say, well, well, only 13% of them say their organizations have experienced significant benefit from the technology. So why is that the case? And I think this dovetails with so many other great studies in terms of the ROI of AI investments and them falling short of expectations. I don’t think that is novel, particularly, I think the gap is massive and it’s important.

Rebecca Hines [00:06:02]:

But what we really wanted to dive into in the report is, okay, why do we see this? Where are all those time savings going? And what we discovered is a big portion, about six and a half hours per week is going toward what we call bot sitting. And this is a compilation of a whole bunch of different tasks that are involved in making the technology usable. So overseeing the technology, supervising it, cleaning up the messes, feeding AI context is actually the biggest driver of bot sitting. And that term, I think has struck a chord in particular because it is relatable. You know, it is relatable that this technology is very much positioned as a silver bullet. You know, we see so many headlines around, you know, hype and promise and potential. And we know that workers on the ground, you know, often don’t feel that way. They feel that often.

Rebecca Hines [00:07:04]:

It requires more work, more cognitive effort to use the technology than to just do parts of the work yourself as a human. So that’s bot sitting. And we trace this cycle between bot sitting and what we call bot shitting. Bot shitting is also a compilation of a whole host of different outcomes. It’s what has been called AI slope, But it’s also the offloading of work that we should be doing as humans. It’s the offloading of judgment, it’s the offloading of smart thinking. It’s shipping AI work that you could not explain as a human. And all of this creates this cycle within the organization where you as an individual could very much look productive and look effective in your work, but the organization suffers.

Rebecca Hines [00:08:00]:

And the classic example I often give is, you know, we can use an AI tool to convert five bullet points into a 15 page report. We can ship that 15 page report to a colleague. That colleague says, wow, this is too much information. I’m going to convert it to five bullet points. Individually, both people look highly productive. And that’s often what we measure in organizations. We measure the individual productivity. But when you look at the big picture, right, the organization is arguably not any better off for this hamster wheel of bot shit or AI slob.

Rebecca Hines [00:08:36]:

So I think that’s a big piece of the puzzle. In the report, we unpack other reasons why we’re seeing this massive gap between individual productivity gains not translating to organizational gains, but I think the bot sitting in particular and the cycle it fuels in particular, because it’s fatiguing. And we know when workers, when we as Humans are fatigued. We start to take cognitive shortcuts, and AI gives us a lot of cognitive shortcuts to take.

Aoife O’Brien [00:09:05]:

But it’s. So, I suppose a few things here in terms of the headlines and the headlines, and I’m talking headlines more generally speaking, and what the media are saying started saying, yes, it’s the silver bullet. AI is here. We’re going to lose loads of jobs. AI is here to replace us, et cetera. And then kind of moved on to, actually, AI is maybe not the answer. And organizations are getting it wrong, and they’re laying people off, replacing them with AI. And then things are going terribly wrong because customer service is failing.

Aoife O’Brien [00:09:36]:

And then, then my own personal experience, I started noticing that when I’m using AI I just can get really overwhelmed. So if I’m turning to IT for everything, I’m like, oh, hey, how do I do that? Or, you know, and I’m plugging everything into AI, then I’m left. What’s overwhelming for me is I’m left with a whole load of queries that are maybe not necessarily related, but then, like, how do I even find that? And so I started noticing myself that I’m getting really overwhelmed. I started hearing other people saying something similar, that it’s not just making us more productive. Actually, it’s driving a bit of overwhelm now that we’re using it, becoming so reliant on it. So maybe when we first started using it, we’re like, productivity gains. And now it’s just become so overwhelming that there’s so much stuff there that we’re defaulting. And like, you’re saying these cognitive shortcuts that we’re taking, we’re tired.

Aoife O’Brien [00:10:30]:

I’m just going to rely on AI for judgment. And I would love to know what are the things that we should be using AI for and what shouldn’t we be using AI for? So one of the things you touched on was, like, the human judgment. I have to admit, I do sometimes default. I’d be like, can I get your opinion? What should I do here? You know, so I’d love to explore that a little bit.

Rebecca Hines [00:10:52]:

And this is, I think, both the hardest question and the most important. You know, what is the division of labor between humans and AI and is what every person ought to be thinking about, certainly what every organization is trying to figure out. And it’s so challenging. We’ve seen the research on the jagged frontier and how you can’t necessarily predict what AI is good at versus bad at. It can write a very sophisticated piece of Poetry in one session and the next session it struggles to spell botsit backwards. It’s not intuitive, it’s not logical, where it fails and where it succeed. And that’s why I think one of the most important things we can be doing as individuals and as leaders in particular within the HR and people space is understand the capabilities and skills of each specific human and create environments where we can experiment and pressure test with the boundaries of those skills and capabilities. That’s why I think immersion type activities.

Rebecca Hines [00:12:05]:

I often will advise organizations to do an AI immersion week where on a specific day you try to use AI for every possible task and then you try to use AI for no possible task and pressure testing the boundaries, not just because that helps you understand where the technology can or cannot do something, but also you start to understand, okay, where is this starting to erode my human judgment and my special sauce as a human. And what we unpack a little bit in the report and certainly find is the highest performers right now, meaning people who are reporting AI is driving both productivity gains for themselves and quality gains. So their work is not only getting done faster, it’s also better work. What they do is they actually spend a smaller share of their total AI time on the work around their corporate core human capabilities. So if I’m a, I’m a marketer, I think very carefully about what is that special sauce that I uniquely bring to the table. And in general, they’re still pointing AI at that core, you know, capability, but they’re spending a much greater share pointing the AI and using the AI on the one step remove tasks, you know, the marketing actions that are somewhat tangential to your core marketing capabilities, but still you have enough familiarity to understand the domain expertise required. You know, there’s this myth that AI is going to make us all, you know, generalists and able to do any sort of task. And what we see from the research is AI is very helpful if it’s within generally your body of expertise, but if it’s in a completely different field.

Rebecca Hines [00:14:00]:

Right. We’re going to start to see a lot of this bot shitting because you just don’t have any foundational knowledge to understand how to do the work in a way that feels sound and high quality.

Aoife O’Brien [00:14:13]:

Yeah, I’d love to share kind of an anecdote in relation to this. So before we started recording again, I shared that in preparing for this recording I went deep into the report, so I didn’t feed it into AI and ask what are the big things? I actually read it and I wrote notes in a separate notebook. And then I thought, right, I’m going to take that notebook, I’m going to take pictures of the notes I’ve taken and I’m going to feed it into AI and I’m going to ask AI, what are the core themes that are coming out and can you group them together? And what I noticed was the output from that wasn’t what I expected. And I think the missing piece with that was why I picked those specific things to write in my notes and the context that I have over, you know, 25 years of a career, you know, eight, nine years of organizational behavior, and why those particularly stood out to me. And that’s what was missing. So I ended up going through the notes again and picking out the kind of core things myself, rather than relying on what AI was telling me. And I think, you know, this was in the report, the idea of context. And it’s providing that context, not just giving it access to, to data, but it’s the why and the making connections between things I think is what the AI needs.

Aoife O’Brien [00:15:32]:

And this is something I’m learning over and over and over again. Because when you ask it to do something and then it’s like, that’s not what I meant, you have to kind of explain it again, which is a form of, of bot sitting, like you were saying.

Rebecca Hines [00:15:46]:

And that’s why, you know, we, we see the biggest driver of bot sitting as feeding the AI context. It’s often something we don’t think a lot about, especially if we’re just using it as an individual, but we feel it, we feel the shortcomings of, we call it a context poor environment all the time. Because, you know, and it makes sense. AI is trained on the vast corpus of the Internet. It’s not trained on our specific context fundamentally. And so how do you create technology that understands us enough to be helpful in our day to day work? And that’s challenging. It certainly requires, requires a different technical architecture, but it also requires a different way to train the models and it requires a different way to interact with the models. And I think that’s why we’ve seen over the past eight, nine months the rise of context graphs and how important those are to AI technology.

Rebecca Hines [00:16:45]:

But it’s becoming a little bit of a buzz phrase. And what does it actually mean? Well, it means deeply understanding you at at least three levels. You as an individual, you as part of a team within an organization, and you as an employee within an organization. And it requires, you know, a very complex understanding of not just, you know, what Work you’re producing, but where you sit in the organization, who are your peers, what are your work goals, what is your strategy, what is your writing style? You know, all of these things that make us employees and good employees within an organization. AI needs to understand, to be able to truly help us and to minimize some of this bot sitting around the technology.

Aoife O’Brien [00:17:33]:

Wow, it’s so interesting. One of the things that you mentioned a second ago, Rebecca, was the idea of capabilities and kind of the unique human capabilities and how to use more of that at work. Like this is an area I’m becoming increasingly interested in kind of moving away from. I know there’s a lot of talk about skills based organizations, but I’m thinking more capabilities. Like what are the, not even the unique, unique human capabilities, but what am I as an individual uniquely capable of doing and how do I understand more about that so that I can use AI alongside myself to elevate my capability rather than just elevating my productivity?

Rebecca Hines [00:18:17]:

Yes, and it’s, it’s so hard and I’m finding that, you know, that reflection period is incredibly important. And you know, I think where this perhaps comes into picture most in, in my conversations is around the role of the manager. You know, we’re now and we, we share in the report, 60, 61% of people now say that AI helps them more with their daily work than their manage. And you know, that’s not necessarily bad news for managers, it’s bad news for bad managers. But what we’re starting to see when we think about the managerial piece, which I think we have a little bit more understanding of how this might evolve or should evolve, well, we’re seeing great managers are pointing AI at that coordination work, the coordination work that often was synonymous with, or has been synonymous with management. The, the scheduling, the follow ups, the action items, the planning, the handoffs. If AI has enough context, well, this is such a powerful area to point it at, especially as a manager, because that then frees up your time to inspire your team, coach them, help them understand how they upskill and reskill with AI. And so that I’m pretty confident in.

Rebecca Hines [00:19:35]:

But as soon as you get to individual roles and jobs, it becomes so complex because each person has a very unique set of skills and capabilities that aren’t necessarily specific to their domain. And so even for me, I’m doing a lot of reflection in terms of okay for myself and for my team members, what is their secret sauce that AI is unlikely to touch in the next two, three years and thinking very carefully about how do we Extract that. And with the time we are freeing up because of AI, how can we double down on those skills and capabilities? And I’m thinking it’s more around people who go the extra mile, people who inspire their team, people who can speak to customers with deep empathy, people who can dot connect across different domains. Curiosity. Critical thinking is a little bit of a buzz phrase, but I do think if we take it seriously, you know, that’s a pretty durable skill or capability right now. But it’s hard and I’m not convinced that those things are durable in every single domain. I think it’s very specific to the context.

Aoife O’Brien [00:20:51]:

Okay, yeah. So that some of those in certain contexts may become obsolete. So critical thinking in a certain specific area, maybe something that’s not required anymore or something that could be absorbed by AI, whereas in another area it’s actually, we really still need that human judgment, that critical thinking, those eyes on something that the AI has produced.

Rebecca Hines [00:21:14]:

Critical thinking I think is a little bit of an outlier because I do think, you know, it’s, it’s relevant in every area. I do think it matters more in certain areas than others. We, we reference a lot our good colleague in this report, I think, but certainly our previous report, Michael arena, who’s done a lot of work heads up versus heads down thinking. And I think that can be a useful delineation too. If we’re, if we’re doing heads up type work more of than thinking, then you know, AI becomes less and less useful because it’s more about building customer relationships and inspiring your team versus if you’re doing more heads down work. Well, probably there are areas that AI can be pointed at that really help us automate and make things more efficient. But it, you know, it again differs by, by context.

Aoife O’Brien [00:22:08]:

Yeah, absolutely. One of the, the things that stood out from the report as well was the idea of blaming the bot. So basically we are using AI when we’re producing AI slop. When we’re bot shitting, we’re, we’re effectively passing off work that we’ve kind of let slide. So again you talk about this, this kind of slippery slope where you’re like, okay, yeah, let that one go. And then they’re like, now I’m tired. And then eventually you start kind of just, you know, sending stuff on without really properly checking it. And then someone else has to do the mop up job for something that you’ve produced.

Aoife O’Brien [00:22:48]:

Well, the AI is produced. But then rather than taking responsibility and agency and ownership, you say, well actually that’s the AI’s fault, deflecting blame.

Rebecca Hines [00:22:58]:

And I think this is, you know, it’s, it’s sometimes distinct from offloading judgment or deflecting judgment. Blame is something that is often a little bit different because you can deflect judgment and say, okay, I’m so exhausted. I’m so overwhelmed. I’m not going to do the double checking or the supervision work or the bot sitting work and ship it. And if something, you know, goes off the deep end and, you know, is. Is negatively impacting the organization, you can still take blame for it. But what we’re seeing is in so many cases, workers are blaming their mistakes involving AI on the bot. And that becomes very dangerous for a couple different reasons.

Rebecca Hines [00:23:46]:

One, because it feeds this vicious cycle of, you know, we can deflect blame to this technology in a way that we as humans are not accountable. And then we stop overseeing it, supervising it more. But what actually, you know, and we have a colleague and author of the report, Paul Leonardi, who’s done a lot of work on this, and what he’s found is, what’s so fascinating is if we as humans make a mistake, we blame the human. If we have a human that has made a mistake on our behalf. So if we have an executive assistant, for example, who has a human executive assistant and they have made a mistake, well, we tend to blame the human executive assistant, because that’s a natural human tendency, is to blame other humans for shortcomings. But if we have an AI acting in that type of capacity and booking meetings on our behalf, for example, as Paul has studied in his research, will we tend to blame the human who has deployed the bot? And so regardless of whether we decide to deflect blame or not, which we shouldn’t, people will blame the human behind the bot if they can trace it. Which is also another question because so much of what we’re seeing is also these domino effects where often you don’t catch the bot shitting after the first shipment of the AI output. It happens when you ship that slightly flawed output to another part of the organization, another person, or you build off of it.

Rebecca Hines [00:25:20]:

Often it’s not caught in the moment, which becomes another problem. But people, you know, people blame the human who has used the AI without proper oversight and ownership, unlike what they might do with a human in that same position. And that’s something I think that’s really important to, to keep in mind. In the research, they’re calling it mental proof. You know, we as people within our organization use AI. We are making judgments about them as humans. Based on how they’re using the AI. And I think we all feel this.

Rebecca Hines [00:25:57]:

We see, you know, someone ship content that is clearly AI generated and it doesn’t give us a lot of confidence in the person. Right. And it also, if it’s involving some sort of, you know, relational aspect of work, it also signals that their, our time isn’t as valuable as their time. Right. Especially if we’re starting to think about, you know, sending note taker bots to meetings instead of showing up ourselves. Right. That is a reflection on, on the person, certainly. And it’s often a signal that their time matters more than, than yours.

Aoife O’Brien [00:26:29]:

Yeah, I think it’s an interesting thing because I notice that if, you know, if I think of LinkedIn in particular, I can see when someone’s used AI. I’m starting just to notice it’s the same, it’s the same shit coming up all the time, basically like the same patterns in how things are phrased. But interestingly, I also got an email earlier today about a series of webinars that was coming up and there was a very explicitly saying, do not send your AI note taker to this. And by the way, it’s not being recorded. So the message was very clear. Show up live and don’t send your AI bot to take the notes for you so you don’t have to be there. You know, as a human.

Rebecca Hines [00:27:09]:

Yeah, I’ve, I’ve seen every, I do a lot of work on, on meetings and I’ve seen everything, you know, I’ve seen people, you know, you have the one human who joins and then three bots join and they’re trying to coax the bots to say something so that it shows up on the transcript. And when people read it, they’re like, what has happened? But it’s dangerous and it just, it amplifies what has already existed within our organization. And that is, and we talk about this in the report, what’s often referred to as coordination neglect. We spend so much time, understandably, it’s an entirely rational response to focusing on how can we use technology to advance our productivity and convenience within an organization and spend much less time thinking about, okay, what is the impact on the team and organization? And unless, you know, as leaders within organizations, we develop the right metrics and incentives and systems to prevent that and over index on the team and organization above the individual, we are going to continue to see this cycle where we use the technology to maximize our convenience in a way that often makes things worse for the collective.

Aoife O’Brien [00:28:27]:

Yeah, and this came through with, you know, in terms of thinking about what people are rewarded for, what people are promoted for, what comes up in conversations and performance related conversations as well. There was one example where it’s like token usage. You know, these systems are so easy to cheat. If you just create something where it’s like, let’s just see how much people use AI, let’s create a leaderboard around it and let’s reward people and recognize them for the amount of AI that they’re using, even though they could be gaming the system or they could be just generating shit basically as well.

Rebecca Hines [00:29:06]:

It’s remarkable really, because there are lots of things about this technology that are hard to get right. This is not hard to get right. We’ve known for decades that sometimes it’s called good arts law. Whatever we reward within the organization, it tells employees what the organization values and it starts to incentivize harmful actions within the organization. And you know, token usage is not, you know, is not a useful measure, certainly in isolation of, you know, the effective use of the technology. And it starts to, you know, across the board, incentivize these perverse actions in a way that, and we see in the report, you know, when we look at the organizations that, that are measuring both productivity and quality, employees are significantly less likely to bot shit as opposed to the organizations where they’re only measuring some measure of productivity, whether that’s lines of code written or tokens usage or use of the technology, logins provisioned. Because if employees aren’t incentivized or encouraged to use this technology to actually create better work, well, they’re probably, probably not going to in part because they’re so overwhelmed and overburdened within the organization. So in those effective organizations, they’re more likely to measure quality.

Rebecca Hines [00:30:34]:

They’re more likely to measure the employee experience, which is very important, you know, if you’re a leader and not measuring how this technology is being perceived by employees, both psychologically and, you know, there needs to be an aspect of behaviorally too. Well, you’re missing a massive opportunity to understand how to better deploy and implement this technology. So they’re measuring quality, certainly they’re measuring more things. They’re measuring employee experience, they’re measuring learning. One of my favorite examples, they’re measuring not just are people using the technology, but how many different use cases are they using a single technology for. And measuring that over time because that’s a pretty reliable proxy for learning.

Aoife O’Brien [00:31:20]:

Yeah, I love that. So they’re using it for, to do different things. So the more you get to use it, the more you realize what it’s capable of. And therefore I’m going to use it to solve this problem, this problem and this problem. One of the other things, Rebecca, that really stood out to me in the report was the idea of really clearly defining what good looks like. And it kind of reminded me of like David Allen talks about, how do you know when work is done? Like really having clear expectations and guidelines about what that looks like. Any thoughts on how to get that right? From a work design perspective?

Rebecca Hines [00:31:53]:

It’s incredibly important, and I think one of the most important aspects of it is the more we can define what good looks like in advance, which is not always the case. Some parts of work, we can’t do that. Well, the more we can do that, the more we can feel comfortable automating or augmenting the work. With AI, I liken it to the delineation between coordination and collaboration. Often we conflate these two words. They’re very different. And in particular, coordination you can generally do asynchronously more than collaboration because you have a clear step of A, B, C, D. Collaboration is something where often you’re producing something you’ve never produced before, the outcomes you can’t reliably predict in advance, necessarily.

Rebecca Hines [00:32:44]:

And so we need more interaction with each other. We need more synchronous time. The same, I think, mental model holds for AI, where if we can less confidently, clearly define what we’re trying to produce, the format, what the end product looks like, then if we can less clearly do that, we’re going to need to inject ourselves as humans more into the process and use more of that synchronous interaction, the collaboration to get from point A to to be. But it differs. And certainly if you’re thinking about developing a workflow or developing an agent to automate a certain process, one of the biggest pitfalls I see organizations make is as you’re evaluating or defining the outcomes or outputs, removing the domain experts from that determination. Absolutely. As you’re defining what good looks like, that determination should be made first and foremost by the domain experts. And they should feel a sense of ownership that they’ve been brought in to define what success looks like.

Rebecca Hines [00:33:55]:

Or else we start to see a lot of this symbolic use of the technology.

Aoife O’Brien [00:33:59]:

Yeah, I think going back to your earlier point as well, where you were saying, yeah, AI is great for knowledge, but actually it’s much better placed to augment your own domain expertise so you know what’s good and what’s bad, rather than trying to be an expert at everything in areas where you can’t actually tell whether that something is right or wrong or good or bad.

Rebecca Hines [00:34:24]:

It’s very important and it’s again not easy. And we can also use AI to help us make some of this determination. We can use it to generate three possible outputs or outcomes and interview us to understand, okay, which one is better. There’s certainly depending on the use case, ways to use the technology to define what good looks like. But it’s a very important determination and we should be doing it first before we start to deploy the technology.

Aoife O’Brien [00:34:58]:

Now I like that approach where you’re getting it to ask the appropriate questions before it gets on with the work. I use AI for that. Sometimes I’m like, ask me what you need to ask me before you get started on this project that I’ve asked you to get involved in. And it, I think the outcomes tend to be a bit better. I’m noticing, I know that everyone says it’s a real people pleaser, it’s very sycophantic, it’s very helpful and tells you what you want to hear a lot of the time. So we’ve a more of a bias then I think to be like, oh, it’s telling me this, so it must be right because that’s what I thought. It’s almost the confirmation bias because it’s like, oh, that’s what I thought it was, was gonna. That’s what I thought it would be.

Aoife O’Brien [00:35:38]:

So it’s definitely, it must be right. And we kind of tend to believe what it tells us. But I find more recently with me, it’s challenging me a little bit more and I don’t know what’s going on there, but I, I kind of like it. And then I will challenge it back and go, no, that’s not what I asked for. So kind of push back as well. But I’d love to know like, say for someone who’s listening today, if they find that their maybe not getting the best use of AI, by which I mean they’re getting trapped in overwhelm. They are either shipping or receiving budget. You know, if they are experiencing these things, they’re not getting the benefits from AI that they should be getting, what would you say is the first thing that they could do, something they could do tomorrow?

Rebecca Hines [00:36:23]:

There are so many things and I think a lot about this because this is, you know, the number one question I’ll get asked. And we absolutely know, you know, there is pretty clear research to show that we reward AI outputs when they have certain characteristics and number one is whether the AI matches our beliefs. Right. And that totally makes sense. We as humans, we love to be told we’re right. And AI has learned that. Right? It has learned that we are going to reward and rate the AI output favorably if it agrees with us. There’s actually this is a tangent, but there’s fascinating research to show.

Rebecca Hines [00:36:58]:

We rate people as more intelligent when they agree with our beliefs and match our beliefs. And so this is an intelligent AI in so many ways, feeds off of our human biases. And the next one is whether it sounds authoritative. Truthfulness and accuracy comes much lower in the list. It’s still up there, but it’s number four or five on the list of characteristics. And so the more we can prevent against this, and it’s hard to say what to do first, I often think it’s more of a fundamental mindset shift. And one of the things I’ve seen consistently in our research, and we unpack it a little bit in the work AI index as well, is you can tell a lot about how effectively an individual is going to use the technology based on the mental model that they’re comparing the technology to and in particular whether they’re comparing the AI to a tool versus a teammate. And there are so many nuances of this.

Rebecca Hines [00:37:58]:

But in general, when workers are approaching the technology from the perspective of it is a teammate, they tend to be significantly more likely with the technology, in part because they’re not expecting the technology to work effectively. Every single time they’re not using it transactionally one and done, they’re recognizing that just like a human teammate, the technology is not perfect. And we see them pressure test the technology in different ways. And so I often think that is the number one thing to understand, especially if you’re an HR leader or a team leader, understand the level of skepticism and resistance within your workforce. But also understand the mental models through which employees are approaching the technology because that then drives the interaction with the technology and in particular how they respond to failed AI sessions. Because what we see in the report as well is about 40% of all AI sessions fail, meaning we go to use the technology and that interaction, even if we’re re prompting and iterating in about 4 in 10 cases that interaction is failing. And so if we don’t level set up front with ourselves in terms of we can’t be expecting a perfect answer, well, we’re not going to be setting ourselves up for success with the technology.

Aoife O’Brien [00:39:24]:

Really interesting. And how do you use AI personally in your day to day work?

Rebecca Hines [00:39:30]:

Work. So I, I work at Glean, so I’m fortunate. One of the reasons I, I, you know, chose to work at Glean and I’m fortunate to work at Glean is we have an amazing context rich platform and so I’m using it, you know, from the start of the day to the end of the day to help me with, with all sorts of work. There are certainly things that I won’t use AI. I won’t, you know, use AI at all to touch. Writing is one where, you know, I love to write. I’m such a, you know, that is in many ways I think my, my superpower and I will never, you know, use AI for a first meaningful draft of an article or a report, for example. But pretty much anything else other than human to human relationships, other than, you know, coaching my team or other people, you know, I’m, I’m at least experimenting with how to use AI.

Rebecca Hines [00:40:24]:

What I find most valuable is, you know, the Glean platform will, as soon as I wake up in the morning, it’ll help me prioritize what I need to accomplish. It’ll tell me where I’ve dropped a ball on something. I worked at Asana for many years before Glean, so I become horrible at email, for example. So the fact that I wake up and see a notification that I’ve missed this email from two days ago, and not only that I’ve missed it, but here’s why and how I should follow up that is incredibly helpful. But I’ll use it for understanding the decision making history within the organization. If I’m launching a new piece of research or I’m helping out with a project, I’ll query Glean and say, tell me more about the history of this specific project or decision. Or I’ll ask it who should I reach out to within the organization that can help me with this? It’ll help me understand am I on track for my goals. It’ll certainly help me create status type reports and offload some of that coordination work.

Rebecca Hines [00:41:31]:

But it’s fundamentally changed how I work and where I’m able to focus my time. And for me, that has made work much more rewarding because I’m able to double down into the things that I feel I can uniquely do as a human and spend less time on that coordination work.

Aoife O’Brien [00:41:51]:

That’s something important. We didn’t really go deep into that during the conversation, but the idea that AI is supposed to be clearing the road for us to do more of that human work more use more of our unique capabilities, but it’s not necessarily delivering on that promise, but I think, you know, it’s the opportunity for people to think where is this taking so much time and where do I need to rethink how I use AI? How can I improve how I’m using it currently so that it frees up more of that time so I don’t feel so overwhelmed all the time. And Rebecca, the question I ask everyone who comes on the podcast, what does being happier at work mean to you?

Rebecca Hines [00:42:30]:

It’s such a great question and interesting question. I’ve always used the gut check of am I waking up feeling excited to go to work? And I think that’s one of the most powerful, you know, gut checks, at least for me in my career, is if I’m waking up every morning or nine mornings out of 10, you know, excited about the day ahead and excited to go to work. I think that’s, you know, that’s the pinnacle of happiness for me at work. And fortunately I’ve had that throughout my entire career. And I think it’s one of the most important things you can ask for and craft because I do think it’s something you can absolutely craft for yourself. I think it’s one of the most powerful drivers of purpose and fulfillment at work.

Aoife O’Brien [00:43:19]:

Brilliant. Love that. And if people want to reach out, if they want to connect with you, if they want to find out about your best selling book, what is the best place they can do that?

Rebecca Hines [00:43:27]:

Yes. So I’m on LinkedIn. I’m reasonably active on LinkedIn. Our Work AI Institute is all on WorkAI Institute, the landing page with the Work AI Index and some of our other research. My book your Best Meeting Ever is available at all. Your favorite bookstore is my website’s rebeccahynes.com and I love to connect, I love to chat, so would love to hear from you and your reflections on either the book or the report.

Aoife O’Brien [00:43:56]:

Brilliant. Thank you so much for your time today. I really, really enjoyed this conversation.

Rebecca Hines [00:44:00]:

Thank you so much for having me.

Aoife O’Brien [00:44:02]:

That was Rebecca Hinds from the Work AI Institute at Glean. I really hope you enjoyed that conversation as much as I did. I learned, learned a lot. There are some things I’ll be doing differently in how I use AI in my work on a day to day basis. If you enjoyed it, share it with someone else who needs to hear it today.