Kristina Williams, Director of Knowledge and Digital Strategy, Lionakis

Talk Description
What's the smartest format for upskilling your AEC firm's talent? Architecture, engineering, and interiors firm Lionakis has spent years refining how it helps people learn new tools, workflows, and ways of thinking. What began as hands-on, instructor-led training has steadily expanded into a more balanced learning ecosystem, one that blends in-person instruction, peer-supported learning, on-demand resources, and hybrid formats designed around how adults actually learn.

Kristina will share how Lionakis is evolving this ecosystem to keep pace with a changing practice, with a new emphasis on on-demand content, simplification and clarity of materials, and the development of structured learning paths that help employees understand what comes next.

She also shares the firm’s early experiments with AI—tools with the potential to guide employees through software questions, reinforce concepts outside the classroom, and provide real-time support during project work. Finally, she reveals how lessons emerging from Design Technology’s learning efforts are beginning to influence the broader firm, contributing to a cohesive, intentional, and modern approach to knowledge and learning management.

You’ll learn practical ideas for designing flexible, scalable learning experiences tailored for AEC talent that meets people where they are—especially during periods of rapid change.

Speaker Bio
Kristina Williams is the Director of Knowledge and Digital Strategy at Lionakis, where she focuses on integrating design technology and knowledge management to better connect people, ideas, and information. She began her career at Lionakis as a Revit Specialist and brings 17 years of experience in design technology. Over that time, she has continuously refined how she teaches and supports teams, shaping her approach to how knowledge is shared, adopted, and applied across the firm. She has carried this perspective into her current role leading knowledge management efforts, where she works to improve how information flows and how teams learn from one another.

Transcript

The transcript has been lightly edited for clarity.

Kristina: I'm very honored to be here on the KA Connect stage and happy to share our experience with all of you today. A little bit of context for you: we're Lionakis. We were that photo that you saw in Chris's presentation. We've been around for a while, even before that photo. Actually, our founder was the first state architect in California in response to the great earthquake in 1906. We've got 240-ish employees across six offices, all in California. We serve the core markets of education, healthcare, and civic, and we do architecture, structural engineering, interior design, and so much more.

I've been with Lionakis for 14 years. I started as a Revit specialist, and as Chris mentioned, I had been a director of design technology for many years. And just up until very recently, two months ago, I went into a full-time knowledge management role. So I'm very excited about that — still have yet to see what this all means. But today, I'm going to share with you why we felt compelled, and crazy enough, to create over 230 and counting short-format videos from scratch, despite having hours and hours and hours of past live recordings for those same topics. Most of them had many versions of live recordings for the same topics.

So, some background on why we did that. Much of that content was created for an annual training program that we hold, our design technology bootcamp training. This is something we've held annually for over 12 years, and it's a leadership-selected opportunity for a small group of individuals. Those individuals are representative of each market, service, and office that we have. We tend to have somewhere between 3 and up to 10, but not often do we have that many. We cover intermediate to advanced topics like model management and design technology stuff, visualization, coordination, production and standards. And these individuals are in studios — they're not on track to be in design technology. They're designers, architects, structural BIM technicians.

When they go through the bootcamp program and graduate, they're all the while still embedded in their studios, but they have the opportunity to become a design technology support specialist, or DTSS, as we call it. So it's an added role for them. These individuals help with tech support, peer-to-peer support on their projects. They help test and advocate for technology that we're implementing in DT. And by doing this, we can grow our collective intelligence around design technology across the firm. We can scale support without scaling headcount. We can put DT expertise closer to the work because they're working on projects, whereas us in DT, we're overhead completely. And then we can start to recognize emerging leaders from this too, because they're seen as go-to people in their studios.

We started bootcamp in 2013 with just a handful of people in Sacramento, and we've been holding it nearly every year since, always growing and adapting how we deliver it, to today where we have 30 DTSS across all offices, and we're hoping to add nine more from this year's cohort. And all of these individuals — we've spent, on the low end, around 50 hours of training with each of them. And that doesn't even count all the people who have been through it and are no longer at Lionakis. So all of this is really just to say that we've spent a lot of time not just delivering this program, but thinking about how we're delivering it and whether we're being effective when we do so.

So just a little bit of a glimpse of how things have adapted over the years — the things we've tried, things that have worked and haven't. Our very first bootcamp was a lot of hours. I still look back and think it's incredible that we even got buy-in for this. It was full days of training, which nowadays seems pretty uncommon, but back then was pretty common, especially for Revit training. Those full days were sandwiched between full days of project work, and everybody traveled to Sacramento to do this in person. We felt like with the curriculum then, maybe the outcome really wasn't the best outcome we were going for. I think we were trying to dump everything we knew into their heads and then put them out in the studios and expect that we've just got more DT members out in the studios, which isn't realistic. And I think because of that, we dialed in on the curriculum a little bit, and it was hard to get buy-in on that program.

We also felt like the full days in-person were really great. I kind of miss those days 'cause they were really immersive and we bonded a lot. But also, the mix of project work and training — we thought maybe we could just focus all in one month, all-in on training, and we could have better retention of the topics that way. So we tried that. We reduced the curriculum a little bit more. With technology training, I started to feel like there was a lot of mimicry going on, meaning we had them follow along hands-on in the course — we're doing something on the screen and then they're following, and we're doing something on the screen and they're following. And I had that experience myself along the way when I was learning Dynamo, and it wasn't sticking, 'cause I was doing the same thing. I was at a conference, watching, doing, watching, doing, and it just wasn't sticking. It wasn't until I struggled through it on my own, doing something fully immersed, that I started to learn it.

So that's where it occurred to us that we needed to introduce that struggle for them to be able to do it on their own, iterate, think about what they did, and then go back to more learning. That's where we introduced these exercises. The exercises are just little handouts that they follow, that they do individually or in small groups, which gets them to do the struggle — or the "desirable difficulty" is a much better way of saying it. And this we've been doing ever since, and I think it was a step in the right direction for sure.

Then, of course, the pandemic came, and we were forced to do everything virtually, which was a blessing in disguise, because though it was really nice to be in person, we were taking advantage of everybody's time and their presence traveling. So we had to cram everything into those full days. Now we could do everything virtually, hybrid — you all know the drill 'cause we're all doing the same thing — but it allowed us to spread out the sessions and shorten the sessions. And we've been actually doing a similar schedule ever since, with about 50-ish hours of training, one- to two-hour sessions.

And then, since this LMS came about, Chris had asked me: how might bootcamp look different now that we have tools like Synthesis LMS and technology like AI available to us? And maybe we would have gotten to this pondering eventually, but that question really just opened up our eyes to a new vision, and we went back to the drawing board and completely revamped bootcamp for this year's bootcamp. This was going on in late last year and early this year, when we were planning all of this. And enter a new era of bootcamp. At the time we were really hopeful of our approach here, and now that we've done it, we're really happy with how it's turned out. This is a new era, not just for bootcamp, but for a lot of different learning opportunities at our firm.

So going back to the title of this talk, "Learning at the Speed of Change," I want to address what we're reacting to with the change. Chris has addressed a lot of it here too — that with social media, you kind of wonder, are our attention spans shortening? I don't know, because we're all able to watch long movies and listen to long podcasts; things that are relevant and interesting to us, we give our full attention, right? So again, Chris had put it really nicely at one point — a new perspective — that maybe our attention spans aren't shortening, but that we have a lower tolerance for information that isn't useful to us at that time. And especially when you're learning something at work — yes, it's useful for your job, but you've got to find when it's relevant. And with AI especially, it's able to distill a lot of information into one answer for us when we're looking for it. So in a way, the learner expectation is changing for how we want to receive information.

And then, of course, doing more with less — the more codes, more documentation we have to do, more litigation, we're all subject to this pressure of doing so much. I heard recently that to deliver a $50 million project, you're now expected to do so in half the time compared to 10 years ago. So we're all being crammed into this, where we have a lot less time to squeeze in that important information along the way.

So here's what we set out to do, and have accomplished so far. Like I said, we recorded these short-format videos, and these were intended for a future audience. Again, back to those live recordings — they're unpleasant to watch, hard to go back to, and no one ever really wants to. So those were not meant for a future audience. Those were meant for the people attending at that time. And even then, was it really great for the people at that time? I don't know. So we recorded a series of videos and, of course, included our exercises as well. This is a course on our learning center, and you'll see more about that in a bit.

With those short videos, we had a choice. We knew that if we sent them off and assigned them these short videos, and then wanted to reconvene to do some exercises together, what's the likelihood that half of them would show up not having watched the courses? Some didn't have time, some forgot, some feel guilty — whatever the reason, no matter what, we're starting the session getting people up to speed, getting people on the same page, and we've now lost time we could have maximized for exercises. So we decided to do a live watch party — just watch them all together. And yeah, it's weird, because I'm the instructor, I'm the one who recorded this, and I'm right here in the room just playing it for you. At first I was worried it would seem lazy in some way — like I didn't feel like coming and talking to you. It was weird at first, but then I got really comfortable with it, and it's been a pleasant experience. So we watch them together, and then we know right then and there that we're all ready to start on the same page with the exercises at that point. And they know they can access them on demand later, because they're recorded.

When we set out to do these recordings, we aimed to say it short, say it clear, be as concise as possible. When I was watching the past live recordings — even watching myself — I would think, "Man, I really have a tendency to tell them a lot of just-in-case things." And I knew that about myself, in the back of my head, thinking, "I don't want them to think I didn't warn them about this thing, so I'm going to tell them all the just-in-case scenarios." That's who I was as an instructor, and I think that really added to the noise they were hearing and prevented them from retaining the truly important information. So watching those, and then recording these, helped me be a better instructor. And I've been teaching these same topics for 10 years — Revit hasn't changed all that much.

Also, when watching the recordings, I felt like, "Man, this is really slow." How many of you watch your YouTube videos on 1.5 speed? I'm pretty sure we're all at that level, right? I figured we can process information faster than we can say it, and some quick searches led me to believe that's true — that we can process a lot faster than we can talk, and it's generally safe to increase your videos to 1.5 speed. So we're conservative, and in our editing we do 1.25, 1.2, sometimes a bit higher, so that when you're playing our videos at normal speed, you're hearing them at about 1.2 speed. We just wanted to, by default, save people time, and then they can slow it down if they'd like.

When we created the recordings, we also had AI search retrieval in mind. In design technology, we're used to it — when we're creating tutorial pages, we know, "Okay, I want to be able to send this URL to somebody when a tech support question comes up." Same thing with these videos: I want to be able to send a URL of a video, and have AI search retrieval pull that same video up. We could do the same with longer videos and chapters that it could pull up. What I like about this is you can also update the videos a lot more easily — if you need to change something, you just swap out certain portions rather than a full course with chapters. So this is an example of a Revit course on linking in a DWG file — more about the case where somebody may have linked in drawings in Revit a bunch of times but forgets the little settings in that dialogue box. Now there's a video we can just send them, or that can come up in search retrieval too. So we went to town creating all of these videos.

And speaking of quality of production — we did not aim for high-quality production. We just turned our cameras on, talked, and went for it. Part of that was keeping in mind the sustainability of this method for other instructors. Everything we were testing, even the recording software, we wanted to be really simple for anybody to use, knowing we wanted to extend this format across the firm. So our general plan was to turn on the camera for a few seconds at the beginning and then turn it off, and we edit it so we're in the lower corner of the screen, not big on the screen. I think if you're asking someone else to record a video and they know they're going to be huge on screen, it's really intimidating. And once you turn off your camera after those few seconds, you're a little more at ease doing the rest of the video. That was our goal, and the quality for us was in the clarity and conciseness of our messaging.

Those videos and exercises came together as a course — this is our course catalog on the learning center, Synthesis LMS. A collection of courses comes together as a learning path. This year we held two bootcamps entirely in this new process: the live watch party, hands-on exercises, and then availability on the learning center afterward. So our regular DT bootcamp program is about 50-ish hours of training, and new this year to us as a firm was a visualization bootcamp training. I have been a little out of touch myself in my role, even as director of design technology, with the visualization process — we have a visualization team, and I was just thinking how we had a new design media specialist on board who recorded these. I thought, if I needed to get up to speed on this, this would be my favorite way — I would actually look forward to watching these, and I'm so glad we captured the knowledge from this individual in this format.

And speaking of instructor burnout — like I mentioned, I've been teaching these classes for 10-plus years. Even though I could teach them in my sleep, every year I would still have to prepare my dataset files, make sure everything's working properly, refresh my memory. We would spend a lot of time preparing for these courses, and I had always wished I had more time to focus on the more enhanced parts of it — the exercises and assessing what's working and what's not. But I'd find I wouldn't end up having time to do that because I had to prep so much. That led to a lot of burnout, teaching the same topic over and over again. So we're hoping these videos have a decent shelf life, so we can start to realize the benefits over years of not having to prepare for those at all, and then focus on the next-level tasks.

What we found, which we should have expected, is that we would deliver the message in the live watch party, and then when they set out to do their exercise, they'd be watching the videos again right away. That was actually my experience in live training in past years too — I'd say something, then they'd start their exercise and ask me questions about what I just said, and I'd always think, "Was it me? Did I say something? How do I explain this in a better way?" But it's just the way we all learn — we need repetition, we need it to sink in, and then we need to do it and go through that cycle. It was actually really cool to see them put their headphones on and just start the videos and do their exercise. They went into the exercise feeling a lot more confident that they could achieve it themselves, and there was way less burnout for us, since we weren't repeating ourselves right away. And now they know, because they're in the habit of going to the learning center, that once it comes up on their project, they can go right back to it again. That was really cool to see.

We really enjoyed this hybrid version, because we could use video for what it's best for — clear, concise messaging that's easily re-watched — and live interaction for what it's best for, which is the engagement between humans, talking through and working through the challenges of the exercises. Some people, I think at first, will think that learning through video takes away from the human experience. I'm now starting to see that it actually enhances the human experience, because it takes away that repetitive, robotic talk — the fundamentals we're kind of repeating over and over again — and allows us to spend our energy talking to each other and problem-solving. That really is where, as an instructor, I feel the most energy: working through those problems. So when we were talking with our president about what we were doing in DT, she was really supportive and opened the doors to carrying this format into other areas of the business. I truly believe that executive sponsorship, as it always is, is just so critical in this situation.

Getting into the practice side, we're addressing different changes and challenges. We've got hybrid work environments and dispersed teams. Project leaders or mentors are subject to the doing-more-with-less pressure, and as Chris said, they're having a harder time finding time for mentorship even if they really want to — the apprenticeship model isn't dead, but it's eroding a little bit. We want to keep it alive, we just need to adapt it. We also have growth on our mind: when we're acquiring a new firm, how do we onboard an entire office more efficiently, and how do we make sure they're not getting disparate voices — is it Brent's way of doing things, or Jennifer's way? How do we clear that up into one message?

So when we were going through our firm calendar of learning events for the year, we thought about where we could apply this new format. It turns out we can apply it in all sorts of areas, even on the practice side, even though some of the tacit knowledge in those areas is a little harder to turn into explicit knowledge. We sat in a room with a lot of our project leaders and started with pain points — in that room, it was more about our gut check of pain points. Since then, I've heard from other firms using RFIs and items that come up in quality checks as tangible pain points to determine where to focus their learning time — if you're going to invest time in creating these recordings, let's make it count. So we came up with some topics for our pain points, and most importantly, what was feasible. On the design technology side, it was easier for us to create a large volume of content because we've been doing it a lot; not as feasible on the practice side, for all the reasons we know. So we just started small — four courses, four topics, one each quarter, and then four on the business side for project management workshops as well. Different teams, so let's go for four. I think it's important to start small here.

We did the same thing — held a live watch party, did hands-on exercises. The watch party was a little different here, because it was a big Zoom call with everybody on the practice side together, versus the smaller cohort in bootcamp. The first one we held was a cartooning course, recorded by one of our busiest and most experienced architects. It was a very engaging training session — when people left it, they were messaging us afterward about how awesome it was and how they had never thought of cartooning that way. It was nice to see people jazzed after a training instead of drained. We held it the same way — videos, breakout rooms for exercises, back to the main room to watch more videos. In the breakout rooms, we had those with less experience share their screen and drive, so we could still have that struggle and not have the more experienced people just chime in and answer it. We did want the more experienced people to attend the course too, because we wanted everyone to get to that same message, not the disparate voices — by attending, they're seeing the documented message and that this is the Lionakis way of doing things, and that's how they should be directing their team. And now it's packaged up nicely, so I can just send them there.

So it's important for everybody to know what's there and to attend the course for that. And in doing so, the person creating the recording is responsible for understanding the Lionakis way and getting the perspective of their peers in different markets, to make sure that when we're documenting and codifying it, it's truly our current best-known practice. So getting on the same page, we can hopefully grow together from there. Similarly on the practice side — I know it's harder to take that tacit knowledge and turn it into explicit knowledge, but I think we can pull down a lot of those pieces and turn them into videos, so we cover the foundation and the energy of the mentor is better spent on the project specifics and the more challenging parts of the topic.

We were really sensitive to the instructor experience at this point, because we know our busiest practitioners — this is going to be difficult. We want to extract the knowledge, but we don't want to drain them in the process. That architect who did the cartooning course — this was definitely a new process for him. He was a little apprehensive at first, it was unfamiliar, but when he talks about it in hindsight, he says it was way better for him to create a few short videos than to prepare for a 45- to 60-minute presentation. He's actually become a good spokesperson for us with others we're trying to get to do the same process.

We also offer part of a knowledge coordinator's role to facilitate this experience for practitioners. If you're looking to do this, I don't think it needs to be a full-time person — it depends on how many you're tackling. With the eight courses this year, this was very manageable as a part-time role. Using AI, the coordinator would take a bullet-point list or a brain dump from the practitioner and put it into our PowerPoint template to start developing the slides. I was even thinking, in the last session, that maybe when they're brainstorming their outline — we had two practitioners recording for the same class — we could record that brainstorm and put the transcript in to help create the outline; use AI to help get their content out. They don't need to sit there and spend time creating slides. So the knowledge coordinator also gets them comfortable with recording, edits the recordings, uploads the courses, and facilitates the live training. I think that was one of the most rewarding parts for our instructors: showing up on the day of training and just answering questions. I really loved being able to see that for them.

What we're trying to do next is come up with templates or prompts that get instructors thinking about their recordings and how to outline them. One of the most challenging things has been getting them out of the mindset of the one-hour presentation. A lot of times, if I'm working with someone early on, they'll think, "Okay, I'm going to take my 60 minutes and turn it into 10 five-minute videos plus some time for Q&A" — which really just gives you the same long course, broken up by videos. What I've found is that letting them brain-dump their whole story first helps, and then we pull out the pieces of what they really need in the training session. Then we start with the exercises — what do we want people to struggle through and do themselves? — and figure out what videos we need to support that. If we want more videos on the learning center for context later, sure, we can create those too, so when somebody comes across them in the flow of work, they can see more content at that time. But the focus is on what we're really trying to do in the live training. This is what we're doing with our specifications training, trying to recreate their "Specs 101" in short videos.

A little bonus that came out of this was that the course was so nicely packaged that anybody could take it on the road. That happened with the cartooning course — a whole office wasn't able to attend, and one of our engineers happened to be going to that office the next week. He'd had enough experience with cartooning and with architectural sets working with our teams, so he just went down there and delivered the same exact course, facilitated the exercises, and answered the questions — he didn't have to prepare a single thing. It was the same single, concise message about how we do cartooning.

About live training more broadly — there was definitely a period in this process where I wondered why we were even doing live training at all. I'm talking about the broadcasted kind — not the leadership training or things around culture and trust, but "here's a topic you're probably not going to come across in the next few months" kind of live training. It made me think about the debate you always hear between universities and firms in our industry, questioning how come when somebody graduates, they don't know how to put a set together. The university's answer is that their job is to teach critical thinking and expose students to topics, but it's really on the job that people learn how to do the work. So I figured there are a lot more outcomes to live training than just skill competency.

Here's what I think some of those outcomes are. One is that it's the firm saying this is an important topic — "we have a way of doing this, a Lionakis way, and we want you to know about it." We reinforce that message through repetition, not just saying it once but carrying it through other trainings and studio meetings. It's also about the single, concise messaging — getting away from disparate voices — because live training gets everybody to attend and see what content's available. It fills in gaps, taking people from what they don't know they don't know to what they know they don't know — once they know they don't know it, they're more likely to go seek it out, because they know there's a way. It gives them a bit of a roadmap. We have a document that helps guide people to what's important for the Lionakis way across all the stages of a project, and I envision we'll be adding some of our learning center content to that too. I really see the learning center and live training as together contributing to that broad knowledge and roadmap of what's important, with the learning center also fulfilling deep learning — when somebody comes across it on their project, they can go off and dive deeper into courses, watch them again — and that, paired with mentorship, time, and experience, can start to get to the skill competency we're looking for.

Bootcamp has its own purpose for live training too. If we'd just had all the Revit users come across the content in their flow of work, we would never have had somebody go through and see the big picture from start to finish, someone who has really connected with us in DT and formed an appreciation for best practices. There's the power of the cohort, the cross-market collaboration and camaraderie among bootcamp-year members, the sense of belonging with DT, the emerging leaders — there are so many reasons to still hold live training in a cohort model. So going back to live training in this format and these topics — it's really just to plant the seed of knowledge, to provide awareness, to touch on skill competency, but not necessarily to achieve it, depending on the topic.

On our learner experience — I touched on how people were kind of raving about the format of the cartooning course. We had somebody message us saying they feel so grateful to work at a place with so many thoughtfully put-together resources. I think that gets at the apathy question someone asked earlier — if you don't have the right resources, how do you feel about your job? I do think it matters to show that you care about the learner's experience too. The learning center is accessible; the content is the Lionakis way, but it's easy to navigate, like YouTube.

As for what's next for us — we hope to take some of our onboarding content and put it into the learning center in the same format, along with reboarding of existing employees, which is everything I've been talking about: getting everybody on the same page. When we created these courses, we had in mind not only search retrieval and updating, but also that we didn't want to repeat courses or duplicate videos across several courses depending on the audience. So we broke up our courses into three topics: Big Picture, Core, and Deep Dive. Big Picture is the what and the why — here's our Lionakis way of doing something generally, and why we do it. Core is how to do something from point A to point B in as few steps as possible, in the clearest way — for somebody doing it for the first or second time. Deep Dive is where we dump everything else — the just-in-case stuff I always wanted to say, the nuance, the project-specific scenarios — so that when somebody's experiencing it in the flow of work and actually seeking that knowledge, they can find a ton of it if they need it. A new hire would just see the Big Picture stuff — they don't need to know how to do it yet, just that it's there when they need it. A bootcamper would watch both Big Picture, since we still need them to know the why, and Core too.

I had the pleasure of delivering our first new-employee orientation, just the DT portion. What we do, and probably a lot of you do something similar, is have a new hire meet with a lot of our department leads to get an orientation to that department, while in the meantime they're meeting with their market leads and supervisors to get to work on their projects. I had been used to doing the DT orientation for every single new hire who came through the firm — talk about bootcamp courses I've been doing for 10 years, I don't know how many new hires I've said the same exact thing to over that time. So I really wanted to package that up. Nowadays, because I've participated in so many of these recordings, anytime I have to show somebody something, I immediately want to record it first and then show them, because I know it'll be worth it later.

So I recorded these, and then had three new hires start last week for our regularly scheduled one-hour DT orientation. What I did was break that hour into 15 minutes of meet-and-greet — my favorite part, where we just get to know each other and I understand their background — then had them go back to their desks and watch the videos. If we'd assigned them a bunch of videos across all of onboarding, that would seem like an overwhelming task, but here, we do it in the hour we already had scheduled, and I know exactly how long it'll take, so when they're done, they come back and we do Q&A. What they were really grateful for was knowing it was available to them on demand on the learning center, because how many people are drinking from a fire hose when they first start and don't retain half of what we tell them — which is totally understandable. Now they can go back to the learning center once they need to.

My favorite part was that the meet-and-greet — which I guess never needed to happen in a conference room — because the videos make it so accessible, I turned it into a walk-and-talk, since we were lucky to all be in the same office. That was so refreshing; the human interaction was actually enhanced, because I had the videos to send them afterward, and I could get more work done and wasn't drained. A lot of times I come to onboarding meetings a little drained, knowing I have to say the same thing again, even though I try my best because I'm so excited to meet them. So I was really excited for this meeting, and it was great. I can't wait for others doing this to follow a similar process.

Another thing we want to get into is what Chris mentioned — knowledge agents. I really feel like knowledge agents can supplement the learning experience by continuing beyond the course: the course gets you the foundational information, and then you can dive into discussing with an agent or assistant how to apply it to your project-specific content. This was just a small example, but in the cartooning course, the first video was about drawing organization — how to break out a building into multiple areas. So with the Sheet Numbering & Naming Assistant, you can give it your project details, and it works with our standards for sheet numbering and naming — it'll ask how many stories, levels, and buildings, and start working with you on your actual project task beyond the course. The ultimate goal is ubiquity around this, so everybody's saying, "There's something on the learning center for that — go check that out first, then we'll talk." And when you're sending someone there, you know it's an experience they can trust and would enjoy — you're not sending them to a one-hour recording just to get five minutes of information. Building that culture of knowledge-seeking is what we want, so it becomes natural for people not only to know they should, but to want to seek out that information.

My last thought to share with you: no matter who you are — you don't have to be a learning and development expert, though I wish we had one on our team, I think we'd have rapidly gone through this whole process — but if you consider yourself a designer of learning experiences, whether you're developing or delivering training, you'll really start to put yourself in the shoes of the people learning from you or from others. The same way our designers consider the user experience for the built environment we're designing, we can consider our own people and their learning experience too. Thank you.

Q+A

Chris: Awesome. So, you have a new job — you're no longer the director of design technology, but there's all these videos with you on them.

Kristina: Yes. I thought about that even for the onboarding. I actually have to explain in the onboarding that I have a video saying, "You're hearing from me, and this is who I am," but really you're going to be working with this new person, because that's where we are right now, and I'd rather just get into it, 'cause I'm going to be saying that same thing to them anyway.

Chris: Yeah, so you're okay with that? You have all those videos — do you think your team will re-record them all next year, or will they leave some of them with you in it?

Kristina: No, leave them all. We talked about that in the beginning, even thinking, what if this is somebody who isn't here with us in a few years? I think back to design technology members who are no longer here with us, and if I would've minded if one of the videos was from them — no, I don't think I would. If the message is still what we want, and it's clear and concise, then does it matter? I would imagine we'd just preface it with something like, "Hey, this was so-and-so — they were part of the DT team at one point, and they were really great and had a lot of knowledge to share." It's okay that people aren't here anymore. With the bootcamp content, it was a little bit of a bummer, because I'll actually not reap the benefit of this next year. But I think of it as a gift that I gave, and that I'm still around — so when they're hearing that voice, they'll be like, "Oh, Kristina knows Revit."

Chris: How did you, in making the short videos and making them clear and concise — did you script them? Did you edit them heavily? Did you do multiple takes?

Kristina: Part of it was because we've been doing these topics for so long, but we did have outlines from before, from what we went through in the full one- to two-hour course. So we'd look at those and try to piece together what a new hire would need to know — for instance, we have a production standards document, and the new hire just needs to know that we have this and generally what it's about. Then we'd look at the steps and figure out what could fit into a recording — just kind of rearrange the outline. If I were starting from scratch and didn't have that outline, I'd do something similar to what I was doing with the practitioners — put the whole story out there and then think about what you want them to do first.

Chris: But start with the exercise and work backwards.

Kristina: Start with the exercise.

Chris: How did they like that approach, of starting with the exercise and working backwards?

Kristina: They like it. Definitely something clicks with them when I say that, but then they're like, "Wait, how do I do an exercise? What exercise? Can we just do a quiz?" And I'm like, I think really you want them to struggle through something, so let's think of something fun that they can do — and we can use AI for that too, thinking through what are some fun exercises or games. They like that, especially when we can offer help with the exercises. And once you tell them the exercises or bounce ideas off them, they get really excited, because they think, "People are going to love doing that." And it's actually going to get them to start thinking about the things we need them to think about, rather than just what we're telling them to think about.

Chris: And then they can work backwards — this is then the content I need to create to set them up to be able to do the exercise.

Kristina: Mm-hm. Plus, usually you need more than just that — you're not just teaching to the exercises and the videos, but you want enough context to do that.

Chris: That busy architect you talked about, who did the cartooning — he went first, and ended up thinking it was a good experience. How did you convince him to be first?

Kristina: This is where executive sponsorship comes into play, and I'm glad you asked, because I wanted to elaborate on that. In bootcamp, in our last session a couple of weeks ago, we usually rely on some of our past DTSS to record some of the topics, and I had a hard time with that, honestly, because it's just coming from us in DT — they're really busy, and there's not that executive sponsor or leader in the room saying they have to do it. So that was rough, and I realized how important it was. Brian, the one who did the cartooning, this came from Laura, our president — he was in the room determining the pain points, in the room deciding we were going to do four courses. We were all talking about, "Okay, you're going to be the first one," and we were all committed at that point, so he felt more accountability to the whole firm that way. It was a lot easier from our perspective to get it — it was more us helping him through than getting him to do it, really. So that executive sponsorship is really key. Can't stress it enough.

Chris: When you were watching yourself for the first time, did you kind of want to do the — but you know, this other thing might come up — did you want to do some of those just-in-case digressions, or was it more, "No, I've got it"?

Kristina: Oh, yeah. Yes, and I edited them out, or I noticed I was about to do them, and then — the way I do my editing, and I tell other people this too — I'll just go through, and if I mess up a sentence, I just pause and keep going, so you don't feel like you have to redo the whole thing every time you mess up. Then you edit that stuff out later; you can do the transcript editing, so you're just deleting words, and that's really fast — Camtasia has a nice Audiate feature for that. But yeah, I would find myself doing that, and I'd stop and go back to what I was saying and not do that, and then edit it out.

Chris: So in making the video, and then watching it with the other folks in the room, did you have that same instinct, or were you like, "We don't have time for this now, Kristina"?

Kristina: I did have that same instinct. I think a lot of times we would end the videos and I would want to tell them "just in case." And I did, many times, but you get used to it, and it's okay, because it's not in the content for later. Part of that is the exercise, where you get into some of the nuance.

Chris: It's cool. We have questions.

Attendee: Hopefully a quick and easy question. You're in all the videos, or someone's in all of them — did you consider using any of the AI training-creation tools where you write a script and it talks for you?

Kristina: We did consider that, and we talked about it a lot in our LMS beta cohort, about how many people were attempting that or not. I think we just wanted to try it ourselves, and we weren't sure how we felt about it, but we knew we could do it, so we haven't dabbled in that yet. I do feel like, if we're concerned about taking away from the human interaction, that would do that a little bit more — it is more efficient. So I'm not sure where we stand on it yet, but we haven't tried it.

Attendee: Regarding those just-in-case topics — is that something you would put in a more Deep Dive video?

Kristina: Definitely, yeah. I thought we would do a lot more Deep Dive in bootcamp, but we ended up not — we focused on the Core, and our plan is to record more Deep Dives later, and actually to crowdsource them more, because I think the Deep Dives are a lot easier to capture. I can go to any of my DTSS and say, "Here's the Core and the Big Picture we recorded — is there anything you'd want to add, because you're doing this on projects, or you have a more market-specific angle?" And they can just say, "Oh, sure." I tell them it can be a minute, it doesn't even matter — just brain dump. You don't have to intro, you don't have to explain all the fundamentals. So that's kind of our plan next, to start curating some of the deeper dives.

Attendee: What's an example of where the Core would drop off and a Deep Dive would pick up?

Kristina: Let's see, I'm trying to think — I think of design options in Revit. Design options are something where you show them how to set up different options, and even though a lot of Revit users should know that, they often don't. So we cover that in bootcamp to make sure they know to do that and not just make a bunch of copies of their stuff, getting on the same page. But design options can be pretty challenging and complex, and people don't want to use them because of that. We actually have a past recording — short — from a DTSS who's not here anymore, where he was able to work design options really well, doing an entire building in design options.

Attendee: Almost like a little case study.

Kristina: It was like a little case study, yeah — case studies is where I feel like a lot of the Deep Dive will be. Even though he's not here anymore, and it wasn't a pre-recorded course, because it's short and very concise, and it's just him flying through how he uses design options, I feel like that serves well as a Deep Dive as it is right now. We might re-record it, but no one does design options the way he does, so I don't know if I could recreate it that way — he was so skilled at it. So I see a lot more flexibility in the Deep Dive, because that's where people are looking for that information — they may be like, "Okay, I'll take anything I can get, I'm going to sift through everything you're putting here."

Attendee: Would those ever be assigned, or would that just be something you discover via search, so you'd know to send it to them?

Kristina: I thought they'd be assigned to at least the bootcampers, but now I think they're just more flow-of-work stuff.

Jess: So I feel like one of the things people are hesitant about, in moving onboarding to video, is losing that in-person connection. So I love the sandwiching on either side of it. Have you gotten feedback yet on how people feel about that, the in-the-middle videos?

Kristina: Yes. I asked them specifically — I told them they're the first people we've ever done this with, and how excited I was about it, and told them why we're doing this, so they were filled in on the whole process. I think they appreciated understanding all of it, so for a while I'll just explain, and almost over-explain, their experience here, so they know this isn't just us sending them to go watch videos — this is something because we care about their experience and care that they may want to re-watch this later. And then I did ask them for feedback — I told them, "I want to know exactly how this is, you can say you hated it, just tell me." And they loved it, they were like, "This was great." They loved going on a walk with me and it felt great.

One thing I noticed, and I was going to mention it on the speed slide, is that we had someone who was an English-as-a-second-language speaker, and I think the speed is a bit of an issue there, which is totally understandable — I can't imagine watching an Italian video at 1.25 speed. So I've asked them to try slowing it down and see if it's better, and I'll get feedback and rethink the speed if that becomes an issue.

When we did the Q&A, I think I want to enhance that part a little more — I want to do an assessment to make sure they understood it, but I don't want it to feel like I'm quizzing them at the same time. It felt like a little of that, and there weren't really any questions, maybe one general question, so you can't really tell how much they got out of the videos — but because they knew they were part of this process, they seemed pretty invested in it with me.

Chris: So in the same way that you pioneered the hybrid learning and the practice side took it, do you think other people might take your onboarding approach?

Kristina: I sure hope so. That's what I'm going to work on next — I know they're going to love, especially the ones repeating it like I am, being able to record and not have to say the same thing again, and to put more energy into that human-interaction side of things. I think they'll really want to. If anybody doesn't really want to, then I'll dig into why and we'll figure out how to make it better.

Attendee: With a lot of focus on videos, curious what proportion of your content is video versus written down. What made me question this was the point about all the stuff you're self-editing out of your videos — all the little details and corner cases and oddities — that feels like a great bullet list on a page. Is that what you're doing, or is it all video?

Kristina: So it's all video currently, and I'm glad you asked, because we've had this on our mind — what do we do with all of the written tutorial pages we have, since currently they're off in "tutorial page land," and this is on the learning center, and how do people know which one is more relevant? So we want to maybe transport those over, underneath the videos. As far as proportion, it's all videos and lessons currently in this new format, but I want to consolidate our written stuff, since we do have that. As for the just-in-case stuff I'm editing out, I don't end up getting far enough into it to actually take it and use it — I stop myself and don't do that, or I've gotten better at not doing it at all. Those are the ones we plan to do next — maybe that's what happens next year for bootcamp, the DT team recording the Deep Dives and showing them what's available. There are some topics where maybe half of them already know it, so I feel bad going over the Core stuff again, and maybe we can do some sort of mix-and-match based on where their level is at that time — half of them looking through Deep Dive stuff and half going through the Core content.

Attendee: We're actually really curious about how you're assessing the knowledge people are gaining from these training sessions. I love the idea of these bootcamp sessions getting people up to a certain level and then transitioning into more integrated DT leadership — but how are you actually assessing the widespread knowledge people are gaining from these sessions, and being able to leverage that?

Kristina: That has been on my mind this entire time through bootcamp. It felt like we were always spending so much time preparing, and had all this other stuff to do, so assessments always fell by the wayside, even though they shouldn't. This year we spent so much time recording that we again couldn't do the assessments. So that's the next-level stuff we can focus on in upcoming years, and assessments are one of them — I really want to dig into that, because currently we don't have a real method of verifying comprehension.

Attendee: So curious — you do the live sessions and then there's the on-demand version. Are you beginning to measure who's going back and revisiting the content, or reaffirming later on, like, "Hey, go back to this"?

Kristina: That's another thing I want to get into — I'm excited about it too. We just delivered this: we finished the courses in February or March and then delivered bootcamp all the way through the last couple of weeks. So that'll be something we'll be curious to dig into the analytics on at some point too.

Chris: Awesome. Okay, Kristina.

Kristina: Okay.

Chris: Thank you so much for sharing. That's an awesome, inspiring story.

Kristina: Thank you for having me. Thank you for listening everybody.