Many of y’all know that I have been using TherapyNotes as our practice EHR for over 10 years now. I’ve looked at others, and I just keep coming back to TherapyNotes because they do it all. If you’re interested in an EHR for your practice, you can get two free months of TherapyNotes by going to thetestingpsychologist.com/therapynotes and enter the code “testing”.
This episode is brought to you by PAR.
PAR offers the SPECTRA: Indices of Psychopathology, a hierarchical-dimensional look at adult psychopathology. The SPECTRA is available for paper and pencil assessment or administration and scoring via PARiConnect. Learn more at parinc.com/products/spectra.
[00:01:00] Hey, folks. Welcome back to the podcast. I’m glad to be here with you. Today, I have a powerhouse group of guests from PAR. I’ve got Dr. Jenny Greene, Dr. Nikel Rogers-Wood, Dr. Cecil Reynolds, and Dr. Randy Kamphaus here to chat about PAR’s new Digital Record Form. So the digital record form is the next step in evolution in the PAR platform in terms of digital administration of performance-based measures like the RIAS-2 and many others in the future.I think of the digital record form, and you’ll hear us talk about this, as almost like a digital clipboard for the examiner to use without altering the test-taking experience for the client at all. So we’ll dive deep into the digital record form: what it is, how to access it, advantages, appropriate use cases, that kind of thing.
We also talk about [00:02:00] the future of digital assessment in general. We talk about the potential role of AI, and how these tools can help our practice, and help us be more efficient and do the work that we’re meant to do, which is being present with the client and using our brains for the complex task of synthesizing assessment data.
So this is a fantastic conversation. We touched on a lot of different areas. I think there’s a lot to take away as usual. In case you don’t know who these folks are, I’m going to do some quick bios and then we’ll go from there.
Jenny Greene is the Digital Assessments Product Owner at PAR. She has a PhD in Measurement and Evaluation from the University of South Florida, and over 10 years’ experience in test development. In this
current role, she drives the strategic development of digital assessment products that hopefully delight psychologists and provide significant insights into the clients with whom they work.
[00:03:00] Dr. Nikel Rogers-Wood is the Project Director. She has a PhD in counseling psychology and is a licensed psychologist with clinical interests including women’s emotional health. She has extensive experience in psychological assessment.She’s been a faculty member at a university. She’s in private practice. So she’s a clinical director in skilled nursing facilities and in college counseling. So she has a wide variety of experience that she’s bringing to PAR. And like I said, she’s the project director in the research and development department there.
Dr. Randy Kamphaus is the Senior Advisor for External Affairs at the Ballmer Institute at the University of Oregon. He is the author and co-author of more than 85 scientific journal articles, 50 book chapters, and 10 books. He has also authored or co-authored several psychological and educational tests, including the Reynolds Intellectual Assessment Scales, Second Edition, the Reynolds Interference Task, and the Mathematics Fluency and Calculation Tests.
He’s received numerous awards throughout his career, [00:04:00] including the Senior Scientist Award, Division of School Psychology, from the APA, and both the Lifetime Alumni Achievement Award and Russell H. Yeany Research Award from the University of Georgia.
And lastly, Dr. Cecil Reynolds is a professor emeritus of educational psychology and a distinguished research scholar at Texas A&M University. He’s the author or editor of more than 45 books and has published more than 300 scholarly works. He’s also authored or co-authored more than 12 tests, including the Reynolds Intellectual Assessment Scales, Second Edition, the Reynolds Adaptable Intelligence Test, the Reynolds Interference Task, and the Test of General Reasoning Ability. You’ve probably heard of some of those.
Cecil has been the recipient of numerous awards as well. Among these are the APA Division of School Psychology Jack I. Bardon Award for a Lifetime of Distinguished Service, as well as the APA’s Nadine Murphy Lambert Award.
So, like I said, a total powerhouse group of folks here, and I’m excited to share my [00:05:00] conversation with them with you.
Hey, everyone. Welcome to The Testing Psychologist podcast. I’m happy to be here with all of you. We have quite a crowd this morning from PAR, so I’m going to do things a little bit differently and jump right into brief introductions so that folks can start to orient to all of your voices and figure out who you are. So Jenny, do you want to go first?
Dr. Jenny: Sure. My name is Jenny Greene. I’m the Digital Assessments Product Owner at PAR.
Dr. Sharp: Thank you. Welcome. Nikel.
Dr. Nikel: Hi, I’m Nikel Rogers-Wood. I am a Project Director here at PAR. I’m also a licensed psychologist.
Dr. Sharp: Fantastic. Cecil.
[00:06:00] Dr. Cecil: I’d say Cecil Reynolds. I’m the co-author of the RIAS-2, and a few other scale folks might have run into it. There’s one out there that’s pretty popular called the BASC. I have 48 other tests out there, so look around. You may stumble over one.Dr. Sharp: That’s fair. Randy.
Dr. Randy: Yes. I’m co-author with Cecil on the RIAS, a licensed psychologist, and research professor at the University of Oregon.
Dr. Sharp: All right. Fantastic. I feel lucky to have all of you here on the same call, on the same interview. This is a powerhouse interview. We’re going to get to talk about a lot of different things related to digital assessment and things that PAR is doing in that realm. So I’m excited to jump into it.
[00:07:00] I know that there’s been this transition to digital assessment over the last several years. There’s a lot of attention that people are paying to this. Maybe we just start with an introduction or laying the groundwork for what PAR’s got going on in this realm, and then we’ll dive into some details over the course of the conversation.Dr. Jenny: Sure. I can take that. So, as the digital assessments product owner, one of the product owners involved in strategy around how we’re going to continue to provide options to our customers to be able to administer tests digitally. So one thing that we’re working on right now, that at the time of this recording’s publication will be released, is something called a digital record form.
The RIAS-2 will be the first test to be able to utilize the digital record form. We are excited about it because it’s going to streamline the administration of performance-based tests. So similar to [00:08:00] Q-interactive, which I’m sure many of your listeners are familiar with, this will be for the clinician to be able to administer performance-based tests and replacing the print record form. Of course, they can continue to use the print record form if that works for them in their practice, but we’re trying to give people more options to use things digitally.
The digital record form will allow you to administer the subtest with the appropriate start points. It seamlessly advances you through each subtest with the administration rules, so the start, reverse and discontinue rules, and allows you to enter everything digitally via a tablet.
The other cool part about this is that it doesn’t impact the client’s experience at all. So your test takers will still be interacting with whatever either eStimulus or print stimulus that you want to use, but it’s revolutionizing the clinician’s perspective of things and giving them more digital options.
[00:09:00] Dr. Sharp: I love this. I have so many questions about this. I’m a details person, a concrete person, so I’m holding back from diving immediately into the practicality of how to do this. Let me zoom out just a little bit. So y’all have taken a bit of a different approach, maybe from say, Q-interactive, which has been out for years. I think people are familiar with that.So I’m curious just on a meta-level or big picture level, what y’all are thinking or how you’re conceptualizing digital assessment on the whole as you’ve seen things play out over the years, and maybe that flows into why you chose this particular way to roll out.
Dr. Jenny: I think the name of the game for us is flexibility. We’ve been focused on providing options without forcing hands on certain tablets. The test taker also has [00:10:00] to have a different experience. So this is device agnostic. Although it is designed to be used on a standard iPad, it can be used on any tablet or device with a touchscreen. So we’ve left it very open and flexible.
There’s no app to download. It’s all through our assessment platform, PARiConnect. So as long as you have a PARiConnect account, you can administer it like any other test on PARiConnect. So we’ve tried to make it as easy to use and start using it as possible, and allow it to be a very seamless experience. If you’re already on PARiConnect using other tests, this does not change your workflow at all.
We have also been interested in not making changes to the test takers’ experience and allowing you to continue using whatever stimulus books you’re comfortable with. If you like print, and you have all your print stimulus books, there’s no need to buy more stimulus books. You don’t have to buy a digital version. You don’t have to have multiple devices. That’s another barrier for people sometimes is trying to get [00:11:00] enough devices to go around. If you have one device, you’re good to go here.
Dr. Sharp: I like that. I’m envisioning it like a digital clipboard. I don’t know if that’s the right way to think of it, but you’re holding an iPad, and doing whatever you’re doing on the iPad, and it’s almost like a digital clipboard.
Dr. Jenny: We wanted to mimic that print experience, but not force people to have to use something with the test taker.
Dr. Sharp: Sure. Cecil, what’s on your mind?
Dr. Cecil: I just want to reinforce that this is consistent with the original philosophy for why the RIAS exists. One of our key goals and part of the philosophy that Randy and I have approached all of this with is that the assessment, administration, scoring, and interpretation of intelligence tests doesn’t have to [00:12:00] be arduous, time-consuming, or any of those things. It should be relatively quick and easy to do.
And so we’ve had a lot of experience with digital administration. In fact, we first launched digital administration of the RIAS in 2015, long before COVID. And if you know, COVID fostered a lot more development of remote and digital testing, but we had been doing it since 2015.
And trying to make it easier and make the work flow a lot more systematic and easier for clinicians, that was a big part of our original design of RIAS. It doesn’t have to be that hard. So it [00:13:00] was a natural extension of that philosophy when we first talked with PAR about a digital record form with just making it seamless if you will.
It does streamline a lot of things for the examiner. If you choose to use the digital record form, it monitors a lot of things for you. It monitors the discontinue rules. It flows directly into the computerized scoring. You can use it to generate any of the options that you want, and you don’t have to reenter anything. It’s seamless. So it saves you time and enhances your accuracy. So what’s wrong with that?
Dr. Sharp: I think that’s what we all want.
Dr. Cecil: Yeah. And it’s consistent with the philosophy that we have been trying to [00:14:00] follow with every aspect of this and with other things that Randy and I develop. It doesn’t have to be hard. You shouldn’t have to be a four-eyed octopus to be able to administer a test accurately and recall your responses. We want it to be easy. We want it to be seamless. PAR has picked that philosophy up and ran with it. You’ll see it in a lot of other things that they’re doing.
Dr. Sharp: That’s great. Maybe it is a good time to paint this picture a little bit more. We’ve dipped in, and y’all have given a little preview and some details about what this looks like in practice, but someone give me a good overview of, okay, what does this actually look like from the clinician’s experience when we go to set up the testing session for that morning? [00:15:00] We’re starting with RIAS-2, what’s it look like?
Dr. Nikel: I can jump in there. So, something that I think is really great is that you can pre-set your entire day of testing. So just like when you go in and create a client or select a client that you’ve already administered tests to in PARiConnect, you can just say, okay, I want to go the RIAS digital record form, the BRIEF, and others, you can pre-kit that for that day or that week.
Then, when you jump in, something that I love that we did is that we’ve tried to make the experience, to go back to what you said, Jeremy, feel like a digital clipboard. So when you open the digital record form, you go to assessment home, and it shows you all of the subtests, how long we estimate that it will take you to administer. You end up back at the assessment home if you need to pause or if [00:16:00] you need to check and see where you are in administration. It gives you a lot of information.
And then, when you get into the actual subtest, something that I love, also going back to Cecil’s comparison, the four-eyed octopus, that was one of the things that I showed our IT department. I was like, this is what you have to do. And they went, wait, you’re doing all of that stuff and you have to keep track of standardization? The digital record form does that.
If you miss a step or something, there is a popup that says, hey, by the way, time to reverse. And now they’ve met the basal time to continue forward and so you’re doing it correctly, which I really love. It feels a lot like the paper.
Something else because I was a subject matter expert. I sat there. I was like, I write all the record form. How can we make that happen?
[00:17:00] Dr. Sharp: That’s a big ask from a lot of clinicians.Dr. Nikel: Yeah. So we have tried to make it feel familiar, but innovative.
Dr. Sharp: Just from a development perspective, what led y’all to make the choice to go with one tablet versus two? We’re going to keep coming back to this Q-interactive comparison because that’s the only thing out there right now that we can compare it to, for better or worse. What led to that choice to go with one device versus two?
Dr. Jenny: I think from what we talked about earlier, the wanting to be flexible. If we were to require two iPads or something like that, not only are we forcing your hand in terms of what stimulus book you’re using, we’re also changing the test takers experience, and that’s not really something we’re interested in doing.
And we wanted to be very flexible. If you like print stim, there’s no reason to stop [00:18:00] using that. Some people really enjoy that. They’ve already got all the materials. Why change that? Don’t fix it if it’s not broken. If it’s already working for you, we don’t need to change it.
Dr. Sharp: That’s fair. That’s like chaos theory; the more devices, the more variables you introduce. There’s more opportunity for chaos.
Dr. Randy: If you wouldn’t mind, I’d like to add my kudos to Jenny, Nikel, and the entire PAR team for how clever they went about developing this digital record form. It is so elegant. And it’s elegant enough that it allows the examiner the cognitive flexibility to focus more on the client being assessed. And particularly with young children, that’s terribly important, as you know. So I think the practicality of design and the elegance, the design [00:19:00] enhances the examiner’s work.
Dr. Sharp: Yeah, and that’s no small feat. Could y’all speak to that at all? Since you brought that up, Randy, tell us what kind of work went into the actual user interface, and how you set it up and made it so elegant. What are some of those features?
Dr. Jenny: Sure. We’ve spent a lot of time working on it even before we started doing any code or any work on the IT side. So we worked with a UX/UI designer and talked through our workflow. Nikel was very instrumental in that process of talking through with the whole project team, what the experience is like to administer performance-based tests, and helping us get in the shoes of people that are going to using this.
So we spent probably about a year in that design phase. We created some clickable prototypes. [00:20:00] They look real, but they’re not real. They’re just mockups of what it could look like. And did several rounds of testing.
We recruited actual real psychologists to take a look at these, click through them and help us see where we were hitting the mark, and where there was room for improvement. So we made several rounds of changes to those mockups based on that. And then those mockups are then what serve as the basis for the inspiration as we’re developing it with our digital team.
Dr. Sharp: Yeah. It’s a process. I was talking about our software before this, that things that you think are intuitive are not intuitive. There’s a lot that goes into building a user experience that makes sense and flows.
Dr. Cecil: And until you start doing it, you can’t anticipate all the things that are going to go wrong.
[00:21:00] Dr. Sharp: Right. It’s so true. This will be released, and people will be able to use it by the time this airs. So let’s talk a little bit more about the main features, benefits, and things that you want to highlight for folks who might be considering jumping over to try it out.Dr. Jenny: Sure. I can start talking through some of the features. As Nikel said, there is what we’re calling the assessment home screen. So once you start the digital record form from PARiConnect, you’ll always be returning back to the screen after each subtest. So it shows you exactly where you are within the RIAS administration. So it’ll show you that you’re in progress on a subtest or that it’s completed.
It’ll also show you all the items on that screen as well, so you can see exactly where you met the basal and the discontinue, [00:22:00] what item you started at. So it’s very user-friendly to just view it and see what happened on each subtest. It also will give you the raw score as it’s calculated as you’ve administered the subtest so you can see in real time, as you’re completing the subtest, the raw score as it’s calculating for you.
Within each subtest, we’ve tried to keep the user interface pretty similar so that there’s really not much difference between the subtests except for the actual content of the items. So we have the ability to make notes, as Nikel had mentioned. There’s a little button that you can add a note to any item that you want to add a note to.
You can also flag items to return to later. So there’s a little flag button. And that’s really great if you want to come back and review something, if you were wanting to come back to score an item, or if you just weren’t sure about a score, you could come back to it later.
For our verbal subtest, at the bottom of the screen, you can write [00:23:00] the verbatim responses there very easily as well. So we’re giving multiple places for you to be able to put feedback on that item.
We also have a nice left navigation venue within the subtest as well. So while you’re in the subtest, you can see exactly where you are. You can open, close that at any time to be able to see where you are. It’s nice too when you know you’re getting close to the discontinue, but you know you’re maybe not quite there yet and you’re waiting for your pop-up to be triggered. You can open it and get a sneak preview of where you are in terms of looking at the zeros and ones to see where you are in terms of meeting that discontinue.
By far, I think the best feature of it is that it guides you through administration very seamlessly. And so as soon as you start a subtest, for the ones that have start rules, basals, and discontinue rules, popups notify you as you’re hitting those rules. So once you’ve started a subtest and [00:24:00] if there’s a sample on there, it’ll navigate you straight to your start point based on that person’s age. It will give you notifications that you just say, okay, and it’ll navigate you there.
It’ll also alert you if you need to reverse. If you haven’t met your basal, it’ll give you a popup that says, do you want to reverse? You also can say, no, I just need to edit this item.
There might be a chance that you made a mistake. You accidentally selected incorrect when you meant correct or something like that, or the person self-corrected and changed the response, so we always give you an out if you want to say, no, hold on a second. I’m not ready yet. So that will start sending you in reverse.
And then once you meet the basal, it’ll also notify you that you’ve met the basal and say, you’ve met the basal. Let’s continue in forward administration. And then that discontinued rule also would be given as a pop-up as well, once you’ve hit that discontinue rule.
So it makes it so you don’t have to think too hard. You can focus on each item, and [00:25:00] talking with the test taker and focusing on that as opposed to, wait, how many did I get wrong in a row? Let me go ahead and count those up again, which is just so much on your brain to think about while you’re trying to work with your client, establish rapport, and things like that.
Dr. Sharp: I’ve been thinking about … Oh, sorry. Go ahead Nikel.
Dr. Nikel: I just wanted to add one more thing that I love that we put in there. There’s also a practice mode even in the digital version. So whether it’s trying to get acclimated to the digital record form or being new to the RIAS-2, you can go in and practice, and it doesn’t take an administration. Of course, that’s not something that will be scored in PARiConnect. Once you exit practice mode, you go back to the standard version, but I love that that’s in there.
Dr. Sharp: That’s huge. I run a group practice, lots of clinicians. We have new interns every year, postdocs, and there’s a huge variety [00:26:00] in tech comfort. So practice is huge. They’re like, how do I get used to this? Before they see a client. So that’s great. Cecil.
Dr. Cecil: I want to go back to a larger view in how I think this facilitates doing better assessments. Randy and I were both taught, and I think taught very well by a guy you’ll recognize named Alan Cawthon, that there’s no such thing as a standard battery of instruments for every kit. So I don’t have a set of tests I give to every kid who comes in.
If I’m suspecting of SLD, IDD or whatever, we were taught individualized assessment and that meant not just doing it one-on-one, but that the [00:27:00] battery of assessments that you use are tailored to the individual child or a guy for that matter.
And so the tests that I might preliminary plan to give change as I get results during the evaluation, I may decide against giving something I thought I was going to use, and use something else. And often, since you don’t have time to get the standard scores, you don’t want to stop and start doing table lookups and all this stuff in the middle of interacting with a child. It could be very disruptive to rapport, and it adds a lot of time to the assessment.
With the digital record form, you’re a minute or two away from having all the scores right in front of you. Once you’re done, it’s a minute or two with a digital record form, you’ve got all the [00:28:00] scores. So you don’t have to guess if that’s going to be a poor scoring process. You don’t have to guess. Okay. He’s six. How’s that memory score going to look? You’ve got it right away. It’s right there.
So it allows you to make much more accurate, real-time changes in your assessment plan. And I like that. To me, that’s more of a big picture benefit of doing it this way because I use performance on all these things to guide my selection of what’s next.
What am I going to do next? Well, it depends. I want to see those scores. And this allows me to see them exactly and right away, so I’m not having [00:29:00] to do any guessing about how this turned out.
And for an intelligence test, that’s usually one of the first things I do just as a matter of course. It depends on the level of rapport with the child or the adult, but it’s usually very close to the front of my battery. So I use it to guide a lot of things that I do. So I like the almost instantaneous nature of having the accurate numbers in front of me before I move on to the next choice.
Dr. Sharp: Sure. I think that’s a huge advantage. This whole conversation over the last few minutes, I think about cognitive load a lot, and what we have to do as clinicians. I think about it a lot with report writing, of course, but with the administration process, and anything that you can do to reduce that cognitive load to let us focus on the clinical work, [00:30:00] and take care of the other stuff, and just get that done. Randy.
Dr. Randy: It’s also of particular benefit for the RIAS because of the extended age range. So if you’re testing five or six preschoolers this week, and then suddenly you’ve got a geriatric clientele in the following week, and you haven’t seen those items in a while, you haven’t seen those discontinue rules in a while, at the top end, it’s helpful, as Jenny alluded, to have the digital record form do much of the thinking for you.
Dr. Sharp: Right. I think that’s huge. That’s the direction we’re going as well. It all dovetails with the AI discussion, efficiency, and software, anything we can do to help us not have to think about random things that aren’t really in our [00:31:00] powerhouse as psychologists. We should be using our brains to focus on those client dynamics in the clinical part, versus managing paperwork, a timer, or whatever it may be.
Dr. Cecil: If you remember, when you’re first learning to administer tests, you spend so much time focusing on getting the administration right that you don’t focus on the behaviors and the things you need to be watching.
And as any experienced clinician knows, if you don’t watch a person carefully as they take a test, there’s a good chance you’ll misinterpret their performance. The scores don’t mean the same thing for every single child. So, how do you know the processing speed if they were paying attention if you didn’t watch?
If you’re so focused [00:32:00] on getting the administration right and you’re not as adept at that yet, you’ll miss things that are important in making observations that lead you to more accurate test interpretation. So our philosophy is to make this easy so you can focus more on the person sitting there and less on the materials. And this digital record form just flows right into that.
Dr. Sharp: Yeah. We have trainees coming through our practice every year, and just talking to them about behavioral observations and, I know you want to get all this scored, but please don’t dip your head and be scoring while the client is doing something important. Just put it aside, and this solves that problem to some degree. You don’t have to worry about tallying, reading, and [00:33:00] is this correct? So it’s nice to see that.
I’m curious, almost zooming out a little bit, why start with the RIAS-2? Can someone speak to that?
Dr. Nikel: What I remember is that it is a performance-based measure that a lot of people start with, and it lets you get the information you need pretty quickly. And so from what I remember, and Jenny, please correct me if I’m wrong, it seemed like a really great starting place.
And then also a lot of the way the subtests are set up are pretty standard when it comes to performance-based tests. We eventually want to expand the library of measures on the digital record form platform. And so the RIAS-2 seemed like a really good starting point from which to build this.
Dr. Cecil: I would just go back [00:34:00] again to the philosophy. It’s such a perfect fit to the original philosophy for why the RIAS exists. So it was a natural place to start this process with something that then brought that philosophy through the front door and got it going. If you look at PAR products, you’ll find that they’ve done a lot to move things in the direction of the philosophy in the RIAS.
Dr. Jenny: I was going to add too; we did ask customers what they would like to see digitized. And so the RIAS was the top choice of the people that we surveyed. There were other contenders as well. Our academic measures are on that list. The ChAMP, which is a memory test, is on that list. But this was highly rated on that list.
People were really excited about it, especially [00:35:00] our school psychology customers were really excited to hear that the RIAS was being considered as a digital record form. So we like to get feedback from our customers and listen to what they’re wanting and needing. And so the RIAS was at the top of the list. So that was yet another good reason to start with it.
Dr. Sharp: That makes sense to me. For whatever it’s worth, I’ll put on a vote for the ChAMP if you want to move on to that relatively quickly and get that available. I know my clinicians would love that as well.
Dr. Jenny: Oh, thank you. We appreciate it. We’re always out trying to talk more with our customers. We know and love you as one of our customers too. So, happy to get the feedback directly from you.
Dr. Sharp: Right. This is exciting. Maybe since we’re talking about this kind of thing, what are some of the future directions? Anything you might be able to share that’s on the radar, on [00:36:00] the roadmap for the digital record form?
Let’s take a break to hear from a featured partner.
Y’all know that I love TherapyNotes, but I am not the only one. They have a 4.9 out of 5-star rating on trustpilot.com and Google, which makes them the number one rated Electronic Health Record system available for mental health folks today. They make billing, scheduling, note-taking, and telehealth all incredibly easy. They also offer custom forms that you can send through the portal. For all the prescribers out there, TherapyNotes is proudly offering ePrescribe as well. And maybe the most important thing for me is that they have live telephone support seven days a week so you can actually talk to a real person in a timely manner.
If you’re trying to switch from another EHR, the transition is incredibly easy. They’ll import your demographic data free of charge so you can get going right away. So if you’re curious or you want to switch or you need a [00:37:00] new EHR, try TherapyNotes for two months absolutely free. You can go to thetestingpsychologist.com/therapynotes and enter the code “testing”. Again, totally free. No strings attached. Check it out and see why everyone is switching to TherapyNotes.
The SPECTRA: Indices of Psychopathology provides a hierarchical-dimensional look at adult psychopathology. Decades of research into psychiatric disorders have shown that most diagnoses can be integrated into a few broad dimensions. The SPECTRA measures 12 clinically important constructs of depression, anxiety, social anxiety, PTSD, alcohol problems, severe aggression, antisocial behavior, drug problems, psychosis, paranoid ideation, manic activation, and grandiose ideation. That’s a lot. It organizes them into three higher-order psychopathology SPECTRA of Internalizing, Externalizing, [00:38:00] and Reality-Impairing. The SPECTRA is available for paper and pencil assessment, or administration and scoring via PARiConnect. You can learn more at parinc.com/products/spectra.
Alright, let’s get back to the podcast.
Dr. Jenny: Sure. We have some features that we’ve not included as part of this initial release that we want to include at some future date. I can share some with you. It would be around integrating with our eStimulus Books. So, providing flexibility for the people that do want to use the eStimulus, not requiring that it be used, but integration with our eStimulus materials is on the list.
Having a timer on a subtest that requires timing is on the list. Being able to record audio, especially during those verbal tasks, especially on the RIAS verbal [00:39:00] memory, you are very quickly trying to write down and hear everything that that person is saying, and it happened so fast. It’s nice to have a recording to go back to and listen to later, and make sure that you’ve scored accurately.
Those are some of the top ones, but we’re going to be talking with customers throughout the next two months after release to get feedback in real time as to what features they would like us to do next. And then future tests as well. So ChAMP is definitely on the list. The Feifer products: the FAR, the FAM, and the FAW would definitely be on the list as well.
Dr. Sharp: That sounds good. This is a fantastic group of folks. Y’all are pretty deep in the testing world and have a ton of experience. I’m just thinking philosophically, where you stand on the move to digital administration. Do you see us [00:40:00] in 5 years, 10 years, 20 years, having paper involved at all in the assessment process, or are we going to be completely digital? What do you think?
Dr. Nikel: I think part of what comes to mind is equivalency. So I think about how a client interacts with paper; some of the drawing tasks that you might have on performance-based, and how that would shift if it was on a tablet or something else. And so it’s more about how do we make sure that we’re going forward, still measuring what the test was intended to measure?
I don’t think that digital is going away. It, in a way, can deepen what we’re able to do clinically if we don’t have, like what you said, Jeremy, that cognitive load of, oh my gosh, did I bring all the right stimulus books? What am I supposed to do next? If the digital pieces can help [00:41:00] us do that, then how much more information can we gather and use to understand that individual we’re testing?
Dr. Randy: I think the digital will increasingly mimic the analog as it’s doing now. So the reason we’re able to offer digital record forms at this point in time is because the technology has improved to the point where it can just pick up some of the duties of the psychologists. And I think with regard to drawing and other sorts of activities where we’ll have our examinees involved, the technology will just get better at mimicking what we used to do with pencil and paper.
Dr. Cecil: I do think that the hard copy is going to be with us for much [00:42:00] longer than perhaps folks in the digital realm might believe because of the demands with certain patients. There are always going to be, and always is not a word I like to use very much, but I think there are always going to be individuals, especially children and perhaps some of our geriatric patients who need to have the hard copy, and for whom the digital just won’t work very well.
And we’ll have to make decisions as clinicians about, yeah, the equivalency study says that on average it’s equivalent, but for this patient, it’s not. And so we’re going to have to make choices like that. [00:43:00] I think there will be children and there will be geriatric clients, for whom having the hard copy is necessary.
I think that the number of people for whom that’s necessary is gradually going to decline, but will it ever get to zero? Probably not, but at some point, it may be that the publishers simply can’t provide the hard copy. That the cost of that just becomes way too much for the number of users. So it may go away because of that too.
Dr. Sharp: That’s fair. I’m just thinking generationally, and there are always going to be folks who are less comfortable with digital means, but I feel like we’re getting to that point. My in-laws and parents are in their late 60s, early [00:44:00] 70s, and they’re pretty comfortable with smartphones. It’s the generation above that, folks in 80s 90s, like my wife’s great grandma, she does not know how to work a phone, but I feel like we’re getting close to digital literacy for the majority of folks that are there.
Dr. Cecil: Well, I don’t disagree with that. I just think their physical condition is going to be such that there may be folks for whom the hard copy is just much more functional. It’s not a matter of digital illiteracy, but simply their state.
And even at the younger ages, you may be able to hold a 3-year-old’s attention much better with a hard copy than with a digital copy, but again, that depends on the 3-year-old. [00:45:00] So that’s why I was saying we’ll need to make that kind of call one person at a time.
Dr. Sharp: We need options.
Dr. Cecil: We need options. Exactly. I hope that we will retain those options even when we get to the point which is going to come where all of these tests are administered and scored with a 3D headset by AI, and there’s no clinician involved. So the clinician will be whatever name you gave the AI. We’ll have the 3D headset, and we’ll tell the AI to administer the RIAS-2 to this young person, and it will.
[00:46:00] Dr. Sharp: Oh my gosh. Even as an AI optimist, that still feels like a big leap, but you’re right.Dr. Cecil: Oh, it’s coming. It’s going to be completely disruptive to the economic models of publishing, to the economic models of clinical practice. It’s going to disrupt everything. And we had better be paying attention to that as it comes. It’s going to change the landscape of how we do what we do.
Dr. Sharp: Well, now that you’ve opened that door, I’m sure everybody listening is like, oh my gosh, what do I need to be preparing for? Is it too much to talk about a timeline for this kind of thing? Do y’all have some kind of insider knowledge about when this might be coming? Are we talking like six months or six years? Ease people’s anxiety.
Dr. Cecil: It’s taking a lot longer than I expected. I [00:47:00] gave an invited address on this very topic to the Association of Test Publishers 10 years ago, and told them to get moving on this, and that it was coming. I thought it would be here by now.
So in terms of when, I have no idea, but I can tell you that when it does come, it’s going to be perceived as something that happened overnight. Of course, it’s not going to happen overnight. It’ll just feel that way. I have no idea when folks are going to wake up and find that, but eventually we will.
Dr. Nikel: I have a question. Now that you’re talking about that, Cecil, where do you think the clinician fits? If we’re able to technologically get AI to deliver the tests, where do we fit after that?
[00:48:00] Dr. Cecil: That’s a really good question that people are going to struggle with in a lot of ways, and particularly when AI reaches the point where it can integrate information better than we can. And when AI can observe the person taking the test and take down those observations, and the large language models have been taught well enough to interpret those behaviors, and to modify the score interpretations individually, then we’re in trouble.But it’s like a self-driving car. There’s still a way for a human to intervene in those. I think that that clinician is always going to be in the background. He’s going to be needed. He’s going to have to review that.
I work a lot [00:49:00] with behavioral neuroimaging. We know that for a number of things, the computer can read the behavioral imaging better than a neuroradiologist. But it’s very interesting, there are certain things that a neuroradiologist does with certain types of masses and certain types of images that we see in the brain, where the neuroradiologist is clearly better than the computer, but there are other things where the computer is much better than the neuroradiologist.
So I think we will find that to happen as well. And that it will make us more efficacious in our practice. It’s not going to take us out of the equation, but I think it will make us much more efficacious. It’ll ease the clinician shortage, [00:50:00] if you will. We will be able to deal with the decision making and treatment planning, and those things, with a lot more patience than we can now, with the assistance of AI.
Dr. Sharp: Yeah. Anybody else want to weigh in on this? It’s always spinning out into AI these days, so let’s just go with it.
Dr. Randy: I think there have been many disruptions, and we’re always okay in the end. So I would not advise pre-stressing about the upcoming change. I suspect it was quite a shock in the 38 or 39 when the Wechsler scales came out with that non-verbal scale, that performance scale, for all those Binet users. Psychologists
probably didn’t see it coming, but adjusted [00:51:00] quite readily to that.
I think Cecil is right. We won’t know when it’s coming or what it’s going to look like, but psychologists have the skills to adjust to that new way of working.
Dr. Sharp: I’m with you. Like I said, I’m an optimist with all of this, but I do feel like there’s always going to be a place for us. I wonder sometimes that it almost spins us back into more of a therapist role where we become the medium between the AI and the patient, where the real skill is in delivering the results of the assessment, which is important.
It’s already important, but it’s also split between the role of administering the test, writing the report, and all the other things that go into testing. And I wonder if it spins us back into that role of having to communicate the results in a way that is [00:52:00] caring and supportive. And that relationship becomes even more important than it already is.
Dr. Cecil: And making the final decision. I’ll go back to one of my favorite words that I use. It’s going to make us more efficacious.
Dr. Sharp: Right. And hopefully, confident in a number of other things. I will steer us out of the AI conversation just for a bit. I love talking about this stuff. I do want to go back, though, and double click on one thing that y’all mentioned a bit ago, which is the issue of equivalency in digital products and testing. Did you circumvent a lot by only doing a digital record form versus altering the test administration or not? What did that whole world look like in this development process?
Dr. Cecil: We already know that the digital [00:53:00] administration is equivalent, though. We have independent equivalency studies that have been done, and not just with the RIAS-2. When we were developing the test of general reasoning ability and the relevance of adaptable intelligence tests, we collected data under both formats. We collected data using paper and pencil, hard copy, stimulus books, and all those things. And we collected data with a fully digital administration.
We had planned to have separate norms. As we worked on it and got down through the item selection, it turns out we were able to create forms that had negligible differences in the score distribution. So we combined the samples into one set of norms because they were, in fact, equivalent within the standard error [00:54:00] of the mean, and the skewness statistics, and things like that. The distributions were virtually identical. So if you plan well, those are not going to be issues.
Dr. Sharp: Right. So, in talking about this whole equivalency in digital administration and whatnot, like you said, you’ve been doing digital admin of the RIAS since 2015, how is that different from the digital record form, just for folks who might be wondering?
Jenny: I think it’s important to differentiate if we’re talking about the clients or the clinician. We have in the past been able to do digital administration using eStimulus books that can be done in-person or remotely via a screen-sharing type service. That has existed for a very long time.
[00:55:00] The new piece of it is the clinician’s experience of having the digital version of the record form, which previously we only had the print version of a record form. And so now, we have ultimate flexibility. You can do your clinician side record form, print or digital, and you could have already done a digital or print stim for your test taker.Dr. Sharp: Got you. That makes sense. I appreciate the breakdown. I think this is a consistent conversation around some of these digital tools, what goes where, when do I use this, and how do they integrate? We’re a very practical bunch. We just want to know, what is this going to look like? Can I get in the room with the client? Where do I have to click? What do I sign into? Do I need a pencil? Do I need the time to handle all these things?
I appreciate this [00:56:00] discussion, y’all. I think everybody probably knows; I’m excited about the integration of technology in assessment, and AI in assessment, and all the possibilities that we have out there to do better work, and free us up to be the best clinicians that we can be. So I appreciate y’all being here and talking through this step, this piece of the puzzle, and the hope is that people will check it out. So, can you share with folks how they can find these assets and start to use them in their practice?
Jenny: Sure. You can go to parinc.com. That’s our eCommerce website. You can go search for the RIAS-2, and it’ll be on the RIAS-2 product page. So you’ll be able to purchase digital record forms right there on the website. It’s instantaneous fulfillment, so as soon as you purchase them and as long as you have a PARiConnect account associated with that account, you should be able to use them [00:57:00] immediately.
Dr. Nikel: And the only thing that I would add to that is that if this podcast wasn’t enough of a training for you, you’ve got practice mode, and there are training materials also available to walk you through how to administer the RIAS-2 on the digital record form.
Jenny: That’s a great point, Nikel. The training materials are posted in our training portal. They will also be available on the digital record form itself under the Help button. So even if you are in the middle, and you want to go ahead and take a peek at those training materials, you will have access to them right within the digital record form.
Dr. Sharp: I just want to say thanks to all of you. I know it’s tough to get everyone together for this amount of time, especially this crowd, so I really appreciate you being here, walking us through the digital record form and these resources, plans for the future, AI and all kinds of things. So thank you all so much for being here.
Dr. Cecil: Thank you for having us.
Dr. Randy: Pleasure meeting you.
Dr. Nikel: Thanks, Jeremy.
Dr. Jenny: Thanks, Jeremy.
Dr. Sharp: Alright, y’all. Thank you so [00:58:00] much for tuning into this episode. Always grateful to have you here. I hope that you take away some information that you can implement in your practice and in your life. Any resources that we mentioned during the episode will be listed in the show notes, so make sure to check those out.
If you like what you hear on the podcast, I would be so grateful if you left a review on iTunes or Spotify or wherever you listen to your podcasts.
And if you’re a practice owner or aspiring practice owner, I’d invite you to check out The Testing Psychologists’ mastermind groups. I have mastermind groups at every stage of practice development: beginner, intermediate, and advanced. We have homework, have accountability, we have support, we have resources. These groups are amazing. We do a lot of work and a lot of connecting. If that sounds interesting to you, you can check out the details at thetestingpsychologist.com/consulting. You can sign up for a pre-group phone call, and we will chat and [00:59:00] figure out if a group could be a good fit for you. Thanks so much.
The information contained in this podcast and on The Testing Psychologist website is intended for informational and educational purposes only. Nothing in this podcast or on the website is intended to be a substitute for professional, psychological, psychiatric, or medical advice, diagnosis, or treatment.
Please note that no doctor-patient relationship is formed here, and similarly, no supervisory or consultative relationship is formed between the host or guests of this podcast and listeners of this podcast. If you need the qualified advice of any mental health [01:00:00] practitioner or medical provider, please seek one in your area. Similarly, if you need supervision on clinical matters, please find a supervisor with expertise that fits your needs.
Click here to listen instead!