Dr. Jeremy Sharp (00:00.568)
Hello everyone and welcome to the Testing Psychologist podcast. I’m your host, Dr. Jeremy Sharp, licensed psychologist, group practice owner and private practice coach.
Dr. Jeremy Sharp (00:13.036)
Many of y’all know that I have been using TherapyNotes as our practice EHR for over 10 years now. I’ve looked at others and I just keep coming back to TherapyNotes because they do it all. If you’re interested in an EHR for your practice, you can get two free months of TherapyNotes by going to thetestingpsychologist.com slash therapy notes and enter the code testing. This episode is brought to you by PAR. PAR offers the spectra, indices of psychopathology.
hierarchical dimensional look at adult psychopathology. The spectra is available for paper and pencil assessment or administration and scoring via PARICONNECT. Learn more at PARINC.COM slash spectra. That’s S-P-E-C-T-R-A.
Hey folks, welcome back to the podcast. Glad to be here with you. Today I have a powerhouse group of guests from PAR. I’ve got Dr. Jenny Green, Dr. Nicole Rogers-Wood, Dr. Cecil Reynolds, and Dr. Randy Camphouse here to chat about PAR’s new digital record form. So the digital record form is, I think, the next step in evolution in the PAR platform in terms of digital administration of performance-based measures like the REOS 2.
and many others in the future. So I think of the digital record form and you’ll hear us talk about this as almost like a digital clipboard for the examiner to use without altering the testing, test taking experience for the client at all. So we’ll dive deep into the digital record form, what it is, how to access it, advantages, appropriate use cases, that kind of thing. We also talk about
just the future of digital assessment in general. We talk about the potential role of AI and of course how these tools can help our practice and help us be more efficient and really do the work I think that we are meant to do, which is being present with the client and using our brains for the complex task of synthesizing assessment data. So this is a fantastic conversation. We touched on a lot of different areas and I think there’s a lot to take away.
Dr. Jeremy Sharp (02:28.534)
as usual. in case you don’t know who these folks are, I’m going to do some quick bios and then we’ll go from there. So Jenny Green is the digital assessment product owner at PAR. she has a PhD in measurement and evaluation from the University of South Florida and over 10 years experience in test development. In this current role, she drives the strategic development of digital assessment products that hopefully delight psychologists and provide significant insight.
into the clients with whom they work. Dr. Nicole Rogers Wood is the project director. She has a PhD in counseling psychology and is a licensed psychologist with clinical interests, including women’s emotional health. She has extensive experience in psychological assessment. She’s been a faculty member at a university. She’s in private practice. She’s a clinical director in skilled nursing facilities and in college counseling. So she has a wide variety of experience that she’s bringing to PAR.
And like I said, she’s the project director in the research and development department there. Dr. Randy Campos is senior advisor for external affairs at the Ballmer Institute at the University of Oregon. He’s author and co-author more than 85 scientific journal articles, 50 book chapters, and 10 books. He’s also authored or co-authored several psychological and educational tests, including the Reynolds Intellectual Assessment Scales, second edition,
Reynolds Interference Task and the Mathematics Fluency and Calculation Tests. He’s received numerous awards throughout his career, including the Senior Scientist Award, Division of School Psychology from the APA, and both the Lifetime Alumni Achievement Award and Russell H. Yaney Research Award from the University of Georgia. And lastly, Dr. Cecil Reynolds is a Professor Emeritus of Educational Psychology and a Distinguished Research Scholar at Texas A University.
He’s the author or editor of more than 45 books and has published more than 300 scholarly works. He’s also authored or co-authored more than a dozen tests, including the Reynolds Intellectual Assessment Scale, second edition, the Reynolds Adaptable Intelligence Test, Reynolds Interference Task, and the test of general reasoning ability. You probably heard of some of those. Cecil has been the recipient of numerous awards as well. Among these are the APA Division of School Psychology,
Dr. Jeremy Sharp (04:47.354)
Jack I. Barden Award for a lifetime of distinguished service, as well as the APA’s Nadine Murphy Lambert Award. So, like I said, total powerhouse group of folks here, and I’m excited to share my conversation with them with you.
Dr. Jeremy Sharp (05:17.102)
Hey everyone, welcome to the Testing Psychologist podcast. I’m happy to be here with all of you. We have quite a crowd this morning from PAR, so I’m gonna do things a little bit differently and just jump right into brief introductions so that folks can start to orient to all of your voices and figure out who you are. So Jenny, do you wanna go first?
Sure. My name is Jenny Green and I’m Digital Assessments Product Owner at PAR.
you. Nicole
I’m Nicole Rogers-Wood. I am a project director here at PAR and I’m also a licensed psychologist.
Cecil.
Dr. Cecil Reynolds (06:00.064)
I’d say it’s Cecil Reynolds. I’m a co-author of the Riaz 2 and a few other scales folks might have run into. There’s one out there that’s pretty popular called the Basque. I have 48 other tests out there. So look around, you may stumble over one.
That’s fair, that’s fair. And Randy.
Yes, I’m co-author with Cecil on the RIAS, a licensed psychologist and research professor at the University of Oregon.
All right, fantastic. Well, yeah, I feel lucky to have all of you here on the same call, on the same interview. This was kind of a powerhouse interview. We’re gonna get to talk about a lot of different things related to digital assessment and things that PAR is doing in that realm. So I’m excited to jump into it. So maybe we start, I mean, I know that there’s been this transition to…
digital assessment over the last several years. There’s lot of attention that people are paying to this. Maybe we just start with an intro or kind of laying the groundwork for what PAR has got going on in this realm and then we’ll dive into some details over the course of the conversation.
Jenny Green (07:20.504)
Sure, so I can take that. So as the digital assessments product owner, one of the product owners involved in strategy around how we’re going to continue to provide options to our customers to be able to administer tests digitally. So one thing that we’re working on right now that at the time of this recordings publication will be released is something called a digital record form.
The REOS 2 will be the first test to be able to utilize the digital record form. And we really are excited about it because it’s going to really streamline the administration of performance-based tests. So similar to Q-interactive, which I’m sure many of your listeners are familiar with, this will be for the clinician to be able to administer performance-based tests and replacing the print record form. Now, of course, they can still continue to use the print record form.
if that works for them and their practice, but we’re trying to give people more options to use things digitally. So the digital record form will allow you to administer the subtests with the appropriate start points. It seamlessly will advance you through each subtest with the administration rules, so the start, reverse, and discontinue rules, and allows you to kind of enter everything digitally via a tablet.
The other really cool part about this is that it doesn’t impact the client’s experience at all. So your test takers will still be interacting with whatever either eSTIM or PridSTIM that you want to use. But it’s really revolutionizing kind of the clinicians’ perspective of things and giving them more digital options.
Yeah, yeah, I love this. have so many questions about this. You know, I’m a details person, a concrete person, so I’m really holding back from digging, just like diving immediately into the practicality of how to do this. But I want to, let me zoom out just a little bit. So y’all have taken a bit of a different approach maybe from say Q Interactive, right? Which has been out for years and you I think people are familiar with that. So I’m kind of curious just on a
Dr. Jeremy Sharp (09:29.868)
a meta level or big picture level, what y’all are thinking or how you’re kind of conceptualizing digital assessment on the whole, you know, as you’ve seen things kind of play out over the years. And maybe that flows into why you chose this particular way to.
Yeah, so I think the name of the game for us is definitely flexibility. So we’ve been focused on providing options without forcing, you know, hands on either certain tablets or that, you know, the test taker also has to have a different experience. So this is device agnostic. Although it is designed to be used on a standard iPad, it can be used on any tablet or device with a touchscreen. So we’ve looked at very open and flexible. There’s no app to download.
It’s all through our assessment platform, PariConnect. as long as you have a PariConnect account, you can administer it like any other test on PariConnect. So we’ve tried to make it as easy to use and start using as possible and allow it to be a very seamless experience, right? If you’re already on PariConnect using other tests, this really does not change your workflow at all. We also have really been interested in not
making changes to the test taker’s experience and allowing you to continue using whatever stimulus books you’re comfortable with. If you really like print and you have all your print stem books, there’s no need to buy more stem books. You don’t have to buy a digital version. You don’t have to have multiple devices. So that’s another barrier for people sometimes is trying to get enough devices to go around basically. If you have one device, you’re good to go here.
Yeah, yeah, yeah, I like that. I almost I’m envisioning it like a digital clipboard almost like, you know, I don’t know if that’s the right way to think of it. But, you know, you’re holding whatever you’re doing on the iPad.
Jenny Green (11:20.038)
Yeah, we wanted to mimic that print experience but not force people to have to use something with the test taker.
Sure.
Sure. Yeah. See, so what’s on your mind.
Well, I just want to kind of reinforce that this is consistent with the original philosophy for why the RIAOS exists. One of our key goals and part of the philosophy that Randy and I have approached all of this with is that the assessment, administration, scoring, interpretation of intelligence tests doesn’t have to be
arduous time consuming, any of those things. should be, relatively quick and easy to do. And so we’ve had a lot of experience with digital administration of the real estate. We, fact, we first launched digital administration of the real estate 2015 law on COVID. as you know, COVID fostered a lot more development.
Dr. Cecil Reynolds (12:32.746)
of basically remote and digital testing. But we had been doing it since 2015 and trying to make it easier and really make the workflow a lot more systematic and easier for clinicians. And that was a big part of our original design of the RIOVS. It doesn’t have to be that hard.
So it was a natural extension of that philosophy when we first talked with with par about a digital record form with just making it seamless that if you will, and it actually does streamline a lot of things for the examiner. If you choose to use the digital record form, it monitors a lot of things for you. It monitors the discontinue rules.
It flows directly into the computerized scoring. You can use it to generate any of the options that you want and you don’t have to re-enter anything. It’s just seamless. So it saves you time and enhances your accuracy. So what’s wrong with that?
I think that’s what we all want.
Yeah. And it’s just consistent with the philosophy that we have been trying to follow with every aspect of this and with other things that Randy and I develop. it doesn’t have to be hard. You shouldn’t have to be a four eyed octopus to be able to administer a test accurately and record your responses. So, we want it to be easy and we want it to be seamless.
Dr. Cecil Reynolds (14:23.694)
And PAR has picked that up, has picked that philosophy up and really run with it. And you’ll see it with a lot of other things that they’re doing.
great. That’s great. Yeah, maybe it is a good time to paint this picture a little bit more. I mean, we’ve dipped in, you know, and you’ll you’ll give them a little preview and some details about what this looks like in practice. But someone give me a good kind of overview of, okay, what does this actually look like from the clinician experience when we go to set up the testing session for that morning? Like we’re starting with a Rias to what’s what’s it look like?
I can jump in there. So something that I think is really great is that you can preset your entire day of testing. So just like when you go in and create a client or select a client that you’ve already administered tests to in pari connect, you can just say, okay, I want to give the Rios digital record form and the brief and others and you can pre-kit that for that day or that week.
Then when you jump in, something that I really love that we did is that we’ve tried to make the experience to go back to what you said, Jeremy, feel like a digital click board. So when you open the digital record form, you go to assessment home and it shows you all of subtests, how long we estimate that it will take you to administer. And you end up going back to assessment home if you need to pause or if you
You need to just kind of check and see kind of where you are in administration. gives you a lot of information. And then when you get into the actual subtests, something that I love, also going back to Cecil’s comparison, the four-eyed octopus, that was one of the things that I showed our IT department. I was like, this is what you have to do. And they went, wait, you’re doing all of that stuff and you have to keep track of standardization? The digital record form does that.
Dr. Nicole Rogers-Wood, (16:30.336)
And if you miss a step or something, there is a pop-up that says, hey, by the way, time to kind of reverse. And now they’ve met the basil, time to continue forward. And so you’re doing it correctly, which I just really love. And it feels a lot like the paper, something else. Because I was just really kind of a subject matter expert. And I sat in there and I was like, I write all the record form. How can we make that happen?
Ask from lot of clinicians.
big
Yeah, so we have really tried to make it feel familiar, but innovative.
Mm-hmm.
Dr. Jeremy Sharp (17:11.266)
Let me just from a development perspective, what led y’all to make the choice to go with one tablet versus two, you know, and I hate, we’re just going to keep coming back to this Q interactive comparison because that’s, think the only thing out there right now that we can compare it to. you know, for better or worse, what, what led to that, that choice to go with one device versus two.
I think from what we talked about earlier, the wanting to be flexible. If we were to require two iPads or something like that, not only are we kind of forcing your hand in terms of what STEM book you’re using, we’re also changing the test taker’s experience, and that’s not really something we’re interested in doing. And we wanted to be very flexible. If you like print STEM, there’s no reason to stop using that. Some people really enjoy that. They’ve already got all the materials. Like, why?
I changed that. You know, don’t fix it if it’s not broken, right? If it’s already working for you, we don’t need to change it.
Mm-hmm.
Yeah, yeah, that’s fair. I mean, it’s kind of like chaos theory, right? Like the more devices, know, the more variables you introduce, there’s more opportunity for chaos.
Dr. Randy Camphouse (18:24.014)
If you wouldn’t mind, I’d like to add my kudos to Jenny and Nicole and the entire PAR team for how clever they went about developing this digital record form. It is so elegant. And it’s elegant enough that it allows the examiner, the cognitive flexibility to focus more on the client being assessed. And particularly with young children, that’s terribly important, as you know.
So I think the practicality of design and the elegance of design really enhances the examiner’s work.
Yeah, and that’s no small feat. Could y’all speak to that at all since you brought that up, Randy? mean, you know, tell us what kind of work went into the actual user interface and how you set it up and made it so elegant. Like, what are some of those feats?
Sure. So, you we’ve spent a lot of time working on it even before we started doing any code or any work on the IT side. So we worked with a UX UI designer and talked through our workflow. Nicole was very instrumental in that process of talking through with the whole project team kind of what the experience is like to administer performance-based tests.
and really helping us get in the shoes of people that are going to be using this. So we spent probably about a year in that design phase. We created some clickable prototypes. they were, you they look kind of real, but they’re not real. They’re just mock-ups of what it could look like. It did several rounds of testing. We recruited actual real psychologists to take a look at these and click through them and help us
Jenny Green (20:16.878)
see where we were hitting the mark and where there was room for improvement. So we made several rounds of changes to those mockups based on that. And then those mockups are then what kind of serve as the basis for the inspiration as we’re actually developing it with our digital team.
Yeah, it’s process. You know, just thinking about, I was talking about our software before this, that, know, things that you think are intuitive are not intuitive. You know, it’s a lot, there’s a lot that goes into building a user experience that makes sense and flows.
And until you start doing it, can’t, you can’t anticipate all the things that are going to go wrong.
Right. Right. That’s so true. That’s so true. Yes. So let’s see this. mean, this will be released and people will be able to use it by the time this airs. So let’s talk a little bit more about just, you know, main features, benefits, things that you want to highlight for folks who might be considering jumping over to try it out.
Sure, I can start talking through some of the features. as Nicole said, there is what we are calling the assessment home screen. So once you start the digital record form for Empire Connect, you will always be kind of returning back to the screen after each subtest. So it shows you exactly where you are within the RE-OS administration. So it’ll show you that you’re in progress on a subtest or that it’s completed.
Jenny Green (21:50.262)
It will also show you all the items on that screen as well. So you can see exactly where you met the basal and the discontinue, what item you started at. So it’s very user friendly to just kind of view it and see what happened on each subtest. It also will give you the raw score as it’s calculated as you’ve administered the subtest. So you can see in real time as you’re completing subtest the raw scores it’s calculating for you.
within each subtest. We’ve tried to keep the user interface pretty similar so that there’s really not much difference between the subtest except for the actual content of the items. So we have the ability to make notes as Nicole had mentioned. So if you select a note, there’s a little button that you can add a note and add a note to any item that you would want to add a note to. You can also flag items to return to later. So there’s a little flag button. And that’s really great if you want to come back and review something, if you were
wanting to come back to score an item or if you weren’t sure about a score, you could come back to it later. For our verbal subtest, you can also at the bottom of the screen, you can write the verbatim responses there very easily as well. So we’re giving kind of multiple places for you to be able to put feedback on that item. We also have a nice select navigation menu within the subtest as well.
So while you’re in the subtest, you can see exactly where you are. You can open, close that at any time to be able to see kind of where you are. It’s really nice too when you know you’re getting close to the discontinue, but you know you’re maybe not quite there yet and you’re waiting for your pop-up to be triggered. You can kind of open it and kind of get a sneak preview of where you are in terms of looking at the zeros and ones to see where you are in terms of meeting that discontinue.
By far, think the best feature of it is that it really guides you through administration very seamlessly. And so as soon as you start a subtest for the ones that have start rules and basals and discontinue rules, pop-ups notify you as you’re hitting those rules. So once you’ve started a subtest and you’ve done like the sample, if there’s sample on there, it will navigate you straight to your start point based on that person’s age. And it will give you just notifications that you just say, okay, and it will navigate you there.
Jenny Green (24:10.798)
It will also alert you if you need to reverse. If you haven’t met your basil, it will give you a pop-up that says, do want to reverse? You also can say, no, I just need to edit this item. There might be a chance that you made a mistake, you accidentally selected incorrect when you meant correct or something like that, or the person self-corrected and changed the response. We always give you an out if you want to kind of say, no, actually, hold on a second. I’m not ready yet. So that will start sending you in reverse. And then once you meet the basil,
it will also notify you that you’ve met the basil and say, you’ve the basil, let’s continue in forward administration. And then of course, that discontinue rule also would be given as a pop-up as well once you’ve hit that discontinue rule. So really it makes it so you don’t have to think too hard. You can really focus on each item and when talking with the test taker and really focusing on that as opposed to, wait, how many did I get wrong in a row? Let me go ahead and count those up again.
which is just so much on your brain really to think about while you’re trying to work with your client and establish rapport and things like that.
Mm-hmm. Mm-hmm.
Yeah, I’ve been thinking about… sorry, go ahead, Nicole.
Dr. Nicole Rogers-Wood, (25:22.67)
I just want to add one more thing that I love that we put in there. There’s also a practice mode, even in the digital version. So whether it’s trying to get acclimated to the digital record form or being new to the Rias 2, you can go in and practice and it doesn’t take an administration. Of course, that’s not something that will be scored in Par I Connect. Once you exit practice mode, you go back to the standard version. But I love that that’s in there.
Yeah, that’s huge. mean, that’s one of those things just, you know, I run a group practice, lots of clinicians. We have trainee, you know, new interns every year postdocs and there’s a huge variety in tech comfort, know, so practice is huge. They’re like, how do I get used to this? You know, before they see a client. So yeah, I think that’s great. Yeah.
Well, I want go back to, I guess, to a larger view and how I think this facilitates doing better assessments. Randy and I were both taught that, and I think taught very well by a guy you’ll recognize named Alan Cawthon, that there’s no such thing as a standard battery of instruments for every kit.
So I don’t have a set of tests I give to every kid who comes in. If I’m suspected of SLD or IDD or whatever, we were taught individualized assessment and that meant not just doing it one-on-one, but that the battery of assessments that you use are tailored to the individual child or a guy for that matter. so the,
the test that I might preliminary plan to give change as I get results during the evaluation. I may decide against giving something I thought I was going to use, use something else. And often since you don’t really have time to get the standard scores, can’t, you know, you don’t want to stop and start doing table lookups and all this stuff in the middle of interacting with your, with your child, particularly a child.
Dr. Cecil Reynolds (27:39.414)
it can be very disruptive to report and adds a lot of time to the assessment. the digital record form, you’re a minute or two away from having all the scores right in front of you. once you’re done, you’re literally it’s a minute or two with the digital record form. You’ve got all the scores. So you don’t have to guess if that’s going to be a poor score and process. You don’t have to guess.
Okay. He’s six. Now how’s that going to be? How’s that memory score going to look? You’ve got it right away. It’s right there. So it allows you to make much more accurate, real time changes in your assessment point. And I really liked that. to me, that’s more of a big picture benefit of doing it this way.
because I use performance on all these things to guide my selection of what’s next.
You know? Right.
What am I going to do next? Well, it depends. It depends. And I want to see those scores. And this allows me to see them exactly and right away. So I’m not having to do any guessing about how this turned out. And for an intelligence test, that’s usually one of the first things I do just as a matter of.
Dr. Cecil Reynolds (29:10.414)
It depends on the level of rapport with the child or the adult, but it’s usually very close to the front of my back. So I use it to guide a lot of things that I do. So I like this, almost instantaneous nature of having the accurate numbers in front of me before I move on to the next choice.
Sure. Yeah, I think that’s a huge advantage. know, it’s this whole conversation over last few minutes. You know, I think about cognitive load a lot and what we have to do as clinicians. I mean, I think about it a lot with report writing, of course, but with the administration process and anything that you can do to reduce that cognitive load to let us, you know, focus on the clinical work and, you know, take care of the other stuff.
You know, just get that done. Yeah, Randy.
It’s also a particular benefit for the Rias because of the extended age range. So if you’re testing five or six preschoolers this week and then suddenly you’ve got a geriatric clientele in the following week and you haven’t seen those items in a while and you haven’t seen, you know, those discontinue rules in a while at the top end, it’s really helpful to have the
as Jenny Allu’s student, have the digital record form do much of the thinking for you.
Dr. Jeremy Sharp (30:37.39)
Right. Yeah. I think that’s huge. mean, that’s the direction we’re going as well. You know, like having clinicians. I think it all just kind of dovetails, you know, with the AI discussion and efficiency and, you know, software and, you know, anything we can do to take that, you know, to help us not have to think about random things that aren’t really in our powerhouse as psychologists, right? Like we should be using our brains to really focus on.
those client dynamics and the clinical part versus managing paperwork or a timer or whatever it may be.
If remember when you’re first learning to administer tests, spend so much time focusing on getting the administration right. You don’t focus on the behaviors and the things you need to be watching. And as any experienced clinician, those, if you don’t watch a person carefully as they take a test, there’s a good chance you’ll misinterpret their performance.
scores don’t mean the same thing for every single child. So how do you know on processing speed if they were really paying attention? If you didn’t worried, if you’re so focused on getting the administration right and, and you’re not as adept at that yet, you’ll miss them.
And if r-
Dr. Cecil Reynolds (32:07.898)
that are really important in making observations that lead you to more accurate test interpretation. So again, our philosophy is to make this easy so you can focus more on the person sitting there and less on the materials. And this digital record form just flows right into that.
Yeah, yeah. mean, I can’t count the number of times that, again, we have lot of, you know, we have trainees coming through our practice every year and, you know, just talking to them about behavioral observations and, you know, I know you want to get all the scored, but please don’t like dip your head and be scoring while the client is doing something important, right? Like just put it aside, you know, and this kind of solves that problem to some degree, right? As you don’t have to worry about tallying and reading and is this correct or is you know.
So that’s really nice to see that. I’m curious, almost like zooming out a little bit, why start with the Riaz 2? Can someone speak to that?
What I remember is that it is a performance-based measure that a lot of people start with, and it lets you get the information you need pretty quickly. so from what I remember, and Jenny, please correct me if I’m wrong, it just seemed like a really great starting place. And then also a lot of the way the subtests are set up are pretty standard when it comes to performance-based tests.
eventually want to expand the library of measures on the digital record form platform. And so the Rias 2 seemed like just a really good starting point from which to build this.
Dr. Cecil Reynolds (33:57.686)
Hmm. Well, I would just go back again to the philosophy. It’s such a perfect fit to the original philosophy for why the Rias exists. So it was a natural place to start this process at was with something that then brought that philosophy through the front door and really got it going. And I think if you look at par products, you’ll find that they’ve done a lot.
move things in the direction of the philosophy of the real.
Yeah, I was going to add too, we did ask customers what they would like to see to be digitized. And so the Rias was the top choice of the people that we surveyed. There were other contenders as well. Our academic measures are on that list. The CHAMP, which is our primary test, is on that list. But this was the most highly rated on that list.
People were really excited about it, especially our school psychology customers were really excited to hear that the re-office was being considered as a digital record form. So we like to, of course, get feedback from our customers and listen to what they’re really wanting and needing. And so the re-office was the top of the list. So that was just yet another good reason to start with it.
Yeah, that makes sense to me. For whatever it’s worth, I’ll put in a vote for the champ if you want to move on to that relatively quickly and get that available. I know my clinicians will love that as well.
Jenny Green (35:33.044)
Thank you, appreciate it. mean, we’re always out trying to talk more with our customers and we know and love you as one of our customers too. So, happy to hear, get the feedback direct from you.
Right, right. Yeah, I mean, this is exciting. maybe since we’re talking about this kind of thing, what are some of the future directions? You know, what’s anything you might be able to share that’s on the radar on the roadmap for digital record form? Let’s take a break to hear from a featured partner. Y’all know that I love therapy notes, but I am not the only one. They have a 4.9 out of five star rating on trustpilot.com and Google.
which makes them the number one rated electronic health record system available for mental health folks today. They make billing, scheduling, note-taking and telehealth all incredibly easy. They also offer custom forms that you can send through the portal for all the prescribers out there. Therapy notes is proudly offering e-prescribe as well. And maybe the most important thing for me is that they have live telephone support seven days a week. So you can actually talk to a real person in a timely manner.
If you’re trying to switch from another EHR, the transition is incredibly easy. They’ll import your demographic data free of charge so you can get going right away. So if you’re curious or you want to switch or you need a new EHR, try TherapyNotes for two months, absolutely free. You can go to thetestingpsychologist.com slash therapy notes and enter the code testing. Again, totally free, no strings attached. Check it out and see why everyone is switching.
to therapy notes.
Dr. Jeremy Sharp (37:20.43)
The spectra indices of psychopathology provides a hierarchical dimensional look at adult psychopathology. Decades of research into psychiatric disorders have shown that most diagnoses can be integrated into a few broad dimensions. The spectra measures 12 clinically important constructs of depression, anxiety, social anxiety, PTSD, alcohol problems, severe aggression, antisocial behavior, drug problems, psychosis, paranoid ideation,
manic activation and grandiose ideation. That’s a lot. It organizes them into three higher order, higher order psychopathology spectra of internalizing, externalizing and reality impairing. The spectra is available for paper and pencil assessment or administration and scoring via PARI Connect. You can learn more at parinc.com slash spectra. All right.
Let’s get back to the podcast.
Sure, so we have some features that we’ve not included as part of this initial release that we want to include at some future date so that I can share some with you. It would be around integrating with our eSTIM books. So again, providing flexibility, right? For the people that do want to use the eSTIM, not requiring that it be used, but integration with our eSTIMulist materials is on the list.
Having a timer on subtest that require timing is on the list. Being able to record audio, so especially during those verbal tasks where, especially on the re-ass verbal memory, you are very quickly trying to write down and hear everything that that person is saying and it happened so fast. And it’s nice to have a recording to go back to and listen to later and make sure that you’ve scored accurately. So those are kind of some of the top ones. But again,
Jenny Green (39:16.632)
We’re going to be talking with customers throughout the next couple of months after release to get feedback in real time as to what features they’d like to see us do next. And then, of course, future tests as well. So our CHAMP is definitely on the list. And the FIFR products, the FAR, the FAM, and the FAW would definitely be on the list as well.
Dr. Jeremy Sharp (39:41.646)
That sounds good. know I have, you this is a fantastic group of folks. Y’all are pretty deep in the testing world and have a ton of experience. Just thinking like philosophically, where you stand on the move to digital administration. I mean, do you see us in five years, 10 years, 20 years, having paper involved at all in the assessment process or are we going to be completely digital? What do you think?
I think part of what comes to mind is equivalency. So I think about how a client interacts with paper, like some of the drawing tasks that you might have on performance-based, and how that would shift if it was on a tablet or something else. And so it’s more about how do we make sure that we’re going forward still measuring what the test was intended to measure?
But I don’t think that digital is going away and it really, in a way, think can really deepen what we’re able to do clinically if we don’t have, like what you said, Jeremy, that cognitive load of, my gosh, did I bring all the right stimulus books? And like, what am I supposed to do next? If the digital pieces can help us do that, then how much more information can we gather and use to understand that individual we’re testing?
think the digital will increasingly mimic the analog as it’s doing now. So the reason we’re able to offer digital record forms at this point in time is because the technology has improved to the point where it can just pick up some of the duties of the psychologists. And I think with regard to drawing and other sorts of activities where we’ll have our
examinies involved, the technology will just get better at mimicking what we used to do with pencil and paper.
Dr. Cecil Reynolds (41:49.454)
I do think that, you know, hard copy is going to be with us, for much, much longer than perhaps, folks in the digital realm, might like, might believe because of the demands with certain patients. They’re always going to be, I think, and always is not a word I like to use very much.
But I think there are always going to be individuals, especially children and perhaps some of our geriatric patients who need to have the hard cup and for whom the digital just won’t work very well. And we’ll have to make decisions as clinicians about, yeah, I the equivalency study says that that on average
it’s equivalent, but for this patient, it’s not. And so we’re going to have to, we’re going to have to make choices like that. And, and I think there will be children and there will be geriatric college, for whom having the hard copy is really necessary. I think that the number of people for whom that’s necessary is gradually going to decline.
but will it ever get to zero? Probably not. But at some point it may be that the publishers simply can’t provide the hard copy. That the cost of that just becomes way too much for the number of uses. So it may go away because of that too.
you
Dr. Jeremy Sharp (43:42.67)
That’s fair. Yeah, I’m just thinking generationally. And there are always going to be folks who are less comfortable, I think, with digital means. But I feel like we’re getting to that point. you know, my in-laws and parents are in their late 60s, early 70s. And they’re pretty comfortable with smartphones. But it’s the, you know, generation above that, like folks 80s, 90s, you know, like my great grandma, my wife’s great grandma, you know, doesn’t…
She does not know how to work a phone. But we’re getting close, you know? I feel like we’re getting close to digital literacy for the majority of folks that are.
Well, I don’t disagree with that. just think their physical condition. Yeah. That there may be folks for whom the hard copy is just much more functional. It’s not a matter of digital illiteracy, but simply their state and even at the younger ages, I mean, you may be able to hold a three year olds attention much better.
Yeah
Dr. Cecil Reynolds (44:50.926)
with hard copy than with a digital copy. But again, that depends on the three year old, right? So that’s why I was saying we’ll need to make that kind of call. One person at a time. So that we need options. Exactly. Exactly. And I hope that we will retain those options.
We need options.
Dr. Cecil Reynolds (45:19.342)
Uh, even, uh, when we get to the point which, um, is going to come where, uh, all of these tests are administered in score, uh, with a 3d headset by AI and there’s no clinician involved. So the, the, the clinician will be, uh, will be whatever name you gave the AI and we’ll put the, uh, we’ll have the 3d headset.
and we’ll tell the AI to administer the RIAOS to this condition person.
Even as an AI optimist, that still feels like a big leap. But you’re right. I think you’re right.
it’s coming. It’s coming and it’s going to be completely disruptive to the economic models of publishing to the economic models of clinical practice. It’s going to disrupt everything. And we had better be paying attention to that as it comes. It’s going to change the landscape of how we do what we do.
Right. Well, now that you’ve opened that door, I’m sure everybody listening is like, oh my gosh, what do I need to be preparing for? Like are we, is it too much to talk about a timeline for this kind of thing? Do y’all have some kind of insider knowledge about when this might be coming? Are we talking like six months or like six years? do we, know, ease people’s anxiety.
Dr. Cecil Reynolds (46:57.122)
Well, it’s taking a lot longer than I expected. gave an invited address on this very topic to the association of test publishers 10 years ago. Okay. And, told them, to get moving on this and, that it was coming. I thought it would be here by now. So in terms of when I have no idea, but I can tell you that when it does count.
It’s going to be perceived as something that happened overnight, but of course it’s not going to happen overnight. It’ll just feel it was. And I have no idea when, when folks are going to wake up and find that, but eventually we will.
So I have a question actually, now that you’re talking about that, Cecil. Where do you think the clinician fit? If we’re able to like technologically get AI to deliver the tests, where do we fit?
That’s a really good question that people are going to struggle with in lot of ways. And particularly when AI reaches the point where it can integrate information better than we can. And when AI can observe the person taking the test and take down those observations and the large language models have been taught well enough to interpret those behaviors.
and to modify the score interpretations individually. Then we’re in trouble. But it’s kind like the self-criminology. There’s still a way for a human to intervene in those. And I think that that clinician is always going to be in the background and it’s going to be needed and it’s going to have to review that. We know, I work a lot.
Dr. Cecil Reynolds (48:59.182)
with behavioral neuroimaging. And we know that for a number of things, the computer can read the behavioral imaging better than a neuroradiologist. But it’s very interesting. There are certain things that a neuroradiologist does with certain types of masses and certain types of images.
that we see in the brain, where the neuro radiologist is clearly better than the computer, but there are other things where the computer is much better than the neuro radiologist. So I think we will find that to happen as well. And that it will make us more efficacious in our practice. It’s not going to take us out of the equation, but I think it will make us much more efficacious.
and it will ease the clinician shortage if you will. We will be able to deal with the decision making and treatment planning and those things with a lot more patience than we can now with the assistance of AI.
Dr. Jeremy Sharp (50:20.844)
Yeah. Anybody else want to weigh in on this? It’s always spinning out into AI these days. So let’s just go with it.
I think there have been many disruptions and we’re always okay in the end. So I would not advise pre-stressing about upcoming change. I suspect it was quite a shock in the 38 or 39 when the Wexler scales came out with that non-verbal scale, that performance scale for all those Binet users and psychologists probably didn’t see it coming, but
adjusted quite readily to that. And I think Cecil is right. We won’t know when it’s coming or what it’s going to look like. psychologists have the skills, I think, to adjust to that new way of working.
Yeah, yeah, I’m with you. I’m with you. Like I said, I’m kind of an optimist with all of this, but I do feel like there’s always going to be a place for us. know, I wonder sometimes if it almost like spins us back into more of a therapist role where we become the, you know, the medium between the AI and the patient where, you know, the real skill is in delivering the results of the assessment, which is important. I mean, it’s already important, but
You know, it’s also split between the role of administering the tests and writing the report and how, you know, all the other things that go into testing. And I wonder if it spins us back into that role of having to communicate the results in a way that is caring and supportive. And, you know, that relationship becomes even more important.
Dr. Cecil Reynolds (52:04.942)
And making the final decisions. Again, I’ll go back to one of my favorite words that I use. It’s going to make us more applications.
Yeah. Right. And hopefully confident, right? A number of other things. Yeah. So I will steer us out of the AI conversation just for a bit. It’s so hard. I love talking about this stuff. I do want to go back though and double click on one thing that y’all mentioned a bit ago, which is the issue of equivalency in digital products and testing. Did y’all, did you kind of circumvent a lot of that by only
only, you know, only doing a digital record form versus altering the test administration or not. What did that whole world look like in this development process?
We already know that the digital administration is equivalent though. We have independent equivalency studies that have been done and not just with the RIAs too. When we were developing the test of general reasonability and the realms of adaptable intelligence, we collected data under both formats.
We collected data using paper and pencil and hard copy and stem books and all those things. And we collected data with fully digital administration and we had planned to have separate norms. And as we worked on it and got down through the item selection, it turns out we were able to create forms that had negligible differences in the score distribution.
Dr. Cecil Reynolds (53:51.682)
So we combined the samples into one set of norm because they were in fact the equivalent within the standard error of the main and the skewness statistics and things like that. The distributions were virtually identical. So if you plan well, those are not going to be issues.
Dr. Jeremy Sharp (54:17.614)
Right. So yeah, and talking about this whole equivalency in digital administration and whatnot, I know y’all have, like you said, you’ve been doing digital admin of the RIA since 2015. How is that different from the digital record form just for folks who might be wondering?
So I think it’s important to differentiate if we’re talking about the client or the clinician. We have in the past been able to do digital administration using a e-stimulus book that can be done in person or remotely via like a screen sharing type service. That has existed for a very long time. The new piece of it is the
So.
Jenny Green (55:02.478)
clinician experience of having the digital version of the record form, which previously we only had the print version of a record form. And so now we have ultimate flexibility. You can do your clinician side record form, print or digital, and you could have already done digital or print stem for your test taker.
Gotcha, gotcha. That makes sense. That makes sense. I appreciate the breakdown. Yeah. And these are, think this is a consistent conversation around some of these digital tools and you know, what goes where and when do I use this and how do they integrate? you know, people just, we’re a very, I think, practical bunch. You know, we just want to know what is this going to look like? Can I get in the room with the client and where do I have to click? And what do I sign into? Do I need a pencil? Do I need a-
Yeah, I appreciate this discussion, y’all. You know, I’m super excited. I think everybody probably knows I’m excited about the integration of technology in assessment and AI in assessment and all the possibilities that we have out there to do better work and free us up to be the best clinicians that we can be. So yeah, I really appreciate y’all being here and talking through this step, this piece of the puzzle and the hope.
of courses that people will check it out. So share with folks how they can find these assets and start to use them in their practice.
Sure, so you can go to parinc.com. That’s our e-commerce website and you can go search for the Rias 2 and it will be on the Rias 2 product page. So you’ll be able to purchase digital record forms right there on the website. It’s instantaneous fulfillment, so as soon as you purchase them and as long as you have a Parahe Connect account associated with that account, you should be able to use them immediately.
Dr. Nicole Rogers-Wood, (57:01.198)
And the only thing that I would add to that is that if this podcast wasn’t enough of a training for you, you’ve got practice mode and there are training materials also available to walk you through how to administer the Rios 2 on the digital record.
Yeah, that’s a great point, Nicole. The training materials are posted in our training portal. They will also be available on the digital record form itself under the Help button. So even if you are in the middle and you want to just go ahead and take a peek at those training materials, you will have access to them right within the digital.
I just want to say thanks to all of you. I know it’s tough to get everyone together for this amount of time, especially this crowd. So I really appreciate you being here, walking us through the digital record form and these resources and plans for the future and AI and all kinds of things. So thank you all so much for being here.
Thank you for having us.
Azure Meeting If.
Dr. Nicole Rogers-Wood, (57:56.546)
Thanks, Jeremy.
next week.
All right, y’all, thank you so much for tuning into this episode. Always grateful to have you here. I hope that you take away some information that you can implement in your practice and in your life. Any resources that we mentioned during the episode will be listed in the show notes, so make sure to check those out. If you like what you hear on the podcast, I would be so grateful if you left a review on iTunes or Spotify or wherever you listen to your podcasts. And if you’re a practice owner or aspiring practice owner,
I’d invite you to check out the testing psychologist mastermind groups. have mastermind groups at every stage of practice development, beginner, intermediate, and advanced. We have homework, we have accountability, we have support, we have resources. These groups are amazing. We do a lot of work and a lot of connecting. If that sounds interesting to you, you can check out the details at the testing psychologist.com slash consulting.
You can sign up for a pre-group phone call and we will chat and figure out if a group could be a good fit for you. Thanks so much.
Dr. Jeremy Sharp (59:21.08)
The information contained in this podcast and on the testing psychologist website are intended for informational and educational purposes only. Nothing in this podcast or on the website is intended to be a substitute for professional psychological, psychiatric or medical advice, diagnosis or treatment. Please note that no doctor-patient relationship is formed here. And similarly, no supervisory or consultative relationship is formed between
the host or guests of this podcast and listeners of this podcast. If you need the qualified advice of any mental health practitioner or medical provider, please seek one in your area. Similarly, if you need supervision on clinical matters, please find a supervisor with an expertise that fits your needs.
Click here to listen instead!