Blog

[00:00:00] Hello, everyone. Welcome to the Testing Psychologist podcast, the podcast where we talk all about the business and practice of psychological and neuropsychological assessment. I’m your host, Dr. Jeremy Sharp, licensed psychologist, group practice owner, and private practice coach. 

This episode is brought to you by PAR. PAR offers the RIAS-2 and RIST-2 Remote to remotely assess or screen clients for intelligence, and in-person e-stimulus books for these two tests for in-person administration.] Learn more at parinc.com.

All right, y’all, welcome back! 

Today’s episode is another EHR review. The star EHR for today is an EHR called TherapyAppointment. Again, many of you may have heard of TherapyAppointment. It’s been around for a few years and with good reason. TherapyAppointment is pretty strong. I’m excited to jump into the review.

[00:00:58] Before I do that,  if you have not checked out the rest of the series, I’ve been doing EHR reviews over the last few weeks and I will continue to do them over the next couple of weeks. If you’re interested in an EHR, and you are having trouble making the choice, these might be helpful for you.

So, I will do a brief episode here where I talk through the pros and cons and a verdict for TherapyAppointment. But the real star of the show for these EHR reviews is the accompanying YouTube video. You can access the YouTube video in the blog post, or there’s a link in the show notes, or if you’re a subscriber to the YouTube channel, The Testing Psychologist, you can go and check out the videos there.

[00:01:45] In those videos, I dive in and do probably 40  to 45 minutes just clicking through the EHR with fresh eyes, looking through the lens of a testing psychologist, and trying to figure out how these EHRs [00:02:00] might work for us. 

Before we get to the review, I want to invite any beginner practice owners out there who might want a little accountability and support in a group setting to consider the beginning of practice mastermind that’s going to be launching in a couple of months. 

So the beginner practice mastermind is a group coaching experience. I’m the facilitator. You will be in a group with five other psychologists who are all in the beginning phases of practice.

It’s really aimed at folks who are about six months away from launching their practice all the way to six months on the other side of launching their practice. And we talk about things that are challenging in the beginning, things like how to find office space, how to set up a business, what materials to buy, how to do it effectively, setting up your schedule, how to get clients, marketing, those sorts of things. Finances, all of those pieces come up at the beginner practice groups.

 So if you’d like some group accountability and help along the way, you [00:03:00] can go to the thetestingpsychologist.com/beginner and get more information and schedule a pre-group phone call to see if it’d be a good fit.

Let’s go ahead and get to my review of TherapyAppointment.

Okay, everybody. We are back here and ready to dive into this brief overview of my TherapyAppointment experience. Now, again, these episodes are just meant to be a really simple, quick pros and cons of each of these EHRs with the idea being that you really go and check out the YouTube videos that go along with each review.

So those are [00:04:00] linked in the show notes and on the webpage. You can jump in there and check it out. Those are about 40 minutes long. 

Now let’s get right to it. Overall, I liked TherapyAppointment. Here are some of the things that stood out with TherapyAppointment. The first is pricing. They do offer tier pricing, kind of like fair and nest. But the tiered pricing starts at $10 a month if you have fewer than10 appointments. That could work for a testing psychologist. And then it bumps up from there. I think their unlimited appointments are $59 a month, which is competitive with the other EHRs.

One thing that I liked about TherapyAppointment was the onboarding process. When you sign up with TherapyAppointment, they have you sign a business associate agreement right off the bat, and that is saved and recorded. You don’t have to dig through and sign it later. And that’s a crucial part of any software that [00:05:00] you work with. It’s right there, it’s signed and it’s in the system from the very beginning. 

And then from there, they take you through a nice little initial walk-through where they force you to input all of the crucial information that you will need to schedule appointments, bill insurance, take credit card payments, that sort of thing.

So, it gets all that stuff out of the way right from the beginning. It doesn’t give you the option to do it later. In some cases, it does if you don’t have that information. It’s very proactive where it walks you through that process rather than just providing say a checklist for you to complete at your own leisure once you get into the system. That was pleasantly surprising for me. I tend to be someone who likes to just jump in and get right to it. But I actually, appreciated that they made me put in this important information from the beginning [00:06:00] so that I could hit the ground running. And it took maybe five minutes so it’s not like it was a huge deal. 

Once you’ve got into the software, it was laid out really nicely. The aesthetics were great. I liked the colors. It’s kind of like a blue and orange color scheme. That may not resonate with some people, but I thought it looked nice. The layout was good. The font was relatively easy to read, not too tiny. So again, aesthetics were on point.

 Now a little more specific to the clinical piece. There was a really easy set-up of CPT codes and add-on codes. It was quite simple to specify which were base, which were add-ons, how many units should be billed for each one, and whether it should be one per occurrence or if you could do multiple, and if so, what the time frame would be, or the [00:07:00] time limit for each add-on unit.  That was great. They had pre-populated a couple of testing codes. It was interesting that they had actually pre-populated more add-on codes than anything.

But it was easy to add my own and pair those with add-on codes. Let me say the appointment scheduling process was pretty straightforward with TherapyAppointment. A little hiccup in the workflow but nothing major. It took me just a minute to poke around and figure out. I think the problem was trying to figure out how to assign a CPT code to an appointment. But ultimately, it’s just ended up that I had missed a button and it allowed me to do that. 

So, setting up an appointment was pretty easy. Let’s see. One thing that really jumped out as a positive, if you bill insurance, TherapyAppointment has a [00:08:00] built-in feature called the claim scrubbing feature.

So, any of you familiar with insurance know that claims have to be quote, and quote clean to get paid. And TherapyAppointment is the first EHR I have worked with to explicitly at least I have a claim scrubbing feature. It goes through and before it let you submit a claim, it tells you if there’s any missing information and it walks you through a process to resolve or correct the things that are wrong or the things that are missing. So that was really cool. I think that’d be really helpful for folks who build insurance.

So many positives for TherapyAppointment.

Let’s take a quick break to hear from our featured partner. PAR has developed new tools to assist clinicians during the current pandemic. The RIAS-2 and RIST-2 are trusted gold standard tests of intelligence and its major components. For clinicians using tele-assessment, which is a lot of us [00:09:00] right now, PAR now offers the RIAS-2 Remote allowing you to remotely assess clients for intelligence, and the RIST-2 Remote, which lets you screen clients remotely for general intelligence.

For those assessing clients in office settings, PAR has developed in-person e-stimulus books for both the RIAS-2 and RIST-2. These are electronic versions of the original paper stim books. They’re an equivalent, convenient, and more hygienic alternative when administering these tests in person. Learn more at parinc.com/rias-2_remote. 

[00:09:37] All right. Let’s get back to the podcast.

[00:09:40] The things that did not go so well. One thing that right of the bat that honestly almost caused me to scrap the review and not do it in the first place is they do require that you put in a credit card to get started. They don’t charge the credit card, but you do have to put one in. I did not like [00:10:00] that. So, this just means that there’s going to be an extra step if you’re doing a free trial that you’ll have to go back in and cancel to make sure that your card doesn’t get charged.

Another thing that stood out is they really didn’t have very many choices at all for gender identity. I think in this time with so many of us being attentive to different aspects of identity and so forth, this was to me just a complete oversight. And maybe I missed it somewhere, but it wasn’t clear. They really just give male, female and other, which I think corresponds to what insurance claims need. But it was disappointing to only see those options. Hopefully, those exist somewhere else in the system, but I could not find them. 

Another thing that jumped out as a big problem with many EHRs that I’ve reviewed from the testing perspective is that there is no clear way to create [00:11:00] a note or a custom session note that would match to the requirements of testing psychologists. It was pretty limited in terms of the note templates that they had. And I again, couldn’t find an easy clear way to create a custom note template. Maybe it’s in there, but I couldn’t find it. 

The last piece is, this is kind of a small complaint about the dashboard. I feel it could be a little clearer regarding notes that need to be done. But that’s again, just a small complaint. They are listed there in the form of appointments. So they list your appointments. It doesn’t explicitly say, write a note for this appointment, but it does list all of your appointments in your dashboard. And when an appointment is there, that means that you need to write the note and do the billing for it. So, it’s a good reminder, but I would like it to be just a little bit more explicit.

In [00:12:00] terms of the overall verdict for TherapyAppointment, when you watch the YouTube video, you’ll see that I really liked TherapyAppointment. Of all the EHRs that I’ve reviewed, this came the closest to TherapyNotes, which is by far my favorite and has been for years and years. But I liked TherapyAppointments. I felt like it flowed pretty well. There are still some hurdles to get over in terms of that note template not being friendly to testing and the gender identity stuff, those are tough.

But all in all, I think this is definitely a possibility for a testing psychologist especially if they could make some updates where you could change the note template, that would be awesome. That would go a long way. So, I would consider this still on the table. If you’re a testing psychologist, you should give it a [00:13:00] look particularly if you’re going to ramp up slowly. They have a pretty nice tiered pricing model that’s barrier friendly. The entry is very low.

All that said along with the walkthrough at the beginning to help you get started, I think this is a decent one. It’s certainly a possibility for anybody who might be looking for a new EHR or launching a practice.

As I said in the beginning, if you haven’t checked out the YouTube video for this episode, I would definitely go do that. It’s about 40 minutes long, a lot more detailed, and you can see for yourself what TherapyAppointment looks like.

And if you are a beginner practice owner who is looking to launch a practice in 2021, or just get some support maybe after you’ve already launched a practice recently, then I would invite you to consider the beginner practice mastermind group which is launching in a couple of months, likely in March.

[00:14:01] We will have a group of five or six psychologists all in the beginner of practice stages. And the idea is that you get support and have some accountability to get your practice off the ground and do so in a way that keeps you sane. You can get more information and schedule a pre-group call at the testingsychologists.com/beginner. 

Stay tuned. I will be back on Monday with a clinical episode and then the following Thursday, we are continuing with EHR reviews. I think that I’m kind of losing track. It’ll either be IntakeQ or it will be an EHR called Jane, which is a new one on the scene that a lot of people are saying good things about.

So, stay tuned. If you haven’t subscribed to the podcast, now’s a great time to do it. Start 2021 right and catch all of the testing related content that’s coming up. Y’all take [00:15:00] care and I’ll talk to you next time.

The information contained in this podcast and on The Testing Psychologist website is intended for informational and educational purposes only. Nothing in this podcast or on the website is intended to be a substitute for professional, psychological, psychiatric, or medical advice, diagnosis, or treatment.

[00:15:39] Please note that no doctor-patient relationship is formed here. And similarly, no supervisory or consultative relationship is formed between the host or guests of this podcast and listeners of this podcast. If you need the qualified advice of any mental health practitioner [00:16:00] or medical provider, please seek one in your area. Similarly, if you need supervision on clinical matters, please find a supervisor with expertise that fits your needs.

Click here to listen instead!

[00:00:00] Dr. Sharp: Hello, everyone. Welcome to The Testing Psychologist Podcast, the podcast where we talk all about the business and practice of psychological and neuropsychological assessment. I’m your host, Dr. Jeremy Sharp, licensed psychologist, group practice owner, and private practice coach.

This episode is brought to you by PAR. The BRIEF-2 ADHD Form uses BRIEF-2 scores to predict the likelihood of ADHD. It’s available on PARiConnect- PAR’s online assessment platform. Learn more at parinc.com

Welcome back everyone to another episode of the Testing Psychologist. Hey, we have a return guest today. Dr. Ellen Braaten is back talking about some of her latest research and a variety of other topics.

We dig into the recent article that she’s written that talks about the relationship between processing speed and a variety of psychiatric concerns, but we also get into a number of other topics like sluggish cognitive tempo, and its relationship to ADHD and processing speed. We talk about the impact that COVID-19 has had on learning and how it has been a different impact depending on what’s going on with the kid. And we touch a little bit on ADHD and the perception of time and some ideas to help gauge time effectively. So, this is a pretty wide range in conversation, and there’s a lot to take away from our talk. So hope you enjoy that.

Let me tell you a little bit about Ellen. If you didn’t catch her back on episode 50, that was a great one as well. She really dove into her book, Bright Kids Who Can’t Keep Up, and we spent a lot of time just talking about the nature of processing speed, what it is, and how to work with it. If you didn’t check that out, please go check that out.

Let me give you a little bio for Ellen in case you don’t know much about her. She is the associate director of The Clay Center for Young Healthy Minds at Massachusetts General Hospital (MGH). She is the director of the Learning and Emotional Assessment Program (LEAP) at MGH as well, and she’s also an associate professor of psychology at Harvard Medical School (HMS).

She is a psychologist, teacher, and researcher whose career has focused on the better understanding and treatment of children with learning and attention issues, especially ADHD, learning disabilities, dyslexia, and autism spectrum.

As she is a mother of two young adult children, as well as a psychologist, she is keenly interested in parenting issues, especially those relating to normal development, education, and parenting kids with behavior and learning differences.  In addition to the book that I mentioned earlier, Bright Kids Who Can’t Keep Up, she has written/co-authored Straight Talk about Psychological Testing for Kids, How to Find Mental Health Care for Your Child, and I know that we’ll talk about this in the episode that she is working on a sequel of sorts. I don’t know if sequels applied in the nonfiction world, but a follow-up to bright kids who can’t keep up, about kids who seem to have low motivation.

She’s also, as I said, a prolific researcher. Lots of publications, collaborations with psychologists and academics around the world. She has also written the book that’s now in its second edition called The Child Clinician’s Report Writing Handbook. So Ellen has done a lot and she continues to do quite a bit. I am very fortunate to have her back.

Before we jump to the episode, I want to invite any of you beginner practice owners out there to consider joining the Beginner Practice Mastermind for testing psychologists. This is a group coaching experience that is meant to increase accountability and help you launch your practice successfully.

This group is going to start in March of 2021. It’s a group of about 6 psychologists and they’ll all be in that beginning phase of practice, either about 6 months prior to launch or 6 to 12 months after launch. So we talk about things like how to set up your business, get office space, tax entities, set up your schedule, purchase your battery, marketing, and a variety of other things. So if that sounds interesting to you, I invite you to check it out. Get more details at thetestingpsychologist.com/beginner.

All right, let’s jump to my conversation with Dr. Ellen Braaten.

Hey, Ellen, welcome to the podcast.

Dr. Ellen: Hi, thanks for having me. It’s great to be back.

Dr.Sharp: That’s right. I should’ve said, welcome back to the podcast. Gosh, I was looking, and it was April 2018, which is nuts. I can’t believe it’s been that long.

Dr. Ellen: I can’t either. It seems like it was just yesterday, but this year worked our sense of time in a way but has passed fast too.

Dr. Sharp: Yeah, for sure. I know. I don’t know how else to put it, but it really is like a time warp. I feel like this year in some ways did not even happen and in other ways, it’s just has been the longest year ever.

Dr. Ellen: Yeah.

Dr. Sharp: Here we are. I’m glad that you’re back. I know that you’ve been busy since the last time that we talked. I’m excited to dive into all things with you. So maybe tell us, what have you been up to over the past two and a half years? What are the big things on your radar?

Dr. Ellen: Since I spoke with you in 2018, I worked as a visiting professor at Charleston University in their Medical School in Prague, Czech Republic was seven months. I was working with a number of ADHD researchers there who are also interested in some of the topics that I’m interested in, particularly Hine perception and ADHD, but I was able to see another medical and psychiatric system, and it was really wonderful. I did a lot of teaching. I’m still going back. I’ve got a five-year appointment there as a visiting professor, so I still have another three and a half years I think on it. And so I go back frequently or did it pre-COVID. So that’s been exciting.

And my lab has published a number of great papers on processing speed which is one of my interests. And one of my books, The Child Clinician’s Report-Writing Handbook, I came out with a new edition of that, and I mentioned to you when we were chatting that I have a new book that is being written right now, frantically written, that the take-off on my book, Bright Kids Who Can’t Keep Up, which is Bright Kids Who Don’t Give a… We’re trying to come up with the right word that fits in there that doesn’t offend people.

It’s about kids who don’t really care and can’t get motivated, but it’s really more than just about motivation. So hopefully, in another year or so you might invite me back to talk about that, but for right now, it’s not finished yet, but it does take my research on processing speed to the next level, like who are these kids as they move on into adulthood. And I know it’s an epidemic that most of us who evaluate kids for a living are seeing, there are lots of kids or just not making it successfully into adulthood, and it’s about both kids.

Dr. Sharp: Sure. So is this overlapping with the failure to launch set? Is there something in there?

Dr. Ellen: Yes, although the book is really written for younger kids, not really for the adults, but to kind of say, if you’ve got a child who you think, even in second grade, isn’t really rolling with the punches and doesn’t seem to have much drive, what do you do at different ages of development? And so, a lot of it really is parents understanding what the motivating factors are, and also understanding through evaluating kids and knowing, I think so much power can come from having a good evaluation and understanding who your child is and what their capabilities and limitations are. I think that knowledge is power and it can be used as a real way of motivating kids to understand who they are.

Dr. Sharp: Sure. That’s interesting. I know this is just a little side path as we get started, but I dove into the literature around motivation, I don’t know, maybe 6 or 12 months ago for a little project I was working on. And tell me, my conception of all that is that the two factors that really make a big difference in motivation are meaning and agency like having it matter, but also having control over the thing you’re working on. I don’t know. Is that accurate or is that what y’all are finding?

Dr. Ellen: Yeah, it is. Although, what I’m finding is there are so many theories of motivation that it’s hard to pick just one. And maybe that’s part of it is that we think that motivation is a thing, but you’re right, those two things are clearly well studied, and they’re, in theory, important components of motivation. As part of that, I think that parents don’t always listen to what their child is telling them. We only have a sense of agency or things that we actually want to do.

And so, I find that a lot of things go off the rails for the parents that I evaluate early in their development because there are assumptions about what their child should do, particularly as it relates to college and it sets a trajectory, and that is sometimes destined for failure. And kids tell us a lot of times what they are capable of doing by not doing what we want them to do. And that’s not always because they just want to misbehave. A lot of times it’s because they’re incapable of it.

So in terms of motivation, that’s why we really have to find out, can they have a sense of agency over it? Are we asking them to do things that are not possible? And that destroys your motivation if you’re asked to do something you can’t do.

Dr. Sharp: Absolutely. There’s so much here, and I’m like, Oh My Gosh, we should be talking about this.

Dr. Ellen: I’ll be here, but it’s great, it’s really an interesting and complicated subject.

Dr. Sharp:  Yeah, absolutely. We’ve seen that just on our own. We have two kids, a 7-year-old and a 9-year-old. One of them is super motivated for homeschool, does all his work and is on top of it and reminds us of his meetings instead of the other way round, and then our little girl, she really couldn’t care less about school and she’s motivated for plenty of other things, but just not jumping onto those meetings and doing the schoolwork, and it’s been really eye-opening. My wife is also a therapist. So we’re pulling our hair out, what do we do?

Dr. Ellen: Nothing more humbling than being a psychologist or a therapist and having kids.

Dr. Sharp: Exactly.

Dr. Ellen: I have more than one undecided because my first was also like, she did everything then my son. I had two that were very different. And they both turned out to be great adults, but during the process, you realize that there’s so little we do that they come out who they are. The best we can do is just help them understand who they are so that they can figure out what they want, what they desire, and how they can most easily do that.

Dr. Sharp: I love that. Right, it’s like long-term vision or something. Delayed gratification. I don’t know, but it’s hard.

Dr. Ellen: Delayed gratification, yeah. 20 years. 25 years.

Dr. Sharp: That’s encouraging though. I just need to keep hearing that. They’re going to be successful adults and it’ll be fine.

I know that you’ve been doing a lot of work. We were talking before we started to record just about COVID. And I know that y’all have been doing a little bit of work just looking at the impact on kids from that. I wonder if we might just start there. I know that you have this recent article as well that I want to dive into about processing speed and its role in different pathologies, but the COVID thing is topical and it’s right in everybody’s face right now. So I’m curious about that. What have you all been finding with COVID and learning?

Dr. Ellen: Well, one of the things that we’re doing is a survey on kids and families and their stress levels during COVID, but we haven’t gotten any data yet. And one of the things that I think we’ll probably end up doing is looking at these kids longitudinally. But it’s interesting because last year we published a study on slow processing speed and Sluggish Cognitive Tempo. And one of the reasons why I got interested in that is because I was always asked about those two things, especially when I had a […] that I was giving to teachers.

Teachers hear this term sluggish cognitive tempo an awful lot. And so, they’re curious about it. I’ll just say that. And I really didn’t have a good understanding of what the difference was. And so, we looked at these two areas in our data set, which we have a large data set that we’re following kids longitudinally over time, and basically, we found that in kids with ADHD, that Sluggish Cognitive Tempo and processing speed are both closely related to ADHD, but they’re not the same.

Sluggish cognitive tempo is a cognitive-emotional phenotype, that’s daydreaming, confusion, kids are staring blankly up the window, they’re kind of sluggish and unmotivated. Teachers described them as sleepy and drowsy, but just sort of underactive. And about 60% of kids with sluggish cognitive tempo will have comorbid ADHD. So that’s a big percentage.

In our study, about 40% of youth with ADHD had comorbid Sluggish Cognitive Tempo. So you’re looking at this either way, but we wanted to know whether Sluggish Cognitive Tempo and processing speed were the same thing. I’ll kind of cut to the chase because the reason I bring this up in relation to COVID is that there are some things that we can take away from our study on this.

We’ve found that there was a significant negative correlation between Sluggish Cognitive Tempo and processing speed. The higher the Sluggish Cognitive Tempo cyst of symptoms, the slower processing speed, but it was a low correlation of like .14. So, we’re not talking about the same thing. Basically, what we found was, we looked at both adaptive symptoms and academic achievement as well as ADHD, and what we found is that there wasn’t a lot of association between the Sluggish Cognitive Tempo and academic skills.

So when you think about Sluggish Cognitive Tempo, those slow-moving kids still perform well academically in general. Not the same thing for kids with slow processing speed. We know that they’re affected by academic skills. And in fact, the study that, we’ll hopefully talk about later, we got into that even more. But processing speed wasn’t associated with adaptive skills.

So what we’ve found is that there’s this double dissociation that there is a separate measurement of both processing speed and sluggish cognitive tempo and ADHD samples is warranted, that kids with this sluggish cognitive tempo had problems with adaptive skills and processing speed is associated with academic skills. Now there are students who have both, but when we’re thinking about what this means, these kids who have slow processing speed need more academic support, more focused on accommodations, but kids with sluggish cognitive tempo need more adaptive functioning daily living skills.

So, what does that mean for COVID? It really means that the kids who are at home who have sluggish cognitive tempo are probably having even more difficulties now during COVID because they’ve got these poor academic or poor adaptive functions. And so that means that they’re home, they’re having to do a lot of things around the house, and they’re also having to adapt to one week on one week off or Mondays and Wednesdays at school all of that. So they’re probably having more difficulties right now.

Kids with slow processing speed are probably more at risk for their academic skills falling through the cracks. And I am worried about kids right now because I’m not sure they’re getting as much of the basics as they need to. So kids with slow processing speed are at risk in that regard. So if you’ve got your kids who are either one, they probably are having unique challenges right now.

Dr. Sharp: That’s fascinating that they are so unique. I would have assumed that there was a lot more overlap between those two, but this distinct difference, adaptive versus academic is super interesting.

Dr. Ellen: Well, I think too. And I should have said this at the beginning is when we’re talking about the Sluggish Cognitive Tempo, we’re really talking about something that someone reports on. Parents are reporting the child seems sluggish because it’s more of a tempo, whereas processing speed, we’re measuring it as a neuropsychological trait. So they’re really are two different things. And like I said, there’s an overlap between the two, but they’re not identical. And I think that maybe the tie in here might be inattention, but we can talk about that when we get to the other study.

Dr. Sharp: Yeah. That might be a good segue because it just got me thinking about… When you describe sluggish cognitive tempo, honestly, in many ways it kind of mirrors the inattentive symptoms of ADHD. If you’re talking about just describing or reporting behaviors, that seems like there’s a lot of overlap there. Is that fair?

Dr. Ellen: Yeah, definitely. But when we have to time someone in something like coding or symbol search or word generation, that’s not necessarily the same as sort of this long behavior trait that you see hour after hour but if those sorts of skills are very important in terms of completing a math worksheet, those sorts of things.

Dr. Sharp: Yeah. So when you say inattention, can you define that for us a little bit?

Dr. Ellen: Yeah, so when we’re measuring it, we’re really looking at the actual inattentive symptoms of ADHD. We’re really talking about that actual criterion. In our research, when we’re measuring attentional skills, usually, the measure that we’re looking at is, some of the factors from the CPT is really just visual attention skills, when I’m talking and the data that we generally analyze, we’re looking at just those attentive skills in the ADHD diagnosis.

Dr. Sharp: I got you. So what did y’all find? I know, again, before we were recording, this dimension of inattention was common across several diagnoses, but maybe getting into the weeds. Could we maybe back up. You just talk through the basics of this most recent article that we’ve referenced and what that was all about, and then we’ll dive into some details?

Dr. Ellen: Yeah, sure. So we want it to look separately from these others cognitive tempo and the processing speed, but really looks like, take a deep dive into what does processing speed mean, psychiatric symptoms or academic and adaptive symptoms. So what we looked at was how processing speed and different psychiatric disorders were linked. And we predicted the odds of having impaired processing speed for different diagnoses compared to kids in our clinic with no psychiatric diagnosis.

For example, we looked at psychosis, autism spectrum, mood disorders, ADHD, and anxiety disorders. And we put this in as a stepwise regression. If you have psychosis, that’s the most significant psychiatric disorder followed by the autism spectrum, mood disorders, ADHD, and anxiety. So, it’s a hierarchical model. So the odds ratios are over and above what would be predicted by age and sex. And we put all of these diagnoses in the model so that they’re essentially accounting for comorbidity. I hope all this makes sense. Interrupt me if you need a little more explanation on the spectrum.

What’s interesting is that even though impaired processing speed is so much associated with ADHD in the literature, I found it’s not the only diagnosis that’s associated with impaired processing speed. The other areas, in addition to ADHD that were associated with impaired processing speed were psychosis, autism spectrum, and anxiety disorders. That was really interesting in and of itself.

We asked them though, what about inattention? All right, so we know those 4 diagnoses are highly associated with processing speed deficits, but we also know that it’s important to consider psychopathology as a dimension, not just these diagnoses. We did the same analysis, but this time we looked at symptoms.

Let’s get back to what you are asking about. How do we measure attention? We measure them as clusters of symptoms. And we didn’t just look at inattention. We looked at regression, depression, hyperactivity, inhibition, anxiety, social responsiveness, mania. We looked at all of those things, and the only one of all of those things that were associated with impaired processing speed was inattention. And so, it’s interesting to think that inattention is that pure attention is something that’s really important in terms of understanding the connection to a lot of different disorders, and processing speed is one way that we can measure that beyond just how well a child attempts on a CPT, for example.

Dr. Sharp: Right. Well, there are so many questions from all of that. Let me sort through and see if I can make this make sense. One seemingly random question, but something that hopefully will tie in is, I noticed in the article you defined lower processing speed as I think sub 85, is that right? Below 85?

Dr. Ellen: Yes.

Dr. Sharp: And was that just one the PSI from the WISC-V or was that from a different one?

Dr. Ellen: Yes, it was from the processing speed index from the WISC-V. And we have really struggled with this. We really struggled with how to define what processing speed weaknesses are. That’s stuff that we’re doing right now, where we’re really trying to figure out how to analyze this is to look at whether or not processing speed is something that is a distinct cutoff, that’s normatively based like an 80, one standard deviation below the mean, or if it’s important for some kids to have a processing speed of maybe 92 but have a verbal comprehension index of 115. And I’m positive, you haven’t seen kids like this. Those are the real puzzles. It’s the difference between processing speed and other factors of intelligence what trips some kids up. And I definitely think so. We are just having trouble trying to figure out how to analyze that.

Dr. Sharp: Right. I talk with parents a lot about just anecdotally sort of this gap that you’re talking about or that discrepancy, whether it’s 80 to 100 or 95 to 125, but just that kind of internal felt sense for a kid of being slower or some things being harder than others, just the idea of their inconsistencies and performance and what that must mean internally for a kid. I don’t know if anybody’s ever actually measured that or looked at that or quantified it somehow, but it seems like it’s a thing.

Dr. Ellen: It definitely does. Anybody who tests kids knows exactly what we’re talking about. It’s hard because I have seen kids who have an IQ or verbal comprehension of 140 and a processing speed of 80. Those kids are at huge risk. And those are a lot of those who I find wind up being the subject of other books that I was telling you about that there’s no motivation in that because you’ve got this incredible intellect, but no ability to actually get your thoughts on paper or do any of the tasks that you need to do to be an efficient learner. But then there are also those kids, like you said, who has a 120 and a 98, what does it mean for them?

But development likes consistency, and personalities like to be consistent, our world likes to be consistent. And when you’ve got an inconsistent profile, it’s going to come with a challenge, and some of those might be in this range that I’ve just talked to you about that deal with psychiatric kinds of issues.

One thing to think about is if it’s inattention that runs through all of these diagnoses, not that might be where we want to start in terms of intervention. And so, maybe medication at an earlier age is important or accommodations that decrease inattention, I should say.

Dr. Sharp: Right. Just for explanation’s sake, could you talk a little bit more about just the relationship between processing speed and inattention? Is processing speed the underlying piece of inattention that’s driving that or vice versa? Are there other components that are involved in inattention? I really want to suss out the relationship between those two a little bit, if we can.

Let’s take a quick break to hear from our featured partner.

The BRIEF-2 ADHD form is the latest addition to the BRIEF family of assessment instruments. Using the power of the BRIEF-2 the gold standard rating form for executive function, the BRIEF-2 ADHD form uses BRIEF-2 score and classification statistics within an evidence-based approach to predict the likelihood of ADHD and to help determine the specific subtype. It can also help evaluators rule in ADHD and rule out other explanations for observed behaviors. Please note that BRIEF-2 parent and teacher form scores are required to use this form. The BRIEF-2 ADHD form is available on PARiConnect- PAR’s an online assessment platform. You can learn more by visiting parinc.com/brief2_ADHD.

All right, let’s get back to the podcast.

Dr. Ellen: That’s such a great question. I wish I had a really great answer, but I do know they’re both multi-determined. And the way we measure them as neuropsychologists it’s pulling in a lot of different aspects of attention. So for most of the tasks we use, we have to have visual attention and auditory attention to some extent, and also motor attention and also processing speed. So we’ve gotta be able to process information visually, sometimes verbally, sometimes meteorically.

So I’m not sure which drives which. I think that’s something that’s completely open for debate. They’re probably somewhat directional, but I think if you don’t have good attention skills to begin with, it’s going to slow down your ability to process that information. And then if you’ve got slower processing speed, it’s harder for you to hold on to attention as you’re doing a task.

So, we don’t really know which comes first. We just know that there’s a relationship between the two, to be honest. Not everybody with attention problems has slow processing speed and not everybody with slow processing speed has attention problems. So, there’s an overlap, but it’s…

Dr. Sharp: Complicated.

Dr. Ellen: […] very well.

Dr. Sharp: That’s okay. That’s validating. That’s something that I’m always wrestling with and to hear that there’s not a great answer makes me feel better. So selfishly, I feel that was a great answer.

Dr. Ellen: Yeah. One of the things we can do as testing psychologists is that we can really get a good sense of where the attention is most vulnerable and where the processing speed is most vulnerable. So if kids do poorly on word retrieval, for example, or hoarding, that gives us different data points. So they have a problem with quickly retrieving words, so that’s going to be an issue or problems quickly processing information verbally versus visually. So, it can give us a sense of where it’s breaking down for a particular child.

Dr. Sharp: Yeah, I got you. So tell me, with this research, with this article that y’all pulled together, how do these results strike you? Were you surprised to find that inattention was the more common dimension across diagnoses or not so much?

Dr.Ellen: I think I would have expected given the fact that the other diagnoses were so significant that it would be more than just inattention that would be important. So it did surprise me a little bit, but also shows the question that you just asked. It’s just how much important it is that we get a good handle on what is attention and what is inattention?

The other thing too that we found in this study is that I don’t know if this is a good time to bring it into play, but we also looked at academic functioning because we wanted to get more, beyond just psychiatric functioning. How does processing speed impact academic functioning? And we want to notice if it interferes with their overall approach to their work? Is it a study skills thing? And I bring this up because I think inattention does come into play here. So we looked at where their processing speed has a general impact on academic functioning broadly, or given that processing speed influences things like study skills, and would that be occurring through inattention?

So what we found were two different things. There was a wrong association between study skills and its relationship to processing speed, and that attention was an important part of that. So processing speed has a direct effect on study skills as well as an indirect impact on inattention. And so that may be why treating attention doesn’t fully resolve the burden of the slow processing speed has on academic difficulties. Do you know what I mean?

So, if you’ll hear kids whose attention is better because they’re on medication but they’re still not getting the work done, and that’s because attention, we can treat that but it doesn’t treat issues with processing speed.  But then we looked at if reading and math are related to processing speed as well, but with reading, it is mostly mediated by working memory in general cognitive ability, but for math, there is a direct effect of processing speed on math.

If I was back a little bit, processing speed has a direct impact on reading but when we look at the cognitive ability and working memory, no, it really doesn’t. But math has some direct effect on… processing speed has some direct effect on math. And that’s why I think we see sometimes these kids with slow processing speed, who aren’t doing that well in math and they don’t really have dyscalculia. They just don’t seem to hold it together. And I think that might be because math demands a lot more attention to detail.

Dr. Sharp: That’s fascinating. I’m fresh off of the second edition of Overcoming Dyslexia. It just came out. I guess what I took from that though, is there still that dual deficit model of dyslexia with processing speed playing a big role. I may be overreaching here, but are you saying that what y’all found was not necessarily true?

Dr. Ellen: So we were looking at just decoding skills. And we were really just looking at the effect of processing speed on reading decoding.I go as far as to say that… I don’t think it goes against anything that is presented in the research on dyslexia.

Dr. Sharp: Okay.

Dr. Ellen: In fact, we’re looking at… I mean, processing speed is still a big factor in dyslexia. And we’re not, I guess maybe this might be one difference here, we’re not just looking at kids with dyslexia on this study, we’re looking at all kids. And I think that’s where when we look just overall does processing speed have an effect on reading regardless, no diagnosis or not, some kids with dyslexia in the mix, no, but it does have an impact just on general mathematical ability.

Dr. Sharp: That’s wild. Can I dial it back just a bit and ask you to clarify something? When you say that processing speed had an impact on study skills, what do you mean by study skills?

Dr. Ellen: Our measure for study skills is we had teachers fill out the BASC Study Skills on the BASC. And on the BASC, there is a study skills scale which reflects those skills that are conducive to a strong academic performance including organizational skills and study habits. It’s got some internal consistency and reliability to it. We use the teacher ratings because they’re independent of the parent-rated inattention symptoms that we were used in the analysis to look at inattention. And we hope the teachers have a unique insight into that construct. So that’s how we were measuring study skills. So it’s really talking about the association, but the teacher evaluated the study skill scale and their academic functioning in general. Does that answer your question?

Dr. Sharp: It does. Yes, thanks for clarifying. I’m guessing people are probably wondering what study skills we’re talking about here.

Dr. Ellen: Yeah. And even though parents report on studying study skills, they’re also reporting on inattention. So those two things are correlated and we wanted something that wasn’t already biased.

Dr. Sharp: That makes sense. So then you found, like you said, the processing speed has a direct effect on general math ability as well independent of working memory or overall cognitive ability?

Dr. Ellen: Exactly. That sort of surprised us. And I guess this is a long-winded answer to when you asked before whether there are some things that surprised you? That did surprise us. Math seems to have, now even untimed math. So this is using the numerical operations of tests from the WIAT. And even in that untimed test, that processing speed still had an effect on a child’s ability to be able to do that, because that was computational math.

Dr. Sharp: What do you make of that?

Dr. Ellen: I think that math requires a lot of different steps and it requires you to do steps quickly. If you’re doing a long division problem, you’ve got to remember where you are in the process. And when you slow down the process, you’re more apt to make mistakes. And then also you’re juggling a lot of things in your head at the same time. You’re juggling multiplication facts and additional facts and procedural sorts of things. We don’t really know, but that’s our hypothesis about that. We’re doing the same thing with reading. I’m curious as to whether reading comprehension is also affected in the same way. So we’ll see.

Dr. Sharp: Yeah, there’s so much to dig into here. I love that y’all are just zoning in on processing speed. It’s such an important piece of the assessment that we do. And it seems like it’s implicated in so many different things as well. Let’s see. What else from this article feels important to highlight? Eventually, I think we can move on to just, what do you do with this? how does this translate to intervention like you had mentioned, knowing where to intervene or what to do? So two questions wrapped up in there.

Dr. Ellen: The article was published in the European Child and Adolescent Psychiatry Journal. And they did an editorial on processing speed in general. It’s really this cross disorder phenomenon that has a lot of clinical value and we need to be looking at it much more. I find that clinically when I see a child with slow processing speed, my antennae are up because I know this is sort of a child.

If it’s a child with dyslexia and slow processing speed, I’m probably going to advocate for more Orton-Gillingham, for example, than less. And they are going to be the kinds of kids that are going to take longer to respond to the treatment. And the moment to moment, hour that that teacher has or tutor, but also over the course of a year or two, and that they are at risk for other psychiatric issues that need to be monitored.

I’ve been at this long enough where I have seen kids who presented to me at age 7 with very weak processing speed, who are now at 27, who just haven’t launched that well or who have had psychosis, and so I think our data backed up what I’ve been seeing in my clinical practice over the last 20 years, that when kids have this really weak processing speed, that they are at risk for some significant sorts of psychiatric and learning challenges that go beyond the diagnosis of dyslexia or ADHD.

Dr. Sharp: Right. Well, that makes me wonder, and I wanted to ask this question earlier. I’m glad it’s come back around. It’s just like we talked about with inattention, just the directionality here. Is it the “psychosis” that’s driving the low processing speed or is low processing speed somehow contributing to psychosis? That connection feels more tenuous to me,  but that’s the big question. Or is it anxiety driving low processing speed or vice versa?

Dr. Ellen: Yeah, that’s exactly right. We don’t know that. But is it slow processing speed a marker? The Real marker of psychosis is that we need to pay attention to that. Of course, the vast majority of people with slow processing speed do not go on to develop psychosis, but it’s highly related. And so, is it something that we need to pay attention to at an early age, and just are these the kids who need to be monitored more closely than we would have thought before?

15 years ago, we would have just said it is a slow processing speed. Yeah, he just doesn’t take notes very quickly, so fairly fine. I’ll get him a typewriter and get him teacher-made notes and he’ll be fine. And what we’ve found is that that’s not the case. It’s an important data point. And hopefully, we’ll be able to answer that question you just posed or somebody’s will.

Dr. Sharp: Right. So is it at the point yet where you would say, okay, so let’s say we evaluate a kid at 6 or 7 or 8 and they show up with pretty low processing speed, is it at the point yet where you would say, we have this ADHD thing, but let’s also look out down the road for anxiety or pay extra careful attention for a psychotic issue or something else, or is it not quite strong enough yet?

Dr. Ellen: I actually do. I guess it depends on the 6-year-old. Sometimes they do grow out of it. And we do find kids who are… And I think this is the case. For some kids with slow processing speed, they might just have a white matter growth in their brain that’s just very slow that catches up. And I think a lot of this might have to do with white matter. And so there are some kids like that. But I do tell parents that it doesn’t seem like very much, but it is something that holds kids back in school and can make a lot of the learning that we do or we ask kids to do these days, very difficult for them. And so, I kind of set parents up to know that we need to take care of this kid and this gives us a sense of that.

Your question before though, the chicken in the egg question, I do wonder in some ways whether or not our current academic environment leads kids with slow processing speed, maybe some that might even be genetically disposed to psychosis to go on to develop psychosis because they’ve been working at capacity stuff for like 15 years, struggling to keep up in an environment that is for a lot of kids, overly taxing. I wonder if that is just one of those you sort of the stress model. There are some kids that are just being stressed in that way. And we know that psychosis does have a pretty strong genetic component, but not everybody goes on to develop it. So, I need to think about it.

Dr. Sharp: That is fascinating. I just have to ask. Is that something that’s come up in the literature, or is this something that you’re thinking off the top of your head because that’s fascinating that our school system is stressing kids enough? If they have low processing speed that might somehow play into that.

Dr. Ellen: No, I don’t find that in the literature we ever really talk about what kids have to really do. We talk about them in terms of diagnoses and symptoms. And then it’s like, well, what about the homework that they were asked to do all last year in 5th grade that was really more appropriate for an 8th-grader? What does that have on them? And I’m not aware of any literature that looks at that because I think it takes us looking in the mirror, all of us in certain ways, parents, teachers, as developers, everything we’ve got to say, what are we doing? I’m not sure if we’re ready to do that, although COVID gives us sort of an opportunity to rethink these sorts of things.

Dr. Sharp: That’s true. Well, I wonder if we could talk a little bit about intervention or takeaways from the article, where you go with the knowledge that you gained from this research or things that we might be able to put into play as we’re working with kids or things to just keep in mind as we’re assessing kids.

Dr. Ellen: I think we talked about some of the more important ones that I think about, which are these kids are really in need of the maximum amount of services. There’s been another study in the last two years. I mentioned that I was in Prague and there is a group of researchers over there, it’s very interesting in time perception. And there’s been a lot of user data on how people with ADHD have trouble with time perception, meaning just perceiving how long time goes by, what do 5 minutes feel like versus 15 minutes? And here’s why processing speed particularly has trouble with time perception.

And so one of the things I think that we can do with these kids is giving them a better sense of time. We spend an awful lot of time getting them planners, organizational tools, calendars, but if you don’t know what time feels like you can’t do that. So it makes it really, really tough. So teaching them about time, having them wear an analog clock, having a clock on the wall that shows them how much time is going by, time a lot of things that they have to undertake. Here’s the stopwatch. Here’s my phone. Time how long it takes to brush your teeth and how long it takes us to get to school in the morning and different from in the afternoon. What do those things feel like?

Dr. Sharp: I love those ideas. And you specifically mentioned an analog watch. Is that purposeful so that they can have a visual cue of time passing with the set the hands on the watch?

Dr. Ellen: Yeah, because you see how long like a quarter of an hour. It’s multisensory in a way that a digital clock just isn’t. A digital clock is just numbers on a clock, but an analog clock really isn’t. And even to change your phone, if you told adolescents to just change their phone to that, instead of seeing it when their phone comes on to see that clock, and I’m amazed at how many kids just do not have that really good sense of time. Infusing that into a family’s life and into the curriculum is important. And then also doing things like teaching kids about their profile. What do they need to do to be successful? What do they need from the environment? And they’d be taught to help other people to slow down, or if you’re talking too fast or those things that are important.

Dr. Sharp: That self-advocacy piece seems important like you just said. So, how old do you think kids have to be to get to gain that capability, to have enough understanding, to be able to communicate that to people? Do you have a sense of that?

Dr. Ellen: I really think kids can start at a pretty young age, like 7-8 years old now, I am not a believer that kids need to be their biggest advocate because let’s face it, we have trouble as adults advocating for ourselves. We’re not getting whatever the grade, the pay raise we want, or whatever. The hardest thing that we have to do is to advocate for ourselves, yet we ask kids to do this all the time. I hear this all the time, especially from teachers.

But that being said, I think we need to think about advocacy from a child with a small a, which is like what feels right to me right now? Did I not hear that? It’s more about self-awareness. That’s the key. And you can instill that even in a 6 and a 7 and 8 years old to know that, hey, I didn’t hear what you just said. Can you repeat yourself? That’s how advocacy starts with those small bits of knowledge and awareness of what you’re feeling, and what’s going on around you.

Dr. Sharp: Sure. I love that. I love these actionable ideas. It’s always nice to take things away. So I’m curious, as we start to wrap up, where are you headed in the future? You’ve got this book that I’m guessing you’re going to be working on a large part of the time. What else are you working on?

Dr. Ellen: I think right now, getting this book done and also getting through COVID, to be honest. I am an associate director and we also have a wonderful director of the learning and emotional assessment program at Massachusetts General, but it’s been really tough to figure out where we’re all going to be at the end of COVID and with virtual testing. And I think right now my mind is really on the next 3 or 4 months. Let’s see how we can best evaluate kids during this time.

And then I think after that, we’re really interested in looking at this construct more broadly. And we mentioned this idea about kids who are just low, a certain cutoff, or kids who have a discrepancy within their profile, something that’s really interesting to me. And eventually, I would love to see treatments come into play. I wish that pharmaceutical companies were interested in studying this outside of ADHD. And maybe if we’re talking about a different type of attentional factor, that might respond to medication. I don’t really know. That’s not my area. But I would love for psycho pharm to get involved in this because these are not always the kids who respond very well to medication, to be honest.

Dr. Sharp: Right.  Well, at least if they got involved, there would be more money presumably dedicated to looking into this stuff. That’s so true. Let me ask. I don’t think I’ve talked with anyone like a large-scale hospital or training facility like y’all have, how are you handling assessment right now with COVID?

Dr. Ellen: The clinic has been mostly a hybrid of virtual and in-person testing. Just to keep the numbers down in the waiting room and the offices as empty as possible at least at the same time, we are shifting now to all virtual for January and February. So it’s really tough because it sounds like things are going to get worse before they get better, but it’s hard to plan.

In my private practice, I’ve been seeing just a few people individually in person, usually not a hybrid but because when you’re in individual practice, you can have things, you don’t have people in the waiting room or coming into a hospital setting where they’re being exposed even more, but even I think I’m just going to go all virtual or just tell people to wait till the spring.

Dr. Sharp: Well, let me ask you what might be a hard question, which is, as such a processing speed proponent and expert, how are you assessing processing speed remotely?

Dr. Ellen: This is such a good question. This is why I like to see my kids in person. We’ve done a few things where we’ve sent the WISC protocol to a tutor and then they’ve administered it with us watching, and then they put it right back in the envelope and sent it. We could also get some things from being able to look at word retrieval, verbal fluency. There are some other tasks that you do get even a sense of processing speed on things like we can’t do block design, but most other tests of non-verbal functioning on the WISC are timed. I’m not sure how valid they are given that it’s on the screen, but still, we are getting a little bit of a sense of that. My personal preference would be to just at this point now just wait and see kids in person.

Dr. Sharp: Yeah.

Dr. Ellen: Because there’s going to be the vaccine enrollment and get back to normal.

Dr. Sharp: Right, that’s exactly what’s going to happen.

Dr. Ellen: It’s a little bit like yes.

Dr. Sharp: Yes, Oh my Gosh. I think that’s where we’re all at.  There’s some hope. There’s a light at the end of a still pretty long tunnel, but I think it’s maybe there now.

Thanks for talking through all this. I think we touched on a lot of different things. We got to dive deeper into a couple of different pieces as well. And it’s just always a pleasure to hear what’s on your mind and what you’ve got going on. I know you’ve got a lot of irons in the fire, so thanks for coming and sharing those.

Dr. Ellen: Yeah, it’s been really good talking to you.

Dr. Sharp: Sure. For people who are interested, we mentioned a lot of resources, and I’ll link to all the articles and books that I can hear in the show notes. The one thing I don’t know that I grabbed was the group out of Prague that you’ve been working with who’s been doing this research on time and ADHD.

Dr. Ellen: I can get a few of their articles if you want them to link to them.

Dr. Sharp: Yeah, great.

Dr. Ellen: Yeah. They do publish in English, but they don’t have any books or anything like that in English.

Dr. Sharp: That’s fair. We’ll take it.

Well, thanks again, Ellen. And like you said at the beginning, if there’s a third time around when the next book comes out, I would be happy to have you back. So I really appreciate it.

Dr. Ellen: Thank you so much for having me.

Dr. Sharp: All right, y’all. Thank you as always for tuning in to my conversation with Dr.Ellen Braaten. I hope that you took some things away from that, as I did. I know that I’m excited for that book coming up on kids with low motivation. So we’ll keep an eye out for that. Maybe Ellen will come back to talk about it when it comes out. There are plenty of links in the show notes, including to Ellen’s existing books and most recent article, and a couple of other things, so check those out.

Like I said, in the beginning, if you are launching your practice or thinking about launching your practice, the Beginner Practice Mastermind group is starting up in March 2021. This is a group that is meant to give you some accountability and support as you go through those beginning phases of launching your practice. It’s a small group coaching experience and there will be no more than 6 psychologists in there to help you out, all at that beginner phase. You can get more information and schedule a pre-group call at thetestingpsychologist.com/beginner.

I will be back on Thursday with another EHR review. I think the one coming up this time is a Therapy Appointment. I think it was a good one just as a teaser. So, you want to tune into that one if you’re looking for a new EHR for your practice. In the meantime, take care. I will talk to you soon.

The information contained in this podcast and on The Testing Psychologist website is intended for informational and educational purposes only. Nothing in this podcast or on the website is intended to be a substitute for professional, psychological, psychiatric, or medical advice, diagnosis, or treatment.

Please note that no doctor-patient relationship is formed here, and similarly, no supervisory or consultative relationship is formed between the host or guests of this podcast and listeners of this podcast. If you need the qualified advice of any mental health practitioner or medical provider, please seek one in your area. Similarly, if you need supervision on clinical matters, please find a supervisor with expertise that fits your needs.

Click here to listen instead!

[00:00:00] Dr. Sharp: Hello, everyone. Welcome to the Testing Psychologist Podcast, the podcast where we talk all about the business and practice of psychological and neuropsychological assessment. I’m your host, Dr. Jeremy Sharp, licensed psychologist group practice owner, and private practice coach.

This episode is brought to you by PAR. The BRIEF®2 ADHD form uses BRIEF®2 scores to predict the likelihood of ADHD. It’s available on PARiConnect– PAR’s online assessment platform. Learn more at parinc.com.

All right, y’all, welcome back. Glad to have you for this first businessy episode of 2021. Hope you all had a good new year.

So I took a break from the EHR review series last week to do the best of 2020 episode, but I am back. Today I’m reviewing an EHR called TheraNest. Now, many of you may have heard of TheraNest. It’s been around for a while. It’s fairly popular.

[00:01:00] And  I’m going to dive into it. For the purposes of the podcast, I’m going to go over the pros and cons. These are relatively short episodes with the intent really being that you go check out the accompanying YouTube video where I spend about 40 to 45 minutes walking through the EHR, and diving in, really getting hands-on through the lens of a testing psychologist to figure out how it might work for us. So definitely go check out the YouTube video that’s embedded in the blog post here. So to the website to check that out. The link is also in the show notes for the podcast.

Before we get to the review, I want to extend an invitation to any of you who are launching your practice in 2021 to check out the beginner practice mastermind group. So this is a group coaching experience specifically for folks who are about [00:02:00] 6 months away from launching their practice all the way to 6 months on the other side of launching their practice.

So it’s a group of 6 psychologists.  I’m the facilitator and we just dive into all those things that come up in the beginning around how to set up your schedule, how to find office space, how to set up a business, how to purchase materials, those sorts of questions that so many of us wrestle with when we’re just getting started. If that sounds interesting to you, you can go to thetestingpsychologist.com/beginner and apply for a pre-group call and we’ll talk about whether it’s a good fit or not. So, check that out if you’re interested.

And without further ado, here is my review of [00:03:00] TheraNest.

Okay. We are back and today, like I said, I am talking all about TheraNest. Now, this episode is part of a series. I’m doing about 6 EHR reviews, looking through the lens of a testing psychologist. So, if you haven’t seen the other episodes, please go back and check those out or look forward to them if you are listening in real-time here. There will be some others after this. But as far as TheraNest is concerned, let’s talk about that.

So TheraNest is an EHR. It’s been around again for several years. It’s one of the, I think, more popular EHR’s. So things that I liked about TheraNest.

I’m just going to jump right into the pros and cons and then give a little bit of a verdict on what I thought about TheraNest.

So pros, fairness is relatively easy to look at. The layout I think is [00:04:00] pretty clear, the colors are nice, it’s generally aesthetically pleasing. So, this is something I can’t help but notice right off the bat when I’m looking at any webpage or software. So if the aesthetics aren’t on point, it’s a big problem for me. But aesthetics are good.  That was not a sticking point for me here. 

[00:04:26] Little details that jumped out as pros with TheraNest, one is you can change the colors of the appointments, which is not always available in other EHR’s. One thing that really jumped out to me is that there is a ton of detailed demographic info and rather… there are many demographic choices that you can pick regarding gender identity for pronouns, that sort of thing. I noticed that TheraNest has a lot more choices, particularly around gender identity [00:05:00] compared to other EHR’s which was really nice.  That, again, jumped out right away when I was entering client information.

[00:05:09] Another little feature that really stuck with me is, they have a box in the contact information for the client that says whether or not you can mention the practice name when you contact the client. I thought that was a nice little touch around protecting client privacy.

And then one of the other things that I noticed that I really liked is that the appointments can be moved around just by clicking and dragging. So this seems like a small issue, but for those of us who might reschedule appointments fairly frequently,  this can be a really nice feature.

So those are just a few pros that jumped out about TheraNest. Now, you’ll probably notice that most of those are just kinds of nuances and little details which do go a long way. You can tell that [00:06:00] that TheraNest put a lot of attention to the details. And just those little pieces that many other EHR’s haven’t really thought about. And that was great. It definitely jumped out how much attention they put to detail, particularly around client identity and privacy.

[00:06:19] Let’s take a quick break to hear from our featured partner. The BRIEF®2 ADHD form is the latest addition to the BRIEF family of assessment instruments. Using the power of The BRIEF®2, the gold standard rating forum for executive function, the BRIEF®2 ADHD form uses BRIEF®2 Scores and classifications statistics within an evidence-based approach to predict the likelihood of ADHD and to help determine the specific subtype. It can also help evaluators rule in ADHD and rule out other explanations for observed behaviors. Please note that BRIEF2 parent and teacher form scores are required to use this form. The BRIEF2 ADHD form is available on PARiCconnect, [00:07:00] PAR’s online assessment platform. You can learn more by visiting PARinc.com/Brief2_ ADHD.

[00:07:11] All right, let’s get back to the podcast. Now, the list of cons is a little more heavy with TheraNest.

So, when I really got into TheraNest and you’ll see this on the review video, I found that the navigation for TheraNest was really not intuitive at all to me. I had a really hard time finding the information that I needed or finding how to execute certain actions in TheraNest. For example, I couldn’t find my agenda like a to-do list for which notes I needed to write for the session.  I [00:08:00] just had a hard time sort of finding what I needed in TheraNest when I was clicking around, like. Like I said, one other thing that jumped out was just, there was no obvious way to write custom notes or create custom notes for testing appointments. This is a big deal. So, as far as I could find, the only way to write a note for an appointment was to use their templates which were customizable in a sense but certainly nothing in the vein of what we need for testing appointments where you document the time and the tests that you use and so forth. So that was really challenging.

And then another thing that jumped out is that billing codes were easy to enter but really hard to figure out how to apply 2 appointments. So, [00:09:00] the way that I was doing it and granted this is just a basic look, this is me just going in with fresh eyes, no familiarity with the EHR  during these reviews, so I just went in and tried to conduct these general tasks that we do; scheduling appointments, billing appointments, entering client and phone and so forth. But I could not find an easy way to actually choose which CPT code goes with which appointment. It was a cumbersome process and I really did not like that. 

[00:09:37] Thus far TheraNest is the only EHR where I had to dive into the help section and actually search through some articles to try and figure out how to complete the actions that I was trying to complete. And for me, that’s just a huge turnoff. I mean, it could be a [00:10:00] great EHR, but the fact that I as a new user couldn’t go in and just sort of intuitively navigate around the system and figure out how to do the major primary tasks that I have to do, that was really a pretty big negative for me. So I know there’s always an argument to be made there like, Jeremy, you have to read the instructions, like give us some time and then see how you do when you’re familiar with it. But like I’ve said from the beginning of these EHR reviews, the whole point is to go in with just a blank slate, check the initial impressions of an EHR, and figure out how intuitive they are to do what we need to do. TheraNest really stood out as being not intuitive, really at all for me. So, it could be me. Your mileage may vary, but [00:11:00] that was a takeaway for me from TheraNest

So the overall verdict is it would take a lot of persuasion and coaching for me to consider using this as an EHR. There were a couple of kind of deal breakers as far as being friendly to testing. One, of course, is not being able to enter billing codes easily and or assign them to appointments rather. And the other was just not being able to write custom or create custom testing notes for testing appointments. So far as testing specific features, that was pretty challenging for me. So as far as I’m concerned, TheraNest to be off my list. But again, your experience might be different and I would encourage you to check out the YouTube video and really dive into some details with TheraNest and [00:12:00] see if it might be a better fit for you.

I think that one of the pieces that I didn’t mention just to throw this out there is that the pricing for TheraNest is really nice. So TheraNest is one of those EHRs is that prices by the client rather than by the clinician. So that was nice. You get a little bit of a pricing break depending on how many active clients you have.

So just want to close on a positive note. But again, generally speaking, I probably won’t revisit TheraNest as a potential EHR, nor would I recommend it to testing folks moving forward unless they make some major changes.

So I hope that [00:13:00] this was informative for you. Like I said, watch the YouTube video to gather some of your own impressions. This one I will acknowledge was a little challenging simply because I really did have to navigate around quite a bit to figure out how to work the EHR. So, moves a little bit slowly in parts, but I think that’s an accurate representation of what your experience might be like. 

Like I said at the beginning, if you are interested in a group coaching experience where you are hanging out with other psychologists and being held accountable for getting your practice started, the beginner practice mastermind is launching in a couple of months for our spring cohort. So you can go to thetestingpsychologists.com/beginner and check that out and see if it might be a good fit for you.

All right, y’all hope your 2021 is off to a fantastic start [00:14:00] and looking forward to another episode on Monday around processing speed. I’ve got Ellen Braaton coming back. She’s talking about processing speed and its relationship to different psychiatric disorders or psychopathology, but we get into a number of topics that I think are quite interesting. So she’ll be back Monday. And then the following Thursday, we’ve got another EHR review of TherapyAppointment. So until next time, take care.

The information contained in this podcast and on The Testing Psychologist website are intended for informational and educational purposes only. Nothing in this podcast or on the website is intended to be a substitute for [00:15:00] professional, psychological, psychiatric, or medical advice, diagnosis or treatment. Please note that no doctor-patient relationship is formed here. And similarly, no supervisory or consultative relationship is formed between the host or guests of this podcast and listeners of this podcast. If you need the qualified advice of any mental health practitioner or medical provider, please seek one in your area. Similarly, if you need supervision on clinical matters, please find a supervisor with an expertise that fits your needs.

Click here to listen instead!

Dr. Sharp: [00:00:00] Hello, everyone. Welcome to The Testing Psychologist podcast, the podcast where we talk all about the business and practice of psychological and neuropsychological assessment. I’m your host, Dr. Jeremy Sharp, licensed psychologist group practice owner, and private practice coach.

All right, welcome back to The Testing Psychologist, and welcome to 2021. I am ready to get started with 2021, and this episode is launching the year with a bang. So my guest today is someone that I am guessing many of you have heard of, or know of.  Dr. Yossi Ben-Porath is a professor of psychological sciences at Kent State University and a board-certified clinical psychologist. He did his doctoral training at the University of Minnesota. He’s been involved extensively in MMPI research and development for the past 35 years. So he is the co-developer of the MMPI -2- RF, MMPI -A- RF, and  MMPI -3. He’s also the co-author of numerous test manuals, books, book chapters, and articles on the MMPI.

[00:01:12] Yossi served as the editor-in-chief of the journal psychological assessment, and he is a member of the APA’s committee on psychological tests and assessment. His clinical practice involves supervision of assessment at Kent state’s psychological clinic and consultation to agencies that screen candidates for public safety positions. He also provides consultation and expert witness services in forensic cases.

So like I said, many of you have probably heard of Yossi. He is relatively active in The Testing Psychologist Facebook community as well. So I am just so grateful to be able to sit down with him and really dig into his experience with the MMPI instruments over the years. So we go way back, we talk about how he got involved in the MMPI and his progression from a grad student assistant all the way up to now a co-developer of MMPI instruments.  We talk a lot about the transition from the MMPI -2 to the 2- RF, and then to the MMPI-3, what that was like both from a development perspective, but also just from a professional standpoint and what that was like to have such a big transition and encounter some criticism around that. And we talk about a lot of the updates to the MMPI -3 and how they continue to improve on the family of instruments. So, if you utilize these instruments at all, I think this is a great episode. You’re going to take a lot away from it, and we just had a fantastic conversation.

So without further ado, I bring you my conversation with Dr. Yossi Ben- Porath.

[00:03:24] Dr. Sharp: Yossi, welcome to the podcast. 

[00:03:26] Dr. Yossi: Thank you. I’m very happy to be here. 

[00:03:29] Dr. Sharp: Yes, I am very grateful that you are here. This is one of those moments where I’m interviewing someone on psych. Wow, this is really happening. So, thanks for the time and for the interest here. I’m excited to talk with you.

[00:03:42] Dr. Yossi: Well, I appreciate the opportunity. Thank you, Jeremy. 

[00:03:45] Dr. Sharp: Yeah, I know we have a lot to talk about. There’s the MMPI-3. That’s the big topic or agenda today. But I would love to start with a little bit of your origin story as it were if you’ll be willing to talk about it. I’m just so curious, how does one even get into something like this? Can you go way back and think, how did this even start? 

[00:04:15] Dr. Yossi: Well, you’re correct in referencing way back because it does go back a while. And when I was preparing for this conversation, I was thinking back in fact and ask myself, when was the first time I ever heard about the MMPI and it was in my first year as an undergraduate. I was 22 years old at the time. I’m from Israel originally. And so after high school, I did 3 years of military service and then took a year off to just clear my head, and so started college at the age of 22 and the year was 1981. So I guess everyone can do their math now and figure out my age if that’s of interest, but in any event, it was 39 years ago.

And the reason I remember I think, first hearing about or first awareness of the MMPI is that our textbooks were in English. We didn’t have Hebrew textbooks, and I  was fairly fluent in English because I’d spent some years in the United States as a child.  My classmates knew this. And I remember one day, one of my classmates coming up to me and asking me, what does this word inventory mean? That was in our introductory textbook and it was in the context of the MMPI, and so I explained to her that basically you’re taking inventory of someone’s personality and translated that then. And that is my earliest recollection of the MMPI. I didn’t really think about it as much for the remainder of my undergraduate years. When I first started as a psych major, it was with the intention of becoming an industrial-organizational psychologist. I was thinking that would be an interesting area, and I had been planning to go to graduate school. This was at the University of Haifa Israel, by the way, I should have mentioned.

And so in my last semester of undergrad, I took an abnormal psychology class and the professor who taught that class was a brand new faculty member who had just arrived with his Ph.D. fresh from the University of Minnesota where he had actually worked with Auke Tellegen who of course is my coauthor on the MMPI -2- RF and an MMPI 3. But now we’re in 1984 and I had already decided that I was going to take a year off before applying to a  graduate school and some research experience that would be helpful with my application. And I just really fell in love with clinical psychology and in the field of abnormal psychology and psychopathology in that class that I took with Michelle Maguire was the name of the professor, someone I still work with and also have a now very long-term friendship with. And I started working with Moshay who as I said, was a student of Auke Tellegen, and what we actually worked on as a research project was translating a psychological test into Hebrew, but it was not the MMPI, it was Tellegen’s multidimensional personality questionnaire, which is a normal personality inventory that is not used at this point in applied settings, so most people are not familiar with it, but in any event, that was the personality measure that I cut my teeth on.

We did the translation and eventually wrote up a publication. But the other thing that happened was that Moshay made some introductions for me, wrote my letter of recommendation and I wound up at the University of Minnesota in 1985 as a graduate student in the clinical training program. My plan had been to work with  Tellegen which I did, and of course, still do till this day. But I wasn’t a US citizen at the time which made me ineligible for many of the training, scholarships, and fellowships that were available but required US citizenship and Tellegen didn’t have search funding.

[00:09:12] So I was kind of poking around and looking into possibilities of working as a research assistant. One of my professors, still at the University of Haifa, probably gave me the best piece of advice any professor ever gave me, which was if you’re planning to go to the US and go to graduate school, learn how to run statistical analyses on a computer. We’re talking about the early 1980s here. That meant computer mainframes, not on your laptop or even a desktop, and that will be a very good skill set for you to have available.

And as luck would have it, right at that time in my second quarter, they were quarters back then, when we took the MMPI  class which was taught by James Neal Butcher, that was just when the data were starting to come in for the MMPI-2 revision that was still under construction at that point. And he happened to ask me whether I knew how to run statistical analysis on a computer because he needed someone to work as a research assistant with the data that were coming in, and I said, sure I do, and within the next few weeks I was processing the data that were coming in for the MMPI-2, started working with James butcher, not just as a research assistant but he wound up being my advisor and working with him on a number of the MMPI-2 projects, among other the construction of the content scales of the MMPI-2.

And at the same time, Tellegen was also involved, he was another member of the MMPI-2 development project. And so with him, I worked on developing the VRIN and TRIN scales of the MMPI -2 and the uniform T scores that have been used since the MMPI -2. And that’s basically how I wound up at that point immersed in the MMPI-2 world. And then when I got my first job and I’ve only been in one academic position since completing my graduate studies, it was at Kent State where Jack Wieland another, of course, MMPI -2 legend if you will, was another member of the development committee. He was on the faculty here at Kent. And so when I came to Kent, I continued to still work with James Butcher with Tellegen on some projects but also began working with Jack Graham. And so, now we’re talking about the early 90s. That’s essentially how I wound up working on the MMPI.

[00:12:12] Dr. Sharp: And then you’re in it from that point?

[00:12:15] Dr. Yossi: I have been in ever since.

[00:12:17] Dr. Sharp: That’s wow. I know that a lot of us have the experience especially in grad school of stumbling into projects that may or may not be a good fit and then we realized, this actually does fit. Did you like statistics? Did you like running all this data or was it one of those things where you got into it and then it grew on you over the years? 

[00:12:47] Dr. Yossi:  I liked running the statistics but I’ve always viewed it as a means to an end, not an activity that I enjoy in itself. What I was really fascinated by was the notion of assessing and measuring and quantifying psychological characteristics and variables. And obviously, the statistics are necessary both when you’re constructing measures, but also when you’re studying them and evaluating the utility, validity, and so forth. I started to become interested in that part while still working with Alma Gore and MPQ at the University of Haifa. And then having the opportunity to work on the MMPI, actually get paid for it as a graduate student, and then continue to work in scale development and validation. I just learned that that really is what I have a passion for in terms of my research interests.

[00:13:52] Dr. Sharp: Sure. And at this point in your career, I’m just personally curious, I think we all know you for the MMPI series, right? Do you work on anything else at this point or is the MMPI kind of all-consuming in your research?

[00:14:09]Dr. Yossi Ben- Porath: I’m a one-trick pony, Jeremy. As far as my own research is almost all MMPI. I do a lot of work and get exposed to a lot of work and assessment because I’ve been for the past 6 years the editor of a psychological assessment, the APA journal psychological assessment. So in that capacity, I have an opportunity to read all of the submissions that come in. I can’t say that I read every one of them to the very last letter because I have a very Productive team of associate editors who do a lot of the work with journalists as well, but it does give me an opportunity to get exposed to a very broad range of work in the area of assessment which I think is helpful to me in my own work. 

[00:15:06] As an  MMPI researcher, I also get exposed and get involved with a very broad range of work that assessment psychologists do because the MMPI is such a versatile instrument. Obviously, some of the work that I do is in the area of psychopathology working with mental health populations. But I’ve also worked and I’ve been very fortunate to have collaborators in the area of health psychology and use of the MMPI with medical patients, for example, in pre-surgical evaluations. We’ve done a lot of work in that area.

And also in the forensic area, I myself have done a lot of forensic work evaluations in the criminal court area, but also I’ve done some research on uses of the MMPI in forensic assessments and also with psychologists who work in the police and public safety domain. So although it’s one test,  it’s really many different applications. And I think that has helped keep me interested in a broad range of various psychology and assessment psychology. 

[00:16:26] Dr. Sharp:  Yeah, sure. When you talk about it in those contexts I suppose that’s true. There are so many applications and different ways that you can use it. It’s hard to get bored I would imagine it.

[00:16:41]Dr. Yossi: This hasn’t bored me. That’s for sure. 

[00:16:43] Dr. Sharp:  That’s great. I know people are just so curious about the new version, and I think we can probably dive into that.

[00:16:56] Dr. Yossi: Sure. 

[00:16:57] Dr. Jeremy Sharp: If you’re ready to do that. But I have just a general question that I think will hopefully lead us in a productive direction, which is, when you go from one version of a measure to another, how do you know when an update is needed? That’s a very naive and general question, but hopefully, it takes us somewhere.

[00:17:23] Dr. Yossi: No, it’s a great question actually. And I can tell you specifically, describe the evolution of, at least my thinking about this. So, as I mentioned, when I first started out as an assistant professor at Kent State, I was working with Jack Graham, and the hot topic back then when the MMPI – 2 first came out, there was some controversy about whether the coat type literature that had been developed with the original MMPI or empirical correlates that were the foundation for interpreting the Kotex. Could that be applied to the MMPI -2 because the norms have changed, and as a result, someone who may have been classified with one code type with the old norms may not actually produce the same code type with the new norms? And that was a fair amount of back and forth about that with some concern expressed that we addressed with empirical studies that show that yes, you actually could apply the MMPI literature to interpreting code types on the MMPI 2.

But what Jack Graham and I did was to initiate the first large-scale new study of empirical codes of the now MMPI-2, collecting some data at a community mental health center in Akron, Ohio, Portage Path Behavioral Health, where we collected a very large sample data from a large sample of outpatients who had received services at this community mental health center were tested with the MMPI -2 on admission as part of the intake process, and when we obtained information from the intake workers and then later from the therapists who saw these individuals, but before they actually were exposed to their MMPI so that we would not have criteria and contempt contamination.

And then, with this data set, we did a very large-scale analysis of the correlates of the code types but also the individual clinical scales. This wound up getting published as a book that we authored along with our post-doc at the time, John McNulty. And I can tell you that when you do that kind of work and again, do the statistical analysis and go underneath the hood as I got to do, what I discovered was that the clinical scales, some of the Kotex made good sense. And some of the correlates of the clinical scales made good sense and others not so much were very counter to what you would expect on the basis of the constructs that were to be assessed by the scales. A really good example of that is the PD scale 4. What we found was that the things that you would expect with that we would be correlated in a mental health sample with higher scores on scale-4, things such as, acting out behaviors, substance abuse, possible involvement with the criminal justice system[00:20:40] were all correlated with scale-4. But so were things such as being unhappy and disgruntled and dissatisfied. And those are not the things that are prototypically associated with elevated scores on a scale- 4. And there were many other examples of that.

So to me, that was the early indication that things are not quite as clear-cut as they may appear to be if you just look at the textbook now. At the same time, one of the members of the MMPI-2 development the re- standardization committee called Auke Tellegen had actually already begun to work on a project that eventually produced the restructured clinical scales, the RC scales.

[00:21:36] I actually remembered hearing Tellegen talk about the need to update and deal with some of the psychometric deficiencies of the clinical scales as a graduate student while the MMPI -2 was being developed. He had actually proposed that to the committee and the committee decided that updating both the norms and the scales at the same time would be too much of a change at once, and that, the two needs and need for new norms was more press sensitive. So they essentially decided to carry over the clinical scales to the MMPI -2 without change.

But Tellegen then shortly thereafter began working on this project that he at that point called streamlining the clinical scales. And he and I had continued to communicate. I think it was in 1995 that I first saw him present some of the data on these scales that he was working on. And one thing that was very clear was that what you get in terms of correlates with these streamlined scales that he was working on was much cleaner in terms of their discriminant validity. So what came to be the restructured version of clinical scale-4 had all of the acting out correlates, but none of the other unexpected correlates having to do with being unhappy and dissatisfied.

And I think it was at that point that it became clear to me that there really was a need to at least update the clinical scales. And I joined Tellegen in that effort and together we finalize the RC scales which were pretty much complete in 1998. They weren’t actually published and added as another set of measures to the MMPI -2 until 2003. We spent several years studying them and making sure with a lot of datasets that we had available at that point that they worked as intended or that we could develop some empirically grounded with solid foundations guidelines for using them in the context of the MMPI -2. But that was really when the realization set in when I had the opportunity to look under the hood and see what really correlates with what. And so that probably was the turning point, at least in terms of my perception of the clinical scale was of the MMPI.

[00:24:09] Dr. Sharp:  Sure. And then how did that process come into play this time around where… I mean, in my understanding, and you, of course fact check me anywhere you need to, is that the MMPI-3 is pretty solidly correlated with the 2 RF, right? I’m not sure.

[00:24:30] Dr. Yossi: They’re very similar.

[00:24:32] Dr. Sharp: So how do you know this time around?

[00:24:35] Dr. Yossi: So there’s one more step that I haven’t described kind of in the evolution if you will. So in 2003,  the University of Minnesota publisher added the RC scales to the MMPI -2 as another standard set of scales.  But as we were studying the RC scales, Tellegen and I, it became very clear to us that it wasn’t just the clinical scales that could benefit from restructuring, but the entire instrument. The restructured clinical scales were never designed to assess everything that could be measured with the MMPI-2 item pool, but rather major distinctive core components of the clinical scales.  And we knew that there were many other constructs that could be reliably invalidly assessed with that item pool of the MMPI -2.

So once we completed the work with the RC scales, we began restructuring the entire instrument and that’s what produced in 2008, the MMPI -2 RF, the restructured form. And what we did with the MMPI – 2- RF was essentially applied a similar methodology to what Tellegen had done with the RC scales but restructure the entire instrument. But we limited the restructuring in the revision to existing MMPI-2  items. And in standardizing the MMPI – 2-RF, we use the MMPI-2 normative samples. So we didn’t collect new norms. We didn’t write new items. And that’s the answer to a question that I sometimes, rather frequently was asked when the RF first came out was why didn’t you call it the MMPI-3? And the answer was because we did not write any new items or collect new norms. It was a restructured version of the MMPI-2. And doing it that way, had the advantage of allowing us to use all of the existing data sets that had been accumulated and not just us, but anyone who had an MMPI -2 data at that point could do an MMPI-2- RF study. Was it possible to go back in and rescore your data as the restructure form and that allowed for a lot of investigators to immediately begin doing MMPI -2- RF research without having to collect new data, and contributed to the fact that we now have close to 500 peer-reviewed publications on the MMPI -2-RF not just from my group, but just across the board.

But the flip side of that is that the two things we didn’t do were to update the item pool or update the norms. And those were the two primary objectives for the MMPI-3: to pick up where the MMPI -2-RF left off, take advantage of all of the psychometric improvements that we had made in the restructuring project, but then fill in some gaps, areas that the MMPI -2 item pool doesn’t cover and therefore, were not adequately or at all covered with the restructure form. And a good example of that is there are no items on the MMPI-2 and therefore no items on the restructured form having to do with eating disorders or any kind of problematic eating behavior which of course is a significant problem in mental health today.

[00:28:13] And there were other areas where the item pool was relatively lacking. Not many items having to do with grandiosity on the MMPI -2 or compulsive behavior. And so, our objective then was to explore adding new item content to the MMPI and update the norms. The norms for the MMPI -2-RF are the MMPI-2 norms. Also, the data that I was working with as a graduate student in the mid-1980s, those data were collected in the mid-80s. And so they’re more than 30 years old.  It was a normative sample that represented the adult population of the United States in the mid-1980s, which was very different from the population today, demographically and experientially.

One way that I sometimes tried to illustrate that is to point out that not a single member of the MMPI-2-RF normative samples had heard about the internet, let alone social media. We live in a very different environment today. And of course, demographically, the adult population was over 80% white in the mid-1980s. It’s now, the latest estimates for the 2020 census are 62%. And of course, individuals have of Hispanic origin, that’s a much larger proportion of the population today, only 2.9% of the MMPI-2 normative sample identified as being of Hispanic origin. And so, demographically, the normative sample was becoming increasingly inadequate and we wanted to expand the item pool content-wise not lengthwise, but expand the coverage of the item pool. So those were the two main objectives for the MMP-3. 

[00:30:20] Dr. Sharp: Got you. Thanks for talking through that. It is interesting to trace the history. And I don’t know that I really thought about it in that stark of terms to think the normative sample had not been updated since the 80s. That’s a long time.

[00:30:39]Dr. Yossi: Yeah.

[00:30:39]Dr. Sharp:  That makes sense. So I feel like I have to ask, I know you have been asked this, but am curious just about that big decision to go from sort of the structure of the MMPI-2 to the RF and abandoning I suppose criterion validity for the content. Well, am I getting that right? 

[00:31:07] Dr. Yossi: Well, the criteria and keying, yeah. one of the questions that has been raised by moving away from the clinical scales that were constructed on the basis of the criterion King approach the empirical, in the approach, are you not abandoning the origins of the test if you will? And I think the answer to that is no because we still use the items that came out of that process of criterion King or, empirical King Stark Hathaway, and Charlie McKinley used in constructing the original clinical scales by contrasting the responses of specified diagnostic groups with a set of nonclinical individuals who were called the Minnesota normal who were not in a part of the treatment process.

And I think that Hathaway and McKinley wound up assembling a remarkably rich and robust item pool, but they were applying the methodology and technology of the late 30s and early 1940s when they constructed the clinical scales. And we’ve learned quite a bit both about personality and psychopathology and test development since then. And so, our goal was to apply more modern test construction techniques. And this is in developing the MMPI -2-RF to apply more modern test construction techniques to that item pool that they assembled and see if we could address some of the psychometric problems of the clinical scales. Now interestingly, the empirical King didn’t work. The folks who will bring that up sometimes ignore that part of it. 

[00:33:11]Dr. Sharp: Can you elaborate? When you say it didn’t work, what do you mean?

[00:33:15] Dr. Yossi: So the idea was that the MMPI would be administered to any new patient coming into the University of Minnesota hospital, where the test was developed and be used as a direct differential diagnostic indicator to indicate whether the patient had one or more of the eight most commonly occurring diagnoses in the patient population at the time: hypochondriasis depression, hysteria, psychopathic deviance, paranoia, psychasthenia which today we would call anxiety disorders, schizophrenia, and hypomania. Those were the eight original clinical scales. And the idea was that if a patient produced the profile that was two standard deviations above the norm, a score that was two standard deviations above the norm, say on the schizophrenia scale, then that patient had schizophrenia just like any other medical test if they exceed the cutoff, then the condition is present.

It turned out that you couldn’t use it that way. The vast majority of the patients who scored above 70, T score 70 was the cutoff used at the time, and the schizophrenia scale didn’t have schizophrenia. And so if you use the test as intended, the false positive rate would have been excessive and unacceptably high. And so very early on I had to make a complete shift, a paradigm shift. This was the first paradigm shift in the evolution of the MMPI instruments and it was actually one of Hathaway’s former students who was a graduate student at the time that the MMPI was being developed and then actually, shortly thereafter became a faculty member at the University of Minnesota in the mid-1940s, a fellow by the name of Paul Miele. And it was really Miele who led this paradigm shift and focus and looking away from the scores on the individual clinical scales to looking at patterns of scores or combinations of scores of clinicians using it.

I think there was such a strong need for a measure like the MMPI that didn’t exist before it was introduced that clinicians began using it. And though they couldn’t use it as intended as a direct differential diagnostic test, they found that certain patterns of scores, certain combinations of scores on the clinical scales tended to reoccur and that the individuals who produced those scores had certain features in common, certain personality, characteristics, or symptoms of psychopathology or behavioral tendencies.

[00:35:48] And what Miele led was then an effort to study this systematically and collect empirical data on these patterns of scores that came to be known as the code types. And so MMPI interpretation really from that point on was no longer all dependent on the empirical King approach. The way in which the scales work was were constructed had no impact on their use when code type interpretation became the primary vehicle for interpreting scores on the MMPI. So when I hear that we’ve abandoned the richness that came from the empirical King, the richness that came is represented by the items, not by the methodology itself.

[00:36:41] Dr. Sharp: Yeah, I think that was the major reaction criticism, I suppose. 

[00:36:49] Dr. Yossi: That’s one. The other one relates to what I was just talking about, and that is the code type. So, another criticism that we’ve heard and that has been expressed is, well, you’re abandoning the richness of the code types because the code types very much didn’t have a rich empirical foundation that helps support their interpretation. And that’s how the MMPI Clinical scales have been used over the years, rather than looking at scores on the individual scales. But the code types themselves were an ad hoc solution to the problem that the clinical scales didn’t work as intended. And the reason the code types worked was that they basically looked at and solved some of the challenges of the clinical scales, all the overlapping items, and the construct overlap.

Going back to the example I gave you earlier of the PD scale-4, the fact that we find scores on the PD scale to be correlated empirically in clinical settings not just with acting out, but also with unhappiness and dissatisfaction, well, if you look at the rest of the profile, not just scale- 4, but the rest of the profile, and along with scale 4 you have an elevation on scale 9 and 4-9 codetype what we find empirically to be correlated with a 4-9 codetype, what we find in therapy will be correlated are the acting out behaviors and all of the things that we typically associate with PD. On the other hand, if another individual has exactly the same score on the PD scale but rather than an elevation on 9, they also have an elevation on scale 2,so a 2-4 codetype, [00:38:51] well, if you look at the empirical correlates, and we did all of this with the data that we collected in the 90s if you look at the empirical correlates of the 2-4 code type, that’s where you find the unhappiness and dissatisfaction.

So you really need to look at all of the other scales in order to make sense of each individual scale. And so the code types really provided this kind of ad hoc solution.  What we did in the restructuring project is basically go back to the building blocks, the items and see if we couldn’t apply these more modern test construction techniques and develop scales that didn’t require all of this additional information. I want to point out that this was not some kind of a unique insight of ours. I think Tellegen and I obviously, we’re the ones who did the work, but with the restructuring of the clinical scales. But, if you don’t mind, Jeremy, I want to read something to you.

[00:40:04] Dr. Sharp: Of course.

[00:40:06] Dr. Yossi: You can just quote, and it’s a quote from Paul Miele who was Hathaway’s student, and of course we came, one of the most famous clinical psychologists of the 20th century, and this is from a chapter that he wrote in 1972 which was part of a book. The book was edited by Jim Butcher. And the book came out of a conference that was held to honor Starke Hathaway and his contributions. And one of the themes of the conference was, how would we go about improving the MMPI? It’s been out for 30 years now at the time and a number of different authors presented at the conference and then written chapters and then Miele wrote the final chapter in the book. And what he’s commenting on here in this chapter is what I was just talking about. So  I’ll read it really quickly here.

He said,  ” Unfortunately, one can achieve a moderate and sometimes rather high elevation on scale 4 without being a sociopath. Not surprising when we look at the items scored for this variable. At an increment of two or three T-score points per Ross core item shift, it takes less than 10 items in the combined areas of family strife and post institutional troubles to achieve a T score of 70. We all recognize today that this kind of thing happens and is one source of error which we attempt to “correct” for mentally by taking the patient’s situation into account as well as looking at the rest of his profile. But it would be nicer if such errors were eliminated from the PD key entirely. As a factor analyst once complained to me during a heated discussion on criterion King, internal consistency, scale purity, and related topics. If you Minnesotans are going to eyeball the profile and do a subjective factor analysis in your head that way, why not let the computer do it better at the stage of scale construction,? Not an easy argument to answer.” He wrote.

And that’s basically what Tellegen and I did. We did some factor analysis looking at each of the scales and some other items we don’t need to get into the methodology in detail here, but that was the idea. And my point here is that this wasn’t a new or novel idea. It was something that people who were very familiar with the MMPI had thought about including Paul Mielle who probably other than Hathaway thought about the MMPI more than anyone else. 

[00:42:50] Dr. Jeremy Sharp: Right. That’s a great quote.  I want to dig in a little bit here because I’m just imagining what this might have been like for you, and I’m doing a little bit of projecting here, of course, but I would think if this were me and I had been working on this project for 15ish years, you go through this major revision doing something that I, in my mind, I would be saying, we’re making this better. We’re really honing this and applying these modern statistical methods, and this is going to be an upgrade, and then the RF comes out and people are like, what are you doing? That’s terrible. All this criticism. What was that like for you as a researcher and a person to go through that process? And am I projecting?

[00:43:42] Dr. Yossi Ben- Porath: No, I think those are great questions. And it is a process. And it’s sometimes can be complicated, sometimes it can be challenging. I remember early on,  I do a lot of trainings, a lot of workshops, and of course was doing a lot of trainings after the MMPI -2-RF came out and at one of these workshops, I had been talking about some of these issues that we were just discussing and someone raised their hand, and actually stood up and said, “I really don’t understand, what did you do to my PD scale?” And I said, “We fixed it.” And there was a little bit of laughter and so forth, but then, of course, I went into the details and explained why and how we did so. But there were. I think it’s natural for many. Clinicians had been using the MMPI- 2 for years and the code types, then developed a fair amount of expertise in doing so. And you really needed to be an expert interpreting the MMPI -2 with the clinical scales and the sub-scales and supplementary scales and the code types and so forth was a rather complex process. And some people became really, really good at it and wrote books about it.

But most clinicians, in my impression and experience and then because I do so many trainings,  I get to talk a lot with clinicians. And that has been one of the most important sources of ideas and feedback for me. And one piece of feedback that I remember hearing when I was still doing trainings on the MMPI -2 is, “I have a lot of other tests that I use and in my practice and in my battery that I administer, if it’s someone who does a batteries of tests and I see clients or patients for therapy, and I don’t have the time to do a Ph.D. on every single instrument that I use. Does it really have to be this complicated?”

And so one of our goals in the restructuring was to make the use of the test less complicated without sacrificing at all its reliability or validity. So, while I certainly was hearing from some of the experts who I think were experiencing the sense of the loss with the introduction of the MMPI -2- RF, they were by far outnumbered by clinicians who gave me very positive feedback about how helpful they found the MMPI-2- RF and how much easier it had become to integrate it into their practice. And the fact that it was shorter was also a benefit. So more efficient not just in terms of interpretation, but also administration time.

And the positive feedback certainly by far outweighed some of the criticisms and complaints. And as many Minnesota PhDs are empiricist in my approach to questions such as this, and the empirical data were quite compelling and I felt quite comfortable with that. And again, we worked on this for many years and studied this test inside and out. And so I was very comfortable that what we were making available to people was from a psychometric perspective of very useful and well-constructed instruments.

I think when you change something, a test that has almost iconic status, in fact back in the 70s and maybe even the beginning of the 60s there was a group of people who used the MMPI who called themselves the mult-cult,  and I think the word cult is maybe, appropriate to some extent.  So obviously, some members of the multicultural were not happy with the restructured form, but the vast majority of users and the feedback that we’ve received from people who use the test in all of the settings that I mentioned earlier has been quite positive. The criticism is out there. We respond to the substance of the criticism when it appears in the literature. And I think that’s the way to deal with it. 

[00:48:42] Dr. Sharp: Sure. Well, spoken like a true empiricist. If you can fall back on the data and know that you can be solid, I think that would be really helpful in that process if I were in your shoes. I remember, I started grad school in 2003 and we took a personality assessment that very first semester with Dr. Chuck David chauffeur, and he was one of those folks who seemed to use the MMPI-2 as a crystal ball. It was magic as far as we were concerned and the nuances that he could pull out. And we were just in awe of that whole process. And knowing that there are other folks out there like that, I can se, it does feel like there’s some loss wired.

[00:49:33] Dr. Yossi: It required a great deal of experience and expertise to do that. That’s what Miele was writing about when he talked about doing a factor analysis in your head. That’s basically what these experts do. They’re deconstructing the clinical scales if you will. And they’re doing it by looking at all the other pieces of information that come in and of course taking advantage of any collateral information. But it doesn’t have to be that complicated. And if it doesn’t have to be that complicated, why not make it a more user-friendly instrument is really the thinking behind that. 

[00:50:10] Dr. Sharp: Sure. So I want to ask some more specific questions about the MMPI-3 of course, but on that whole process. Do you feel like moving from the criteria and keying to the sort of the new structure, were there any trade-offs? I mean, were there any compromises sacrifices that you had to make? 

[00:50:33] Dr. Yossi: Not really. And again, I think, as far as the criterion King is concerned, I don’t think it’s a coincidence that the clinical scales of the MMPI were the last set of applied assessment measures constructed that way. And this was again in the late 30s and early 1940s. It’s a methodology that has some limitations. And the way it was applied had some additional limitations. So I don’t think, and again we were able to still take advantage of the richness of the item pool that Hathaway and McKinley had assembled which I think is unparalleled. The reason why the MMPI has survived all these years is because test users are able to glean some clinically relevant information from the responses people give to those items.  How you organize those responses into the scales is really what we’ve focused on.

But your question is a very good one because one of the things that we focused on very much while developing the MMPI -2-RF was the question, is there anything missing? Is there anything in not just the clinical scales, but all of the scales that had been developed over the years for the original MMPI and MMPI -2, are we missing any important content?

And I’ll tell you a little story here. It’s not one that you’ll find in any of the books. It turned out that we were missing some content because when we thought we were done constructing the MMPI-2- RF,  I was already at that point doing trainings on the RC scales. I had just done a workshop for the Louisiana psychological association, and one of the people attending that workshop was a neuropsychologist who I had known previously  Kevin Grieve who’s in New Orleans, and a few weeks after the workshop this is while the MMPI -2- RF  materials were still being developed and produced, but the RC scales were already part of the MMPI-2 and Kevin had done a pre-surgical evaluation of a spine surgery candidate, and he sent me the profile which included both the clinical scales and the RC scales of restructured clinical scales.

[00:53:14] And on the clinical scales, the individual had a very clear 1-3 code type, and on the RC scales there really wasn’t anything at all showing up and Kevin said, “If I use just the RC scales, I would’ve missed something very important here.” And my response was, “Well, the MMPI-2-RF has additional scales.” So he sent me the item responses. I was able to score the MMPI-2- RF and really still didn’t find what  1-3 code type was picking up on. And that sent Tellegen and myself back to the data. And what we found was that there’s a set of items that appear on those scales that were not adequately represented in any of those scales we had developed for the restructured form. And that led us to develop the final scale of the MMPI -2-RF which was the Malays scale, which is one of the specific problems scales and a very important scale that would have been very important item content to have missed or left out of the test.

Fortunately, we were able to stop whatever was already ongoing and make some changes and move some items around, re-replace some items, and include the Malays scale. And so clearly it was possible and it’s still possible that there’s some important content that’s missing but we were pretty systematic and so I didn’t think we had missed much. But what was very clear was that we were constrained by the MMPI -2 item pool for the restructured form. We had decided not to write new items which would have necessitated collecting new norms. And so we always knew that we were going to explore as a next step if the MMPI -2-RF proved to work well using that as the starting point for expanding the content and that really as I mentioned earlier was one of our two primary goals for the MMPI -3 was to add item content, not that was missing from the RF but was missing from the MMPI -2, things like, eating disorder-related items and the others that I mentioned earlier.

[00:55:35] Dr. Sharp:  Yeah. What were some of the highlights from that in addition to eating disorders? Can you remember some of the…

[00:55:46] Dr. Yossi: So I mentioned earlier that there’s a surprisingly limited number of items on the MMPI -2 dealing with grandiosity. 

[00:55:53] Dr. Sharp: That’s right. Yes. 

[00:55:54] Dr. Yossi: And impulsivity and compulsivity. And there’s really also a limited range of items dealing with anxiety-related experiences, for example, very few if any, that would be associated with Panic attack and those kinds of experiences. So what we did for the MMPI-3 in developing the MMPI-3 was to go systematically through. The first thing we did actually was to survey experts, MMPI -2-RF researchers, and clinicians who use the test and ask them, what do you find lacking? What areas would you like to see strengthened? And then we looked at other measures and looked back to, I’ll give you an example when we were developing the MMPI -2-RF, we wanted to develop two separate measures, one focusing on stress-related experiences and the other on worry, but we didn’t have a sufficient number of items to develop two separate scales. And they were highly correlated with each other.

So we constructed a single scale called stress and worry. For the MMPI-3, we wrote additional items dealing with stress-related experiences and worry. And our goal was, and what we did in fact do was to develop separate measures of stress and worry and write additional items dealing with anxiety-related experiences. Some of the items dealing with Pescatore beliefs and the scale, the restructured clinical scale 6, RC6 which is called ideas of persecution, most of the items done that scale are quite extreme. They deal with very significant substantial Pescatore beliefs. And as a result, all it takes is responding to a small number of those items in the key direction and the T score is already in the clinically elevated range.

So we wanted to write some items that weren’t quite as extreme. Dealt with a sense of being mistreated and singled out for criticism, but not necessarily somebody who’s trying to kill me level of persecutory beliefs. So we had a range of objectives for writing new items and exploring the ability to add them to the MMPI-3 to expand the item coverage of the test.

[00:58:30] Dr. Sharp:  Right. Thank you for digging into all of that. I know people are curious, how is this different and what changed of course. We sort of crowd sourced some questions from a Facebook community, and there are a few of those hanging out there that I want to make sure I ask. A lot of folks who are curious about the forensic applications of the MMPI-3. Can you speak to that at all? And if it’s going to be useful or valid? 

[00:59:04] Dr. Yossi: Absolutely,  you know, anytime a new version of the test is introduced, forensic practitioners, and as I mentioned, I for many years did hundreds of competency to stand trial and insanity pre-related evaluations. I would spend my Fridays at the summit County jail in Akron Ohio doing these assessments. So I have a lot of experience both doing this work and research that informs this work. And anytime a new version of a test is introduced, the practitioner faces a dilemma. If they use the previous version of the test, they can be challenged for using an old outdated instrument, and if they use the new version of the test, they can be challenged for using this new unproven device.

It’s an adversarial system, anything you do can and will be used against you in a court of law. The cross-examiners task is to weaken the impact of your testimony and so the introduction of a new version of a test opens the door wide to that. There’s only one way to avoid that situation and that’s to never update our tests. I don’t think that’s a satisfactory solution. So we do need to update the tests from time to time. We shouldn’t be using technology that’s 70 years old, I don’t think. And so I think what the practitioner needs to do obviously is become very well informed about the nature of the change and the potential impact.

[01:00:51] Now in the case of the MMPI-3, we paid particular attention to not necessarily in the forensic context, but one of the things that we did that I think is very relevant to forensic practitioners is, as we were updating the MMPI -2- RF scales, most of the MMPI-3 scales are on the MMPI-2-RF as well,  but as we were updating these scales, we were looking very closely at the data to make sure that we weren’t making changes that reduce their validity. In some cases, we substantially increased their validity. We shortened some scales. So you want to make sure that by shortening the scales to make room for new items, you’re not sacrificing validity. And we paid very close attention to that and included in the technical manual for the MMPI-3 there’s an appendix E that has I think over 38,000 correlations just in this appendix alone.

And what these correlations do is compare the correlates of MMPI -2-RF versions of the scales with the MMPI-3 versions of the scales using collateral information that we collected in different sites in mental health settings and medical settings and others. And what these data demonstrate very clearly,  and they’re out there for everyone to examine and see for themselves, is that the correlates that have been identified of the MMPI-2- RF scales apply to the MMPI-3 as well, which means that the literature, the peer-reviewed literature that’s accumulated with the MMPI -2- RF can be relied upon when interpreting scores on the MMPI -3. And we go through and explain and provide all the data to support this. As a result, a forensic practitioner using the MMPI-3 doesn’t need to rely only on the validity information that’s in the MMPI-3 manual although there’s extensive validity data there as well.

[01:02:57] Another appendix, Appendix D with another set of 38,000 empirical correlates, we can continue to rely on the now close to 500 peer-reviewed publications on the restructured form and the studies that are beginning to appear in the literature on the MMPI-3 so that the answer to the question, are you using a new unproven devices? No, I’m using an updated version of the MMPI -2- RF that is proven, that does have solid empirical data validity data. When questions of admissibility come up certainly in States where the Dobber criteria are the ones that govern judges decisions about admissibility. It really boils down to the scientific validity of the technology or technique that you’re relying on. And I think there’s a very solid foundation that practitioners can rely on when using the MMPI-3.

[01:04:02] Dr. Sharp: I appreciate that. And I appreciate you giving an example of how you might respond if kind of a question is there. I think people are really curious about that. Another question that came up. I don’t think this is specific to the MMPI-3 or the MPI series, but personality measures, in general, seem to struggle to accurately represent folks with autism or autistic folks, depending on what you might call yourself. Just given the true/false dichotomy and sort of interpreting questions literally, did y’all take that into account at all? Or do you have any thoughts on that particularly?

[01:04:52] Dr. Yossi: That’s a great question, and in fact, that’s one of the areas that we contemplated when we were looking at adding items. Can we develop a measure that would be helpful in assessing manifestations of autism spectrum types of difficulties? And we’ve studied this very thoroughly. It was something that we definitely wanted to do if we could. We wound up concluding essentially the same thing that you just said. It’s difficult to assess autism spectrum-related manifestations by self-report. I don’t think it’s the true false structure of the response options so much as the capacity for introspection and insight and self-awareness that at least some of the variables that are related to the autism spectrum disorder entail.

And that’s not to say that there isn’t information on the MMPI-3 that could be helpful.  I think there is. There hasn’t been a lot of research, unfortunately. There’ve been a couple of studies looking at the RC scales, with individuals with an autism spectrum disorder. And not surprisingly, the one  RC scale that you do find elevated is RC-2, low positive emotions, dealing with anhedonia- that lack of positive emotional experiences. Although it’s by no means diagnostic in itself, but it captures features of autism spectrum disorder that is found in some not by no means all individuals with that condition. And some of our interpersonal scales disaffiliatedness  on the MMPI -2- RF and, and social avoidance, I think also have some relevant information. 

This brings up what I think is an important point is that the scales of the MMPI-3 really our measures of what we call Transdiagnostic constructs. These are psychological variables that are not unique to one diagnosis or another, but maybe relevant in assessment of functioning of individuals with various types of conditions and disorders. So if there are some Transdiagnostic constructs relevant to autism spectrum disorder that can be assessed by self-report, then I think we have some scales at least on the MMPI-3 that are relevant to that, but others that really don’t lend themselves well to assessment by any kind of self-report at least from my perspective.

[01:07:40] Dr. Sharp:  That’s fair. I know we’d maybe meant to talk about sort of that shift to the more of a hierarchical model.  Our time is short, but I wonder if there’s anything more to say on that that we haven’t already talked about. 

[01:08:01] Dr. Yossi: I guess what I would say related to that, Jeremy, is that it’s interesting with the MMPI -2-RF that was introduced in 2008, was introduced as a hierarchical model essentially with 3 levels of measurement represented by the various scales. And probably the most significant development in the field of psychopathology research in the last 15 years has been a recognition in the literature that it has nothing to do with the MMPI that psychopathology is perhaps best understood and ultimately diagnosed in the context of hierarchical, dimensional models. The most prominent model being the high-top model that has gotten a lot of attention in the literature. And so the MMPI-3 is very consistent with that movement in the field towards assessing constructs that are both dimensional and transdiagnostic, not unique or specific to any one diagnosis, but also hierarchical. Some are broader and others are more narrow just as are the scales of the MMPI-3. 

[01:09:15] Dr. Sharp:  I’m glad you mentioned that. I heard Dr. Katherine Jonas from HiTOP on the podcast maybe a year ago, I want to say.  And it was fantastic. I like seeing things moving in that direction, I think.

[01:09:29]Dr. Yossi: Interestingly in the article that they published, the first introduced the HiTOP model to the psychiatry community, they had a table that listed tests and measures that assess the HiTOP constructs in the MMPI -2- RF. I don’t think it was a coincidence. It was included in that list of measures. There’s a good synergy there.. 

[01:09:53] Dr. Jeremy Sharp: Absolutely. So one other thing that those folks were asking about is the, how would I phrase it? Gender dynamics, I can’t quite capture it but hopefully, you’re nodding, you see where I’m going here with the MMPI-3. And I think we’re living in a time where I don’t know if gender is literally more fluid or there’s more coverage or exposure to fluid gender dynamics, but did you all think about that realm?

[01:10:29] Dr. Yossi: Yes, and it’s manifested in several ways. Beginning with the MMPI -2- RF, we’ve been using what we call non-gendered norms with RF and now with the MMPI-3 rather than the traditional gender-based norms of the original MMPI and the MMPI-2 which of course require that gender be specified in order to score and convert the raw scores into T scores. That’s no longer necessary with the MMPI -2- RF and the MMPI-3. And one of the challenges that clinicians faced was that in order to even using Pearson software, for example, score the MMPI -2 you needed to specify male or female gender. The software wouldn’t work without that. That’s no longer necessary with the MMPI-3. There’s no need to specify gender in order to score the MMPI-3 because gender is not a factor in converting the raw scores into T scores.

The other thing that we’ve done with the MMPI-3, we have interpretive report for clinical settings. And that interpreted report by default uses gender pronouns. But there is an option if you’re printing the report to produce what we call a gender-neutral version of the report that does not use gender-based pronouns at all. And that certainly would be consistent with individuals who are non-binary or gender fluid. I think that would be the more appropriate version of the interpretive report to produce; it doesn’t change the interpretation, just the language used.

 We still have a lot of work to do in this area. I think this is obviously a population that’s been under-researched including with the MMPI instruments. There actually is some research ongoing at this time with the MMPI-3 and transgender and gender non-conforming individuals. And I Iook forward to seeing that published in the not-too-distant future.

[01:12:53] Dr. Sharp: Right. Yes, I think we all have a lot of work to do in that area. It’s nice to hear you talk through the considerations that you made though to this point. I feel like we’ve covered a lot of ground in this conversation and I really appreciate you digging into some of the personal components and sharing the story. I really enjoyed getting to hear some of that.  Is there anything else before we wrap up that may be important for folks to know or just things that you want to highlight? And if not, that’s totally okay. But I always like to give that opportunity. 

[01:13:33] Dr. Yossi: Well, I guess I’ll take the opportunity to extend an invitation to anyone listening to our conversation today. When I do training, I always make sure to include my email address on the handout and I tell people that that’s an invitation to contact me if they have questions or if they encounter an interesting case that they would like to discuss. And I’d like to extend the same invitation to our listeners here today. I’m very easy to find. If you Google my name, you’ll find my email address. There aren’t that many psychologists named Yossi Ben- Porath, and so, please do feel free to write and I always try to answer questions. And I’m happy to hear from MMPI users and non-users what questions they might have.

[01:14:31] Dr. Sharp: Fantastic. I’ll make sure to put that and a bunch of other resources that we’ve talked about in the show notes so that folks can access that if they’d like. And I know you’re in our Facebook community as well, The Testing Psychologist Community.

[01:14:43] Dr. Yossi: Yeah, I’m happy to respond there. I try to respond. If someone tags me in that community, then I know to respond and  I try to do so.

[01:14:51] Dr. Sharp: Fantastic. Well, thank you so much. This was a great conversation. I really appreciate it. It was great to talk to you for a little while.

[01:14:58] Dr. Yossi: Thank you, Jeremy. I really enjoyed it.

[01:15:01] Dr. Sharp: Okay everyone, thank you so much for listening to this episode. I hope that you took a good bit away from it. Yossi’s fantastic interview, super knowledgeable, kind, articulate, and I really enjoyed being able to talk with him. So everything that we mentioned during the episode is in the show notes. You can check those out.

Otherwise welcome to 2021. Here we are. I hope that this is the beginning of some better times for many of us. So I will be back with you on Thursday. We’re going to resume our EHR review series. And I hope you enjoy. So take care and we’ll talk to you next time.

The information contained in this podcast and on The Testing Psychologists website are intended for informational and educational purposes only. Nothing in this podcast or on the website is intended to be a substitute for professional, psychological, psychiatric, or medical advice, diagnosis or treatment. Please note that no doctor-patient relationship is formed here. And similarly, no supervisory or consultative relationship is formed between the host or guests of this podcast and listeners of this podcast. If you need the qualified advice of any mental health practitioner or medical provider, please seek one in your area. Similarly, if you need supervision on clinical matters, please find a supervisor with expertise that fits your needs.

Click here to listen instead!