Blog

[00:00:00] Hello, this is The Testing Psychologist podcast, episode 77.

I’m Dr. Jeremy Sharp. Today, we’re going to be talking about the best episodes of 2018. I’m looking back a little bit on all of the interviews I’ve done, all of the solo episodes, and pulling the best of the best.

Now, before we get to that, I want to reannounce The Testing Psychologist Beginner Practice Mastermind Group which starts in January 2019. This is a group coaching experience for those of you who are just getting started in your private practices with testing, and you’d like support from myself and others who are in your exact same situation.

We did a mastermind group back in the spring that turned out pretty amazing and people found it super helpful and I’m bringing it back for 2019. So if you’re interested in that Beginner Practice Mastermind, a group coaching experience, head on over to [00:01:00] thetestingpsychologist.com/consulting, scroll down, and just apply to join that group. I’ll give you a call and we will talk about whether it’s a good fit. Would love to have you join.

All right. Let’s talk about the best of 2018 from The Testing Psychologist Podcast.

Okay, y’all, here we are. Welcome back. This is episode 77, and I’m Dr. Jeremy Sharp.

Today, like I said in the intro, I am going to do a little recap of 2018. 2018 was my second full year doing the podcast. And it is frankly pretty amazing to look back and think.

I still remember very well some of those early [00:02:00] conversations with Joe Sanok, who was my personal practice coach for several months 2 or 3 years ago, and we were having this conversation about, I wanted to do something different. I knew that I really enjoyed teaching. I taught professional issues and practice development at a local grad school program. I was just looking for something different to shake it up and share knowledge in that way, and Joe, who runs the Practice of the Practice podcast, one of the most successful if not the most successful mental health podcasts out there, pushed me in this direction.

And my gosh, it was a ton of work in the beginning: getting everything set up, figuring out my system, figuring out how to get people to come on the podcast, and how to interview. It was a huge learning curve. But now here we are two years later. We’ve got now 77 episodes after this one is [00:03:00] published and it has been awesome.

This is still one of my favorite parts of the week. I really look forward to recording these episodes. I love interviewing folks. I’ve gotten to connect with some of the truly preeminent folks here in our field, and just to have that privilege to bring those folks and their knowledge to such a wide audience is really fulfilling. I’m so glad that I’ve gone this direction and will continue to go in this direction.

Let’s see. Let’s take a look back at some numbers for 2018. I’m a numbers person. Many of you are numbers people as well with the work that we do.

The total unique downloads for this year were 35,134 downloads. Now, this is nothing, to be [00:04:00] honest, compared to some of those larger podcasts. I had to get that out of my mind when I was looking at the numbers that we’re never going to be like Joe Rogan or even Practice of the Practice podcast or some of those podcasts with a broader breadth of clinicians that they’re pulling in to listen. And that’s totally okay. We have this little niche and I’m truly amazed that there have been 35,000 downloads of this podcast over the course of the year. Testing is such a specialty, and back in the beginning, I had no idea who would be listening or if anyone would want to listen, and yet here we are.

So that’s super cool. Just over 35,000 downloads. It’s been downloaded all across the world. I think the last time I checked it was over 30 different countries [00:05:00] that had downloaded the podcast. I don’t honestly really know how that works. I’m assuming it’s all English-speaking assessment folks in these different countries. So, it’s across the world and it’s just really cool to know that it’s reached that many people.

So total downloads as of right this moment, I checked right before I started recording are at 49,596. That is really close to 50,000, which to me is a milestone. So that’s cool too that we’re probably going to hit 50,000 before the end of 2018.

Now, let’s see. Most downloads in a single day, that was 404 on October 22nd. That is something to really pay attention to as [00:06:00] well. Those initial downloads often reflect the number of subscribers, and that says that there are about 400 people out there who are subscribed, who are tuning in each week, and who are looking for the podcast whenever it comes out.

I would love to continue to grow that number. So if you have not subscribed, it does so much good to help bump this podcast up the charts, so to speak, and try to show guests that it is worth coming on. So if you haven’t subscribed, take like 30 seconds and go into your podcast app and it should be a pretty obvious button in there. Just hit subscribe and you’ll get those episodes whenever they release.

Okay, now, the best of 2018. This is the part that I really like to talk about. In my best of episodes last year, I counted down the top five most downloaded podcast episodes, and we’re going to do that again this year. So [00:07:00] drum roll.

Without further ado, the number five most downloaded podcast of this year was episode 55, All about dyslexia assessment with Dr. Robin Peterson. If you did not catch this episode, Robin was a fantastic guest, as they all are. She talked all about dyslexia and assessing dyslexia.

Robin is a co-author of two books. She studied with Bruce Pennington, who is a pretty prolific neuropsych researcher, specifically on learning disorders out of Denver. She has a book coming out in January 2019. It’s all about the neuropsychological perspective on dyslexia and learning disorders, but she talked about her ideal dyslexia battery. 

We dove into the discrepancy model, [00:08:00] the PSW model Model for assessing and diagnosing learning disorders, patterns of strengths, and weaknesses. We talked about her key components, like what are you looking for in a dyslexia assessment? And she gave us a ton of resources. That was something that really jumped out. She had an excellent working knowledge of resources out there that could really help with dyslexia assessment, so she dove into that as well. And then as a bonus, I managed to get her to tack on her one-hour concussion assessment battery because she works at Children’s Hospital Colorado and does a lot of that work.

So that was episode number 55, Dr. Robin Peterson, All about dyslexia assessment.

Okay, number four, episode number 44 with Dr. Benjamin Ben Lovett: Rethinking ADHD assessment. This was the first of two [00:09:00] appearances on the podcast with Dr. Ben Lovett. I think he remains the only repeat guest. And this one was a really interesting one.

We meant to get into testing accommodations for ADHD and so forth, like for the MCAT and LSAT and those sorts of things because that’s what he has studied and written books on over the years, but we detoured down this rabbit hole of what are we actually trying to do when we’re assessing ADHD and what parts of those assessments are important from a test reviewer perspective or accommodations request reviewer.

Dr. Lovett raised some eyebrows and raised a lot of discussions when he got into the idea that many reviewers are not looking so much for the cognitive assessment that we do, and the cognitive assessment, at least from an accommodations request perspective [00:10:00] is not that important. So he talked a lot about the role of behavior checklists, good interviewing, and how to document ADHD symptoms in the context of accommodations requests.

Let’s see. What else did he talk about? I think that’s about it. We really focused on how to assess ADHD from a testing accommodations perspective and people had a lot of discussion around this. It’s continued to generate a lot of discussion in the Facebook group since then. And it is a pretty enduring episode. So that was episode number 44, Dr. Ben Lovett on Rethinking ADHD Assessment in Adults.

All right, moving on to the number three most downloaded episode of 2018.

Number three is episode 50 with Dr. Ellen Braaten – All about processing speed. [00:11:00] I had so many guests this year where when I was typing out the show notes, I was able to write, this person has literally written the book on blank topic and Ellen is one of those. Ellen wrote a book called Bright Kids Who Can’t Keep Up, and it talks all about processing speed, what it is, how it shows up in real life, and how to help with it.

She is so good. She was such a good interviewee. I had a nice connection with her. We both went to Colorado State for our doctorate degrees and we had a great conversation about processing speed. We talked about what it is, how it’s related to executive functioning and working memory. We dove into how you “fix it or if you can fix it”, and we talked about how to measure processing speed with neuropsychological testing.

We covered a lot of ground and [00:12:00] she shared a ton of knowledge with us about processing speed. So, like I said, Ellen wrote a book, Bright Kids Who Can’t Keep Up, and she also, many years ago, wrote the Child Clinicians Report Writing Handbook which I still recommend to people as a guide for writing effective pediatric reports. So check that out. Episode number 50, Dr. Ellen Braaten- All about processing speed.

Now, we’re really getting down to it. The number two most downloaded episode of 2018 was episode 73 with Dr. Celine Saulnier – Research-informed autism assessment. Now, this was, I think one of my personal favorite interviews because, much like Ellen, Celine was just a fantastic interviewee.

She was warm. She was very responsive. She was very [00:13:00] knowledgeable and I felt like we had a great connection just right off the bat even though I had never met her before. And she also responded to my interview request, which came totally out of the blue, and she was very excited about it. So, from that perspective, she was a great interviewee, but the knowledge that she shared about her experience in autism research over the years was really just phenomenal. I kept asking questions thinking that I was going to stump her, and that never happened.

We covered a lot of ground in terms of autism assessment, both in research and practice. So we talked about early identification of autism during infancy and things you might look for even in very little babies.  We talked about genetic research in autism and where we’re at with that, we talked about gender differences in autism, and we talked about her ideal battery for assessing autism. We covered a lot of breadth in this interview [00:14:00] and obviously that that showed up. People enjoyed it and this was the second most downloaded episode of 2018: Episode 73 with Dr. Celine Saulnier – Research-informed autism assessment.

Okay, now’s the real drum roll. What episode was the most popular in 2018? Take just a few seconds and try to think to yourself, if you’re a longtime listener, if you’re a subscriber, think to yourself, which episode do you think might have been the most popular?

Alright, here we go. The number one most downloaded episode of 2018 from The Testing Psychologist podcast is episode number 71: Dr. Steve Feifer – Learning disorders are not created equal. A lot of you were aware [00:15:00] that this episode was coming out because I got off the recording and immediately went to the Facebook group and said, oh my gosh, y’all don’t want to miss this one. This was a fantastic interview and that showed.

Steve Feifer, if you have not heard of him, has really done pretty much everything in our field over the course of his career. So longtime school psychologist, now a practicing clinical psychologist, also a test developer, and a researcher. He is an author of the Feifer Assessment of Reading, the Feifer Assessment of Math, and as he talks about in the interview, the Feifer Assessment of Writing.

Steve and I talked about a lot of different things. Another great conversationalist. He was an easy guy to interview. Talked about the nuances of test development and how you might get into that as a psychologist. We really dive [00:16:00] into the different types of reading disorders and the different types of math disorders. I do ask him about assessment of writing and that’s when he disclosed that he was coming out with an assessment of writing because I think we need a better one in our field. And then we talked about how to intervene appropriately and what kind of recommendations might be helpful for kids with different types of learning disorders.

So we really parse out different types of reading disorders, different types of math disorders, and which interventions might be most helpful for those. So, this was the top downloaded episode of 2018, Dr. Steve Feifer, episode 71 – Learning disorders are not created equal.

So it was really interesting. There were a few things that jumped out as I was putting together the list of the top downloaded episodes. One is that our top two most [00:17:00] downloaded are also two of the most recent which tells me that they were just so good that even without the benefit of time, which typically equals more downloads, these episodes really leaped to the top. So both of those were within the last two months. So that’s awesome. It also tells me that the subscribership is going up, and more people are downloading right off the bat, which is also really cool. Like I said, I would love to get that number up.

But the biggest thing that I took away from this top five list is that you’ll notice that all of these episodes are interviews, and not only are they interviews, but they’re interviews with experts in our field. So I think this has been the trend as time has gone on with this podcast is I’m really finding that there are a lot of podcasts out there about general practice development and business [00:18:00] and the coaching podcasts in the mental health world, and what really sets us apart is this ability to interview experts in the field and bring relevant knowledge on specific cases, diagnoses, research, things like that.

There are no episodes here in the top five about business, and none of my solo episodes are here in the top five. I had to deal with a little ego blow there, but I think I’m good with that. What that tells me is that I’m going to keep heading in that direction. I will still do business episodes, of course, and I will still do solo episodes, but I am really going to double down and try to focus on finding experts to interview for our field.

So with that, if you have any requests, any particular areas that you’d like to hear about, any experts you’d like to try, and [00:19:00] get on the podcast, please reach out and let me know. You can email jeremy@thetestingpsychologist.com and shoot me some ideas. I would love to hear what y’all want to hear and I will do my best to continue to bring some excellent content to you folks about testing and assessment.

There you have it. That’s the 2018 wrap-up. I am pushing it down to the wire. I think I mentioned before that I typically record mostly in real-time, so this is absolutely in real-time. It’s going to be released probably around noon Eastern time, maybe a little bit later. So on New Year’s Day. So here we are last day of 2018.

I hope it was a good one for y’all. This is also a time for me when I like to reflect back and just look at the practice run the numbers, but really think about what worked for me this year. What was an [00:20:00] improvement? What do I still want to accomplish in 2019? What do I want to change?

In my case, I’m focusing more and more energy on running our group practice. And a big part of that is cutting back my clinical load even more. So I’ll be doing less clinical work, more administrative work with our practice and really trying to build that. I’m also going to be dedicating a lot more time here to the podcast and consulting.

One of the things that is on the horizon down the road is a Testing Psychologist Membership Community where for a relatively low monthly fee, you’re going to have access to tons of good information, very specific testing-related resources, paperwork, interviews, case consultations, training,[00:21:00] all sorts of good stuff. So keep that on the back burner. That’s going to be where I’m putting a lot of my energy in the coming months. I will keep you posted as things develop with that.

Like I said at the beginning, if you are interested in the Beginner Practice Mastermind Group, I would love to have you. We have 3 spots left out of 6, and it starts in three weeks. So, if you’re interested, you can go to thetestingpsychologist.com/consulting, scroll down, and just hit Apply now, you’ll fill out a short questionnaire, I will schedule a call and then we will figure out if the mastermind could be a good fit for you.

All right, y’all. That’s it. That is a wrap. We are wrapping up 2018. I wish y’all the best. I hope you have a great holiday season, a great break. I hope you’re taking some time off, rejuvenating, relaxing. [00:22:00] I will look forward to seeing you in 2019 with more awesome guests and great content.

Alright y’all, thanks so much to all the listeners for all the downloads, for all the support, and for all the positive words. I really appreciate it. It really keeps me going. I love doing this and I look forward to doing more. Take care, y’all.

Click here to listen instead!

[00:00:00] Dr. Sharp: Hey y’all. Welcome to episode 73 of The Testing Psychologist podcast. I’m Dr. Jeremy Sharp.

Our guest today is pretty incredible. Dr. Celine Saulnier is talking with me all about research and practice in the assessment of autism spectrum disorders, neurodevelopmental concerns, and adaptive functioning.

Gosh, this is going to be a long bio. She has done a lot in the 20 years that she’s been in our field. She started with her PhD at the University of Connecticut under the mentorship of Dr. Deb Fein. She then went on to postdoc at the Yale Child Study Center and she became the training director and then clinical director of the Yale autism program.

After several years, she was recruited down to the Marcus Autism Center and Emory School of Medicine in 2011, and she worked there to do many things. She was an Associate Professor within [00:01:00] the Department of Pediatrics. She was an investigator on many grants studying autism spectrum disorder. In 2012, she made a huge contribution and worked on the team to help get one of the national institutes of health autism centers of excellence grants.

She’s published in many leading journals for autism. She’s a member of the scientific advisory board for Autism Science Foundation. She’s also co-author of the Vineland Adaptive Behavior Scales, Third Edition and she’s co-authored two books on autism spectrum assessment and adaptive behavior assessment.

Now, Celine has moved on to the next chapter and she founded Neurodevelopmental Assessment and Consulting Services in Atlanta where she practices and completes assessments for neurodevelopmental disorders. She also consults with groups and [00:02:00] agencies across the country on a variety of topics.

So I’m very lucky to have her here today. It was a great conversation. This was a great interview and I’m happy to present it to you.

So without further ado, Dr. Celine Saulnier.

Hello and welcome to another episode of The Testing Psychologist podcast. I’m Dr. Jeremy Sharp. We have a fantastic guest with us today, Dr. Celine Saulnier. She is a licensed psychologist in private practice. She also does a lot of consulting around autism spectrum assessment.

We’re going to talk about many things, but I think we’ll probably center around ASD assessment and all of your work in that area. You have [00:03:00] a storied career, I think, from what I can tell with all of that. So I’m excited to have you on the podcast. Welcome.

Dr. Saulnier: Thank you so much. It’s such a pleasure to be here. The Testing Psychologist is my new favorite obsession, so I’m really excited to be part of the podcast.

Dr. Sharp: Oh, that’s great. It’s good to hear that from people. I started the group, it’s probably a year and a half ago now, and it’s just grown and grown. There’s so many people but I never know if it’s helpful or not helpful.

Dr. Saulnier: Well, my friend, Harry told me to go on because he said at least four or five times a day, someone is asking about ASD and it would be great to have you on here, but I’ll tell you, I’ve learned so much being in private practice, which is a new world for me, is a learning curve. So I just throwing out a shout-out to everyone in The Testing Psychologist. Thank you because they’re awesome. Love it.

Dr. Sharp: Nice. Oh, that’s so good to hear. [00:04:00] That is a cool thing about it. It seems like there’s a lot of give and take and there’s as much learning as teaching going on. So it’s really cool to hear that you’re getting some of them.

Dr. Saulnier: Yeah.

Dr. Sharp: And we’re lucky to have you. We were talking before we started to record and I said, when you joined the group, I was like, oh, this is a new person. You posted your website and I always go and check out people’s websites and just see what they’re up to.

And the more I read, I was like, ooh, we need to talk. You’ve done some great stuff in this field and have been connected to some really cool folks and sounds like you have a lot to share. So I’m thankful that you are willing to take some time and talk with me.

Dr. Saulnier: Thank you. I did recently just take a 180 in my career. And I, from graduate school, have been in academia of my entire career. So I spent nine years at the Yale Child Study Center [00:05:00] and then my mentor, Ami Klin, was recruited from Yale. After 21 years at Yale, he decided to leave and come to the Marcus Autism Center, which is a huge clinical care center in Atlanta and affiliated with Emory University School of Medicine.

So I relocated with him and spent seven years helping to pretty much transform the Marcus Autism Center into a center of science. So build up an academic research program there and did that. And then quality of life, midlife, time to reintroduce myself to my children, my husband, thinking, you know what, I’m going to go off on my own.

I opened up my own company, Neurodevelopmental Assessment and Consulting Services here in Atlanta. It’s been a wild ride so far, but what a change in my quality of life. I can’t even explain it. It’s wonderful.

Dr. Sharp: That’s why we do it. I feel like that’s the motivation for so many people. So that’s a huge shift, right?

Dr. Saulnier: It is.

[00:06:00] Dr. Sharp: Well, it’s been, I’m just going to immediately go off script and ask what has been the hardest thing about jumping into private practice so far.

Dr. Saulnier: Without a doubt, leaving an academically rich environment where I always have an expert and a friend and colleague over my shoulder to say, hey, come look at this kid or what do you think about this, and just having all sorts of multidisciplinary minds surrounding me to help conceptualize something, even just for fun, even if it wasn’t about a case or a research project, just being immersed in that environment.

So, fortunately, I kept connected through grants and through consulting and through some organizations I’m affiliated with to academia and to my colleagues. So I still have access to that but definitely, on a day-to-day basis, it’s something that is a huge shift.

Dr. Sharp: Sure. Yeah, I think that that’s one of those things that people talk about a lot, like the isolation of private practice and especially coming from a place like that [00:07:00] where it sounds like y’all had a lot of give and take and just a nice community going on.

So let’s back way up. You’ve given some hints here and there, but I would love to hear about what your career has looked like, like where has your research focused and why autism and then, again, this transition to private practice.

Dr. Saulnier: Sure. So my career started prior to graduate school. I spent four years working with children with autism within treatment programs at various places. I fell in love with the disorder, with the people, with their families and just wanted to offer them something more than what existed. So that was my motivation to go into academia. I wanted to do research.

So I found Debbie Fein, begged her to get into her graduate program. So my interest in autism was the focus [00:08:00] of my clinical psychology degree. I knew through my entire training, I’m never going to be a therapist, I’m never going to be a quintessential psychologist. I was always in this for autism.

And so I fell in love with assessment. When you work with a neuropsychologist like Debbie Fein, you just fall in love with diagnostic testing, and then the natural transition from there to the Yale Child Study Center, working with Ami Klin, Fred Volkmar, and Sarah Sparrow.

I just honed my skills in early detection but lifespan comprehensive developmental assessments. It was at Yale that I met Sarah Sparrow and naturally got involved in the research on adaptive behavior profiles for individuals with autism and the extreme deficits that they have.

In 2010, when she passed away, completely unbeknownst to me, she literally bequeathed me the Vineland Adaptive Behavior Scales. [00:09:00] And so it was a huge honor to take on that legacy. And then I just started working with Pearson Clinical and over the next six years developed the third edition of the Vineland.

Dr. Sharp: Oh my gosh. Wow. So you found yourself all of a sudden thrust into a test developer role. It’s fun. I’m curious, I interviewed Steve Feifer a week or two ago, and we got into like, how do you even make that transition? And he was saying it was a wild ride. I’m curious what that was like for you to jump into that role.

Dr. Saulnier: Well, what was crazy about it is, suddenly you’re in a business environment, six years, I’m immersed in a business environment coming from academia. So every time I say something like, but the research shows, Pearson sitting here going, we don’t care. We’re a business. We’re selling a product.

And so you just have your mindset is like, wait a minute. And you just have to take yourself out of that academic [00:10:00] role and stick yourself into this business role. And that has been a struggle for me.

There are a lot of things about the Vineland that are absolutely wonderful and beautiful and a lot of things that if I had my say, would be refined a bit more, certainly the computer administration and the platform of it but overall though, it’s been a fantastic experience and a learning curve that I never would have expected.

Dr. Sharp: Mm-hmm. Oh my gosh, yeah. I could see that being really tough. I wonder, so are you allowed or can you talk about like any of those, were there any major sticking points where you were like, but the research says, and that just couldn’t work with the test development?

Dr. Saulnier: That they pretty much right off the bat, nipped in the bud. I think probably one of the sticking points for me was the computer platform. In 2010, knowing that the things that [00:11:00] Q-global couldn’t do then, and foreseeing in the future, we’re looking at 10, 15, maybe even 20 years that the Vineland is now on this platform. If it already has these limitations in 2010, what are we looking for in the future? There was just no wiggle room there.

I even brought in a consultant that I knew from another computer programming company that was saying we could do this kind of thing. It just didn’t come to fruition, but I’m hoping and I’m hopeful that Pearson has been wonderful to work with, that as the Vineland evolves as Pearson evolves, that maybe those platforms can improve a little bit.

Dr. Sharp: Yeah, that’s the hope. I think that’s probably true across the board. Everyone’s going to online or digital administration in some form or another and I have to think that’s where all the money and time and attention are being devoted because it’s…

Dr. Saulnier: Well, can I tell you this? And [00:12:00] I would love your perspective knowing so many psychologists. From a Pearson sales perspective, no one’s buying on online platform. Everyone is sticking to the old paper form.

And is it because we as psychologists are so antiquated that we have a hard time, like those of us meeting old, like mid later career don’t want to shift to something that novel whereas maybe if we only knew computer programming from graduate school on, we’d be more apt to use like the WISC-V Integrated, those types of platforms.

Dr. Sharp: Yeah, for sure. I think that’s a big part of it. I do. I think that the, how would I phrase it? The familiarity is a big factor because even for me, I’ve talked a lot on here about, I was a very early adopter of Q-interactive. I love technology. I like new things and shiny things but I was definitely trained in [00:13:00] paper administration and it’s been hard to totally shift over even though I want to, and I think there’s something to be said for just being locked into your way.

Dr. Saulnier: Well, I’ll tell you, I have the WISC-V and several other assessments on Q-interactive for my new practice, but I haven’t had to use them yet. And every time the phone rings, I’m like, oh God, I hope I don’t have to give the WISC-V because I am that creature of habit that, where’s my paper form.

Dr. Sharp: Sure. Well, I think too, there’s still some, I don’t know if suspicion is the right word, but skepticism may be around the transfer to the digital platform. I think that’s less so with maybe something like the Vineland, but certainly the more cognitive tests, it’s hard to know for sure if it’s going to be equivalent or measuring the same thing. So there might be some hesitation there too. That’s interesting to hear that perspective though.

So [00:14:00] let me jump back a little bit to your research. Can you talk specifically about what kind of research you’re doing in the field of autism?

Dr. Saulnier: Sure. So in adaptive behavior, my research has focused on the gap between cognition and adaptive functioning in autism, particularly in individuals who don’t have cognitive impairment. So that gap is associated with poor outcome into adulthood because we just naturally assume if a person has intact cognition and intact language, they are translating all those skills to everyday life and they’re not whereas if someone has significant intellectual disability, we’re really good as clinicians to make recommendations to say, hey, teach that person to dress themselves, to clean themselves, to feed themselves, to take the bus but we don’t do that for intact cognition individuals.

So that research has been a strong focus. And then working with Ami Klin and my colleagues at Yale and then at Marcus, we really ended up focusing on early detection [00:15:00] of autism symptomatology in infancy, which was fantastic. It was really eye opening to see the earliest emerging signs of developmental derailment.

And so using not only clinical measures and behavioral measures but biomarkers like eye tracking and neuroimaging and genetics and looking at, can we actually detect the onset of autism before the full blown symptom arises by age two? So that research became a big focus of mine.

And then I’d say a third area of research that I got involved with through a network study with colleagues at UCLA and St. Louis was in African American, so cultural differences and cultural diversity in autism. So there’s so many disparities in access to care and delays and diagnosis and then obviously cognitive [00:16:00] presentation. So that area research was something that I was super passionate about.

So now that I’m in private practice, I kept my adjunct associate professorship at Emory. So I still have contact with these colleagues and hopefully will continue to publish and keep one foot in academia in that regard but it’s just I’m a little bit farther removed from it now.

Dr. Sharp: Right. I feel like there’s so much that we could unpack from all of those areas. So I’m like sorting through in my mind, like what is most… could we focus on the markers in infancy for autism? I think I’m super curious about that.

At least in our clinic and I probably should know how this compares around the country but we’ll go down to two-ish, two and a half is when we start to really do assessment with kids for autism. So I’m curious, what were y’all finding from infancy? Is there anything that we could look for as [00:17:00] clinicians or things that…?

Dr. Saulnier: Yeah, in us as clinicians and the thing is, all of us as clinicians, the burdens on our shoulders to make that call. And nobody wants to make a call on a beautiful 12, 15-month-old to say, we’re giving you a lifelong behavioral disorder here. Not to mention that DSM criteria are atypical behaviors that were created for adults and then downward extended to children certainly not infants and toddlers.

So research shows that a lot of the early risk factors are not the presence of these atypical behaviors but the absence of typically developing milestones. So what would cause us concern is limited and different vocalizations; the limited eye gaze, the lack of joint attention.

You’re not going to see hand flapping because 12-month-olds who are typically developing will flap their hands when they’re excited so that’s not going to be a risk flag but [00:18:00] posturing or clutching onto objects would be, and that would emerge into the more stereotypical behaviors that we think of for autism.

And lack of responsivity to name, lack of social contingency. So you smile at a baby, they smile back at you and then it becomes a dance in the same way that we vocalize with infants, that is a reciprocal dance. So ba, ba ba, and then the baby goes, ba, ba, and you go, ba, ba, and it just goes back and forth. That reciprocity is absent or impaired.

So it’s collectively looking at these things, how many of these risk factors are you seeing. By the time they all converge and you have full blown autism at two, when they’re coming into your clinic and most clinics, what we’re finding with research is it’s still optimizing outcome because we’re treating young enough to get rid of language delays in an affiliated behavioral disruption [00:19:00] and self-injury but it’s still a lifelong disorder.

Maybe less than 10% of people even diagnosed at age two will have an optimal outcome. And that’s Debbie Bond’s research. However, we’re wondering if you catch it in infancy as the derailment’s starting, before the full blown disorder is there, maybe you can course correct.

Dr. Sharp: Were you able to think about… could you identify any interventions that early that are more helpful than others?

Dr. Saulnier: People are starting to develop them and they’re still in the experimental phase. They’re very parent oriented, of course, parent coaching because you want that natural interaction between the parent and the baby. I think that it’s just too soon to tell how effective those will be.

Downward extending things like ABA or even some of the NBDIs, the Natural Behavioral Developmental Interventions, Early Start Denver model and JASPER and [00:20:00] things like that that were created for toddlers, even downward extending those to infancy, it’s still premature to say whether or not they’re effective.

And then everyone’s focusing on biomarkers to see if there is some type of fix in the brain. But I think a lot of people in our field, and this could be a podcast in and of itself, is the nature of… we don’t talk about cure as much anymore for something like autism because of the whole neurodiversity movement.

And we are just talking about optimizing outcome and fostering the strengths that every person has and not saying to take away the autism because that in a way is taking away the essence of who the person is. It all depends on who you talk to and how they feel about that.

Dr. Sharp: Oh, of course. We or I talked to Joel Schwartz about neurodiversity two podcasts ago, which was really illuminating. It was good for [00:21:00] me. It’s just that exercise of bending your brain a little bit to think about things totally differently.

Can we talk about biomarkers, where are we at with biomarkers for autism? Is that legit or?

Dr. Saulnier: Well, so my colleagues were focusing a lot on eye tracking. And so, for example, when you have babies watch movies or interactions with caregivers playing, singing to the baby, talking to the baby, even as early as the first six months of life, this is Ami Klin and Warren Jones’s research, babies who will go on to develop autism later in life don’t look at the eye region of the face. They focus on mouth, body, and object.

And so that biomarker detecting that early is more predictive of their future autism than even the ADOS is with itself over time. So that becomes a question, will that biomarker be a [00:22:00] diagnostic tool? So can eye tracking be something in a pediatrician’s office that can predict autism later so that maybe you can do something to course correct?

So my colleagues’ research is actually unfolding into an FDA trial that’s ongoing right now. And if that FDA trial is successful, then you could see this eye tracker be used commercially in a pediatrician’s office for examinations.

Dr. Sharp: Okay. So it is on the way to a product, I suppose.

Dr. Saulnier: Exactly. And then there are so many others that people are doing at other labs and even at Marcus, neuroimaging genetic biomarkers and EEG biomarkers. So it all depends on what type of research, what they’re predicting as what you do once you identify it.

Dr. Sharp: Mm-hmm. Absolutely. I’d like to ask about each of those too but the thing that popped up for me that I imagine others maybe are thinking about is, do you foresee us being out [00:23:00] of a job at any point as behavioral assessors?

Dr. Saulnier: We only hope, if it got to that point that it would improve the lives of people with autism so much they didn’t need us, then absolutely I would bow out and say, I’ll go do something else. However, even if you diagnose autism with a biomarker, you know nothing about how to treat that child. You don’t know that child’s cognitive profile, their language profile, their strengths and weaknesses, their adaptive profile, all of that still requires us.

Dr. Sharp: That’s relieving and well said. Nice. So what about those other biomarkers? Where are we at with genetics and you mentioned EEG. I know that’s out there. Is anything else showing promise in terms of being truly discriminatory for us?

Dr. Saulnier: Sure. I’m talking completely outside my wheelhouse. This is not my area of expertise, [00:24:00] but let’s take genetics. Over 100 up to maybe 900 genes have been identified as being associated with autism.

Each one of those in and of themselves maybe associates with a very tiny percent of autism. So maybe 0.5%, but you put all those together and they account for maybe 20 to 30% of the autisms. So there’s a 30% chance that if you do genetic testing on a person with autism, you’re going to identify a genetic association.

Some of those genetic disorders, you can pinpoint what genes are impacted by the deletion or mutation that they have. Phelan-McDermid syndrome, for example, is a mutation on the SHANK3 gene. There are some experimental treatments, certainly with rodents. [00:25:00] They’re doing the mice models first and then do rats.

If you can course correct the genetics using these treatments. So in a way that the genetic biomarkers would probably be thinking about ways in which you could intervene to change the genetics of an individual.

Sarah Schultz and Longchuan Li at Marcus memory are doing infant neuroimaging. So if they do multiple scans over the first six months of life and they identify the derailment in neurodevelopment and then at the same time you intervene with these parent-focused interventions to keep that infant engaged when they naturally want to disengage, maybe you can change the neural firing of what’s going on in the brain to course correct the brain. It’s too early to tell but we’ll see.

Dr. Sharp: It’s exciting stuff. That’s super exciting. I know that we… there was a discussion in the Facebook [00:26:00] group the other day about the role of genetics in autism. It’s a little different, we’re taking a little different tack and I imagine that a lot of people maybe didn’t see that discussion so I’d like to get into that a little bit, because this is something that has been controversial for me in the past. There was some discussion, differing opinions in the group too.

So what I’m alluding to, for those who didn’t see it, is what is the role of a genetic syndrome or disorder in the diagnosis of autism and do we diagnose autism independently of a genetic concern or does the genetic concern service the umbrella that encompasses all autism symptoms and that is the standalone diagnosis? So I’d be, yeah, if you’re willing to dive into that again, I’m curious.

Dr. Saulnier: Well, I started it, so I guess I have to clean it up. I posted about a case I’m going to see of a [00:27:00] girl with Phelan McDermott syndrome who also has autism and it sparked a debate. Well, when the genetic syndrome of Phelan McDermott override the autism diagnosis. I said, not necessarily because these are the identified genetic disorders that are associated with autism.

So we are basically uncovering the genetic cause that’s controversial but some people would say the genetic cause for that girl’s autism. From eligibility perspective, this girl’s not getting appropriate intervention in her schools because nobody knows what Phelan McDermott Syndrome is or what the symptoms are and so if she’s tagged as other health impaired, she’s going to get whatever type of treatment for the genetic syndrome that no one understands but if she’s listed as autism spectrum disorder with this genetic condition as a specifier [00:28:00] then she’s going to get more targeted social communication interaction treatments that she wouldn’t otherwise get under the other health impaired.

So there’s a debate either way. I now will play devil’s advocate and say that I’ve worked with enough of these genetic disorders to know that when you assess for the autism symptomatology, many times it’s a different autism. I’m still checking the boxes. I’m still giving the diagnosis of ASD but it’s a qualitatively different autism than idiopathic autism, meaning like an autism that doesn’t have a known cause. And so that then can be the argument, why would you then call it autism?

Dr. Sharp: Of course. I’m glad that you bring that up because I was going to ask that question either from your research or colleagues just being steeped in that area; is it fair to say there is a classic autism that [00:29:00] has a certain maybe genotype, if that’s the right word versus this other, like you said, alternate, I don’t know, I forget the word to use, different kind of presentation of autism that still checks the boxes. The symptoms are there but it’s qualitatively different. Can you make that distinction?

Dr. Saulnier: I think because of whatever decisions were made to make it this spectrum. That you have people who are nonverbal all the way to exceptionally verbal, from extreme cognitive impairment to superior cognition and then autism symptomatology that could be mild to super severe, then behavioral symptoms that run the gamut.

How can you say all of that is the same disorder? Well, the common thread is this social disability, the social communication interaction deficits with the restricted and repetitive behaviors. They all have those.

[00:30:00] So maybe it truly is that we’re seeing autisms and they’re all qualitatively different because they all have a different underlying etiology or as we discover them, should we be just naming them something else? And that I think the field of genetics is moving so fast that we almost can’t keep up with it in that regard. So I think that’s still to be determined on where the field will go in that way.

I know there’s a huge controversy right now in the field of how can you call the nonverbal person versus the self- advocate who’s on TV and saying, nothing about us without us and how can you say these are both autism and have the self-advocate advocate for the non-verbal or minimally verbal.

I know that families and even researchers will say, just please call these two things something [00:31:00] different, just so we can move on and actually target and research these specific areas of commonality in those groups, but I honestly don’t know where the field is going to go regard.

Dr. Sharp: Yeah. So do you know if we’re at a place with the genetic research, I know you said this is a little outside your scope but are we at a place with that where we can say like, yes, there’s a well-known enough genotype that equals typical autism versus some of these genetic disorders having a different, now I’m losing the right words but it’s…

Dr. Saulnier: No, it’s hard to say because when we identify a lot of these genetic disorders, by the time your genes take a hit; you’re impaired. A lot of these genetic disorders have intellectual disability, they [00:32:00] have medical comorbidities, seizures, cardiac defects, they can have a host of other things.

So in a way, that is qualitatively different in and of itself than classic autism that really we’re just seeing the core social disability and that’s it, not murky by anything else. So maybe in that regard, these genetic disorders should be classified by their condition and not by the autism but then how do we account for their overarching social disability on top of it?

Dr. Sharp: Yeah, it’s a complicated question. I appreciate you being willing to dive into it a little bit. It is complicated. I’m really curious to see where we go.

I have a mentor down at our local children’s hospital, who I consult with frequently. Her view is similar to yours, if I’m understanding right, that [00:33:00] we would diagnose the autism on top of the genetic disorder to provide that extra layer of specificity or intervention. Exactly. Complicated question, though. It’ll be interesting to see where things go.

Before we totally leave that topic, could we talk about… I guess I have two questions: One, some of those common genetic disorders that may look like autism or produce autism-ish symptoms. And then the other question is kind of the reverse; if we’re seeing a kid, how would we know to refer for genetic testing?

Dr. Saulnier: Sure, the easier answer is if you see the dysmorphology and a certain genetic syndrome comes to mind, then you’re specifically referring, I want this child tested for Williams syndrome or Angelman syndrome or Down syndrome or whatever, based on the dysmorphology that you would expect Fragile X.

[00:34:00] On the flip side, I think there’s enough genetic evidence now that a paper was published. I think the first one came out in 2009 and the second in 2013 by the American genetics. I should know it, and I don’t. It’s like the medical genetics journal and it’s huge and basically said it’s now a recommendation that every individual who’s diagnosed with autism should receive genetic testing.

And so when I make a diagnosis, it’s one of the first recommendations in a report. Go to your pediatrician and ask for an autism screen. So it’s usually a chromosomal microarray and ruling out Fragile X because that’s one of the most common associated genetic disorders unless there’s a dysmorphology that would make someone think just to do a specific gene search.

I was just saying this to a family [00:35:00] yesterday, with a 20-year-old. He had genetic testing three times in the past but when he was a young child, he’s now 20. We know so much more now than we did even last year. So I recommended genetic testing again.

The question that comes most often with a family is why? What difference is it going to make and would we do anything differently based on the findings? A lot of these genetic conditions because of the medical comorbidities is reason enough for me to recommend saying, yes, if your child might have a heart defect, might have X, Y, or Z, that I think is worth investigating.

Not to mention that some of these, not a lot, but some are inherited mutations, so that has an impact on the whole family, even the unaffected siblings of the child with disability in their own family planning. So in that regard, it makes sense.

Dr. Sharp: Absolutely. Yeah, [00:36:00] that does make sense. That’s a little bit surprising, actually. I should know that but it sounds like it’s almost standard of care then.

Dr. Saulnier: I think its standard care.

Dr. Sharp: That’s great. Okay. That’s good information. Well, so that might be a nice segue into more clinical discussion. You’ve transitioned to private practice, so what does that look like for you? What are you doing in your practice?

Dr. Saulnier: Well, it’s fairly new. I just set up the shop in June. I should say I spent the summer having a very slow transition, given the 180 in my career that I did. I took a lot of time off, vacations, and set up my office. I have a beautiful space. I have multiple rooms here. This is my office.

Dr. Sharp: That’s your office, oh my gosh. I thought you were at your house.

Dr. Saulnier: No, it’s my office. I have a testing room. I have a room that’s unused. I have a kitchen. I have a waiting area. It’s big. [00:37:00] I set that all up and then the calls have been trickling in and I have focused so much on infants and toddlers for the past 10, 15 years. I’ve been pleasantly surprised that the majority of the clients I’ve had so far have been adults.

So to me, it’s so exciting to delve into these older individuals and their presentations and really working with families on the flip side of the spectrum than I have been focused for the past 10 years. So that’s been exciting.

And then I wasn’t expecting that the consulting opportunities were going to come in as much as they have, so that balance of trying to figure out, okay, I can only see so many patients because I have these consulting opportunities, several genetics grants that need oversight for [00:38:00] the psychological assessments that go on for the grants.

Then also, so many clinical trials are now using the Vineland as an outcome measure. So being asked to consult in that regard on the new Vineland-3. And I’ve always given workshops around the country on assessment and autism in general. So next week I’m in Alabama and Florida for two different workshops to give.

So I love having that balance because it keeps me in so many different worlds and not just like you were saying, like in that isolated confounds of my actual practice, because then I think I would probably start having a paradigm shift reaction like, oh no, what did I do?

Dr. Sharp: Yeah, I know, it’s nice to keep a foot in both worlds. So let me ask about, we do focus on businessy stuff a fair bit and practice development. I’m curious how you’ve [00:39:00] been building your practice; where are you marketing? How are you marketing? Where are you getting your referrals just starting out?

Dr. Saulnier: Yeah, so I’m still in the process of doing that. You’re reminding me that I had a brochure made that I haven’t even printed yet.

Dr. Sharp: I’m sure you’re not alone.

Dr. Saulnier: I should do that. So right outside my window here is a pediatrician practice and the owner of that practice is my landlord. So I started with her group of pediatricians and educating them on the importance of early detection and to take the burden and onus off of them of saying this might be autism and sending them to me.

The downside is that for now, because this is so new to me and I have been immersed in academia, I’m not taking insurance. A lot of their families who are in need, I can’t see right now or I’m not seeing right now because I don’t take any insurance.

Dr. Sharp: It’s hard.

Dr. Saulnier: So it’s word of mouth. The Marcus [00:40:00] Autism Center, my colleagues there, they have 700 to 900 people on their wait list for assessments. So…

Dr. Sharp: 7900?

Dr. Saulnier: 700 to 900, 700 or 900 people on the waitlist.

Dr. Sharp: That’s still a lot

Dr. Saulnier: Is remarkable. There’s no shortage of need in Atlanta. There’s an Emory Autism Center. There is the Marcus Autism Center and there are a host of people in private practice and there’s still a huge demand.

And then others are just word of mouth through my career. So the person that I saw yesterday is an international family and that was solely through connections of being in my career.

Right now, they’re just coming in at a slow pace, which I’m okay with. When I need them to start coming in more, I’m going to be on The Testing Psychologist asking for recruitment advice.

Dr. Sharp: We would welcome that. It’s a whole another level. I feel like there are levels to this, [00:41:00] right?

Dr. Saulnier: Yeah.

Dr. Sharp: You start and then you figure out how to grow. That’s great. I was going to ask about the battery. So let’s get real practical. You’ve done all this research; you’ve got that side. How does that translate to clinical practice?

Dr. Saulnier: So my ideal battery for autism would involve a developmental and or cognitive assessment and adaptive behavior assessment, a diagnostic interview and then the diagnostic assessment. So I would always do an ADOS. I wouldn’t necessarily do an ADI unless I feel the need to, but I would do a very comprehensive diagnostic interview. And then…

Dr. Sharp: I’m sorry, can I interrupt you real quick and just break that out a little bit. What would convince you that you need to do an ADI-R?

Dr. Saulnier: So, for example, I had an adult two [00:42:00] months ago where I wasn’t sure based on the ADOS that this person had autism. He was a head-scratcher and I felt, I can sit here for an hour and just think of questions to ask about diagnostic symptomatology with this mom or I can just give an ADI and have the ADI do it for me.

That was super helpful because he had a prototypical early development for autism that I can now use this measure to help corroborate my diagnosis and yet I wouldn’t recommend that for traditional clinical practice. It’s too cumbersome. It’s never reimbursed but for research purposes, it’s ingrained in my soul because in research, you have to do the ADI and the ADOS.

And so I’m very much about that comprehensive view. [00:43:00] What I’m missing that I am so used to having in academia is a speech pathologist standing right here next to me, especially in infants and toddlers. That differential is critical about the speech language and communication impairments.

I have them on speed dial in the event that I need them for my evaluations. Ideally, I got this space so that eventually I can have multidisciplinary staff in the clinic so that I can get back to that.

Dr. Sharp: Is there anything that we can do as psychologists to do a lay person’s speech assessment if you don’t have that luxury, anything we would look for with those younger kids?

Dr. Saulnier: That’s a great question. If you know they’re going to have impairments, then doing a battery like the PLS or the self, those are going to flesh out your wrote speech and language [00:44:00] areas of vulnerability.

It’s the higher-functioning kids that are going to ace those the same way they’d ace a WISC. They’re going to have average to above-average scores. They’re going to get average to above-average scores on the language measures too.

Good speech pathologists know the measures to get into the theory of mind and the linguistic skills that trip up our higher cognitive folks, like the Test of Narrative Language or the DELV and some of these other ones.

I guess, as psychologists, we can learn them and administer them. It’s just the expertise of the speech pathologist that I feel is critical. Also for the nonverbal and minimally verbal, the augmentative communication assessments, it’s just something I’m not going to be able to do.

Dr. Sharp: Sure. A whole another world. What do you like for that early developmental assessment measure? So those kids who or maybe can’t do the WPPSI or might [00:45:00] be just too young or too nonverbal?

Dr. Saulnier: This is going to cause a ruckus in The Testing Psychologist but I come from the research world where the Mullen Scales of Early Learning is the go-to test and it’s so antiquated. The Mullen are from the late 1980s. It has wonderful strengths and I can do it with my eyes closed. So it’s my go-to measure.

Bayley, obviously it would be the go to test, except the Bayley only goes to a much younger age. I don’t even know if it’s four whereas the Mullen goes to 60 months. And so for your preschool-age kids that are impaired, you can still get standard scores with the Mullen whereas you wouldn’t be able to get with the Bayley. You’d have to move to something else.

If they are a little more advanced than the Differential Ability Scales; that’s my go-to cognitive because it has the teaching items and the Wechsler [00:46:00] scales do not. The Wechsler scales and even the Stanford-Binet are so verbally loaded that it’s very hard for people with autism, even with intact speech, to comprehend that level of instruction without you being able to demonstrate what you’re asking of them.

So you might misinterpret a cognitive impairment when it’s really, they don’t understand what you’re asking of them. They have this skill, the ability, they just need to be able to show you.

Dr. Sharp: Right. Sure. Yeah, that’s an ongoing question in the group but also, it’s just relevant right now in our practice, I keep talking with our folks about what do we get for those younger kids where we need to. It’s hard to find the one, and of course, we’re trying to balance costs with effectiveness and it’s hard to find the one measure that works best.

So in addition to the clinical practice, you do, like you said, some consulting. It sounds like on [00:47:00] a bigger level, though. You’re consulting with groups and not so much individuals necessarily on their own practices.

Dr. Saulnier: Oh, right. No, with academic groups or professionals like educational institutions, I do a lot of my workshops for the field of school psychology. So I’m going in and training the school psychologists who are assessing the kids within the school systems, which is a whole world in and of itself; school psychology versus clinical psychology can be a different world and I love school psychologists, and I love going to NASP. It’s so much more fun than APA. I just love it.

Dr. Sharp: And it’s going to be in Atlanta next, right?

Dr. Saulnier: I know. I’m doing a workshop.

Dr. Sharp: That’s right next door. Nice. I’m thinking about going to that one.

Dr. Saulnier: Oh, let me know so we can meet. [00:48:00] I would love to get into individual consulting and working with families and especially on an international level.

I have some wonderful colleagues and friends who do that already and they go all over the world and they’re doing assessments but also informing treatment and then they do a lot of telemedicine once they get back here. I would love to move into that arena a little bit more at some point.

Dr. Sharp: Sure. I can hear people as they’re listening, thinking, oh, that sounds great. How could I do that? Do you have any idea how to do that? How would someone move from practice…?

Dr. Saulnier: What I found through speaking engagements, it’s all, you give one, someone hears you and likes it. They refer and you get another and get another. So I would imagine that the international consulting is the same way; you work with one [00:49:00] family, if you can get the inroads to one family and you do a good job, that family is going to talk and refer to another family.

Dr. Sharp: Right. I’m going to ask a dumb question, which is, do you have any idea where to find those international families? Are they here for school or…?

Dr. Saulnier: It’s through probably conferences and speaking engagements is the best way. If you have an opportunity to present at an international conference that has parents, family members or school providers there, and they see you present and say, you know what, this is worth us traveling to find out more about.

And then places like Marcus and certainly Yale, they were just international in and of themselves. People would come from all over the world to be evaluated at the Yale Child Study Center. So I just benefited in that regard by being in the institution and being [00:50:00] surrounded by the colleagues that I was, so got that exposure in that regard.

Dr. Sharp: Sure. As we talk, it seems like you’ve either, I don’t know if it was deliberate or not, but you’ve been in so many fortunate situations to really set up your career path. That’s really…

Dr. Saulnier: Absolutely. I wouldn’t change a thing.

Dr. Sharp: That’s awesome. That’s really cool. And the way that you speak of it, it’s clear that you care about what you’re doing and are just highly invested.

Dr. Saulnier: Oh, thank you.

Dr. Sharp: I know that people can’t see us. We’re doing a video interview as well and your face is just beaming. And this whole time you’ve been super animated. It’s really cool to see that.

Before I let you go, I would be remiss not to ask about the difference in boys and girls with autism.

Dr. Saulnier: That is another topic near and dear to my heart. [00:51:00] Clinically and anecdotally, every time you do an assessment on a girl, you’re thinking, wow, they’re so qualitatively different than boys. Every once in a while, you’ll get that prototypical classic girl but that’s rare.

And so even those you think this still is closer to the prototypical girl because they made it to the clinic to be evaluated, the majority of girls are out there not getting detected and falling through the cracks because they have enough of a social ability that people think that can’t be autism and yet they are struggling so significantly.

And what happens is because they go undetected and they have enough social ability to know that they’re struggling, their psychiatric comorbidities are striking. So the anxiety and depression and even suicidality is really severe. And that’s usually what brings the person [00:52:00] into an assessment rather than the autism symptomatology.

And certainly in the workplace, when there is a lack of structure and support that they have in the educational system, then they’re going to fall through the cracks there. So there finally are initiatives, both research-wise and clinically, I think, to better understand the female presentation.

I sit on the scientific advisory board for the Autism Science Foundation. That organization was founded by Alison Singer, who is a mom of an adult girl. She started the Autism Sisters Project, which is a research-based initiative to focus on the unaffected girls in families of people with autism that get missed or why are they unaffected. How are they spared? What’s the female protective effect that we have so many more boys than girls?

Finally, people are doing research on this, and I think [00:53:00] the answers are still yet to come, but we know there’s a qualitative difference and we know that there probably should be differing diagnostic criteria. Some may argue differently from me but we’re getting there.

And so I think that if there are anyone out there that has a girl and they’re perplexed and they’re scratching their head, don’t discount that it could be autism.

Dr. Sharp: Right. I know you mentioned anxiety, depression, suicidality, perhaps. Are there other things we might look for in girls to at least be curious about autism? Other things you might see that we’d want to pay attention to?

Dr. Saulnier: I’m trying to think, if there’s anything qualitatively different in stereotypical behaviors, they might not be as pronounced or prototypical. If a girl has a genetic defect, she takes a harder hit. And that [00:54:00] raises questions about the protective effect.

So on the flip side of the higher cognitive girls that fall through the cracks in school, you have the really impaired girls that probably are just carrying a diagnosis of intellectual disability in their genetic disorder and no one’s thinking autism’s the forefront either.

So I think just knowing that there are those two varying presentations of girls is good and just not relying so much on one diagnostic measure and using your clinical judgment and knowledge because a lot of the girls that I’ve diagnosed with autism wouldn’t meet criteria on DSM-5 and/or an ADOS, for example.

Dr. Sharp: Yeah. I’m so glad you said that. That question was sort of formulating, but it really crystallized when you said that. I’ve seen many girls who, like you said, would not strictly meet the criteria but they had [00:55:00] autism, so I’m curious how you might phrase that in a report to justify, I don’t know if that’s the right word, but explain, yes, this is a girl with autism but it looks different than… how do you work around them?

Dr. Saulnier: That’s when I might pull the ADI because their diagnostic history, their early developmental history is often telling that they’d have enough criteria and remember the DSM-5 is current or by history, so you can use the history to advocate in that regard.

Other measures, if they’re old enough to do self-report measures of things like the Social Responsiveness Scale because the SRS actually has norms for males and females, which is nice. The ADOS doesn’t, other measures don’t go by gender.

And really it’s clinical judgment. Making a strong case [00:56:00] for your whole comprehensive assessment and why you think, and then educating the reader on presentation in girls, and maybe including some references in there of some research that’s been done, that this is a messier presentation for lack of a better way to describe it and it’s not going to be prototypical.

Dr. Sharp: Yeah. It’s good to hear you say that. That’s the approach that we take here at our clinic. It’s like an extra paragraph that goes in the report that basically says autism in girls is different than autism in boys. Here’s what it might look like and here’s why it might not be caught. Okay. That’s so great. It seems like we’re on the same page, which makes me feel good.

Dr. Saulnier: Me too. It’s nice to hear that from you.

Dr. Sharp: Nice. I’m glad we can agree. We’ve covered so much and I just really appreciate all the time that you’ve given us. Before we wrap, are there particular resources or researchers or projects out there that if people want to learn more, you might point them in that direction? [00:57:00] Where can people go to learn more?

Dr. Saulnier: The Autism Science Foundation is a phenomenal resource for scientific research going on and also for anyone with questions about vaccines, there’s a whole page of resources in the literature out there to say vaccines don’t cause autism. So you just have to send a parent in that direction to say, look here, that’s it.

Autism Speaks and Autism Society of America for Families. And then, for exciting research going on, then there’s the International Society for Autism Research, INSAR. They have an annual conference every year, and this year will be in Montreal, in May.

And then certainly places like the Yale Child Study Center and the Marcus Autism Center and lots of other wonderful places throughout the U.S. and the world.

Dr. Sharp: Nice. And if anybody happens to want to check in [00:58:00] with you or ask you questions, are you open to being contacted?

Dr. Saulnier: Anytime. So my email is celine@nacsatl.com.

Dr. Sharp: Got you. That sounds awesome. I’ll put all of that information in the show notes so that people can find that and continue to learn more about this topic, which is fascinating.

Dr. Saulnier: Thank you so much. It’s been such a pleasure being on.

Dr. Sharp: Oh my gosh. Yeah, no thank you for being willing to come and chat. This has been fantastic. I really appreciate it.

There you have it, everybody. What an interview, my gosh. I kept coming up with these questions and I was like, this one’s going to stump her but nothing ever did.

And y’all couldn’t see, I’ve been doing video on my podcast lately, which has been really a nice enhancement to the whole process and I think helps with [00:59:00] the conversation flowing naturally, but y’all couldn’t see but Celine, like I said, at one point, just had this amazing smile on her face and presented such a positive attitude throughout the interview.

She was really a pleasure to talk with and I highly recommend if you ever had the chance to cross paths with her to make that happen. She’s very knowledgeable and very personable.

I hope you took a lot from this interview, I sure did. Got some additional interviews coming up that I think are going to be pretty fantastic as well. In the meantime, if you are building or growing your testing practice, feel free to reach out for any support that I might be able to give you. There are a number of paperwork packets that might be helpful in your private practice journey. I put together an administrative packet, a clinical packet, and a psychometrist training manual that are all available at [01:00:00] thetestingpsychologist.com/paperwork. You can use the code podcast to get 20% off your entire order.

And if you’re more in the mind for one on one coaching, reach out to me as well. I have, I think at this point, maybe just one more spot available for one on one coaching. And then I’m going to close that down and really transition more to doing mastermind groups because that’s what I love to do and I think people really benefit from that format. So probably one more spot for individual coaching before that closes for a few months.

And if you’re interested, reach out, again, at thetestingpsychologist.com and we can talk for a few minutes complimentary and just see if that’s a good fit for you. And if not, talk about what might be a good fit.

All right, y’all. Take care and enjoy the fall, whatever that looks like for you. We did not get a whole lot of fall. We got 8 inches of snow a week ago and all our leaves are gone. And we’re just headed full [01:01:00] steam into winter, but hopefully, you’re having a little bit of fall. I know my family in the South is actually in an unnatural state of heat right now, so maybe some of y’all are too.

Either way, take care. Talk to you soon. Bye bye.

Click here to listen instead!

[00:00:00] Dr. Sharp: Hello, welcome to The Testing Psychologist podcast. I’m Dr. Jeremy Sharp. This is episode 72. Today, I’m talking with Maureen Werrbach. Maureen knows everything about running a group practice. So if you have any aspirations of hiring other psychologists or clinicians in your practice, or if you already have hired and you’re thinking about hiring more or just need to tune up your systems, Maureen is a great person to know about.

She has a group practice in Chicago that now has 25 clinicians. Their revenue is incredible and the profit margins are even more incredible. She really knows what she’s doing. She has all sorts of resources for group practice guidance. She has a Facebook group, she has a membership community and she does coaching as well.

She shares a lot of her knowledge with us today. This is a little bit of a departure from our expert series on [00:01:00] testing topics, but a lot of people ask about how to run a group practice and how to start hiring, and you’ll get those answers today.

All right, onto our conversation.

Hey, y’all. Welcome back to another episode of The Testing Psychologist podcast. I’m Dr. Jeremy Sharp. Our guest today is a little bit of a break from some of our recent interviews. Today, I have Maureen Werrbach on the podcast. Maureen is the group practice guru experts, I don’t know, what word do you like? What are you?

Maureen: I don’t know, coach.

Dr. Sharp: Coach, don’t sell yourself short. [00:02:00] So Maureen’s going to talk to us all about group practices and hiring and expanding and growing. She is amazing. Like you heard in the introduction, she hosts The Group Practice Exchange. She has a ton of resources for folks who are doing group practice and she’s talked me off the ledge two times so she knows what she’s doing and is really good at this stuff.

So I’m just thankful that you’re here. Welcome to our podcast.

Maureen: Thank you for having me. As you know, you’re my only guest who’s come on my podcast twice, so we have a nice little relationship going here. So I’m really excited that I finally get to jump onto your podcast.

Dr. Sharp: Yeah, absolutely. I know. I like to repay the favor. It’s so good to see you again. It’s hard to know where to start. I feel like you certainly have been doing this for a long [00:03:00] time. You have a relatively large practice in the Chicago area. You have therapists, psychologists, you are doing some testing. You also psychiatric practitioners. You really have mastered the group game, I think.

But I get a lot of questions around group practice and how to expand and when to do it and how to, all that kind of stuff. So we’ll just see where it goes and hopefully pull together some useful information for folks with testing practices.

Maureen: Yeah. So when it comes to expanding, I’ll start from the beginning a little bit and then happen to maybe expanding once you’re already a group, but if you’re a solo practitioner like I was, did you start as a solo practitioner too?

Dr. Sharp: I did.

Maureen: You did. Okay. I know some people who go straight to starting a group so I wanted to make sure. [00:04:00] I started as a solo practitioner and some of the things that … Initially I never thought I was going to own a group practice. I’m pretty introverted. I like to keep to myself. And so if you looked at my history, you wouldn’t have guessed that I would have been someone who was leading and managing other people. You would have guessed I had my own business, though. I was one that likes to do my own things and be in charge of myself.

When I went into solo practice, there was a part of me that, one, realized that there was only so much I could do for the community and for some weird reason that struck a nerve with me that I could only do so much and I realized that there was such a need just in my area, where my main hub is located. Part of me also felt that loneliness factor which a lot of solo practitioners talk about. And [00:05:00] also I had a lot of referrals coming in, and so I knew I could or I assumed I could fill a therapist’s schedule.

I like to stop here because this is probably what most people think about when they think about starting a group practice. And now that I’ve done it, I like to talk to what would I say to my younger self sort of thing. I’d like to say that the best thing to do when you’re thinking about expanding into starting a group practice is getting some business plan in place. I didn’t.

I was lucky enough because I probably wouldn’t have done it for a very long time but I was lucky enough that Chicago, the Department of Treasury, was having a small business plan competition for the city of Chicago. If you won, you were to get some amount of money to help you expand. And so I was like, I’ll do it.

And it really helped me really put together what I wanted my group practice to look like, what I thought it could [00:06:00] look like that would be in line with who I was and what I wanted the community to see in me and my group practice. And that really helped me solidify, one, that if I should have gone the group practice route because at the time I had one person I’d hired already.

So I feel like the thing that people miss most is doing that business plan in the beginning and they wing it similar to how I did. And you’re just going to learn from a lot more mistakes doing it that way. It helps you really figure out what direction do you want your group practice to go. How big do you want it to be? Is it something that you’re wanting to scale in the future?

And obviously a business plan is, it’s ever changing, it’s evolving, it’s not static, it moves but it moves with you as you grow. So mine has changed along the way but I think I would have made a lot more random detours and adjustments had I not had that business plan.

Dr. Sharp: Sure. [00:07:00] I think, at least for myself, I got scared off by doing a business plan because the ones, the templates I looked at didn’t seem like they made sense for us and it was overwhelming. I was like, I don’t know these terms. So what does a business plan look like for someone who’s thinking about building a group practice then in mental health?

Maureen: You can go on the IRS website and they have a template that lists the main things like your business description, your market analysis and your financials. The business plans will all look the same whether it’s in group practice or in banking or in any other kind of line of business.

What I found that a lot of people don’t know about is, yes, you can go on the IRS. If you google IRS business plan template, it’ll give you a template of what a business plan should look like but it’s hard to know what information to put in the categories when it comes to our line of work because some of the terminology just [00:08:00] doesn’t connect with the things that we do.

And so what I ended up doing was, and every state has this. They have business where, what did I go through? I’m trying to think of the name of the website, but it’s like a … For free you can use, your city has people who help small businesses write their business plans. That’s what I ended up using, was a Chicago business association who helps people write business plans and that’s what I used initially. They were able to help me. They didn’t know much about my industry, but they were able to help go into detail of what each piece means.

What I see is that the group practice owners who do write a business plan end up writing a very simple one, that’s one or two pages. A really good business plan requires you to do a little bit of work. There’s a section that’s called market analysis and people usually just write what their marketing strategy is going to be [00:09:00] but market analysis includes going to your local chamber of commerce and asking for statistics on who the people are in your community where you want to have your group practice or where it’s already established.

It helps you know what is … if you’re … I’m just maybe throwing things out in the air right now, but if you’re a child therapist and you want to do testing for children but you find out that later on, because you didn’t check with your chamber of commerce or you can google sometimes, some areas will have the comps on Google, is that you might find out that your average household size is two and the average age is 45 which tells you that it doesn’t seem to be a lot of families there. And if you’re a child therapist, that might be a little rough.

I’m just giving one example, but this is one of the things that really [00:10:00] going in and doing the research on your area which is one piece of the business plan helps figure out what is my community need and can I make that happen because that’s part of what makes you grow a successful group practice is that you’re offering things that are in line with what your community needs.

Dr. Sharp: Right. I don’t know if this is relevant or not, but have you run across any kind of hard data on how many therapists are needed for a particular population level or anything like that?

Maureen: I haven’t, but I like to go from that abundance mindset that we have way more people in the world than therapists so it’s very likely that it’s not going to matter much even if you’re in a very heavily saturated therapist area, but I don’t have any real hard data. I haven’t seen anyone who’s done the work, the back end work, the research even those that are in heavy therapist areas who weren’t [00:11:00] able to be successful and expand and have a group practice.

Dr. Sharp: Sure. The only way that I’ve dived into that was when I was trying to request increases from insurance companies and I did the math around how many contracted providers there were versus children in the school district and stuff like that. And it was way more than I thought. There was something like 5,000 kids for every therapist who was in network with that insurance company or something.

Maureen: Did you use that information?

Dr. Sharp: Yeah, I did.

Maureen: Did you get an increase?

Dr. Sharp: I did not.

Maureen: What?

Dr. Sharp: I know.

Maureen: Oh my gosh. I was waiting for a success story because you know me with those rate increases.

Dr. Sharp: I know. Well, I got that from you. I put a lot of work into that particular one, but it’s like our holdout company. It’s Unite or Optum, and they just won’t.

Maureen: And when was the last time you did it?

Dr. Sharp: Six months ago.

Maureen: Try again. [00:12:00] Optum gave us a huge increase just very recently.

Dr. Sharp: Really?

Maureen: Mm-hmm.

Dr. Sharp: Okay.

Maureen: Very big double.

Dr. Sharp: Double?

Maureen: Yeah, so I would check. I know that it’s likely very specific and it was across the board. If you asked for it, they give it in Illinois. So if you’re in Illinois, listening and you haven’t gotten an increase with United, ask.

Dr. Sharp: Yeah, I know we have listeners from Illinois

Maureen: Yeah, I would try again with them. The way I do it, I know we’re going off topic slightly with the rate increases, but most people give up after they write a rate increase request to an insurance company and they go, oh, okay, they said no. Take it a step further. Probably half the times I will get an increase when I ask for it, but the other half of the time I get a no.

And out of that half, I send another letter saying something along the lines of, I’m going to have [00:13:00] to reconsider my status with you guys. We have X amount of providers that just, financially we can’t work with the rate that you’re giving. I’m a little bit more stern and a little bit more … there’s a little bit more fire under my butt with that second letter, and it’s just usually just a response. It’s not a whole detailed letter, but a good chunk of those will say, let me move it up the chain. It’s as far as I can. I can’t do anything anymore, but let me move it up to my superior, maybe they can do something about it.

I think of it as oftentimes insurance companies just assume that people will give up and so they’ll say no at first and then people won’t ask for it again. Try again.

Dr. Sharp: Sure. I like that. I need that encouragement. I should say, I’ve had success stories too. At the same time that rejection was happening, I got an increase from another panel. That was fantastic and it was really cool.

I did that thing that you’re talking about. [00:14:00] They actually came back and offered something, some raise. And I was like, eh, testing’s really specific and it’s a specialty. Can we get those rates a little higher? We can leave therapy the same, but can you raise the testing rates? And she came back and gave a raise with the testing rates in particular.

And I think that’s applicable for a lot of people listening too, that testing is such a specialty. In a lot of communities, we are in pretty high demand, especially people that take insurance for testing.

Maureen: And oddly enough, testing is underpaid by insurance companies.

Dr. Sharp: In lot of places they are.

Maureen: I’m really actually surprised, when I brought on my first testing person, I assumed insurances were going to pay out more than they do for traditional one-on-one therapy. That isn’t always the case in terms of reimbursement by insurance companies. So I think more than anything, if you have a testing practice [00:15:00] and you’re in network with insurance, requesting those increases often.

And if you get a no, you respond again right away. And then if they say no, again, you wait six months and you request again. You annoy the hell out of them until they give it to you.

Dr. Sharp: I like it. Number one strategy, annoy them to death. All right, let’s dial it back a little bit. So for folks who are really thinking about expanding, you mentioned getting a lot of referrals and you thought you could probably bring someone else on. Is there a way to gauge that for sure? How do you know when you can expand and hire someone referral-wise, business-wise?

Maureen: I like to say that this really can depend on your risk level as a person. There’s going to be people. I was just coaching someone who is starting a group practice. He’s starting from zero, not a solo practitioner. I think that’s higher risk. He hasn’t yet figured out if it’s going to [00:16:00] work. He hasn’t yet got his own caseload where he knows a little bit on how to market himself for whatnot. He’s going jumping right in.

So there’s certain people that are willing to take a little bit more risk when it comes to business and then there’s those that take less risk, which I feel like … and anyone who wants to start a group practice has some level of ability to take risks, otherwise, you wouldn’t own a business. My mom always says, I would never have a business. I just had nothing. I have no willingness to take any risks beyond what I know is 100% possible. So obviously, anyone who owns a business, they have a little bit of ability to go into the unknown.

But when it comes to your specific question, I truly think that you can go the slower risk route, which is what I did. I did one person at a time. Once I solidified that starting a group practice was something that I really wanted to do, that I had the means and the ability to supervise [00:17:00] people and to make executive decisions that maybe not everyone would be happy with.

There’s a lot more than just bringing on someone to work your off hours, which is what a lot of people do initially. I just will bring someone on and they can work the days I’m not there but a group practice is more than that.

And so once you figure that out, if you’re someone who isn’t, not risk-averse but it’s low risk, likes to take little risks at a time, similar to how it was in the beginning, the way I did it and I’m a metrics person, I know you and I talked about that before we went live today, is I like to have metrics that show me if something’s working or not. Otherwise, I tend to be …

I have ADHD so my brain is on all cylinders at all time thinking about this thing and then this thing and then let’s do this. And so for me, I need to have metrics that show me if all of these crazy things that I’m doing, are they working? Are they not working? And so [00:18:00] I think if you can figure out a way to have some level of a metric, whether that be, I get five new calls a week that I can’t schedule because I’m full, that’s a metric. It’s telling you that there’s a need beyond what you can offer.

And I think with hiring your first person, the easiest thing is to have that. Is to just have calls coming in. It doesn’t have to be 100 calls. It can be two calls a week coming in that you cannot fill that you are too full to take. And that, at least, will let you know that you can give that to someone else.

And now there’s a whole other side to the story of making sure that you’re hiring the right person, looking at what kind of calls are coming in. So being risk taker or not, you can hire someone because you really want a Reiki therapist and no one’s asking for it, but you’re willing to market for it. That’s a little higher risk.

Lower risk would be, wow, I have a lot of couples calling for couples counseling. I think my first hire is going to be a couples therapist. The risk is [00:19:00] a little bit less in that you’re more likely to be able to more quickly fill in. And it’s not to say that one is better than the other, but I think it comes back down to what are you comfortable risking? Are you able to be comfortable in higher risk situations, which some people are? Like my coaching guy that I’m telling you about, who’s starting a group with five people he hired all at once.

Dr. Sharp: Oh, my gosh.

Maureen: It’s not impossible. It just means you have to have a certain level of something in you to be able to muster through a longer period of stress figuring out what a group practice is going to look like when you get it plopped into your lap. I don’t think one is any better than the other.

And I do think that as you hire, once I hired my second person, I actually hired them two people at once. And then once I was at then four or five, including me, I was able to hire three people within a month [00:20:00] period. It gets a little bit easier to expand. I look at it as like a snowball effect. Once the snowball is rolling down the hill, it gets bigger on its own.

Dr. Sharp: Absolutely.

Maureen: It’s the beginning parts that most people have the most concern with when it comes to expanding. For me, I always say, if it’s a concern, then why not go the easy route and you hire a person when you have some referrals that you cannot fill yourself? That’s the lowest risk version and you have the metrics to show you that you can do it. You got calls that aren’t able to be scheduled that you’re referring out.

Dr. Sharp: Yeah. I like that you break it down like it’s doesn’t have to be complicated. You basically just track your phone calls for a few months or weeks or whatever it might be to see where they’re coming from and what they’re for. It’s a great place to start.

Maureen: That’s the easiest route. Obviously, if you’ve got [00:21:00] time and you’re a little bit more business savvy, most of us are not business savvy from the get-go, and so going this easy route is just easier. But there’s obviously 100 other ways that you can do it.

I’ve seen success in group practice owners who start physically as a whole group versus starting solo. You then have to have that grit inside to be able to work through adversity and potentially things not going well in the beginning, that’s also a possibility.

Dr. Sharp: Yeah. I wonder, are there any things that you’ve done or things you’ve read or ways that you’ve engaged in personal growth to be more of a business owner to deal with some of those things over the years?

Maureen: Yeah, I think the one thing that I do consistently is because I’m not a business major and I’m pretty introverted, in terms of leadership skills, [00:22:00] it was not my initial. I’m a very nice person. I never yell. So I have all of those great qualities. When I first started my group practice, I didn’t have this innate take charge and lead type of mentality. I am in my own life but not when it comes to …

When I was in school, if I could choose between working with a partner or working by myself, I would work by myself. So I knew my strengths and I knew where I needed to have work done. And so something that I still do and I’ve been doing it for years is reading books on leadership. If you’re thinking about starting a group practice, you should read Radical Candor. It’s a great book on leadership, on how to manage difficulties that’ll come up with interpersonal staff issues.

I also read the book Grit very recently. And that talks more about this innate level of grit that everyone who’s a business owner needs to have to be able to be [00:23:00] successful. There’s only so much that your smarts can do and then there’s this whole underlying level of this grit. It’s like, if you say you’re going to do it, you’re just going to do it. You’re going to figure out how you’re going to do it. And so that book was really good because it just talks about grit as a business owner.

Profit First, obviously, most of your listeners know about that.

Dr. Sharp: I have talked about Profit First book

Maureen: Yeah, most of us are not finance gurus and so we likely as a new group practice owner will pay everyone else first but yourself. You pay your expenses first and you pay your payroll people first and then you pay your taxes and then you’re like whatever I’ve left over, okay. I guess I have $100; I’ll give that to myself.

Profit First was a great book in helping me shift how I looked at money and my relationship to money. So those are the first three books that I’d recommend when it comes to just getting into a leadership mindset and into a business [00:24:00] mindset.

Dr. Sharp: Awesome. I’ll put those in the show notes for sure. I’ve read two of those; Radical Candor and Profit First and love both of them. I thought Radical Candor was awesome.

Maureen: Yeah, me too. It changed how I communicate with my staff and it changed part of the structure of my business because that whole idea of rock stars and superstars, that you’ll have some staff that are rock stars who are just … they’re rock solid in their work and they do their work really well but they have no desire to move up or move forward. And then the superstars are those that do want to move up to move forward. They tend to be the people that leave the practice to start their own practice.

In our industry as group practice owners, tend to not do anything about those superstars and then we get angry because they leave. And so I’ve shifted everything. I have a huge management team now, from site supervisors to clinical director to onboarding coordinator.

I literally, probably since I last talked to you, made a lot of shifts in having more people who are [00:25:00] invested in their business, who do great work, who have certain skills that I was like, I could find a good useful position to help you feel like you’re, that take your investment in my business and give you something special, something different than just a therapist and it’s played a huge role in connecting the therapists and giving them a greater sense of purpose.

Dr. Sharp: Sure. I think that’s super important. I know that’s getting down the road a little bit for folks thinking about having enough staff to even do something like that, but it totally makes sense to at least be looking at your folks in that light, like what might they need besides just doing the work or are they doing the work?

Maureen: I got to say, my clinical director, I had her become a clinical director with clinician number three. I was in my first year. Most people don’t do it but …

Dr. Sharp: I did not know that. I thought it was later on. Okay.

Maureen: No, it was [00:26:00] myself, her, the first person I hired, so she was the second person I hired and then a third person. She only had to manage two people. I knew where my strengths were and I knew where they weren’t. I knew what I liked doing and I knew what I didn’t. I don’t like supervising people. I just don’t. I like to do the business stuff; the metrics, the background, the growth, and the expansion.

I don’t like the day-to-day helping them figure out how to do a note in therapy notes or I’m having an issue with my couple, what other skill can I do? Those are things I don’t want to do. And so from the beginning, I decided, as I hired people, that I would look out for that person that could fit that role. And so when I hired her, she was a clinician for about six months or a year, I can’t remember. It was back in 2012 or 2013, but she was working for a little bit with me and then I was like, I think she’s the one, and [00:27:00] she’s still the one.

Dr. Sharp: That’s awesome.

Maureen: It’s something to think about from the beginning because one of the biggest struggles group practice owners have in the beginning is that they wear every hat. They’re doing the supervision, they’re doing the hiring, they’re doing the staff management, they’re seeing their own clients, doing testing, whatever it is that they’re doing.

Then they’re doing business management stuff, making sure they’re doing payroll and marketing and networking, everything. And so one of the key things that we need to do is to let go of some of those hats and wear the hat that works the best for us, that we feel energized by, that we’re good at and to give those other hats to someone else.

The issue comes up is that people think they can’t afford it. And this is why I like to use my clinical director as an example because I was barely there. We just had three people, and they were still growing at the time. In my head, I thought, if someone is meant to be in a higher level [00:28:00] position, whether it’s a supervisor or clinical director, if there’s something that you can’t do or you feel like you’re not good at when you’re a new business owner is the idea that if that’s something that is their strength, they’re going to be so excited about that, that they’re going to be willing to grow into that position.

So I didn’t hire her to do 40 hours of clinical director work. She did one hour a week. She got paid for four hours of clinical director work a month because we only had two people. They were fully licensed so they didn’t need supervision every week. She was there to be available if issues came up. Obviously, almost every group practice owner can afford to pay one hour of clinical director work a week knowing that that gets taken off of their shoulders so that they can do something that generates more income.

Dr. Sharp: Exactly.

Maureen: Just a little tidbit.

Dr. Sharp: Well, no, I think that’s really important to be thinking about from the very beginning. I talk so much on here about delegating and outsourcing and all that kind of stuff, that we shouldn’t wait [00:29:00] to start doing that, think from the very beginning, like what do I really want to do? What am I not good at? What am I good at? What do I have time for? What do I not have time for? And be thinking about who can do those other jobs for you. And maybe it’s a clinical director, maybe it’s a VA, maybe it’s a biller person, but having that in your mindset from the beginning is important.

Maureen: Exactly. And every time I’ve done it, every time I’ve taken a risk, financial risk, I should say, by bringing someone else on to do something that I could be doing for “free” not really free because my time is worth money but I would feel this way and I’m sure you have and every other new to group practice owner will say, well, if I’m doing it, then I’m saving money because I don’t have to pay someone to do it but your time is worth the most. You’re the business owner, so you got to put a place of value on that.

But every time I made a jump to hiring someone or bringing someone on that wasn’t a clinician because obviously clinicians, essentially, [00:30:00] are not very high risk financially because you pay them for seeing clients, which means you’ve been paid as well. VAs and billers, clinical directors, supervisors, those kind of positions are a little bit more high risk because you’re paying them whether or not the clinicians are seeing clients. They’re there for the most part maybe a VA. Well, even with VAs, you usually pay for a package or a minimum so you have to pay whether you use it or not.

Every time I’ve taken a risk like that and now the risks are much smaller because I’m a large practice and can make additions like this a little more easily but when I was newer, it was a big deal to bring on a clinical director, even though it was just one hour a week. It was a little bit of a risk but every one that I took always yielded in me having more time and being able to use that time to grow the practice in a way that brought in more income.

Dr. Sharp: Absolutely.

[00:31:00] Maureen: I want to put an aside on here. A key thing is if you do hire someone as a support person, whether you’re solo still and you decide to hire a biller or a VA to answer your phones for you or whatever, the thing that makes it worth it because it’s really easy and I’ve heard this a lot where people hire someone but then it feels like their time is still just constantly being used, is that you have to purposefully out loud or on paper, say what you’re going to be doing.

So if I hire a VA for five hours a week of something that I was doing, I am legitimately saying what am I going to do with these five hours? I either am leaving an hour early five days a week so that I … Literally am working five hours less or I’m going to say, I’m going to take those five hours and I’m going to blump those five hours on to X day to do marketing only. I don’t know, whatever it is, get my nails done, whatever it is that you want to do.

I think the issue that people have is that they get support but then they replace [00:32:00] that time with menial things that aren’t helpful to the group practice or the growth and still make you feel anxious and overworked. And so when you do make these changes where you’re going to spend money on someone else for support, is that you say, okay, these extra X amount of hours that I’m going to get back, what do I want to do with it? Do I want to use it for myself? And then use those hours, actually leave work five hours earlier, do something that, so you feel it.

Dr. Sharp: Absolutely. Good point. Oh, you’re so right. So let me try to touch on just some basics for folks who may not know, let’s say they’ve decided to hire their first person. I like lists. What are three things they need to think about before they even hire? They know they need someone, now what? What do you look at? What do you research? What’s important?

Maureen: Step 1, have a business plan, but we’ll pretend that that’s not step 1, because I already [00:33:00] brought that up. So step 1A is find an employment attorney and have them draft an employment contract or an IC contract or an offer letter. Don’t …

Dr. Sharp: Can you talk about that IC, employee difference? I feel like a lot of people get tripped up with this, I know it’s a huge thing, but briefly, IC versus employee.

Maureen: An employee is someone that you hire. An independent contractor is someone that has their own business that contracts their work out in your business. So think of a painting company that hires a guy who has his own painting supplies and he doesn’t work for that company. That company will call if they have a project for him to do, and he’ll say yes, I’m available that day. I can do that project. This is my rate, and this is what I charge. And that company will pay that painter, who will then go out with his own supplies and do that work.

An employee is someone [00:34:00] who the employer has more control over. They can say when and where they need to be. They supply them with everything like couches and chairs and pens and papers and testing materials. But going even beyond that, one, there’s a lot of changes happening in several states, California being one of them where they’re essentially writing into law that it’s nearly impossible in our industry to have contractors anymore. So that’s why I say to find an employment attorney because people want to cut corners in the beginning and kind of copy someone else’s contract and then just use it.

But it’s really important to know based off of your personality, based off of your leadership style, based off of your vision for your group practice, is it better for you to have employees or contractors? We’re finding out that in more and more states, as time goes, contractors is really hard to make happen.

And so I have employees, [00:35:00] but my testing person is an independent contractor. All my therapists are employees but for testing, she has her own counseling business. She does her own counseling and testing at her own private practice somewhere. She doesn’t have any set hours. There’s a room that’s available to her two days a week for a certain block of time from 8 to 12. And if she has someone for testing, she comes. If she doesn’t, she doesn’t. She schedules her own appointments. I pay her business for doing that work. She’s not a part of our team and our team culture of therapists who are W-2s or employees.

So with employees, if you want to supervise, if you want to tell them when they can and can’t work, if you want to make them work full time; these are all things that you would need an employee for. If you really want to be hands off and not provide direction, not supervise, not have policies and procedures that they need to follow, allowing them to [00:36:00] come and go as they please and see clients and they have their own business that you can pay and they want to work very part-time, then contractors might work for you.

This is why I say the first step should be to find an employment attorney because they will tell you in your state, what’s possible and what’s not because you may be in one of those states where contractors just doesn’t work but also even if it is possible and something that can work in your state, you talking through what you want from your practice will help them say, that sounds a whole lot more like you need to have employees.

The issue only arises that people have contractors who they’re really treating like employees and if the IRS finds out you’ve misclassified; you can owe a lot of money. There’s never going to be an issue with you having employees who you treat like independent contractors. So if you want to go the safe route, obviously, having employees is safer because you can say, [00:37:00] and this was me in the beginning is I was like, I really want to be hands off. I don’t want to be setting a lot of rules, obviously, as I’ve grown in my business and gotten comfortable with my role, that’s changed.

But there’s no rule book that says if you have employees that you can’t allow freedom or allow them to make decisions on their own. The problem only can arise if you have contractors that you’re treating like employees. There’s never going to be a time where you would get in trouble for having employees who you’re treating like contractors.

Dr. Sharp: That makes sense.

Maureen: So that would be my first tip, is an employment attorney. Don’t use these Facebook groups to ask what’s better. Don’t use the Facebook groups. I feel like my group is the one, if anyone’s a part of that group, people ask it all the time and I wish I could zap that question off so that they can’t ask because it’s a legal question. You need to [00:38:00] have an attorney answer that for you and draft paperwork that fits your state’s rules and your particular …

Even your county or your city, laws change just by counting area. Here in Chicago, just last year, new law went to an effect where if you have employees, you have to pay for sick time, but literally about three blocks away from me is a suburb called Park Ridge. If my practice was over there, I would not be required to … So even just having …

Dr. Sharp: Oh that’s amazing.

Maureen: Yeah. And so it’s really important that you have an employment attorney because she was the one that emailed me and said, hey, just want to let you know, new laws going into effect in July of 2017 it was, and I was prepared in November of 2016 for it that I actually put it in place on January 1st, six months early.

But you don’t get that information if you don’t have an employment attorney. And when you manage staff, whether they’re contractors or employees, I think the biggest disservice you can do for [00:39:00] yourself is not having someone like that, that you can talk to. And most people don’t because they don’t want to spend the money. Attorneys cost money.

Dr. Sharp: They do.

Maureen: As you and I know.

Dr. Sharp: They sure do. I got a bill from my attorney on my desk right now.

Maureen: So do I.

Dr. Sharp: Okay. So step 2.

Maureen: Oh, so then let’s say you’ve figured out who you want to have; contractors or employees. I feel like a second step, there’s like 10 things at step 2 that can happen and not in any particular order but I would say from the perspective of me being a group practice coach, what I tell people to do next is to figure out your ideal clinician before hiring because we were talking about next step before hiring.

There’s a lot of other things like figuring out if you want to take insurance or not as a group and figuring out if you need a group insurance contract, but those things are not as important and you can do them [00:40:00] later, too. If you’re thinking about hiring someone, you want to know who you want to hire. I think a lot of us have it in the top of our heads, like, yeah, I want a fully licensed person to work 10, maybe 15 hours, and are pretty vague.

And then you start interviewing, and more often than not, if you connect with someone because you like them, you’ll hire them. It doesn’t necessarily mean that they’re the best fit for your practice as a culture, personality-wise, needs-wise, which is why knowing what your community needs is important.

I have an ideal clinician worksheet that’s on my website somewhere and it’s free. Anyone can take it. It walks you through some common questions like would you want someone who’s provisionally licensed or not? It’s an important question because you will get 100 provisionally licensed people applying for your position and it’s very easy in the beginning to be antsy and jump in to saying, oh, you know what? I didn’t really want a provisionally licensed person but [00:41:00] why not?

And that’s where I see a lot of people making mistakes because then they see all of the extra work that might come in with a provisionally licensed person that they didn’t think about. And then figuring out what are your non-negotiables? If you want someone fully licensed and not provisionally, don’t interview someone who’s provisionally licensed because you might like them, and then it feels weird to not take them on and you’ll feel the pressure of wanting to take them on.

If you want a couples therapist, don’t interview people who aren’t couples therapists. Make sure to have this job description be directed to a couples therapist. If you’re wanting someone who, like part of mine and it evolved into this, is I have a really heavy emphasis on company culture. And so I want clinicians who want to be engaged in the practice with the other clinicians. I don’t want someone who just wants to see clients and [00:42:00] go home.

You might want people like that and I think that’s important because it’s going to have a different culture. You’re going to have people who are very independent and kind of coming in and going. I wanted a place where when I walked into the office, that people were sitting on each other’s couches and talking between seeing clients.

I wanted a practice where clinicians would want to come to staff case consults. We have a once a month case consult where we get pizza and Starbucks and we just have a good time and connect. You’ll find very easily and I’ve had this happen before where I didn’t put an emphasis on it and then I hired people who never showed up to the staff outings and just weren’t really a part of it and felt like, who is this person? And for me, it was something I didn’t like.

And so that was part of a non-negotiable for me moving forward after I realized that is, I need to make an emphasis on company culture and that I need someone who really likes to connect with the other staff members. If you’re one that likes to close your door between sessions and lay on the couch and relax or [00:43:00] leave the office and not connect with anyone, you’re probably not going to be a good fit.

So the step 2 is really figuring out in detail what are your non-negotiables, I call them, because it’s very easy to find someone who ticks only a few of things that you want, and then for you to be like, that’s pretty good. I’ll still take them. I think the hardest thing that group practice owners have is when there’s a staff issues.

Every person that I’ve ever, businesswise, in any business, the biggest struggle isn’t figuring out a metric in your business. If you have staff, the biggest issue ends up coming with problems relating to staff, whether it’s communicating concerns, whether it’s clinicians leaving a lot, whether it’s hiring people who say they can do something that they can’t do. It always ends up relating to the staff management piece. And so if I’ve learned anything, it’s to know your non-negotiables ahead of time before [00:44:00] interviewing.

And then, I guess, 2A would be to have a good recruiting strategy in place and interviewing strategy. Don’t just interview them once and then make a decision. I have multiple steps. I obviously have other people now that can help me with that. I have a clinical director who can do an interview. Some people who don’t have that use existing clinicians to do a peer interview.

But if you’re starting from scratch and it’s just you, you can still have a multiple-step process. What it does is it lets you get them in different lights. So if something doesn’t pop out at you at one point, it might pop out at the second interview.

So if you’re a solo practice owner still and you’re hiring your first person, and it’s just you, my suggestion is to have something in place that they have to have, they need to give you a resume and a cover letter. I feel like nobody reads the cover letter, but it’s part of can they follow directions, for me. If they don’t send a cover letter, for me, it’s just indicative [00:45:00] of, they don’t fully read through things. I’m probably going to have to tell them things multiple times. That may not be the case but here’s where I don’t like to take risks. So if they can’t follow that, they’re out in that first round.

I then have a Google Form questionnaire that I made that outlines my non-negotiables again because you can put it in your job description and people will be like, of course I can work 20 hours a week if that’s what you want and of course I’m fully licensed. And then all of a sudden, they’re like, well, I will be fully licensed in three months from now. And I’m like, no, you’re not then.

So I have a Google Form that puts stuff from my ideal clinician worksheet or my non-negotiables on there. It says, please confirm you have this license or this license. I put the fully licensed ones on there. And then I put, group culture is very important. If you’re one that likes to work individually and come and go as you please, this isn’t going to be a place that you’re going to be happy at. Can you confirm? You can look mine up, it’s on my group practice [00:46:00] webpage right at the top, but I pretty much list out those non-negotiables.

I list like, what are some of your strengths? Why Urban Wellness, which is my practice. I think it’s important that they’ve done their research and they know my business. If they don’t, I feel like they’re not very invested now, they probably won’t be later. So that’s the second one. And if I find anything in that form that they fill out that doesn’t fit or jive with what I’m expecting, then I don’t move forward.

The third thing you can do is then have an informal 10-minute, very quick phone consult. It’s really meant to check that they fit all the requirements that you’re looking for, but also can they connect with you? For me, I feel like if we can’t connect on a phone call and it’s weird and just not right beyond nerves, obviously we all know that when people interview, sometimes they’re nervous, but if it goes beyond the nerves piece and they’re just like off, you know that [00:47:00] they’re going to have a hard time establishing rapport with new clinicians or with new clients at that first intake.

And so for me, I’m like, if that’s just weird too, then I don’t move forward with an in-person. And then the lastly would be an in person. So having multiple steps to see them in different lights, it would be that part 2A.

Dr. Sharp: Yeah, for sure. I know it sounds like there’s a third step that I want to hear about. I think it’s important to highlight that because a lot of what I hear in the testing group is that it’s just so hard to find people period who are good at testing or good candidates that then we end up in a scarcity mindset and hire people who then are not good fits at all.

Maureen: Exactly.

Dr. Sharp: Desperate, so just holding out.

Maureen: I think the issue, one, is holding out because there is someone that’s good for you. Maybe they haven’t moved to your city yet. Maybe they haven’t seen your Facebook [00:48:00] ad or your Indeed ad yet, but there is someone that’s a good fit. It is so much more painful to be unhappy with your business if the people that you have frustrate you and aren’t aligned with your business. No quicker way will you want to give up having a group practice than having clinicians who don’t fit in with what you are wanting in your practice.

The other thing that I could mention relating to that, especially in rural areas or if you’re looking for someone very specific like testing or EMDR certified or Gottman completely certified, the ones where it just narrows the search down a lot is that you can look at alternatives and I’m going off the cuff here of either hiring someone who can do it remotely, either some practices that do it. It’s a [00:49:00] whole another level of work because you then are employing someone who doesn’t even work out of your practice, but works remotely, maybe an hour away where they’re located.

It also might mean taking a look at what your non-negotiables are and re-reviewing them. There have been a few cases where I’ve seen, truly, based off of them being in this very rural area where there’s like 100 people total in a 100-mile radius, and there’s probably only going to be 10 therapists and you’re wanting one, that you may have to adjust something.

If it’s a case where you’re in a place like that, if you’re in Colorado, no. If you’re in Chicago like me, no. You can step your non-negotiables and hold off and you’ll find someone. But I guess for those listeners who might be in that outlier area where there’s not a lot of people, you may have to shift and look at what those non-negotiables are.

[00:50:00] If you’re looking for someone who’s, let’s say like I said before, EMDR completely certified, could you adjust that to having someone who’s done all the EMDR training and has been doing it for 15 years? They maybe just didn’t pay for that certification but they did all of this pre-certification stuff. It might mean making an adjustment like that.

And with testing, maybe you’re looking for someone who can do neuropsychological testing but that’s a lot harder to find, but finding someone who can do regular child psychological testing is a little bit easier. Maybe you make that shift for that reason.

Dr. Sharp: Sure. I think too the hurdle that I had to get over is I found when I was hiring folks that I really preferred a known quantity. I wanted either someone who went to my graduate school program or a friend of a friend or I needed some familiarity but 2 out of my 12 [00:51:00] clinicians just moved from out of state. They saw the ad and they applied and it turned out great. I think it’s just speaking too to being open to whomever may come through the door as long as they are a good fit when you really get down to it.

Maureen: Yeah, exactly.

Dr. Sharp: Is there a third step? Did we talk about the third step?

Maureen: That was the third. I did three for if you’re solo. Still I have a few more steps but that’s for those that are established and have other clinicians working for them, you can then have a fourth step where you do a peer interview where two or three clinicians can do a pre-interview before it gets to you. They’ll definitely get a different version of an interview than you would as the group practice owner.

If you have any management or you have a front desk receptionist, they’re going to be dealing with them. You can always have them do an [00:52:00] interview. I have my clinical director do it and then it goes to me. So those would be just two extra steps that we do.

Dr. Sharp: Cool. Well, gosh, I feel like we’ve covered a lot of ground here. Definitely some stuff around the basics of when to hire and how to hire and what to think about. Other things that you’ve learned from coaching others and doing it yourself that people might forget about or not think about when they are starting to grow and expand and bring people on.

Maureen: One last thing I would mention is marketing. I brought it up before, but I feel like if you’re a solo practitioner, your marketing scheme needs to change a little bit because what works when you market solo might not necessarily work for marketing a group practice. One of the main reasons being when you’re marketing yourself solo, you’re marketing yourself. When you have a group practice, you have to market the brand, the group practice. You can’t market yourself anymore, otherwise [00:53:00] people will be calling to talk to you and you’ll just have a harder time convincing them to go see your clinicians.

And so I think that’s just something that every new group practice owner will have to learn to look at is how they can shift their marketing strategy where it actually markets the brand itself or the group practice itself and not necessarily me, the group practice owner and figuring out what you expect when it comes to your clinicians when it comes to marketing.

I call it the two-pronged approach to marketing, which is, in person, one-prong and digital is the other prong. I focus on the digital marketing mainly. The in person stuff comes more for my clinicians. I pay them to do speaking engagements. They don’t have to market.

If you’re a new group practice, I don’t think it’s a bad thing to require your clinicians to market. It’s helpful when you’re starting group practice. [00:54:00] As you become established, it was something I liked being able to take away as a benefit to working in a group practice. They don’t have to anymore. But when they’re new, I think it’s really important that they learn. You don’t have to carry the burden all by yourself of trying to fill everyone up, and so teaching them to market themselves is that one-prong, that in person prong.

Doing speaking engagements or running workshops outside of your office at other businesses, those are all that one prong of face-to-face. Being a part of your local chamber of commerce is a way that you can do it because you get connected with all the local business owners. And then the second prong is digital marketing and that’s doing things like making sure you have a good website, that it’s optimized for SEO, that you are potentially doing something like Google AdWords if you want or Facebook ads or having a social media account, if that’s what you want. These aren’t all things you have to do but that’s fits in that prong of that digital marketing.

[00:55:00] For me, I find that the digital marketing is easiest because it doesn’t have my face in front of it and so I can do the work without clients seeing me and thinking, oh, I want to see her. So the point I’m at and I think it’s easily doable to navigate that way even as a new group practice owner, is to focus more. People are just on the internet. Most of our clients when they’re calling us, I can see calls through their mobile phone because they found us on our website and called straight through our website.

Until your group practice gains traction, you’re not going to have as many word of mouth. And so focusing on that digital stuff is going to be really helpful. And so that’s where I focus my time.

And now I’m at a place where I can pay clinicians a good … I pay them more than double what they make an hour seeing clients to do marketing in the community because we found that community is a [00:56:00] really, you get a high level return on investment by having clinicians go out in the community and do like a speaking engagement. So we have a lot of presentations that we give and that’s that face-to-face prong that I was talking about.

Dr. Sharp: Nice, very cool. That’s a whole topic. We talk with people a lot about, I don’t know if it’s more so with testing and counseling or not, but it seems like that’s a common problem where the practitioner starts out solo, people get used to those reports and that style of testing and then it’s hard to spread that to their clinicians and let the clinicians get referrals because everybody’s like, no, I want your testing.

Maureen: I think with testing; I can see that it would be much more about the person. My daughter has autism. I remember when we were getting her tested when she was younger, I was very specific on who I wanted to see. And so coming from a consumer perspective, I can [00:57:00] see like if someone says, Dr. Jeremy Sharp is amazing at testing my child. It was a great experience, that I would be so laser-focused on wanting to see you.

Just because I’ve had that experience in my own life with my daughter and so I think I can imagine that you have to make sure that your testing psychologists are doing some kind of work in the community, getting themselves known in the community because I can see testing being something that people, it’s connected to a name. And so they need to go out there and make sure that they’re doing something to get their name out there so that people connect with them as well and not just you.

Dr. Sharp: Well, I can say too, at least from my perspective, that’s it circles back to the IC versus employee thing. And that was a big reason that I wanted to have employees, which is because I knew I wanted consistency in our report style. I wanted to [00:58:00] be able to control that. Still, I look at way too many reports that I need to do quality control, so to speak. Even though I fully trust my clinicians, I like to take a look and make sure that what’s going out is what we …

Maureen: What you want. Right. Exactly. I get that.

Dr. Sharp: Well, goodness. You shared so much with us. This is just a tip of the iceberg. Your website has a ton of information. I will link to that in the show notes. You also do a podcast. You also have a membership community for folks who are really focusing on building a group practice and need support around that. We will have links to all of that if people want to check that out.

Maureen: Awesome. Thank you. I had a good time talking.

Dr. Sharp: Good, me too.

Maureen: As usual.

Dr. Sharp: Right, I know. I like this podcast relationship going on here.

[00:59:00] Maureen: I know. Like you said, it’s really is the tip of the iceberg because there’s so much and we’ve been talking for a little bit now and I feel like there’s so much more I could give information wise but hopefully they’ve gotten two tidbits of information that helps them make a decision if they’re thinking about starting a group practice, whether it’s a testing group practice or whatnot.

Dr. Sharp: Awesome. Thank you so much.

Maureen: Thank you.

Dr. Sharp: Okay, y’all. I hope that was super helpful for you. Maureen and I ended up covering a lot of topics and I hope that you’re able to distill a few things from that conversation to get you going if you are trying to hire some folks. Like I said, I know the testing piece, but she really knows the hiring and the growth piece. We did not even get into how she uses metrics in her practice but that as a whole other topic that I would love to bring her back for.

[01:00:00] If you would like some testing paperwork to supplement your practice, I’ve put together three packets. There’s a clinical packet, there’s an administrative packet and there is a psychometrist training manual. If any of those sound interesting to you for your practice, you can check them out at thetestingpsychologist.com/paperwork.

And if you are interested at all in coaching around the testing aspects of your practice, I would be happy to talk with you about that too. You can find out more at thetestingpsychologist.com/consulting. All right, see you next time.

Click here to listen instead!

​

[00:00:00] Dr. Sharp: Hey y’all, this is Dr. Jeremy Sharp and this is The Testing Psychologist podcast episode 71. I think you’re going to like this conversation. Today I’ve got Dr. Steve Feifer here on the podcast.

Steve has done many things over the course of his life. He’s been a school psychologist for nearly 25 years now. He has co-authored the Feifer Assessment of Reading and the Feifer Assessment of Math, two academic measures that have gained a lot of positive recognition in the field over the last few years. He is now in private practice where he assesses kids and splits his time between the speaking circuit, giving presentations and whatnot, and continuing to work on tests and books. He has also co-authored a few books on bridging neuropsychology and school psychology.

So Steve has a wealth of information and he is [00:01:00] a great guy to talk to. So I hope y’all will enjoy our conversation and let’s get right to it.

Hello, and welcome to another episode of The Testing Psychologist podcast. I’m Dr. Jeremy Sharp.

Today, I’m here talking with Dr. Steve Feifer. If you haven’t heard of Steve, you have probably heard of some of the measures he has co-authored. If you haven’t heard of those, you might be living under a rock in our field.

So we’ll talk all about Steve and his career and his measures and his thoughts on a lot of things. We have a lot of good topics to cover today. So Steve, right now, he’s had a long career like you heard in the [00:02:00] introduction. He has had a long career but he is currently in private practice where he is assessing kids and he has a lot to talk with us today about.

So Steve, welcome to the podcast.

Dr. Steve: Good morning, Jeremy. Thanks for having me on.

Dr. Sharp: Yeah, of course. I have to have to disclose, this has been a long time coming in this. You’ve been one of those folks that when I started the podcast, I was like, I hope I can get this guy someday. So the fact that you have agreed and we’re here and we’re doing it is pretty amazing. So I’m thankful for your time and what you’re willing to share with us today. I’m excited to get into it.

Dr. Steve: Me as well and looking forward to it. Thanks for having me. It’s about time we’ve done one of these, so let’s get …

Dr. Sharp: Let’s do it. Well, listeners of the podcast know that with guests, I typically, I’d like [00:03:00] to start and just hear your story of how you got to where you are today and what today even looks like for you. What your day-to-day life looks like and how you’re practicing psychology, so can you tell us that story?

Dr. Steve: Well, where do we begin, huh? I started out, Jeremy, believe it or not, this is year 25 for me. My first year as a school psychologist was in 1993, school year, working in a small town in West Virginia that I’m sure nobody has ever heard of but that’s really where I cut my teeth. I spent the first six years of my career in West Virginia and in the next 13 in the state of Maryland, where I currently reside as well.

But starting in West Virginia, to be perfectly honest, after my first year working as a school psychologist and into my second year, I came to the following conclusion; I don’t think this job is for me. I was [00:04:00] pretty discouraged because I had trained, like most of us, to learn a lot of different things in graduate school, only to find out that I was just someone who tested and came up with a number, and we called that number a Full Scale IQ score.

That’s all anybody wanted from me here; what’s the Full Scale IQ? And that’s either going to justify or not justify taking a student from class and marching them 20 feet down the hall into a room called the special education classroom to work with them.

I was struggling to find the value and meaning of my work. And like most of you, I was spending a lot of time reporting. I’d like to say I could write a report in two hours, that never happened. Three hours, maybe on a good day, but I’m taking all day to write reports. I’m getting buried in referrals.

And frankly, I could go to most of my meetings, not with a report but with a 3×5 card. And on that 3×5 card, all I would need to [00:05:00] write is yes or no. Do they qualify or not? That’s all anybody cared about.

So I was getting pretty discouraged. I don’t want to tell, I’ll be quick, but to tell a quick story. I ran into a student named Jason and I was asked to do a re-evaluation on Jason. He was in a regular elementary school but in a classroom for severely and profoundly impaired students. Jason was non-ambulatory. He was in a wheelchair.

His last IQ test, Jason had scored a 36. He had an IEP that was just basically functional life skills at best. And I’m asked to do a re-evaluation on Jason. His teacher pulled me aside and said, hey, Steve, before you retest Jason, give him a reading test. He’s the strongest reader I have in my class. And I thought, are you kidding me? The kid, he’s your strongest reader? Okay, [00:06:00] I’ll play along.

I remembered, and I’m going to go way back on you, Jeremy, an old test from graduate school called the PIAT-R that I thought might help me out, Peabody Individual Achievement Test. It had a reading/spelling recognition section. I sat down with Jason and I flipped over the first page, four words on a page. Okay, Jason, can you find the word ball? And he scans it and picks it out, no problem. I flip over the next page, there’s four words on a page: boy, girl, apple, orange. Okay, Jason, can you find the word apple? He picks it out. No problem.

This is the only way I could think of Dominican reading on a student who didn’t even speak. And lo and behold, he’s getting all of these correct. I couldn’t believe it. So I flip over to the 9th-grade section of the PIAT-R, there’s not four words on a page, there’s eight words on [00:07:00] a page, eight phonologically irregular words: pterodactyl, psychoanalysis, reconnaissance, words of that ilk. And I said, Jason, can you find the word reconnaissance for me? He picks it out, no problem.

And that’s when I realized I have a problem here. I cannot go back in two weeks to this conference room at this school and deliver my test results the way I’ve always done them. In other words, I couldn’t go back and say, we re-administered the Stanford-Binet scales of intelligence and once again, Jason scored a 36. That’s severely and profoundly impaired. However, on an academic achievement test in reading called the PIAT-R, he scored 118. My goodness, we have an overachiever with us.

I couldn’t do it. And that’s when I had the epiphany of I need to stop looking at learning through the [00:08:00] lens of an IQ score. Once I stopped doing that, because there’s too many blind spots there, and looked at learning through the lens of brain-behavior relationships, an area that I was always interested in but never had much formal training in graduate school, Jason was very easy to explain.

Jason had a condition called hyperlexia. Hyperlexia is the opposite of dyslexia. It’s when you have a very compromised IQ but you can read, but you’re really not reading your word calling because while you can identify the words, you really have little comprehension skills.

I went into that meeting, believe it or not, a school meeting, and that’s the first time I talked about the brain and the brain and learning. And the reaction was unbelievable. We actually rewrote Jason’s IEP. He went into a regular 5th grade class for part of his language arts day.

His mother came up to me after the [00:09:00] meeting and said, “Thank you so much for explaining what hyperlexia is. Actually talking about the brain and explaining how this is even possible and for allowing my son to be with same age peers and non-disabled peers for part of his day. Thank you for providing that.”

I don’t hear that kind of feedback very often. And that’s when I realized if I could use neuropsychology to explain a puzzle like Jason, well, what about your typical kid who struggles with multiplication or spelling or writing or has a little ADHD or maybe an undercurrent of anxiety? It wasn’t long after that, I realized I need to go into a neuropsychology training program, which I did. I was fortunate enough to do.

Dr. Sharp: And how did you do that, Steve? Did you do a formal program or a postgraduate program or how did that work?

Dr. Steve: [00:10:00] It was a little more than two-year program that at that time was coming out of the Fielding Institute. It was a clinical neuro program that has since been taken over. It’s gone through many changes. We’re going back now over 20 years but has been taken over by Dr. Dan Miller. Dan runs that program out of Dallas as a postgraduate certification program for school neuropsychology so you can become a diplomate in neuropsychology.

If you’re interested in working in a hospital or rehab setting, probably you want to go more towards clinical PhD and do a postdoc in a hospital setting. But I, one who did everything backwards in my own education, I did all the neuropsychological stuff first I was so interested in and then [00:11:00] I used that and that was a springboard. I went back to get my doctorate in a traditional doctorate program, but they did have a neuropsychology strand, but I had already done the certification first. So I went in a way postdoc first and then the doctorate, I wouldn’t recommend that, but that’s how circumstances dictated.

And to be perfectly honest, I have spent my entire career of trying to bridge neuropsychology and what’s going on in neuroscience, which I find fascinating, into the world of education and to say that this is really important stuff and to really understand the craft that we do in terms of understanding learning and the obstacles for learning but more importantly, using neuroscience to help develop and to help with our intervention selection, which I find is really trial and error. Well, let’s try this. It worked for Bobby. Let’s try for [00:12:00] Billy. And then we wonder why that doesn’t work.

I’d say the biggest pitfall is that we fall in love with one intervention and want to apply it to everybody and wonder why it doesn’t work for everybody. The answer is all […] that’s the message I’ve really tried to spend my time.

Dr. Sharp: I think that’s so important and I think there’s a lot to unpack there. I just want to briefly touch on the fact that you are doing it all. You’re a practitioner. You’re not just a researcher but you are a researcher and you’ve done some test development. You’ve really seen it from all sides, which I think is a cool perspective to have to be able to talk about all of these.

Dr. Steve: It wasn’t by design, Jeremy. It just happened that way. [00:13:00] I take a lot of pride in being able to do a lot of things in psychology, but if you pin me down to what is it I am at my core, I’d say I’m exactly what you are. I’m a clinician. I like to work with families and I like to work with kids and test them and try to determine what are the barriers to their learning?

People ask me, in fact, you asked me before the podcast, are you a psychologist or a school psychologist, a neuropsychologist? I’ll tell you what I feel like I am. I’m a cognitive detective. A crime has been committed, and that crime is student underachievement. I think it’s our job to figure out why? This is what we do. And I am very intrigued by that. The test development came in a little bit later.

Dr. Sharp: Yeah, let’s talk about that.

Dr. Steve: Well, as I started working and being that bridge for neuropsychology [00:14:00] into the world of school systems and I stumbled and bumbled my way into presenting. So another quick story. I would probably rather volunteer to go on the front lines of Iraq than to stand before a group of people and do a presentation. A major anxiety when it came to presenting. However, when I was doing my neuropsychological training, part of that training is we had to all pick a topic and study that topic, become an expert in that topic and after two years, write a paper on that topic and present it.

At that time, frankly, I was the youngest person by a mile in my training program. Everybody else was picking like Alzheimer’s disease and dementia.

Well, I don’t deal with that. I picked dyslexia and I pulled every article I could over those two years [00:15:00] and developed a subtype model for dyslexia, something that most teachers already know that there’s different kinds of reading problems that kids can have. Some struggle with the accuracy, some with fluency, some with comprehension. And I just helped organize the science around that to say, this is why you would have trouble with one aspect of reading but not another.

I did that and I presented to my class on this subtype model of dyslexia and people went crazy. They really liked it and they said, you’ve got to present it. For instance, over my dead body would I ever do something like that but my instructor at the time, I agreed to do it only if he would do it with me because of my age.

And we presented at a NASP conference. I want to say it was almost 20 years ago. It was a while ago. The one in Orlando. I want to say that was [00:16:00] 1998. I’m taking it way back now. It went great.

After that presentation, someone in the audience said, “Would you come out to my district in Ohio and we’ll pay you X amount of money to do this for my staff?” And I thought, you mean you’re going to pay me for this?

I got over that stage right really quickly. You’re going to pay me for something like this. I had no idea you could make money presenting. So I continued to present with a partner until I felt comfortable after the first two years. And then it took off from there and it […] and I think I presented pretty much at every NASP conference since 2000.

Dr. Sharp: Oh, that’s great.

Dr. Steve: Well, it’s cited up here in Atlanta in 2019 but the topics that I thought have always been about neuroscience into the world of learning and to show brain scans and just show pictures [00:17:00] of what your work does and to say your work changes brain chemistry and enhances pathways here. Sal, look at this picture. Very powerful. It validates our work. It validates teachers work. People are very interested in that.

So I had been doing these presentations for a number of years when I was approached, I want to say 2010, 2011 by a representative from PAR, a testing company in Florida. I’m sure many of us use PAR products.

Again, a quick story. I’ll try to be quick. I was in Phoenix, Arizona. I was doing a workshop and the way they did it in Phoenix, this is for the Arizona Association of School Psychologists, is you do a morning workshop, they give you about a 45-minute lunch break, and you repeat. Well, it’s [00:18:00] very hard to repeat workshops. I had 45 minutes space in between to collect my thoughts to make sure that I was fresh and I could give a good performance in the afternoon as well as the morning.

And that’s when I was approached by a representative from PAR who really wanted to talk to me and about possibly developing some tests. I told him you’re catching me at a bad time and I am not interested to see you later and I blow them off completely. But they persisted and we met up at a NASC conference, I want to say it was maybe San Francisco, around 2011 or so. We had meetings and they said, “We love your presentation on this subtype model of reading, there’s different kinds of reading disorders, we’d be interested in helping us develop a test for that?” I said, no, not really. I’m happy where I’m at but you want me to be like an advisor or something like that? Sure, why not?

And we kept the conversation going, [00:19:00] and even after the conference, there were phone calls and emails back and forth. And finally, they came to Baltimore, Maryland, and we met outside of BWI Airport. I was intrigued, and it captured my interest, and I said, let’s give it a shot. I can tell you, when I signed the dotted line, my life changed completely. I found myself in the world of test developments. I fell in love with test developments, and I realized, oh-oh, it’s hard to be a test developer and to be a full time school psychologist.

And something I had to give and that’s when I made my exit from the school system into the world of private practice, so I could maintain and do all the things that I love, which is continue to test kids but perhaps more at my own pace, develop tests and continue to do workshops and presentations on neuropsychology. And [00:20:00] that’s the story, Jeremy. That’s how we are.

Dr. Sharp: It sounds like an incredible journey and the part that you just talked about is pretty important to highlight that sometimes we take leaps in our career, maybe that’s moving to private practice, it’s quitting the full time job, it’s jumping into something that you maybe had no idea about what test development and finding that you love it and then like following that passion. Sometimes we got to do that.

And I know that a lot of folks listening maybe struggling with that too, working in the schools and thinking about private practice and not knowing where to go. You took the leap and it worked.

Dr. Steve: It was very scary but looking back at the time, I had just won a NASP award in 2009. I was very honored to have won the National School Psychologist of the Year award. [00:21:00] I felt okay. I’ve hit the top here. This is great. Wait, go? I’m not even halfway into this career yet. What’s next for me?

And I’ll be honest, I found myself back in the schools going through motions and I knew I was going through the motions. I knew I needed a change but I didn’t have the guts. I didn’t have to make that move and it was scary to me and I have three kids and a mortgage like everybody else. What if this doesn’t work and I fall out of my face, but those are some of those hard life decisions. I hemmed and hawed, but as circumstances surrounded myself, I made that decision and it was very scary at the time but looking back, I feel good. It was the correct one.

Dr. Sharp: Yeah. Can I just ask? How did you [00:22:00] talk yourself through that fear and get through some of the tougher moments in that process? Do you remember?

Dr. Steve: My family, number one. The school system that I was working for at the time, we did not have a great relationship and it was borderline adversarial. They weren’t very supportive of a lot of the outside things that I was doing. They certainly weren’t supportive of, where’s Steve? Oh, he’s off in Arizona giving a workshop. Well, they didn’t like that. So it made it easier in a way because I knew we were butting heads and that was a source of stress.

And because I was terribly supportive of the things I was doing, I knew I had to, it was time. I just knew it was time. I think it’s very hard though to [00:23:00] leave wherever you are, even if your circumstance is not great, it’s very hard to leave but once I saw what the next thing could be, then it made it easier.

And that next thing was the world of test development, an opportunity to go into private practice with one of my best friends who owns a practice here in Frederick called the Monocacy Neurodevelopmental Center. He was so gracious and says, Steve, come and work for me. And you sit down and say, my gosh, I think we can make this transition. I think it might work. You’re never sure if it will. And there’s a few sleepless nights in there. No question about it but I think he tried to weigh all of those things.

Dr. Sharp: Sure. So let’s talk about your measures. I’m so curious. At this point you’ve done the Feifer Assessment of Reading and the Feifer Assessment of Math. I think they have almost like a cult following in our field as [00:24:00] alternatives to those bigger, the WIAT’s and the KTEA’s and so forth. What makes them different from those traditional achievement tests?

Dr. Steve: You’ve got to be wondering, Jeremy, why does the world need another reading test? We’ve got a bunch of them like the Woodcock-Johnson, maybe you like the WIAT, the KTEA, the GORT, take your pick, a lot out there. And why does world need another traditional reading test? Because it’s not a traditional reading test.

What we found, reading tests and all of those measures I mentioned, their sole purpose is to validate, where are you in reading? We use those measures to come up with an overall reading score. Oftentimes we compare that score to an IQ test. I know our field is moving away from a discrepancy model, but that’s what the purpose of those tests were, come up with a singular [00:25:00] value to compare to an IQ test.

What the FAR is what I hope to be representing a new genre of testing. It’s not a traditional reading test. It’s a diagnostic reading test. The goal of the FAR is not to say where you are in reading. I think most teachers already know that anyway. The goal of the FAR is to say, why are you struggling in reading? It’s to answer the why question.

If you can answer that why question, then when it comes to interventions, I don’t think it’s a trial and error approach. I think then you put yourself in a better position to be a good consumer in selecting an appropriate intervention. To answer the why question, what we did is we built the processing into the test.

For example, a lot of us might give a separate test of executive function or a separate test of working memory [00:26:00] or a separate test of orthography, whatever that is, rather than do all these separate tests of the academic skill you’re interested in, as how it is, we built that all into the test so it gives it better ecological validity.

So I’m not going to say here and say I can measure the totality of executive functioning. I don’t know who can, but I’m not going to even try that. What I can tell you is that I can use two subtests on the FAR and in about five minutes tell you precisely how executive functioning impacts reading because that’s the question I’m interested in. I can’t answer any other questions regarding executive functioning but with respect to this skill, we built the processing into the test. So again, not only has better ecological validity, but as an evaluator, it’s going to save you a lot of time. You don’t have to give all these other standalone measures.

Dr. Sharp: Yeah. Can [00:27:00] you give me an example or illustrate that a little bit when you say we can tell how executive functioning impacts reading. What does that even look like and how are you measuring it in the test?

Dr. Steve: Good question. I’m pretty much convinced, you get called in all the time. I know this happens with you as well. Got to look at Billy’s memory. I’m telling you, Billy’s got the worst memory in the world. What I find is that most students do not have memory problems unless they’ve had some cerebral organic deficit, whatever. Most students don’t have memory problems.

They have strategy problems and strategies can be taught. They have things that impact memory, either they have poor attention that impacts the encoding or front side of memory or poor storage and retrieval, which is more of a strategy executive function problem but the memory is intact.

And here’s how we measure that in about two or three minutes off the [00:28:00] FAR. For those neuropsychological gurus listening, this is going to sound exactly like the California Verbal Learning Test in your right. Jeremy, I would say 12 words and I would ask you to repeat back these words in any order that you can. I’d say the words and maybe you get four or five. That’s not that great.

Let’s try it again. I’d say the exact same word list but this time I’m going to say, now repeat back all the parts of a bicycle you heard. Now repeat back all the fruits and vegetables you heard. Now repeat back all the animals. When I play the role of your frontal lobes and organize the information, you’re able to fill it in. Now these kids are getting 10 or 11 out of 12, guess what? The problem with reading comprehension isn’t bad memory, it’s bad storage and retrieval.

These kids need [00:29:00] graphic organizers, need discussions prior to reading. They need some architectural framework to help themselves organize in advance because the prefrontal lobes can’t do it in real time while they’re reading but you provide the structure in advance, all of a sudden the comprehension emerges.

Dr. Sharp: Yeah. I like that.

Dr. Steve: Another …

Dr. Sharp: Sorry, I’m going to interrupt you just for a second for an insight. So we do do the CVLT, we pair that with a lot of other things but I’ve never really made that connection between CVLT performance and reading comprehension, for example. So that’s illuminating for me. I just want to put that out there. I’m going to be thinking about things a little bit differently. But you have a built in. That’s the important thing with the test is that it’s already there with this reading test, which is great.

Dr. Steve: Yeah. Otherwise you have to [00:30:00] infer how did these processing measures impact reading or writing or math because that’s the issue. I’m in the same boat as you when you give processing measures independent of the academic skill, it takes quite a clinical leap to figure out, okay, what does low score here have to do with why Billy can’t conjugate a verb in 4th grade? I’m not sure I get that connection unless we build it right into that particular skill in question.

Dr. Sharp: Sure. Nice. So what else are you looking at with the FAR? You’ve mentioned the different types of reading disorders two times and how it can help separate those out. What are some of those different types and how’s that playing out in the test?

Dr. Steve: The goal of the test is not to so much say where you are in reading, but why? So what the test is going to do is tell you the four subtypes of reading disorders, if [00:31:00] any, that you might have and what to do about it. The four subtypes are if you have struggle with your decoding and phonemic awareness skills and we try to measure phonemic awareness and decoding in every possible context, whether it’s just straight manipulation of sounds, rhyming words, things like that, to how well you can actually decode let’s say a nonsense word versus an actual word versus a word in context.

So for example, you might have the word giraffe and the word giraffe would be on a … you would do a subtest called Isolated Word Reading Fluency. It’s very much like letter word identification from the Woodcock-Johnson. It’s just a list of words. Just read down this list. Let’s see how many you can get in a minute. Ready, set, go.

But later in the test, this is where we get a little diabolical, you’re going to have those exact same words we wrote a story about. There might be a [00:32:00] story with the word giraffe in there. So we can make a direct comparison about your ability to identify and decode giraffe in isolation versus context.

And let’s say you can do it in context, great. You can decode in the story, great but not in isolation. What’s happening is that these kids rely on context clues to figure out what the word is but you show it independent of that and they’re still struggling, we still have a little bit of decoding work to do.

Without running down every single subtest, we have a collection of subtests that are going to look at the phonics side of reading. We have a collection of subtests that are going to look at the fluency and speed side. And the argument that I make is that phonics allows for reading accuracy but orthography allows for reading speed.

So we try to measure orthography within some of these fluency subtests, and that’s just the ability to [00:33:00] see the entire word as a whole, as a gestalt. There is a reason that every 1st grade teacher has $100 class to their bulletin board. They really don’t want a kid to Ta ha ee. No, just say the word the, okay? You just bang out some of these words.

So we’re trying to get at orthography, and we have a section just on comprehension, and that’s the part that would involve vocabulary development, working memory, some of the executive functioning pieces we talked about earlier.

Dr. Sharp: Yeah, that’s fantastic. So you mentioned the orthography. I’m going to do a shout out to our Facebook group. Someone had asked about orthographic dyslexia. Is that a thing? If it is, what is it and what do we do with it?

Dr. Steve: Yeah. The confusing part about neuropsychology and frankly, our field as a whole is we have 27 different names for the exact same thing.

You’ll see this [00:34:00] in the literature called dyseidetic dyslexia, dysfluency, the term we use is surface dyslexia. We’re all talking about the same thing. These are kids who are really good at sounding out individual letters. They might see the word, debt, and read it as d-e-b-t.

So they’re sounding out the individual letters but they don’t get it as a whole. They’re letter by letter, sound by sound readers. They really don’t need so much of a phonics program because they can decode anything. They can’t put it all together. What’s suffering is speed and fluency.

So no matter what the term you use, dyseidetic or orthographic dyslexia, surface dyslexia, it’s all getting down disfluent in reading. The argument we’re making is because of poor orthographic skills and that’s where we’re going to [00:35:00] focus our interventions.

Dr. Sharp: Sure. That makes sense. I would love to circle around and talk about interventions. Maybe now is a good time to do that because it sounds like that’s a big motivation for your measure is being able to identify exactly what type of reading issue is happening and then that guides treatment or intervention. And it’s not a one size fits all.

Dr. Steve: Correct. And that’s the gist of all of these measures. I can say that to me, this is what being a 2018 psychologist is, and when I go back early in my career, back in the early and mid-90s, all I needed to do as a psychologist is come up with an IQ score. I was just an IQ tester. That’s all anybody wanted out of me.

And I realized that was unfulfilling nor was it very helpful for kids nor did it translate into an appropriate goal or objective, I tried to beef up my evaluation, say, well, maybe I’ll just help teams, [00:36:00] give them more information so they can make better qualification decisions but the fact of the matter is it’s taken me a while to get here.

And to me, at this point in my career, I think that a good report, a good evaluation is predicated on one thing and one thing only, and that is the interventions that you can recommend. And if you cannot take that report to the finish line and not provide families and teachers with appropriate interventions, I feel like I haven’t done my job.

So what we did with the FAR and also the FAM is we have the interpretive report writer where you can just take your scores and plug it right into the computer, yours truly will write the report for you but it’s a computerized report. It’s pretty wonky. It’s pretty stiff. You don’t need it for that. You can write it up yourself probably much better.

Just go to [00:37:00] the end and you’re going to have … Our research team probably looked at close to 100 different reading programs for this report writer. The goal is not to give you 100 reading programs. You’re going to get five or six reading programs that might be very helpful. Whether it’s Orton-Gillingham or Lindamood or Read 180 or Read Naturally or Fundations or Wilson or whatever, you’re going to get a few of those but more importantly, you’re going to get real in depth strategies for parents, for teachers, for tutors. That’s what you want to steal, paste, put it in your reports, be the rock star at the meeting. That’s what people want.

Dr. Sharp: Yeah. Oh my gosh. I can hear listeners cheering in my mind right now because I think that’s the thing. It’s like what recommendations do we actually make that are going to be useful, not cookie cutter and relevant for each person?

Dr. Steve: Yeah, if you could pull a report earlier in my career, [00:38:00] you would laugh because I had such cutting edge recommendations as Billy needs preferential seating in class or accompany verbal directions with visual cues. Our teachers are laughing at that stuff, and that’s all I had. That’s all I was given. It just was taking away from the value of what we do. And I think what we do is a very valuable. And this to me is what teachers really want.

Dr. Sharp: Sure. I think that a lot of us are maybe guilty of that in some regard where, like you said at the beginning, we get stuck on interventions and especially with reading, maybe this is different for others but I’ve had the experience that we’re taught that Orton-Gillingham and its variants are the gold standard and what is there beyond that? It’s important to know that there’s more beyond that and we can tailor recommendations a little better.

Dr. Steve: I [00:39:00] think so. Absolutely.

Dr. Sharp: I appreciate you talking so much about this test. The way you describe it feels very different than some of the others out there. Just like a very practical question for folks who might not know, how is it scored or what kind of scores does it produce? And can you compare those to other measures? I’m just going to throw in the big question that I ask everybody just to stir the pot. How are you identifying a learning disorder without a discrepancy model and how do you look at that?

Dr. Steve: Okay. Wow. We’ll take the first part. The scoring itself is pretty much … PAR does a wonderful job at norming these instruments. So you’re going to have a mean of 100, a standard deviation of 15. You’re going to get standard scores. You’re going to get percentiles.

I think it’s just so [00:40:00] important for psychologists, let’s not lean too much on the numbers. I’ve been at a lot of meetings where we say Billy has a standard score of 86 and people will say, okay, what does that mean? I say, well, they’re reading at the 18th percentile compared to peers. Okay, what does that mean? Well, they’re below grade level. Okay, got you.

That’s that terminology we get so sometimes into statistics. I really think people like the qualitative information to provide a richer description of how a kid is doing. In terms of scoring, the FAR with the interpretive report writer, if you don’t, I’ll put a little shout out, have a PARiConnect account. You can go to PAR and get one for free.

When you get a free PARiConnect account, you will get, I believe, five free scorings of any instrument you want to use, anything that’s on the PAR [00:41:00] platform. If the FAR is one of the instruments you choose, you simply will take your raw score and just enter it right in the computer. You’ll never have to look at the manual. The computer will score it up, write the report for you, give you all the recommendations. There you go.

Dr. Sharp: Awesome.

Dr. Steve: That’s the beauty of a report writer, people, if you’re a little reluctant, I write my own reports. I’m fine with that. I’d say go with that. Your reports are probably better written but what this will allow you to do is never look at the manual if you find it. I spent so much time writing the manual and nobody looks at it anyway. You can score it immediately on the computer but it’s going to give you the interventions as well.

The second part of the question is, how are you determining a learning disability? Jack Naglieri and I have been doing quite a bit of workshops recently and some papers as well showing [00:42:00] that I think the way to go, and most people are going into a PSW, Processing Strengths and Weaknesses model. However, within the PSW approach, wow, there’s all kinds of models and they’re very complicated and there’s all these computer formulas and we’re saying to us, that’s complete overkill.

I used the Discrepancy/Consistency model and it couldn’t be more simpler. You simply, with your cognitive scores, you want to see some variability in cognitive functioning. I think what we have to retrain ourselves to do, and this is really hard, is not look at the Full Scale IQ. We all our eyes go to that. That’s how we were trained. We always look at full scale but I think the real power in these tests are in the index scores, and you want to see some variability, some cognitive strengths, some cognitive weaknesses.

And then when you administer, let’s say, the FAR and it doesn’t have to be the FAR, whatever achievement test you’re using, you’re going to see some strengths [00:43:00] and weaknesses probably in academics, but not always. So what you’re looking for for the Discrepancy/Consistency model; do you have variability in cognitive functioning? That’s the discrepancy part, not the discrepancy between ability achievement, we’re not doing that. Is there now consistency between the weaker part of cognitive functioning and poor performance on an academic task?

So for example, if you say, yeah, I’m doing processing and I gave a VMI or a Bender, and the student bombed a VMI or a Bender, well, that’s very interesting, but a VMI or a Bender has nothing at all to do with reading. You have to have some element of processing phonological awareness, language, working memory, executive, some element of processing that you can make the argument that the weak academic skill is because of a roadblock that we’re going to call [00:44:00] processing.

So the model that I use is pretty much, it’s a PSW approach, but it’s the most simplest one out there; and that is the discrepancy/consistency model.

Dr. Sharp: Got you. I appreciate your perspective on that. I know that folks are all over the place and there’s a lot of research being done to figure out how do we do this. There’s a lot to sort out there. So it’s always nice to hear that perspective.

Dr. Steve: And it goes back to day one, I’m sounding like an old man in this field when I say I go back now 25 years but when I broke in 1993, Jeremy, here was the question. What’s a learning disability and how do we measure it? We’re still there in many ways. We’re still wrestling with that. I think you’re exactly right. There’s a lot of competing models out there [00:45:00] within PSW, not to mention folks who say not go back to the IQ testing or go back to RTI or whatever.

So there’s a lot of disagreement. I do travel quite a bit, I do a lot of presentations at school systems throughout the country, and I feel most people are going in the PSW direction. They’re just not sure which of those models to lean into. And I say, for me, I need to keep things as simple as possible because I need to explain this stuff to parents and teachers, and I can’t get too wonky with the statistics. No one’s going to know what I’m talking about.

Sometimes, I don’t even know what I’m talking about so I try to keep it in the most simple way possible because that’s what people want to know; why can’t Billy read and what are we going to do about it?

Dr. Sharp: Yeah, absolutely. I know that you also have the FAM. We’ve talked about that a little bit, which is the sister test, I suppose, [00:46:00] that’s really looking at math abilities. Is it safe to say the structure is the same and you’re looking at different types of math disorders as well?

Dr. Steve: Yeah, exactly. I think that’s a very appropriate description. It’s the sister of test of the FAR. Now, granted, math is not as sexy a subject as reading, reading gets all the attention. Math is sometimes is that ugly stepchild of education.

Basically three types of math issues with kids that we’re looking at. One is more that difficulty with the automatic fact retrieval even though you conceptually understand math. We also look at what we call the procedural subtype, and that is, you’re fine with your automatic fact retrieval but you start getting into problems that require three, four, five steps, long division, fractions, decimals, you struggle following a set of [00:47:00] procedures or what we call a math algorithm.

And oftentimes that’s because of limitations in working memory, and oftentimes that’s because of anxiety as well. We get very anxious when we are problem solving. That’s an area that we overlook so often. I know I get so hypercognitive in my approach, you forget about that emotional side too.

And there’s something about math that brings the anxiety out in many of us.

And then the last subtype is more that, we call it the semantic subtype, the conceptual understanding. So we try to break it down. Math is a little slippery because you could be really good on one element of math and maybe not so good on another.

Dr. Sharp: Yeah. So the first two are pretty straightforward in terms of how you would assess that. The last one, I’m wondering if you could clarify that a little bit. What types of math problems might illuminate semantic type weakness?

[00:48:00] Dr. Steve: Well, I’ll talk about two subtests on there. One of my favorite is called equation building. So equation building is, you’re presented with a word problem and as the examiner, you read the word problem while the student is reading along on their side of the easel. So reading is not an issue here.

And the student does not have to solve the word problem. They just have to set it up and it’s all multiple choice. Billy has four apples, Mary gives him two more, how many apples does Billy have altogether? You’re going to see four equations underneath that, and you have to pick the one that best represents and sets up the problem you just read.

It starts out very easy but it will get up to … the test goes up for H21, so, yes, there are calculus items on there as well. But we’re looking at, from a conceptual standpoint, your understanding about what exactly is this problem asking me and what formula or [00:49:00] strategy do I deploy to come up with that answer.

We also have the various, what we call the knowledge subtests; addition, subtraction, multiplication, division, where you’re going to have a page of problems with the answers already there. What might be missing is a missing addend, so it might be a very simple one; 5+_=9. So the answers are already there, you’ve got to fill in the blank and it’s timed.

And what we find in math, and you can think about it especially with multiplication, kids memorize the script. They memorize answers, but it doesn’t really mean they conceptually understand math, so it does a good job of really exposing those students who are just memorizing answers because we provide you the answer. You have to fill in the missing addend, and that’s where we zap you as to you really understand the concept behind this.

Dr. Sharp: I hear you. [00:50:00] I love this. So then is it the same deal where there’s an interpretive report that also gives intervention recommendations based on these specific deficits?

Dr. Steve: Kind of yes. The report writer is finished. Lots of interventions and you’d be surprised and we were surprised as well until we dove into the research. There’s a lot out there for math. Maybe not as many programs as what you’ll find with reading, but there’s probably more than you think in addition to just math programs; apps, websites, specific strategies, all of that is built into the report writer.

The publishing company has not yet released it but it is finished and I can just tell you that I’m hearing shortly. So you can tell me how many weeks are in a shortly [00:51:00] but hopefully that will be out soon.

Dr. Sharp: Got you. I know that was a question that some folks from the Facebook community had so I’m glad that we stumbled into that. So that’s great. It sounds like it’s coming. It’s finished. You just got to polish it up and somehow get the publisher to act, which is great.

Dr. Steve: That’s the plan.

Dr. Sharp: Nice. Off the top of your head, could you talk just a little bit about how we might approach each of those different math subtypes and what interventions could be helpful? We don’t have to go into a ton of detail, but I’m curious if you have anything right off the bat.

Dr. Steve: The first thing that comes to mind and what we talk about just to mentioned too just in general rather than specific programs, what we do it and I do a lot of this in the math workshops that I do is we try to present math interventions by way of games and activities. [00:52:00] I know this is going to sound funny and maybe I’m an outlier on this, but I think math can be a lot of fun. Presented in the format of games and activities, it is a lot of fun. It brings anxiety down, which is paramount because as anxiety diminishes, then we can maximize working memory.

I truly believe one of the major hurdles with a lot of kids with math, we can think back to our own math careers. I know it was certainly true for me is some aspects of math we just freak out about and get very anxious and uptight about, one of the reasons, the first thing we do is we show a lot of different games and activities that can help us build and there’s a fun game we do to help kids see the difference between a fraction and a decimal in a picture.

It’s a fun game that we played because for a lot of kids, the fact that there could be a number between one and two, are you kidding me? There isn’t. Yes, [00:53:00] you can subdivide space to infinity, so to speak but we play a little game that helps kids with that.

The other thing that I find that is helpful for math, and I think this is why we get so anxious in the first place is we work with kids to help them visualize magnitudes and amounts. If you cannot visualize math, then it really gets reduced to being nothing more than a bunch of abstract signs and symbols that are very intimidating, very cold, very harsh, and the anxiety builds up because these symbols that we teach don’t correspond to any meaning or value or magnitude.

When we can visualize math, it becomes a lot easier. A great curriculum that allows for that is something like the Singapore Math Curriculum. Singapore Math is all based on bar models and learning to [00:54:00] represent magnitudes and amounts through pictures first, then we’ll get to the equation but we never lead with the equation that always has to be represented with a picture. And it’s amazing when you can do that. The clarity, oh, that’s what we’re doing here. Then we can bring in the symbols and it makes a lot more sense.

Dr. Sharp: Yeah. I’m totally with you. Our kids are both in Montessori school and I was struck by how much time they spent doing math without writing anything. They have a ton of manipulatives and pictures, like you said, and just those visual representations of magnitude and quantity and it’s been really cool. Both of our kids have really taken to math and they seem advanced compared to a public school curriculum, and I think a lot of it has to do with what you’re talking about, and it’s really cool.

Dr. Steve: Yeah, I think [00:55:00] Montessori does it right in bringing all those manipulatives, as you said, and allows kids to visualize math. Let me ask you a question, are your kids fearful of math or is it kind of fun?

Dr. Sharp: No, they love it. We always ask, what was your favorite part of the day? It’s always a math thing for my son and our daughter’s probably half and a half, but they love it.

Dr. Steve: Great.

Dr. Sharp: So that’s good. I’m glad we’re on the right track there. My gosh, we’ve covered a lot here today. I was just looking back and I know that there’s plenty more that we could dig into, but I do want to ask the … gosh, so many people in the Facebook group asked about are you ever going to come out with a writing test? Because I feel like we, as a field, are struggling with writing tests. And if you’re not coming out with writing test, do you have anything new on the horizon that we should know [00:56:00] about?

Dr. Steve: I think I can answer both of those questions. So we have the FAR and we have the FAM. If we did a writing test, what do you think that would be called? You are correct. It is the FAW. I admit it’s …

Dr. Sharp: Fair enough, consistency is nice.

Dr. Steve: Yeah. There’s a story that I’ll tell you off air about the names of those tests, but the FAW is due to be released in about a year, I’m going to say a year to a year and a half. The test is finished. It is currently in norming. I just spoke with the publishing company earlier this week, we’ve got about half of our cases back. It takes a little time to norm 1,200 kids on a writing test but we are right on schedule and I am slotted next summer to actually write the manual and usually when you [00:57:00] write the manual, that’s the final step in the finishing of a product.

I’m super excited about it. It does divide math into various subtypes. The main thing, as a clinician, we always have to determine with writing; is it a motor skill problem or is it a cognitive linguistic problem? There are subtests that are simply looking at apraxia and dexterity and whether you can prejudge how big or small to make the writing and things like that. I think an OT would really appreciate that element to the test.

Of course, the rest of it is more looking at the different kinds of writing disorders that students could have. And most of them come under this broad category that we’re going to call executive dysgraphia, which is, there’s something with executive functioning, whether it’s planning, organization, word [00:58:00] retrieval, how you syntactically arrange your thoughts, there’s something that you’re struggling with there that’s inhibiting you from producing it out on paper here.

So we’re super excited about the test and it is a coming. I can tell you that I’m with you. I’m very frustrated by our current writing measures. I used to give the WIAT and Jeremy, I gave up because I don’t know how to score it. It’s just killing me.

Dr. Sharp: No kidding. To be honest, I stopped giving the essay. We were giving the sentence composition tasks, but I’ve switched to the KTEA only because I do a psychometrist model and we have a lot of graduate students that start each year and it takes six months, it feels like, to train them how to score those tests and then they’re leaving. And then I have to find somebody new. So anyway, it’s challenging to know that you’re doing it right. [00:59:00] I can’t even think about how much variability in scoring is happening just because it’s so complex, but they own the WIAT particularly.

Dr. Steve: Yeah, I was finding, at least for me, that every student was average. I know it was me because I was not scoring it probably correctly or appropriately. I was really struggling and finding that my achievement test of choice these days is the KTEA-3. Do you use that one for writing, the KTEA?

Dr. Sharp: Yeah, we switched to that just a few months ago because the writing subtests are a lot easier to score and more straightforward and I think were real-life applicable. Actually, they seem to mirror actual academic tasks a little better.

Dr. Steve: Yeah, with the story booklet that the kids worked their way through is … all I know is that I feel like I understand how to score it. So I feel like I’m a little more confident from my end. [01:00:00] It is a difficult issue in our field is there’s not a lot out there with writing.

Dr. Sharp: Right. Well, we’re kind of touching on this a little bit, I’d like to expand that if we could, what other measures might you use in your own battery in addition to your own, of course? How would you go about assessing some of our common concerns as far as learning disorders or ADHD and whatnot?

Dr. Steve: Well, let’s take a hypothetical example. What’s our common referral? Maybe a 4th grade student reading on a first or second grade level. Maybe there’s a little undercurrent of anxiety as well. Let’s say those are the main issues.

So for me, I think it’s important when you craft together a test battery and [01:01:00] we talked a little bit about this off the air. We hear a lot about how to assess kids from a lot of folks who don’t assess kids but it’s important to, and I’m always asking others this exact same question because I assess kids. I want to learn what other people are doing and what works and doesn’t, time is everything. So if you see a kid twice, for me, that’s a luxury and each testing session is maybe two hours at the most because if a student isn’t tired in two hours, I am. So we’re stopping.

What can I work in to that three to four-hour slot? How do I prioritize things because that’s the reality of that’s the amount of time we’re going to have with the student? So my battery would be a cognitive measure. That might be a WISC–V or a cognitive assessment test, something like that. My achievement test is often the KTEA-3 these [01:02:00] days. I like it very much.

If they score low, let’s say on the reading or the math or something like that on the KTEA, then I’m going to really follow that up strong with either the FAR or the FAM to get into the whys, why they scored low. We said in this referral there might be an undercurrent of anxiety, so I might do something like a Reynolds Child Manifest Anxiety Scale that takes five minutes or the MASC 2, the Multidimensional Anxiety Scale for Children that takes five minutes.

I’m going to try, if I can, to do a classroom observation. I have a developmental history form. That’s it. We’re done. That’s the battery, but I know with the FAR and the FAM, if you’re saying, wait a sec, you didn’t cover working memory, you didn’t cover executive function, you didn’t cover all these things, I know that’s built into these tests. I can write this up in a lot of depth and looking at all these areas of [01:03:00] processing because the FAR and the FAM built those in.

I don’t have to give the standalone tests unless I have questions about executive functioning, for example, in other arenas besides reading or math. If I have questions about executive functioning and behavior, now I’m pulling out a BRIEF or a CEFI or some sort of rating scale such as that.

Dr. Sharp: Sure. Okay. That sounds good. I had a question about that but I forget. That’s a brief battery that gets at a lot of the core pieces. So you’re not going overboard but you’re getting the important information.

Dr. Steve: If you know what you’re looking for, I think it allows you to get the most essential information. People might say, well, you’re a neuropsychologist, why don’t you get the NEPSY too on every kid that you see? I’m sure [01:04:00] Pearson’s not going to like the following comments, but the NEPSY …

Dr. Sharp: You’re safe here.

Dr. Steve: I’m safe here. To me, it’s like going to a restaurant, let me make us my silly analogy. If you’ve gone to a Denny’s or a Bob’s Big Boy or those kinds of restaurants, it’s the only thing open at midnight and you’re hungry. You walk in, you get the menu. That menu is 20 pages thick. It has breakfast items, lunch items, it has Chinese food, it has Italian food, it has American food. It’s got everything on there. No matter what you select, the food’s going to be mediocre at best, but at least you have everything on there.

The NEPSY is the Denny’s of assessment. It’s got everything and it does it all very mediocre. It’s got memory on there, but frankly, the WRAML-2 is much better. The ChAMP is much better, but the NEPSY’s got some memory. The NEPSY’s got some phonological processing, but you know what, the [01:05:00] sea top blows it away.

The NEPSY has a little bit of everything on there. It’s a one stop shop but the fact is, I think there’s other standalone measures and this, I can think of 1,000 school neuropsychologists who now want to just shut me up through their computer and say, no, we love the NEPSY and that’s fine because I think testing is such a personal decision. I use subtests of the NEPSY, but I’m saying, I think you can get that information through other tests as well that sometimes do a better job.

Dr. Sharp: I hear you. I appreciate your willingness to put that out there. I’ve never heard anyone say the NEPSY is the Denny’s of assessment so I’m going to be considering that. That’s pretty good.

So the one thing, I do want to go back, I noticed in your ideal battery that we were talking about, you didn’t say anything about the CTOPP or the GORT or some of those other reading measures [01:06:00] that I know folks will use quite a bit. Is that because the measures you are using cover all those constructs already or what?

Dr. Steve: Yeah, primarily, I can say that I feel that the CTOPP is the gold standard out there and that thank goodness for people like Joseph Torgerson, who brought phonological processing to our awareness. Thank goodness for people like Nancy Mather, who’s the first one to introduce orthographic processing. Dean Delis with California Verbal Learning Test.

These are phenomenal instruments. If you would ask me this question prior to the FAR being released three years ago, you would have heard a completely different battery. You would have heard me talk about the California Verbal Learning Test, the GORT, the CTOPP.

I like Nancy’s test called the Test of Orthographic Competence but I had to shop around to get all of these various instruments which as a school [01:07:00] psychologist can be very difficult because your office is the trunk of your car and you’re hoarding all these different assessments in there. What we tried to do with the FAR is make it one-stop shop. And believe me, those instruments very much were inspirational for me in developing the FAR.

Dr. Sharp: Very cool. I was curious about that. I think that was a great way to put it. I’m just conscious of your time and I know we’ve been talking for a while and we’ve hit so much important information. How would you like to wrap up? Things that are important for you on your horizon, areas that people could go or places people could go to find out more. What else feels important to you?

Dr. Steve: Well, I suppose it wouldn’t be a proper wrap-up if I didn’t tell you one more quick story. It [01:08:00] will also tie into what I’m doing these days, and I should thank you for allowing the time to talk about some of these things. But earlier this year, following the school shootings in Parkland, Florida, my son’s in college, and my two daughters, I started every day dropping them off at school. They asked us to take them down to Washington, DC, and we live right outside of Washington, for the March for Our Lives rally.

We’re not real political people, but we said, sure. We took them down. There were 800,000 other fairly nonpolitical people like us. These are just families. These were kids and all they wanted was a safe place to learn. I was very touched. I was very inspired. It’s so struck a chord with me that the …

[01:09:00] I don’t sing or dance, to me, my writing is my art and that’s the only expression I have that I’ve really started to research and write quite a bit about stress and trauma and how difficult it must be nowadays to go to school. With that in the back of your mind, how do you learn when you’re fearful of your safety?

So the project I’m working on now, very pleased about is a new book on the neuropsychology of stress, trauma, and learning. And that’s been the big focus. I will be presenting on the topic at NASP in 2019 in Atlanta. Two workshops surrounding that. It’s just an area that, I’m not sure I would have gone naturally in that direction if I wasn’t a part of that march. I know it was occurring all [01:10:00] over the country in many cities. It struck a chord, that’s all I can say.

Dr. Sharp: Sure, it sounds like an incredibly powerful experience. I think sometimes that’s how it works best when you have those experiences and then it drives the work that you do. I’ll look forward to that book.

Dr. Steve: The two main projects at this point, as we wrap up a bit the FAW, the writing test, it will certainly be … hopefully, we’re going to put the finishing touches on that next year about this whole notion of stress and trauma. I know we talked two weeks ago and it sounds like you start your day very similar to how I start my day. And that is, I drop the kids off at school and I wouldn’t be truthful if given you try not to think the worst, but [01:11:00] I think everybody in that car line sometimes is thinking the same thing.

Dr. Sharp: No, I think you’re right. I think we’re fortunate to live in fairly what we think are safe places but I think a lot of parents probably think they live in safe places too. It’s sad and pretty disturbing that we have to think about all of this now and it’s in the back of a lot of our minds.

Dr. Steve: Right. That’s what is keeping me occupied these days, and certainly still testing and still hitting the speaking circuit quite heavily, and you’ll probably see me out and about. I’m sure people are going to come up to me for those comments about the NEPSY and let me have it. And you win. I apologize right now for even saying that.

Dr. Sharp: Well, that’s probably a great segue to ask how people can get in touch with you if they have questions or comments [01:12:00] or would like to learn more about what you’re doing.

Dr. Steve: You’re more than welcome. My email is feifer@comcast.net. I get a lot of questions about the FAR, quite naturally if you’re that vain to put your name on a test, you better own it and answer the questions. I try my best to respond to emails. The FAR and the FAM are sold through PAR, but if you’re interested in some of our books or I do try to post my speaking schedule, that is our websites, www.schoolneuropsychpress.com.

Dr. Sharp: Fantastic. I’ll have all that information in the show notes and I’ll have links to your tests. We didn’t even touch on your books, but I know you’ve [01:13:00] co-authored books as well. I’ll have links to those in case folks want to go check them out and get in touch with you.

Dr. Steve: Super.

Dr. Sharp: Yeah. Well, Steve, thank you so much for this time. You’re doing excellent work and I feel very fortunate to have been able to pick your brain for an hour or so, and I hope that others will find it useful as well. So thank you so much.

Dr. Steve: It’s hard to believe an hour’s gone by, but Jeremy, thank you, any time. I really appreciate your interest in some of the things I’m doing. Thanks for having me today.

Dr. Sharp: Yeah, of course. Take care.

All right, thanks y’all so much for listening to the show today. I hope that you enjoyed this one. This was one of the best ones so far. I walked away with a lot of good information and really just feeling like Steve is such a personable guy and easy to connect with and clearly knows what he’s talking about. I’d be remiss not to say that I’m really [01:14:00] excited about this writing test that is coming out in a year or so. So thanks for listening.

If you have not subscribed to the podcast, I would love for you to do that. That’ll make sure that you don’t miss any episodes in the future, any interviews, any business tips, anything that we come out with. So you can do that in iTunes or Stitcher or Google Play or wherever you get your podcasts. It helps to spread the word about the podcast, which is great.

Otherwise, if you need any coaching or consulting in your practice, that’s a large part of what I do here. So if you are a testing psychologist and you’re looking to build a testing practice or grow a testing practice or hire new folks or really anything on the business side, give me a shout and see if coaching could be helpful for you. You can do a complimentary 20-minute phone call. You can book that at thetestingpsychologist.com/consulting, and we [01:15:00] can talk and figure out if consulting is a good idea for you. If not, I’ll point you in the right direction of something else that might be a better idea. I’d love to work with you if you need that.

All right, stay tuned. Great interviews coming up over the next few weeks and I hope to have you back. Take care.

Click here to listen instead!

[00:00:00]Dr. Sharp: Hey, what’s up y’all? This is Dr. Jeremy Sharp. This is The Testing Psychologist podcast, episode 70. I think you’re going to like this conversation. This is maybe a topic that you have not heard about.

I am talking today with Dr. Jenni Pacheco who has a Ph.D. in Cognitive Neuroscience and she is a Program Manager at NIMH, where she does extensive research in RDoC, Research Domain Criteria.

If you don’t know what that is, that’s totally okay. We’re going to talk all about it and let you know what it is. I will say that it is a research-based way of looking at psychopathology and differences in functioning that shifts our paradigm of how we typically see people and mental health concerns. So Jenni’s going to talk all about that with us and it’s a pretty cool conversation.

[00:01:00] Before we jump to it, I want to let you know about the clinical, administrative, and training paperwork packets that I have put together specifically for testing psychologists. You can find them at thetestingpsychologist.com/paperwork. Like I said, there are three different packets or if you want all of them, you can get them bundled for a little discount. Go to the website, check it out, and see if any of those will be helpful in your practice. You can use the code “podcast” to get 20% off your entire purchase.

All right, y’all. On to the conversation with Dr. Jenni Pacheco.

Hey, y’all. Welcome back to another episode of The Testing Psychologist podcast. I’m Dr. Jeremy Sharp. Hope y’all are doing well today. My [00:02:00] guest today is someone who I’ve been looking forward to speaking with for a long time, and it’s been a long time coming.

Dr. Jenni Pacheco and I have been in conversations trying to make this happen for two or three months now. We’ve had to work through a little bit of bureaucratic red tape, but I think that that is, it’s going to be totally worth it.

Jenni is going to be talking with us about RDoC. If you don’t know what that is, that’s totally okay. We’re going to tell you exactly what it is and why it’s relevant to our field. I’m super excited to talk with her.

Jenni, welcome to the podcast.

Dr. Jenni: Hi, thanks. Happy to be here.

Dr. Sharp: Very cool. I’m so thankful that you were able to take the time and we could make this happen. I mentioned the red tape. So Jenni is a Scientific Program Manager with the RDoC unit at NIH. [00:03:00] She also manages a portfolio of other grand projects but working at this government agency, it comes with some hurdles, I suppose, to be able to talk about what you’re doing with people not in that government agency. Is that right?

Dr. Jenni: Yeah, for sure. There’s definitely things that are going on here that we have to wait until we get final approval to let everyone know what we’re working on.

Dr. Sharp: Sure. I’m just so thankful that we were able to pull it together and craft hopefully a meaningful conversation for folks around RDoC and what that is, where it came from, how we can use it, and all sorts of cool things.

Dr. Jenni: Yeah, I’m looking forward to it.

Dr. Sharp: Nice. So, Jenni, you have a PhD in Psychology. That’s about as much as I know about your background aside from what you do right now. So could we maybe just lead and talk a little about how you got here, [00:04:00] what your experience looks like and what you’re doing day to day right now?

Dr. Jenni: Yeah, for sure. My route is probably a little bit more convoluted than most, although lately, that seems to be the norm rather than the exception. I have a Bachelor’s degree in Biochemistry and I got a Master’s degree right after that in Psychology. I stopped and worked for a little bit mostly doing fMRI and structural MRI data processing before I figured out exactly what I wanted to do with a PhD.

So I went to the University of Texas in Austin for my PhD. It was a Cognitive Psychology PhD. I was doing a lot of, like I said, structural and functional MRI, looking at healthy aging and various memory components of healthy aging and how the brain changes and how that changes [00:05:00] function. I actually was looking a little bit at minorly touching on resiliency, and so what are the differences between older adults that were having issues versus older adults that might have been performing or functioning more like younger adults.

So I did that for a bit. I went on to do a postdoc with the National Institute of Aging, which is up in Baltimore. I went down the rabbit hole of how difficult it is to study something like dementia. It plays a role in my career track and now being in mental health, there’s a lot of heterogeneity in the onset and development of dementia, which of course makes studying it a bit tricky. There’s some nuance there of how to be sure what you’re measuring.

I spent some time as well working on traumatic [00:06:00] brain injury, looking at changes structurally and how we might be able to treat the effects of traumatic brain injury before I ended up here at NIMH. It was, I don’t want to call it happenstance, but I think I was just talking to the right people at the right time when there was a position here with the RDoC unit.

Because I don’t come from a strictly mental health background, at the time I had heard of it but I wasn’t intimately aware of what RDoC was, but as it got described to me, it made a lot of sense given my background with imaging and trying to tie in a bunch of different measures to understand what was happening in some areas that are pretty broad and variable given dementia or traumatic brain injury, it all made sense.

So the opportunity to work with the NIMH and help [00:07:00] look at the broader field and the bigger picture and where we should be heading was exciting and intriguing for me as opposed to continuing to do my own research. So I’ve been here for not quite 4 years and it’s been great.

Dr. Sharp: Very cool. It sounds like you had the right place at the right time and stuck.

Dr. Jenni: Yeah. I’m glad for that.

Dr. Sharp: Sure. So I have to ask, when were you in Austin? I did an internship at UT.

Dr. Jenni: Oh, I was there from 2007 to 2011.

Dr. Sharp: Oh, that’s awesome. I was there 2007-2008 for a predoc internship at their counseling center. Very cool. That sort of thing just blows my mind. I think about, oh, that’s wild. Did we ever walk by each other on campus or something?

Dr. Jenni: Right, it’s strange. It’s funny how all of these little fields end up being really [00:08:00] small. And as you talk to people, they’ve all been in through the same places at some point. So I’m sure we did.

Dr. Sharp: That’s fascinating. Nice. I always say if I were to leave Colorado, Austin is at the top of the list. I enjoyed my time there.

Dr. Jenni: Yeah, it was a great place to, especially for graduate school, it was just a real nice place to be.

Dr. Sharp: Yeah, that’s wild. Anyway, we could go down that rabbit hole, I had some friends in graduate school there. Anyway, we won’t do that. So here you are, you’re the program manager with RDoC. I think there’s a lot to unpack here so I’m just going to get right to it. Can you just tell us what is RDoC? What does it stand for? What is it? Anything you want to tackle.

Dr. Jenni: Yes, so RDoC stands for Research Domain Criteria. [00:09:00] It’s always hard to answer what is RDoC. I think it’s helpful to have an understanding of where the need for RDoC came from or how it came about, but ideally, RDoC is a framework or a way to structure your research where instead of beginning with a syndrome and trying to drill down to find underlying mechanisms, we’re trying to focus on research that starts with disruptions in neurobiological or behavioral mechanisms and work across systems to see what the connections are among the disruptions and how those would relate to clinical symptoms and start with this more mechanistic or systems-level understanding of what’s happening as opposed to starting with a broader defined syndrome that might be defined by clinical symptoms or observable signs.

[00:10:00] I think the need for it came from, if you think about the burden of mental health or where we are in terms of developing new treatments or new understanding, I think our research there had, I don’t necessarily want to call it a plateau but I think that we had stopped making a lot of big advances and I think that was hindering the ability to come up with new treatments or new therapies that would be effective if we were slowing on our understanding of what the differences of these disorders are.

As NIMH’s main mission is to help research both basic and translational to further treatment and diagnosis of patients suffering with mental health, I think our inability to make a lot of progress was seen as a big problem. And [00:11:00] so in 2008, there was a new strategic plan for NIMH which called for new ways of classifying mental illnesses based on more dimensions of observable behavior and neurobiological measures.

And so in 2009, we’re almost coming up on the 10 years of RDoC, it’s when this whole idea was given a name and tried to give some meaning to this idea of how would research look different or how can we start looking at the same questions but maybe from a new way that would help uncover something that could lead to treatments or therapies that we haven’t thought of yet.

Dr. Sharp: So my understanding that this was born out of maybe an observation or a feeling that we had hit the wall a little bit with diagnostic research and being able to nail down what was going on with folks from a mental health [00:12:00] standpoint.

Dr. Jenni: Yeah. I think that’s exactly it. If you think about the psychopathology research was maybe just not keeping up pace with clinical neuroscience or behavioral science and that perhaps our cycle of research was getting hampered or slowed down by having these disorders with names that are pervasive throughout the entire cycle of research, publications, treatment, clinical trials et cetera.

They’re using terms like depression or schizophrenia, all of these diagnoses are helpful in clearly and easily describing who your subjects are for a research design but by using this sample, you’re [00:13:00] restricting the types of questions you can ask or the types of answers you can get because we know there’s going to be some variability in your groups.

If you’re just looking at things group by group, there’s going to be variability in your control population. There’s going to be variability in your patient population. And so what does that say for your results?

And so NIMH and RDoC is trying to ask people to maybe look at things in this more dimensional way instead of a categorical way. So if you think about the domain of functioning that you might be interested in to look at that, perhaps across different disorders, perhaps just across the full range of functions, so including your control or your typical group versus your patient population.

If you start looking at these domains of function, it’s a little bit easier to start classifying people on a continuum as opposed to trying to fit [00:14:00] them in discrete boxes or categories based on what the diagnoses would be in the clinic.

Dr. Sharp: Is it a stretch to say that we’re talking more like bottom-up versus top-down, I don’t want to use the word classification but that’s the only word that is coming to mind.

Dr. Jenni: No, I don’t think it’s a stretch and those terms get thrown around in a lot of different scenarios, especially recently. No, I think you’re getting at it and we can start with disorders that we want to try and explain, but if our underlying mechanisms, if our behaviors and our biology isn’t matching up with where those boundaries have been placed, we do a disservice if we don’t look beyond them.

So if we’re only looking within these boxes that were set up and not trying to [00:15:00] see, are there different boundaries? Are there new disorders within this one disorder? Is depression comorbid with anxiety, two different disorders, or is that one disorder? Are there different boundaries and can we use our tools that we now have to be able to look at systems within the brain or more genetic relationships or other ways of measuring things? Can we look at those to give us more signs as to where this function is happening?

Dr. Sharp: Just as we start to talk about it, I’m starting to reckon with the weight of what y’all are doing because I’m thinking to myself, how do you even know what to call whatever you’re finding when we’ve called it something this [00:16:00] particular disorder for so many years. How do you know what you’re researching? It’s just like upsetting the whole system that’s been in place for so long. How do you start to tear that down?

Dr. Jenni: I think you’re hitting on exactly the tension that came out with the introduction of RDoC is this overall sense of why, how do we do that? What do we do now? And so I think that for a while, that’s what NIMH and RDoC have been trying to address.

Our big focal point is the RDoC Matrix, which is on our website. The matrix came about as an understanding that we can’t just change the way everyone’s looking at research and not provide some guidance. We can’t let everyone just flounder around on their own trying to figure out exactly what we mean.

And so the RDoC Matrix came about as [00:17:00] a framework for people to be able to look at and understand what we mean when we’re talking about a domain or function, what is a psychological construct, what would that look like if we’re talking about measuring physiology or measuring behavior or neural circuits, what are the key players there?

And so our matrix that has … it’s broken into functional domains and each domain has underlying sub-constructs of a psychological phenomenon that can be measured. We’ve put up several units of analysis or ways to measure that construct. So we’ve got from the very basic molecular cellular levels all the way up to various physiological or behavioral measurements of that construct.

And the idea here was that this would help researchers do exactly what you’re saying is figure out what to call things or what [00:18:00] are the things that we’d want to look at when we’re setting up our studies or when we’re asking the questions that we want to ask. Certainly, we’re not asking that anyone completely ignores all the work that has been done in the past or even pretend like these disorders don’t exist or aren’t a thing, we’re just asking perhaps, and there’s many different ways to set up a study that would be RDoC but we were just asking people to think perhaps a bit more broadly and a bit differently.

So you might still be recruiting subjects from a clinical setting who come in with a particular clinical problem but you might use a different type of measure to set up your research group. So you might not just do patients versus controls. You might get people in who have a certain issue that you’re interested in, but then maybe there’s a different measure that you give to everyone in your [00:19:00] breaking on the performance of attention or cognitive control.

There’s something that you’re interested in and that’s going to define your groups. And now we’re going to start to look at what are the underlying mechanisms or systems that play a role in the different performances on that measure you’re looking at.

Dr. Sharp: Yeah. I was just looking at the matrix and if people haven’t checked this out, I’ll definitely put this in the show notes. I think my first reaction is that it is thorough and maybe on the foot, the other side of that coin is complex or complicated, but this is just my …

I don’t work on this every day by any means, but it also looks really cool. Can you explain that a little bit and say more about how y’all break things down and these [00:20:00] different, I’m not even sure you call them, dimensions or domains and how they all fit together in your research?

Dr. Jenni: Yeah. When RDoC first came about, it ended up being six but we had different workshops for each domain. And so the domains were decided on as these broad areas that are important for psychological functioning or psychopathology research. We had large groups of experts come together for several days to discuss how to break that domain down and what are the kind of psychological constructs that go into them.

And we wanted to try, there’s always this desire to find the right grain size, so we don’t want to get too specific and we don’t want to be too broad in what our constructs are but we do have some criteria that we tried to keep in mind of having things that were discrete from each other so they weren’t necessarily [00:21:00] overlapping or just measuring the same thing and that they did have some independent or separable system driving it so that you could independently interrogate a construct and that you weren’t necessarily always going to, by default, have another construct be getting in the way.

So that was how we tried to break things down. For sure, those groups, we tried to have a wide variety of expertise from the field, so both animal and human researchers, clinicians as well as non-clinicians to try and get as many sides of the coin as we could. Ultimately, our aim was to have everyone come to a consensus and this was an attempt to, like I said, give a starting place for people to see on paper what it was that we were talking about when we tried to introduce RDoC.

[00:22:00] The idea is that these are examples and that it’s not exhaustive. There’s, of course, things that are important for psychopathology that are probably not in the matrix. And that’s not because we think they’re unimportant, it’s just at the time, it was kind of we have to start somewhere and here’s where we’ve started.

The the plan was always for this to be an evolving updatable entity. It didn’t happen for a long time that we didn’t really update it and I think it’s for exactly what you’ve mentioned of, there’s this real big shift that we’re asking people to do and it almost felt irresponsible to put up this matrix and then start changing it right away. We needed to give the field and everyone more time to understand what was happening here.

[00:23:00] A lot of this happened before I joined NIMH, but I think it took some time for NIMH and RDoC itself to get comfortable and familiar with the matrix and understand what was there.

In the past two years, we’ve formed a council work group. So the NIMH has an advisory council, and through that advisory council, we’ve formed a council work group for RDoC whose purpose is to help us with changes to the matrix so that when we do want to add something or when we look at it and think, well, something’s missing here, we now have a group of experts who are… their job with us is to look at the changes that are being proposed and figure out how to make sure that they make sense, how to make sure we’re not missing something, change one thing and we make something worse. We want to try and update for the better.

And so in the past year, we’ve actually made several changes. [00:24:00] This May, they submitted some changes to reorganize the positive valence domain and make it a little bit more, the word I want to say is efficient but it’s just a little bit more straightforward and maybe follow some of the research a little bit more closely than what was there originally.

And actually just last week at our council meeting, we had approved an addition of a sixth domain. So currently there’s five domains. They’re negative valence, positive valence, cognitive systems, social processes, and arousal and regulatory. We’re just adding a sixth domain, that’s the sensorimotor system.

And so we had a big group work through that one and figure out what construct should be in that domain. We pushed that through our advisory council and they all approved that that was a good change. [00:25:00] They wrote up an extensive report and so we’re working on getting that report up to the website and then we can update the matrix to add that domain.

So we’re we are trying to update it and change it as the field moves and changes. It’s probably more indicative of the fact that some of this research is just slow, so we aren’t going to be making rapid changes. We’re going to probably be making slower, more thoughtful changes but that’s where we’re trying to be able to keep up with what’s going on in the field.

Dr. Sharp: Yeah, that makes sense. Of course, things move slowly sometimes. I’m just looking at this and I’m kind of a visual person so I’m thinking of folks who might be listening in the car or something, I want to describe this a little bit because I think it maybe hangs together when you can see it a little bit more where you [00:26:00] have this overarching umbrella, it sounds like of the matrix. And then within that, you have these five, now six systems that you’re looking at. You said negative valence, positive valence, cognitive systems, social processes, and arousal and regulatory, and then you said sensory and perception; is that right?

Dr. Jenni: Sensorimotor.

Dr. Sharp: Sensorimotor, sorry. So you have those systems and then under each of those systems, there are these constructs or subconstructs. For an example, under cognitive, there’s a construct of attention, language, and working memory, for example. And then for each of those constructs, you break it down across a number of different dimensions, I suppose, there’s the molecular, there’s cellular, there are circuits involved, there’s physiology, there’s behavior, and then there’s self-report and paradigms, which I’m not totally sure what that is.

There are many [00:27:00] different levels of, like you said, you get pretty granular with this, it seems like.

Dr. Jenni: Right. The matrix is exactly a matrix. So the way it looks on our website is the rows are these psychological constructs and the columns are what we call units of analysis and those are areas that you can interrogate each of these psychological constructs and ways that you can look at them. And like you said, they go from molecules, cells, circuits, physiology, behavior, self-report.

And then we do have this last unit of analysis that’s called paradigms which is actually, it’s not quite the same as the other units of analysis that overarches them, but that’s our attempt to offer some either behavioral tests or maybe an fMRI task or some other [00:28:00] measure that you can use with your subjects that would actually measure the specific construct that you were looking at.

We had a large work group several years ago that went through each construct in the matrix and looked at through the literature and what’s being done and tried to pull out the best types of tasks or measures that could be used for that construct that was reliably measuring what you would expect it to measure for each of the constructs of the matrix. So we have that listed as well.

Our push there is at some point, it would be great if we could have some continuity of measure and common data elements so that at some point we can combine smaller data sets into larger data sets, and there’s some systematic way that they’re measuring different RDoC constructs so that we can really start to pull some larger data and see [00:29:00] what’s happening.

Dr. Sharp: Sure. I wonder if we could make this a little more real. Can you talk to me about just the research that y’all are doing day to day and maybe even give a concrete example of how all this comes to life when you’re looking at different levels of functioning in these domains?

Dr. Jenni: Yeah, sure. Of course, at the NIMH, we’re the funding agency. So we are funding researchers out at different universities and in different situations who have research questions and we’ll fund those projects.

So RDoC itself, we had at the, I think five or six years ago at this point, a series of funding announcements where people could submit grants that were specifically looking at an RDoC [00:30:00] focused type of project. I want to say they can be on anything, so there’s a lot of focus on trying to, like I’ve said before, look at distinct psychological constructs as opposed to entire disorders and try and track the functioning across a whole spectrum of people who clinically might be normal or not have a diagnosis all the way through people who might be have severe disorder and try and see how the functioning or how the systems that work together for that construct might change or might be different depending on where you are on that spectrum.

There’s a lot of work that’s happening that’s looking within one disorder category trying to look at, are there different subtypes or [00:31:00] different biotypes that we can pull out by looking at some of these psychological constructs. So could we get maybe genetic differences in playing with a functional circuit that’s working differently in one population or subpopulation than another?

And does that help us either further explain a current disorder and how we might give us any examples of some better way to focus treatment or perhaps it would help us figure out this subset of person would respond better to one treatment versus another, and so that might help find new targets for new treatments or therapies?

So I think there’s a lot of different ways that the work is being done. Within NIMH, we have several different divisions. I work for the Division of Translational Research, and within our division, [00:32:00] our off-the-cuff number is that about half of our studies do have some kind of RDoC influence to them at this point, which is, for me, it’s great to see that people are trying to understand what’s happening.

Again, I think it will offer a lot of payoff down the road because if we can find subtypes or different cut points for different constructs that’s going to give us a lot more information about how to treat or who to treat or when to treat.

My program is in development and so there’s a lot of differences of doing research in a development mindset as well because some of these constructs will look very different in a five-year-old versus a ten-year-old versus an 18-year-old. And so trying to understand how some of these disorders develop [00:33:00] and change over time with the environment that the kids are in or just the different developmental stage that they’re at is also really important to see how that plays into ultimately hopefully treating or preventing some of these disorders down the road.

Dr. Sharp: Absolutely. Could we maybe take like a, I don’t know if it’s a specific disorder or some real-life example of what this might look like in the research? I’m aware that I’m afraid of saying something too simple-minded here, but like we mentioned before we started recording this overlap between depression and anxiety, for example.

I know that a lot of folks that listen probably are child-focused, I don’t think everyone by any [00:34:00] means. I run into a lot of kids who seem to have this overlap of, it’s like high anxiety with irritability with maybe some explosiveness but they don’t seem to be bipolar but they’re in that disruptive mood territory but it doesn’t quite fit. There’s a lot of these symptoms that seem to co-occur, maybe there’s some ADHD but I’m not exactly sure what’s going on. It seemed this model would be really helpful in that but I’m trying to translate it to real life, like how you might approach that or how you might design some research to get underneath that.

Dr. Jenni: Sure. I’ll talk about one thing first, then I’ll come back to some of the work that I’m familiar with that’s happening in the developmental world. There’s a large [00:35:00] project, it’s called BSNIP which I believe stands for Bipolar and Schizophrenia Network for Intermediate Phenotypes but we call it BSNIP.

And so that’s been done looking at bipolar and schizophrenia but they have some really elegant work that looks into these biotypes. If you were to group people based on these disorders, there’s several different clusters of people that would be included in this group if you just looked at everyone with bipolar and schizophrenia.

And so what they were able to do looking at some genotyping, looking at some measures of cognitive control and two other things that are right in the RDoC matrix, they were actually able to, and I’m [00:36:00] sure I have the statistics wrong, but I’m going to call it cluster. They may not have been doing exactly a cluster analysis, but they were able to cluster these groups based on the different and converging information and that has been helpful to identify subpopulations within this broader class of schizophrenia, bipolar, and psychosis disorders and to really help them figure out what are the specific deficits that these patients are having and ultimately, how can we address them in perhaps a more precision medicine type of way?

There’s a figure that we use a lot of times in our talks about this that I think comes right out of the BEAST Network. I’m happy to share that as well so you can put that in the notes of this podcast [00:37:00] too.

Dr. Sharp: Great. Thank you.

Dr. Jenni: And in a similar vein, the work that’s in my portfolio, like I said, I oversee a portfolio of primarily ADHD work. There’s some irritability or other disruptive behavior focus work, but a lot of it is focused in ADHD.

A lot of that is trying to understand more specifically what are the deficits and I know classically there’s like an inattentive type and a hyperactive type of ADHD, but what does that mean and what are the systems there and are there signs that we can see that might indicate which children would go on to have issues with their ADHD as they get older or are there some that we would see that they would have a remittance of their symptoms and maybe not need as much intervention?

I know that there’s some [00:38:00] work that’s looking at some of these cognitive deficits and have been able to isolate comorbid disorders and identify that some of these deficits are indicative of the ADHD and not any of the comorbidities and that can be used as a sign point as to where they are in this spectrum or dimension with their ADHD and how it will be evidence as who might need to be treated sooner rather than later in terms of ADHD.

Dr. Sharp: Mm-hmm. So I’m thinking about how you put this research together. Do you take folks who maybe already have existing diagnoses and then, I’m not sure whether measure [00:39:00] across each of these different constructs and dimensions and see what shakes out compared to controls or are you going at it from the other side where you just take everybody and measure across constructs and then see what emerges in terms of differences or deficits or how do you structure the studies?

Dr. Jenni: There’s any number of ways to set up your study that would be considered RDoC in nature. I think the most important thing as with most research, is to have a really clear question that you’re trying to answer. A lot of the questions that are out there now lend themselves pretty easily to looking at things in a more RDoC-centric way so that we might look at something as opposed to comparing a control group [00:40:00] to a patient group, you might just look at your whole group and look at comparisons of this dimension of functioning and how does that relate to, say, a clinical outcome or a real-world outcome or …

So depending on what your question is, I think your variables will be pretty easy to select but there are certainly people who are trying to do this more less categorical, less group comparisons, and maybe something that’s more continuous in nature to try and see again, just with the way that people are normally classified into their different diagnoses, there’s probably a large portion of your control group that is probably subclinical, so they’re not passing a threshold to have a diagnosis or they maybe don’t have enough dysfunction for it to be really severely getting in the way of their daily activities but does that mean that [00:41:00] they’re completely without, are they completely asymptomatic or is there some range there?

I think for most of our mental health disorders, our control groups are people who don’t have a diagnosis. There’s going to be still a range within that group that we’re probably missing a lot of information by grouping them all together and saying that they have none of the issues that your patient group has.

Dr. Sharp: Yeah, that’s a great point. I think that’s something that we run into a lot from an assessment perspective is that clinical threshold. So kids or people who have who have symptoms but they’re maybe not exactly meeting criteria as they’re set forth right now. That’s hard. It’s like there’s something going on, but it’s, what do we call it because it doesn’t quite meet the criteria.

Dr. Jenni: Right. I [00:42:00] was listening to one of the past episodes where you were talking about testing in a university setting. I think they were mentioning that sometimes after doing the testing, you might not be able to come up with a diagnosis, but he was saying he was still keeping notes about what the functional deficits were so that just because I haven’t given this student a diagnosis doesn’t mean that everything is going to be completely easy for them and they can go on like normal, there may still be things we can do to help make some things easier.

As I was listening to that, I thought, oh, that really lends itself well to our RDoC focus where we’re looking at different systems and how they’re interacting or not interacting. And so you might be able to identify a deficit that doesn’t have a name and may not for [00:43:00] diagnostic or billing or anything else be able to be called anything, but it certainly doesn’t mean that they’re performing the way you would think someone would perform if they’re not having a difficulty.

Dr. Sharp: Right. I’m just thinking through, I have this question that feels vague but I’m going to throw it out there and trust that you can help sort through it a little bit. My question is, how does this all translate to real life and to practitioners where … it seems like a lot of the research that you’re doing or a lot of the underlying pieces that you’re looking at are physiological, molecular. Measuring those things seems to require a lot more than we have at our disposal in real-life practice, outside of a research setting.

So [00:44:00] how would we translate all of this, even if we know that there’s some type of ADHD that looks like this across these different dimensions, how we get at that in our practices. Do you have any thoughts on that?

Dr. Jenni: I have some. I myself am definitely trained as a research scientist, not a clinician and so I haven’t had to spend a lot of time thinking about it on that side. Of course, ultimately that is the goal of all of this research is to help patients. And so we do have to keep some of that in mind.

One of my longer-term answers is that the goal of RDoC is to get a better understanding of what’s happening so that ultimately, and again, this is a slow-moving process, but ultimately we might be able to help with modifying or [00:45:00] updating some of the diagnostic criteria that are happening in the clinic. So ultimately we might be able to get some tools into the clinic.

We’re certainly not there because research is slow moving and we need to make sure things are the way that they look before we can put them into the clinic. I think there’s a way to think about things on this more RDoC level when you’re in the clinic. I’ve recently come across a paper where they’ve done a case study of someone based on the clinic and they tried to look at it through an RDoC lens.

And so they did all of their testing and went through the normal procedure that they would do anyway and then they looked at each of the [00:46:00] domains of the RDoC matrix and said, okay, what are the issues that this patient is having in the negative valence system? What are the issues in positive valence? And they tried to go through there and pull out the things that that patient was having trouble with.

The end of this case study basically said they were able to almost more acutely or precision treat that patient because there were a few subconstructs that came out as oh, this is something that this person is having trouble with that might not have come out from just a diagnosis of say generalized anxiety.

And so it might not have been a focus of their treatment plan moving forward, but having looked at all of these psychological constructs, they could identify a few extras that, oh, we can work in treatment on this, or we can work this into our plan and ultimately have perhaps a better success at [00:47:00] the end because we looked at some of these different constructs that might not come out from a regular treatment plan.

Dr. Sharp: Yeah, absolutely. I would imagine that’s where we’re trying to go. Anything that we can do to increase our diagnostic precision and accuracy is the name of the game.

This whole thing is so fascinating to me because right now we have a lot of discussion, there’s a Facebook group that corresponds to the podcast and it’s about assessment clinicians and there’s a lot of discussion about how right now we don’t know that there’s a cognitive profile that defines ADHD, for example. We just have all this data at our disposal but it’s hard to know what it means.

Dr. Jenni: Yeah. [00:48:00] Two of the projects that I’ve seen that are funded through NIMH in the last few years are trying to make this association between what you referred to as the tests are things that are easier to do in a research setting and not so easy to do in a clinical setting. So I think one of the pieces here and how RDoC can really help in maybe a shorter term is by using say imaging or EEG or any of these things that we can do in a lab that are not so easy to do in the clinic, if we can understand what those are showing, we can start to figure out better, what are the tests that are easier to do in a clinic that would tell us what’s happening in this EEG?

So if we can understand what’s happening in the physiological systems, how do we now get a test that we can [00:49:00] administer in the clinic that might give us an idea of what’s happening underneath? And those are probably going to be really useful because you’re not going to roll an fMRI machine into your clinic and ask every patient to get in there so you can look in their brain but if we can understand how do results on certain tasks or paradigms that are more easy to administer relate to the systems underneath. I think that that’s probably more attainable goal of how we can get something that’ll be more informative.

Dr. Sharp: Yeah. I would love to see that. I would love it. I think a lot of people would. This is really cool. I love that this is happening. So that brings up like a, I don’t know what you call it, maybe a publicity question. I feel like I had to dig a little [00:50:00] bit to find any information about RDoC or even stumble across it. How do y’all approach that in terms of letting people know? Maybe I’m just living under a rock, but is it more well-known among researchers or practitioners?

Dr. Jenni: My guess is it’s probably more well-known for researchers mostly because as a funding agency, most of them feel like they have to do what we want so that they can get some money. So I think it started with the research and that’s our goal with it. I think more and more, as it’s becoming a little bit more commonplace in the research and in training researchers and clinicians, I think they’ll come across this more as they’re getting into it.

I think [00:51:00] we’re perhaps hitting the new generation of scientists and psychologists or psychiatrists as opposed to maybe from the clinic coming in and getting the people who’ve been there for a long time, we’re coming in through the bottom.

I think that, as we said from the beginning, it’s a big overhaul of how everything has been done and so it’s hard for us as the RDoC unit to always know how to market RDoC and how to do it because we can’t answer everything yet. We don’t have anything to hand over to clinicians right now to say, here’s how we can make your life easier. It’s a little tricky to how extensively do we really want to go after clinicians if we can’t quite help them yet. [00:52:00] We can help researchers try and find and answer questions that will ultimately get there.

As I said, we’re almost at 10 years of RDoC, I think the first 10 years was really trying to get this to take hold in the research world and now I think we have to start redirecting our focus because we have to always be looking ahead. We have to always be looking at where this is going or where the field is going and how can we help that.

And so I think now we have to start looking, someone asked recently how we know when RDoC is done and we can update diagnostic criteria. I don’t have the answers to that, but I think as a field now and us as the NIMH, we can start thinking about, okay, what would it take for us to be sure that we have something that we can pass into the [00:53:00] clinic or pass back to people who are in the trenches trying to help people who have disorders that they need help with.

Dr. Sharp: Right. So maybe we could start to close with a question around that, which is, at this point, is there anything that you feel like we can take from RDoC to incorporate in our day-to-day practice? Even if it’s just a mindset shift compared to a concrete, like a measure or a diagnostic criteria. Is there anything that we could take from it right now and use that in our day-to-day practice?

Dr. Jenni: Sure. So we actually just had on the past two days, we had a meeting here of training directors and some of their trainees. So some of them are more clinically focused and some are more research-focused. One of the things that someone said to me that struck me was that, as a clinician, you’re [00:54:00] taught and trained in these very almost rigid diagnostic boxes and you’re tested on them to be sure what’s happening, but that day to day in the clinic, people don’t always fit into those boxes and you’re not sitting there thinking in terms of those things as more as you’re probably are sitting there thinking more of these psychological constructs and what are the building blocks that this patient who’s in front of me is having issues with and where do we need to focus?

I think it sounds like a lot of the differences between RDoC research and what’s happening in the clinic is perhaps just like a bit of a language thought organization thing in that RDoC is probably a little bit more focused on neuroscience [00:55:00] than things in the clinic or how we’re testing people are but I think some of the methods are similar.

For me as part of the RDoC unit, I find it helpful to get some feedback of, if we called it social communication, what would someone else call it or this is the term and the circuit that we’re using but are we thinking of the same thing? We’re just calling it different things. How can we bridge this gap between what terms you’re using and what terms we’re using?

I think that trying to look at the smaller psychological constructs and how they relate to each other in each patient is probably the first step in trying to think RDoCwise in the clinic.

Dr. Sharp: Sure. [00:56:00] I really appreciate that you were willing to come by and chat about all of this. I feel like it opens up a whole other set of criteria for looking at people. I think a lot of us will probably say that this is long overdue.

Dr. Jenni: Yeah, I’m glad to do it. I think there’s often a lot of, I don’t want to always say it’s misconceptions, but again, depending on how you’re trained and how these things are taught initially, this is a big shift in how to think about it, but ultimately I think it makes a lot of sense to a lot of people on the research side, people on the clinic side that this is following a little bit more of how people are.

Of course, when we started doing a lot of this research and our tools weren’t where we are today, a lot of [00:57:00] the assumptions and the way that things were set up made a lot of sense, and I think that RDoC is just trying to help the field keep up with the tool development and how we can implement that to try to push forward with new treatments or new therapies that might be able to help more people.

Dr. Sharp: Sure. Well, I think that’s the way that things are going. Everything these days, rightfully so, is more, it’s really looking at biology and neuroscience and it just doesn’t make sense anymore to be doing the work we do without that scientific grounding that we just haven’t had for the past several years. I’m glad you’re doing it.

Dr. Jenni: Great. Thank you.

Dr. Sharp: I know that you have a little bit of an exciting [00:58:00] announcement, I guess. Y’all just published a funding announcement and I would love to have you talk about that if you would like to.

Dr. Jenni: Yes, we just had a funding announcement come out on Friday. I had made mention of a work group that two years ago focused on tasks and measures and how we could really measure each of the constructs in the matrix.

One of the big conclusions of that group was that there are a lot these tests that are being used or tasks in research, and we haven’t done a lot of the groundwork that’s needed, so we don’t always know what the psychometric properties of each of those tasks are. We aren’t always sure that we know that it’s measuring what we think it’s measuring. A lot of that work, of course, it’s not pretty. It’s not going to get you a giant publication [00:59:00] but it’s needed.

And so we finally have a funding announcement out that’s asking people to focus exactly on that. So it’s for new task development or task optimization to try and make sure that we have tasks that are here and able to be used, that are working, that are measuring what we want to measure in a way that we understand and that we can trust the data that comes out of them.

I know you said you could post the link to the funding announcement but we’re excited to see some of the work that can come in because I think moving forward will be helpful for the field to have a bunch of tasks at their disposal that they can be sure they know what data they’re collecting from them.

Dr. Sharp: Yeah, absolutely. I’m happy to do that. So that will be in the show notes too for any of you out there who are researchers [01:00:00] and might want to participate there. I know that we have a fair number of university folks out there and faculty members who are involved in research programs so happy to go that link out there as well.

I wonder if we could close just with if people have questions, what’s the best way to get in touch with you or is there someone else to get in touch with? If they just want to get more information about RDoC, where is the best place to do that?

Dr. Jenni: Sure. I would be happy to take any emails. We do have a website on the NIMH website that’s focused solely on RDoC. We also have an RDoC Twitter feed. Our Twitter handle is @NIMH_RDoC. And so you can find us on Twitter.

We do have on our website email address listed too that’s [01:01:00] just goes to a general inbox so that a bunch of us could check it if no one else is answering email. So that’s rdocadmin@nih.gov. We’re always welcome to get any feedback or hear any questions or if anyone is having trouble getting started, we’re happy to… that’s what we’re here for. We’re here to support the researchers and anyone else in the field who’s trying to move things forward.

Dr. Sharp: Fantastic. Thank you so much for coming on and talking with me. This is super enjoyable and it’s piqued a lot of interest for me. I’m going to look at that matrix a little more and see what’s going on there. I’ll be looking forward to hearing more about RDoC in the coming years.

Dr. Jenni: Great. Well, thanks for having me.

Dr. Sharp: Okay, y’all. Thank you so much for listening to this episode as always. I haven’t given a shout out [01:02:00] for subscribe rate and review in a while. If you haven’t done that, that would be awesome. That’s how the podcast gets bumped up the charts in all of those podcasting places. Take 20 or 30 seconds and subscribe. You won’t miss any episodes, and rate and review if you are feeling extra generous, I’ll always appreciate that.

If you have any thoughts about how to build your testing practice, if you need any coaching, any advice, I would love to help you with that. That’s what I do. You can explore that option at thetestingpsychologist.com/consulting. We can schedule a 20-minute complimentary pre-consulting call just to see if that is appropriate for you. If it is, that’s awesome. I’d love to work with you. If not, I’ll try to hook you up with any other resources that might be more helpful. So that’s thetestingpsychologist.com/consulting.

All right, [01:03:00] y’all, take care. Got some great interviews coming up on the horizon so stay tuned, subscribe and we’ll see you next time. Bye.

Click here to listen instead!