Many of y’all know that I have been using TherapyNotes as our practice EHR for over 10 years now. I’ve looked at others, and I just keep coming back to TherapyNotes because they do it all. If you’re interested in an EHR for your practice, you can get two free months of TherapyNotes by going to thetestingpsychologist.com/therapynotes and enter the code “testing.”
Thanks to PAR for supporting our podcast. The BRIEF2A is now available to assess executive functioning in adult clients. It features updated norms, new forms, and new reports. We’ve been using it in our practice and really like it. Learn more at parinc.com/products/brief2a.
[00:01:00] Hello, testing psychologist audience. Welcome back.Today, I’m talking about the idea of the ADI-R and the ADOS-2 as the “gold standard” in autism assessment. To do so, I am reviewing an article by Somer Bishop and Catherine Lord from 2023.
If you’ve ever wondered how tools like the ADOS-2 and the ADI-R fit into the bigger picture of autism evaluations and whether they are truly considered the gold standard, this episode is for you. We’ll be discussing their intended use, their limitations, and best practices based on this 2023 article. So, if you’ve used the ADOS, used the ADI-R, run into issues with insurance requiring one of these instruments, this is definitely the episode for you.
Before we get started, let’s talk about support for your practice.
At this point, mastermind groups are full; the coaching [00:02:00] experiences that I run twice a year, but you can still get a strategy session. To do so, you can go to the website, thetestingpsychologist.com/consulting, and you can book a strategy session right there. This is an a la carte hour that we can dive into any questions that you might have about your practice. We’ll troubleshoot. We will brainstorm and hopefully send you away with some helpful ideas to move forward.
All right, without further ado, let’s talk about the ADI-R and the ADOS-2 as the gold standard of autism assessment.
Okay, everybody, we are back. Again, we’re talking about the ADI-R and the ADOS-2, how they came to be known as the gold standard, and whether that is an accurate description.
Before we dive into the nuances of how these [00:03:00] tools should and shouldn’t be used, let’s take a quick step back and discuss what standardized diagnostic instruments actually are. I think we all have a good idea of this, but I want to set some groundwork here before we totally dive in.
Standardized diagnostic instruments are structured tools designed to aid in the assessment of autism.
Now, when the ADOS came onto the scene, I think it was a little bit of a revolution because it tried to standardize and structure the assessment of reciprocal communication, stereotype mannerisms, repetitive behaviors, and other aspects of autism that, to that point, were captured with checklists and behavior questionnaires. And so this was, I think, the first large-scale attempt to operationalize some of these behaviors, codify these behaviors in a scoring rubric, and [00:04:00] standardize the observation of these behaviors. So huge deal. This is back, I forget if it was late 90s or early 2000s, but when the ADOS-2 burst on the scene, this was a bit of a revolution.
But right now, I think the most well-known examples of these standardized instruments, like I said, are the ADI-R and the ADOS-2; now in its 2nd edition, has been for probably 15 years now. There are some others that are on the market, of course. I’m not going to count the MIGDAS; it’s not standardized necessarily, and it’s more of an extended interview, and other things like TELE-ASD-PEDS, those are out there for virtual assessment. We’re going to stick with the ADI-R and ADOS because this is the framework that the article adopted.
The ADI-R, if you don’t know, is a structured interview conducted with parents or caregivers assessing the developmental history and behavior [00:05:00] patterns of a child. It is relatively long and pretty nuanced to score. There are some very detailed questions that dig into all the different aspects of potential autism spectrum behaviors.
Then we have the ADOS, or Autism Diagnostic Observation Schedule. A direct observation tool assessing social communication, restricted interests, and repetitive behaviors in structured play and interaction settings.
Again, why were these developed?
They were created to try and bring some consistency and reliability to the autism diagnosis. They provide a structured method to gather information across different settings and clinicians. If you’ve taken the research training for the ADOS, you know that a big goal is to get your scoring to the point that it’s research-reliable, where you could match with other clinicians. This is one of the big problems with the ADOS: the subjectivity on [00:06:00] different ratings and how different clinicians can rate very differently.
These were initially intended for research, but they have been widely adopted in clinical settings. In those settings, clinicians typically administer the ADOS with the child and the ADI-R with the caregivers to try to systematically assess ASD traits. And then the results, of course, help to provide some structure to clinical evaluation. They’re just meant to be one component of a comprehensive assessment, but along the way, that changed a little bit over the last 20 years or so.
Looking back at the article written by Somer Bishop and Catherine Lord, Catherine Lord was a developer of the ADOS, one of the key messages from the article is that these tools were never meant to replace clinical judgment.
What was their intended role?
They were intended to help organize [00:07:00] data without necessarily dictating diagnosis. So, clinicians should still use their expertise to interpret the results within the context of developmental history, real-world behavior, environmental factors, ecological factors, and so forth. These instruments were designed to try to reduce some of the subjective bias in the diagnostic process and organize these observations and caregiver-reported concerns.
As I said earlier, it’s also meant to increase reliability in research settings to improve the research process and characteristics of autism, diagnosis of autism, and so forth, and then by virtue of all of that, strengthening the empirical understanding of ASD symptom presentation.
Now, I’m not going to go down the rabbit hole of the limitations of ASD research. I think we [00:08:00] are mostly familiar with the idea that autism research has heavily favored boys and males and that we’re working hard to catch up on research on autism with girls and women. So I’m not going to go down that rabbit hole, but again, the original idea with the ADI-R and the ADOS was to increase reliability and conduct better research. Whether you agree with that or not, as the outcome, that is another conversation.
Let’s talk about the limitations and misapplications of these instruments.
We have an idea of what they were intended for, but like I said, along the way, over the past, probably, I don’t know, 15, 20 years, I can’t remember when the term gold standard first started to be applied to these instruments, but that did start to happen, and I think what came of that is an over reliance on these instruments in [00:09:00] diagnosing autism.
What are some of the pitfalls of relying too heavily on these tools?
Some clinicians treat the ADOS or the ADI-R scores as definitive rather than being part of a holistic evaluation, which, in either direction, risks the increase of false negatives or false positives if the other clinical context and clinical judgment is ignored. There is also a lack of validation for certain populations.
In the article, Somer Bishop and Catherine Lord talk about how the ADOS is not well validated for individuals with severe motor or sensory impairments, those with challenges in verbal expression. So, assessing minimally verbal individuals or those with complex co-occurring conditions can also be pretty tough.
During COVID-19, we ran [00:10:00] into the issue of using personal protective equipment, or PPE, during ADOS administration. That certainly did not do us any favors. Back then, I think Catherine Lord came out and said that any ADOS results obtained while using PPE were not valid.
And then another factor, of course, is that compliance varies. Kids are going to be at differing levels of interest in the ADOS. And so, when you have behavioral compliance varying, that’s also, of course, going to affect kids’ performance on measure.
Let’s take a break to hear from a featured partner.
Y’all know that I love TherapyNotes, but I am not the only one. They have a 4.9 out of 5 star rating on trustpilot.com and Google, which makes them the number one rated Electronic Health Record system available for mental health folks today. They make billing, [00:11:00] scheduling, notetaking, and telehealth all incredibly easy. They also offer custom forms that you can send through the portal. For all the prescribers out there, TherapyNotes is proudly offering ePrescribe as well. And maybe the most important thing for me is that they have live telephone support seven days a week so you can actually talk to a real person in a timely manner.
If you’re trying to switch from another EHR, the transition is incredibly easy. They’ll import your demographic data free of charge so you can get going right away. So, if you’re curious or you want to switch or you need a new EHR, try TherapyNotes for two months, absolutely free. You can go to thetestingpsychologist.com/therapynotes and enter the code “testing”. Again, totally free, no strings attached, check it out and see why everyone is switching to TherapyNotes.
Our friends at PAR have released the latest addition to the BRIEF2 family. The BRIEF2A lets you [00:12:00] assess your adult clients with the gold standard and executive function. It was developed with a three-factor model from the BRIEF2, which characterizes executive functioning deficits more clearly. It also offers updated norms, new forms, and new reports. Learn more at parinc.com/products/brief2a.
All right, let’s get back to the podcast.
There are also some ethical and equity considerations here in overrelying on the ADOS. One of those is that mandating these tools can limit access to diagnosis. The article does talk about how some of the insurance panels are mandating the use of these instruments and the authors argue against that. It can limit access to diagnosis.
Financial and logistical barriers can prevent a lot of families from accessing [00:13:00] these instruments as well.
Let’s talk a little bit more about the implications of mandating specific tools like the ADI-R and ADOS.
A major issue that was raised in the article is this problem with requiring specific instruments for diagnosis. So there are some consequences there, right? Some insurance panels, again, require the ADOS for reimbursement, even when it might not be necessary or feasible for a clinician to administer the ADOS. This puts clinicians in a bad place because there are many clinicians who either lack access to training or materials, or, for any number of other reasons, just cannot properly administer these tools. And then they’re backed into a corner and can’t get reimbursed for the work.
There’s another issue with equity and access to assessment, like I mentioned earlier. Relying rigidly on a few tools can delay diagnosis for [00:14:00] a lot of individuals who don’t fit “standard profiles”. Again, we need a flexible assessment approach that can accommodate these diverse presentations of autism.
There is a policy consideration here. So we’re starting to trend into what to do about it. I think that, as a field, the article suggests that we advocate for policies that recognize clinical expertise in autism assessment and not over-rely on these instruments. They encourage a more inclusive approach to diagnosis beyond just using the ADOS and the ADI-R.
What else can we do as clinicians? How do we use these tools effectively without falling into these traps?
There are several best practices in this article. One is training competency in autism assessment. If you’re going to do the ADOS and the ADI-R, they emphasize [00:15:00] that formal training is crucial. There are a lot of trainings for both of these at this point. I know of, gosh, it seems like there’s at least 4, 5, 6, maybe even 8 to 10 trainings that pop up over the course of the year. There are options for virtual and in-person. So, you can likely find training on the ADI-R and the ADOS. It’s going to cost you, I don’t know, $500, $600, maybe $700, depending on where you go for ADOS-2 training, but those options are out there. And then, of course, ongoing professional development helps keep you sharp.
My favorite option for this is the GAIN program through Cornell. They do a quarterly focus on a different ADOS module. It’s a pretty deep dive into administration for each of these modules with video, real clients, and working with experts to score the ADOS together. [00:16:00] So if you haven’t heard of the GAIN program, definitely go check that out for ongoing professional development within the ADOS.
Now, what else do they recommend?
Integration of multiple data sources. Combining the standardized testing results with parent reports, teacher observations, and clinical impressions. You always have to consider cultural and linguistic diversity in assessment. I did a podcast a long time ago with Bryn Harris from Denver, and she talked about culturally and linguistically appropriate ADOS or autism assessment, so you can go check that out. She has continued to do research in this area, so you can look up her work as well.
This makes intuitive sense, but I know that a lot of us either work in settings or maybe get into the practice or habit, I’m not sure what to call it, of it’s easy to default back to the ADOS or the ADI-R and take those as gospel. [00:17:00] What we’re hearing from this article written by two individuals highly involved in these instruments and their development, was, you got to combine those results with many other aspects and sources of information in the assessment process.
They also talk about contextualizing standardized scores. Interpreting the results within the broader context of adaptive functioning, for example. Recognizing that autism traits might present differently across settings.
Finally, they talk about avoiding over-pathologizing or under-diagnosing. You don’t have to rigidly adhere to the cutoff scores in either of these instruments. You can consider the full clinical picture and be mindful of subtle social communication differences that may not register on formal assessments. I think this is their nod to the idea that some individuals who require less support may fly under the radar a little bit and [00:18:00] not reach the cutoff score on either of these measures.
Let’s wrap up with a little bit of a recap, I suppose.
What they’re saying essentially is standardized tools like the ADOS and the ADI-R are valuable, but they should not replace clinical judgment in the assessment of autism. These instruments have limitations, particularly for certain populations. Best practices involve integrating multiple sources of information for accurate assessment. They also argue against the idea of policies requiring these specific tools because they create barriers to diagnosis and should be evaluated. We can do some advocacy there to argue against the requirement of these tools.
This is an interesting article coming from individuals who are highly involved with these instruments. They’re essentially saying, Hey, please stop saying this is the gold standard and [00:19:00] overrelying on our instruments because the assessment process is much broader than that and requires us to think critically as clinicians and integrate many different data sources when we’re doing an autism assessment.
Essentially, assessment is as much an art as a science, and we have to balance the use of these structured tools with our clinical experience and do a holistic assessment process.
Now, another conversation that we could have and may have one day is then what do we do? This is where I get to a lot with these conversations. If we can’t totally rely on these standardized instruments, then what are we left with? Especially with a relatively subjective and moving target presentation like autism, where the diagnostic criteria has maybe not changed that much over the years, but the clinical [00:20:00] presentation and expansion of that diagnostic umbrella is certainly a factor where it’s getting wider and wider, and we have to stay on our toes in terms of what is considered what is considered to qualify as autistic right now.
That’s another conversation, another podcast, and like I said, this is getting to be a bit of an existential dilemma, I think, in the assessment world, where many of us like to rely on standardized instruments, and there is a place, and the authors are saying, let’s expand our repertoire and make sure not to rely too heavily on those instruments. So, an ongoing conversation. My guess is that many of you have wrestled with this and will continue to wrestle with it, but this is just a piece of the puzzle that you can incorporate into your assessment.
[00:21:00] All right, y’all. Thank you so much for tuning into this episode. Always grateful to have you here. I hope that you take away some information that you can implement in your practice and your life. Any resources that we mentioned during the episode will be listed in the show notes, so make sure to check those out.If you like what you hear on the podcast, I would be so grateful if you left a review on iTunes, Spotify, or wherever you listen to your podcasts.
If you’re a practice owner or an aspiring practice owner, I’d invite you to check out The Testing Psychologist mastermind groups. I have mastermind groups at every stage of practice development: beginner, intermediate, and advanced. We have homework, we have accountability, we have support, and we have resources. These groups are amazing. We do a lot of work and a lot of connecting. If that sounds interesting to you, you can check out the details at thetestingpsychologist.com/consulting. You can sign up for a pre-group phone call, and [00:22:00] we will chat and figure out if a group could be a good fit for you.
Thanks so much.
The information contained in this podcast and on The Testing Psychologist website is intended for informational and educational purposes only. Nothing in this podcast or on the website is intended to be a substitute for professional psychological, psychiatric, or medical advice, diagnosis, or treatment.
Please note that no doctor-patient relationship is formed here, and similarly, no supervisory or consultative relationship is formed between the host or guests of this podcast and listeners of this podcast. If you need the qualified advice of [00:23:00] any mental health practitioner or medical provider, please seek one in your area. Similarly, if you need supervision on clinical matters, please find a supervisor with expertise that fits your needs.
Click here to listen instead!