[00:00:00] Dr. Sharp: Hello, everyone. Welcome to The Testing Psychologist podcast. I’m your host, Dr. Jeremy Sharp, licensed psychologist, group practice owner, and private practice coach.
Many of y’all know that I have been using TherapyNotes as our practice EHR for over 10 years now. I’ve looked at others, and I just keep coming back to TherapyNotes because they do it all. If you’re interested in an EHR for your practice, you can get two free months of TherapyNotes by going to thetestingpsychologist.com/therapynotes and enter the code “testing.”
This podcast is brought to you in part by PAR.
The NEO Inventories Normative Update is now available with a new normative sample that is more representative of the current U.S. population. Visit parinc.com/neo.
Hey, folks. Welcome back to The Testing Psychologist podcast. Glad to be here with you as always. I have a fantastic clinical [00:01:00] episode with a return guest who you all likely know and love. Dr. Stephanie Nelson is a pediatric neuropsychologist who specializes in complex differential diagnosis. She is board certified in both clinical neuropsychology and pediatric neuropsychology, and she is currently the president of the American Board of Pediatric Neuropsychology.
She has a private practice in Seattle, Washington. She also has a consultation practice called The Peer Consult, through which she provides consultation to psychologists and neuropsychologists who specialize in pediatric assessment. If you listen to today’s episode and you want to reach out and talk to Stephanie more, you can find her at thepeerconsult.com.
Today, we are talking about many things. The title of the episode is Beyond Diagnosis. What do we mean by this? That’s a little bit of too brief of a description because it’s not just beyond diagnosis, but I would say [01:02:00] based on the content, this should be titled Beyond Diagnosis, Numbers, and Accommodations.
So Stephanie talks about the ongoing evolution from Reports 1.0 to Reports 2.0, where we are not just focused on diagnosis and the testing data in our tried and true recommendations, but we’re trying to make a shift toward a more qualitative, immersive, holistic experience of the evaluation and learning more the nuances and more of a depth-based approach to assessment than what we’re used to.
So we talk about shifting toward what Stephanie calls the four E’s: experiences, expectations, exceptions, and experiments. As usual, Stephanie brings a fantastic style to this podcast where she punctuates with stories, quotes, and [00:03:00] examples from the literature, and there’s a ton to take away from this. You’ll hear on the interview that I talk about how there are many points that Stephanie brings up that I feel like we could implement this afternoon or tomorrow in our practices to improve the way that we are conducting our assessments.
Before we get to the conversation with Stephanie, I think when this is released, there still might be time to register for one of The Testing Psychologist’s mastermind groups. These are group coaching experiences aimed at helping you build and grow your private practice. So if you would like some support there and some accountability to get some things done that you have had some trouble with, check it out. You can go to thetestingpsychologist.com/consulting and schedule a call to chat with me about whether it might be a good fit or not.
But in the meantime, let’s get to this conversation with Dr. Stephanie Nelson about going beyond diagnosis, [00:04:00] numbers, and accommodations.
Stephanie, hey, welcome back to the podcast.
Dr. Stephanie: Thanks for having me.
Dr. Sharp: I’m glad to have you. It’s been a little while, but you are a celebrity podcast guest, and here we are again.
Dr. Stephanie: Fantastic.
Dr. Sharp: I’m excited to chat with you. As always, we have a brand new topic today to talk about that is going to be super relevant and interesting for folks. But I’ll start with the question that I’ll always start with, which is, especially for you, out of all the things that you could be spending your time and energy on, why are we talking about this today? Why is this important?
Dr. Stephanie: I would like to start by saying that this podcast is [00:05:00] entirely your fault. I was talking with you and some clinicians recently, and you threw off a question right at the very end where you pointed out that our reports now, we’ve moved from report revision 1.0, where people have made their reports shorter, more streamlined, more concise, getting to the point.
And then you said, so what do reports 2.0 look like? What’s the next revolution there? And I thought that was such an amazing question, and I stumbled off an answer about my hope for reports that focus less on what I sometimes call the DNA; the diagnosis, the numbers, and the accommodation.
I think I fumbled through some sort of answer about how I hope to focus more on personalized explanations, real strengths about the child or the young person, and a [00:06:00] more expansive view of the client’s capacities, but I’m not sure I articulated it very well because the other clinicians that we were sitting with gave me a little bit of a look as if I’d suggested replacing reports with Hipster wheels full of radishes or something like that.
And I thought, based on their expression, which said something like, Oh, that’s really interesting, Stephanie, but what on earth are you talking about? It would be fun to come on your podcast and see if I could articulate that a little bit. So today is my impassioned plea for reports that might focus a little bit less on the diagnosis, the numbers, and the accommodations, and a little bit more on the other things that we have to offer.
Dr. Sharp: This sounds great, and if this is my fault, then I will take responsibility for that because it’s important to chat about.
[00:07:00] A lot of this is born from my experience, I think we’ll touch on some of these things, but my experience of writing reports that are “strengths-based” but not knowing exactly what that means, and feeling like it’s a little bit empty, and then getting the feedback sessions and telling people many things that they probably already know or maybe are already doing. I feel like we could do better. I feel like there’s something more.
Dr. Stephanie: Exactly. I think most experienced clinicians get to that point where you feel like there must be something more than this, but the path towards it is a little bit unclear.
Dr. Sharp: Hopefully, we’re going to illuminate that path a bit here today.
Dr. Stephanie: We’ll see what we could do about that. I wondered if maybe I could start with a story. Would that be all right?
Dr. Sharp: Of course.
Dr. Stephanie: So I want to tell you a story about the late 1940s and the [00:08:00] U.S. Air Force. I got this story from the Todd Rose book, The End of Average. I don’t know if people may have read that. It’s a very quick read if anyone wants to.
He opens with this great story, which is that the U.S. Air Force has a problem in the 1940s, which is their pilots cannot keep the planes in the sky. They keep crashing. On the worst day, 17 pilots crashed all in the same day.
They were having a problem with this, and they couldn’t figure it out because it wasn’t the planes that were malfunctioning. So they kept saying, oh gosh, it must be pilot error. The pilots very strongly disagreed with that and thought, no, it’s definitely not pilot error, but no one could figure out what the problem was.
Eventually, attention turned to the design of the cockpit. The way the cockpit had been designed in 1926 was that they took 100 Air Force pilots, all men, of course. The idea of a [00:09:00] woman or non-binary pilot did not exist at that time.
They took 100 men, they measured them all in a million different dimensions. They averaged them and they made the cockpit for the standard pilot. They were feeling like this wasn’t working. So they were wondering, gosh, have pilots gotten bigger in the last 20 years? What has happened that the cockpit doesn’t seem to be working for these pilots?
So in 1950, at one particular base, they decided to get a huge sample of 4,063 Air Force pilots. They measured them in 140 different dimensions to see how many of them were average? How many of them could fit into the cockpit design?
They picked the 10 most important dimensions, and they decided to see how many of these pilots are [00:10:00] average on these 10 dimensions? What percentage of their pilots do you think they found were average?
Dr. Sharp: I don’t know. 17%
Dr. Stephanie: They were expecting it to be quite a bit. Their most pessimistic member of their testing brigade thought that it would be maybe about 30% of pilots, and it turned out to be 0. They didn’t find a single pilot who fit the standard cockpit.
In fact, when they reduced it down to only three dimensions to see how many of their pilots were average in all three dimensions, the number was 3.5% of their pilots fit into the standard cockpit. By designing a cockpit for the average pilot, they’d actually designed something for no one.
Dr. Sharp: Okay. [00:11:00] How do we connect this to what we do as psychologists?
Dr. Stephanie: So I started thinking, when I read this story about what I would do based on my training? What I would do based on my training is pretty simple. I would bring in all of the pilots that crashed. I would get a lot of numbers and data about all of them.
I would pretty much ignore all of the times that they had managed to stay in the sky and only focus on the 0.1% of times when they crashed. Really, wouldn’t be paying attention to, wait, why isn’t this problem worse? How on earth are you managing to stay in the sky so often? Instead, I’d label every time the pilot wasn’t meeting the cockpit’s expectations as a diagnosis.
Then, to be kind, I would probably label every time the pilot’s arms were long enough to meet the [00:12:00] controls as a strength, even if that didn’t help the pilot fly, or even if it wasn’t really something that he had any control over. And then I would probably give a list of accommodations. Honestly, probably not paying that much attention to whether or not they were realistic, feasible, or helpful in that situation. Maybe I’d recommend extra time for this pilot to reach the controllers, and then I would write that all in a fancy report.
I think that’s the kind of report that you have figured out feels empty. That focus on DNA; that focus on the diagnostics, the numbers, and the accommodations, and not really on a more expansive view of how that pilot fits into the cockpit.
So what if instead, we had brought [00:13:00] in the pilots and said, what is this experience of flying in this cockpit like for you? Where are your pain points? Where are your successes? What is your expectation of flying in this cockpit? How does it compare with other planes you’ve been in, with your history, with what you thought you were going to get? What’s the experience like? What are your expectations like?
And then what are the exceptions? When are you able to make it work somehow? Why isn’t this problem worse? What other planes have you been able to fly that worked out fine? And then asked about maybe doing some experiments. What do you think or feel that you might need? Can we try that? Does it work?
Or hey, the last 10 guys all said this works for them. [00:14:00] Are you open to trying that? Let’s see if it works. Let’s do an experiment right now to see if this accommodation is useful. So what I want to talk about is assessment that focuses on experiences, expectations, exceptions, and experiments.
Dr. Sharp: Fantastic. I’m excited. This sounds great. That’s a great story. I had no idea where you were going, and then all of a sudden it became clear. Yes.
Dr. Stephanie: Why are we talking about the Air Force? Well, I’m getting around to it eventually.
Dr. Sharp: No, I’m right with you. You always have these cute little shortcuts and things, this DNA approach; diagnosis, numbers and accommodations. The story illustrated some of the limitations there, but are there any others to talk about, just to put a fine [00:15:00] point on it?
Dr. Stephanie: Absolutely. I do want to emphasize that the diagnosis, the numbers, and the accommodations are going to be important. I’m not suggesting you write a report that never includes a diagnosis that’s appropriate, or that doesn’t have any numbers in it, or that doesn’t make accommodations. I want to make sure that I’m clear about that.
I want to talk about ways to reduce the emphasis on those things because they do come with some concerns. They do come with some problems and they are leaving us feeling a little bit empty. I want to talk first about some of the limitations of diagnosis and numbers, and then maybe save some of the limitations of accommodations for later, because I think that’s maybe my most controversial point.
Let’s talk about some of the limitations of diagnosis. I think you were hinting at some of those, which you were talking about your experience with evaluations. I think [00:16:00] everyone who’s been doing evaluations for a while figures out pretty quickly that with at least some clients, the diagnosis is not a main issue.
For example, you are asked to evaluate a child who is in the middle of a crisis. The diagnosis is clearly not important, given that we need to put out this fire that’s right in front of us. Or you’re asked to give a diagnosis of a learning disability on a 3-year-old or a 4-year-old, and you’re like, that’s not possible.
Or they’re already getting all of the services and doing everything that you would in the case of that diagnosis, and so a diagnosis won’t change anything. Or you’re being asked to give a diagnosis to a child, and you can tell that it’s really a family systems issue. The wrong person is in your office, or [00:17:00] there’s a systemic issue, a person who’s facing marginalization or some other type of systemic concern that makes a diagnosis for the individual inappropriate.
Or a diagnosis that’s statistically not very likely, a young adult who thinks that they have a diagnosis that’s usually diagnosed 20 years earlier, and often in a more extreme presentation. I sometimes refer to this as the tornado problem or the tornado presentation.
And by that, I mean not all bad weather is a tornado. You can have really bad weather, and it can still be something that is distressing, that’s worthy of our attention, that’s worthy of sympathy, and it is worth trying to understand, predict, or plan [00:18:00] around. And still it isn’t necessarily a tornado.
In the same way, you can have symptoms that are distressing, worthy of attention, understanding, sympathy, and planning that still doesn’t quite meet criteria for a diagnosis. It can sometimes feel uncomfortable when you want to tell the person that the diagnosis is probably going to be unlikely, but you’re not sure what else you would have to offer. Have you had any of those experiences?
Dr. Sharp: Oh, sure. Yeah. I’ve had all those experiences at different points, and a lot of the last experience that you described here over the last 2 to 3 years. I would add another circumstance there, and I’m curious if you would lump this in with this discussion, but I’ve done many feedback sessions just over the [00:19:00] last 10 days or so, for varying reasons.
In a lot of those feedback sessions, I’m finding myself saying the same thing over and over, which is, this doesn’t fit neatly into a diagnostic box, so let’s muddle our way. I’ll try to describe it in a way that captures something meaningful. So there’s, I don’t know what you call that, the inaccuracy or imprecision of our diagnostic system that doesn’t, in my mind, do a great job capturing different presentations.
Dr. Stephanie: Absolutely. That is what I sometimes call the sunset problem. I was going to transition to talking about that, which is, that you’re pointing out that the diagnoses that we have available to us are heterogeneous in their presentation. They have low reliability, low [00:20:00] validity.
We all know this if we’ve been in the field for a while, but the DSM is a grab bag of all different types of diagnoses. Allen Frances, who is the chair of the DSM-IV Task Force, points out how some of our diagnoses are lifelong, while others are transient. Some focus on emotions, some are on behavior, or social relationships. Some are things you develop early. Some are things you only develop very late in life. Some are about inner misery; some are about bad behavior.
They’re this huge grab bag of things that we can notice about a person, and because of that, they don’t have a lot of ability to stand up to scientific scrutiny. We don’t have very good inter-rate route agreement. There’s a lot of different ways a person can meet a [00:21:00] diagnosis. Two people can meet the diagnosis and even have identical symptoms, and still have different causes for why they’re experiencing that diagnosis.
Co-occurrence is more the rule than the exception. Trying to figure out where one ends and the other begins is really difficult. What I’m speaking about here is that diagnoses are nebulous and hard to distinguish.
The author of that book that I got the story from at the beginning, Todd Rose, talks about how diagnoses are a problem because they’re one dimensional. They focus only on one aspect of the person or even one dimension of a person, but don’t really embrace their dynamism and complexity.
They also are essentialists. They assume that that same person is going to show up the same way in [00:22:00] every situation. That if you’re an introvert, you’re going to be an introvert in every single situation. If you have social anxiety, you’re going to be anxious in every social situation that you’re in, when that may not be the case.
Even when we frame them as strengths, even if we say your diagnosis is actually your strongest point, or a superpower or something like that, they still embrace a normativeness. They still assume there’s a normal way to be, a normal pathway to take, a normal way that skills are developed, or that personality unfolds.
Any of us who have worked with patients for a long time realize none of these things really hold. We are dynamic, contextual, and idiosyncratic. I think it makes about as much sense to talk about a normal person as it does to talk about a normal [00:23:00] sunset.
I’m stealing the sunset idea from Carl Rogers. Some of the young folks listening might not know who Carl Rogers is, which is a little bit horrifying, but he is a very famous psychologist. Go read some of his work. Look, can I read you a quote from him?
Dr. Sharp: Sure.
Dr. Stephanie: So he says, “People are just as wonderful as sunsets if we let them be. Perhaps the reason we can truly appreciate a sunset is that we cannot control it. When I look at a sunset as I did the other evening, I don’t find myself saying, Oh, soften the orange a little on the right-hand corner. Put a little bit more purple on the base, and add a little bit more pink to the cloud color. I don’t do that. I don’t try to control the sunset. I watch it with awe as it unfolds, and I like myself best when I can appreciate people in that same [00:24:00] way.”
Dr. Sharp: That’s beautiful.
Dr. Stephanie: Isn’t it beautiful?
Dr. Sharp: It is.
Dr. Stephanie: And I think about diagnosis sometimes, I can feel as if I’m trying to diagnose a sunset. I feel as if I can be saying, Ooh, you see that subtle shift from saffron to burnt orange right there, what do I call that? Where am I deciding that one color ends and the other begins? How much of the sunset is due to the sun versus the cloud cover versus the atmospheric conditions versus the landscape?
If I went tomorrow, even to the same place at the same time, would the sunset look the same? It wouldn’t
because we’re nebulous. We’re dynamic. We’re contextual. Even if you can narrow your focus down onto a specific [00:25:00] feature that you want to look at. You and I can both look at this specific spot in the sky, 30 degrees up, and focus on it, but we still run into the limits of labels, and we risk getting distracted by definition and disagreement.
I feel like you and I might get into an argument about blue versus green. When we live in this world of turquoises and aquamarines, I feel like we’re trying to diagnose sunsets a lot of the time.
Dr. Sharp: That’s such a good example. That resonates, and especially here recently. Maybe it’s just timely, coincidence or something, but lots of these cases where I just feel like it’s trying to paint individuals or kids into specific boxes that just don’t fit.
Dr. Stephanie: And it might be [00:26:00] that the world is getting more complex or the children that we’re seeing are more complex. It might also be that we’re becoming more complex as clinicians, as you see more and more individuals and start thinking more about their problems and start hitting the limits of our testing, for example.
What can happen though, I think and you can let me know if this is happening for you. It’s certainly happening to a lot of the people that I speak with in my consultation world is that when we’re trying to argue over blue versus green in a world that’s full of turquoises and aquamarines, it narrows our vision and it stresses us out. We start doing what I call fear-based decision making.
We start worrying that we’re going to get the diagnosis wrong, and that can take over our entire evaluation. Either we start worrying that we [00:27:00] don’t know enough or we feel that we have some imposter syndrome. Isn’t there some other continuing education class I could take, a weekend seminar, a test I could learn, a book I could read, a person I could consult?
Or we start over-relying on our own judgment. We start feeling contempt for our colleagues who never see the blue when we see it everywhere and are constantly calling things green when we’re sure that it’s never green. Or we get into this place where we can’t even make a decision. You do 12 hours of testing, you spend days with a child or a young person, and you still can’t make a decision, or you’ve made your decision in the first minute, and nothing could dissuade you from that.
That way of talking about what fear-based decision making does to us reminds me of a quote from the poet, James Richardson, who says there are two kinds of people in the [00:28:00] world, and who among us is not both. And I often feel that way. I feel like I’m both of these people. I have impostor syndrome because I’m worried that I won’t get the diagnosis right, but also a little bit of judgment about other people who are seeing the client and saying that they see the color green when I see the color blue.
Dr. Sharp: Yes. Two sides of the same coin.
Dr. Stephanie: Exactly. I know once that worrying starts, it just takes over. We’re concerned that we’re going to miss it or get it wrong, and we start chasing diagnoses, or chasing rare possibilities, or trying to confirm our hypothesis, no matter how much of the data doesn’t seem to be pointing in that direction.
We start thinking about what our colleagues would think about our diagnostic decision [00:29:00] making rather than the person right in front of us. There’s always someone who would diagnose it differently or diagnose a case that you said was aquamarine as a clear case of blue, or someone who will make accommodation recommendation that you didn’t even think of or that you wouldn’t have thought was appropriate. And this makes us chase after shadows and miss the person who is right in front of us.
Dr. Sharp: Well said. I would imagine that we’ve sufficiently raised the anxiety level of everyone listening through the roof at this point. We could reliably diagnose panic in nearly all of the audience at this point. And so if we’re overemphasizing diagnosis, and it’s fraught with all these concerns, I wonder where we go from here.
[00:30:00] I’m reflecting on our conversation, Stephanie, and I’m often asking myself this question as we talk, which is, what am I supposed to be doing with my life then? Because you raise these questions and these concerns with the work that we do, that makes me examine what we’re doing very closely.
Dr. Stephanie: I do want to reassure everyone who’s listening that if you resonate with some of these ideas or musings, you are certainly not alone. If you’re not resonating with them, that’s fine. If you feel good about making your diagnoses, definitely continue to focus on that. If you’re starting to question it, so are a lot of people. These arguments about diagnoses are not new.
There’s a whole new book out called Conversations in Critical Psychiatry [00:31:00] by Awais Aftab. He also has a great Substack called Psychiatry in the Margins, and it’s a whole bunch of his interviews with other thinkers in this field who are worried about the weight that we’re putting on diagnosis and some of the other features. So you can read that for solace and constellation.
Dr. Sharp: Fantastic
Dr. Stephanie: But in terms of solving the problem, what do we do? If I’m not supposed to over-focus on diagnosis, what am I supposed to focus on? I think that the next thing that most of us turn to is, we think, oh, okay, the numbers are going to save me. I’ll focus less on the diagnosis and more on creating a blueprint or roadmap of the child that I get through this profile of numerical strengths and weaknesses.
That this [00:32:00] will allow me to either see the diagnosis more clearly, have more backing for it, or even if I just have to give a not otherwise specified diagnosis or no diagnosis at all, at least I’ll have numbers to rely on. Is that something that you resonate with?
Dr. Sharp: Oh, sure. Yeah. It’s interesting, I’m just coming off of a million internship interviews for next year. And this is something that comes up a lot in internship interviews because I think a reason that a lot of us are drawn to testing is that it has the illusion of being more concrete, “analytical,” and data-driven. And the numbers are a comfort; sometimes a false comfort.
Dr. Stephanie: A comfort, exactly. I sometimes talk about that as [00:33:00] the psychology assessor self-selection problem is you take a lot of people who want to get A+ on things, and then promise them that the answer is really in there, and you can find it if you get enough numbers and tweak them in enough different ways that you’ll be able to find it.
And then many of us start realizing that the comfort, the concreteness that was promised by numbers, is a bit of an illusion. And specifically, I’m going to talk about three quick shortcomings with numbers. I think most people who are still listening, who haven’t turned this off in horror, will recognize some of these illusions, which is that numbers give us the illusion of inevitability, the illusion of impartiality, and the illusion of importance.
And by illusion of inevitability, I mean we often treat our numbers as if they [00:34:00] are stable, reliable, and will always be the same. That if you test a child’s fluid reasoning right now that you got a good read on it and that it will be similar in the future, and that if you subtract his fluid reasoning weakness from his Full Scale IQ strength, you’ve learned something important and stable about him.
Unfortunately, the literature is not suggesting that this is the case. If you look at, for example, I’m just going to pick one here, which is, Watkins, Canivez, Dombrowski, McGill, Pritchard, Holingue, and Jacobson 2022, but you can find lots of others if you go to that one and follow their work. They looked at, for example, WISC-V scores in a clinical [00:35:00] sample.
What they found is that you might be able to use the Full Scale IQ, the VCI, and the VSI for clinical decision-making purposes. They might be reliable enough over two years’ time to make clinical decisions, but the other indexes, fluid reasoning, working memory, processing speed, and the subtest scores and the scatter that you find, even if it’s really unusual scatter, is not consistent.
It does not hold over time. You won’t get the same score the next time. You won’t necessarily find the same discrepancy the next time. And that feeling of permanence, solidity, and inevitability that the numbers were suggesting probably is an illusion.
But I think [00:36:00] more importantly, you start realizing that even if the numbers were right, is that really important? Is it useful? Does it give me everything that I want to know about this person? It feels objective and scientific, but is that the same thing as useful to the family or important, or reflecting their lived experience?
I saw a headline in a Forbes article that I thought the headline was amazing, which said, storytelling beats statistics, but only 100% of the time.
Dr. Sharp: That’s great.
Dr. Stephanie: Which is because we can’t understand numbers without context. If you talk about a 20% chance of rain tomorrow, is that a higher or lower chance of rain? That same 20%, if I’m saying there’s going to be a tornado in the next year [00:37:00] that’s going to hit your house, is that same 20% now high or low?
Dr. Sharp: Sure. It’s such a good point. There are a lot of layers.
Dr. Stephanie: There are so many layers. Once you get that information, it doesn’t necessarily translate immediately to insight or the person’s priorities or their goals. It doesn’t necessarily lead to anything important about the person that you might want to know. It starts becoming a little bit of the tail wagging the dog. Your test stores can start telling you, Oh, because you can do this test, it’s important to measure, or because you got a high score, it’s automatically a strength.
Can I give an example here? I’m asking specifically because I’d like to use you as an [00:38:00] example.
Dr. Sharp: What? Sure.
Dr. Stephanie: Okay. Because you and I have known each other in some capacity for probably 10 years now.
Dr. Sharp: Yeah. Getting real close.
Dr. Stephanie: In that entire decade, I have never, not once, wondered what your visual short-term memory score is. Not even once. Never.
Dr. Sharp: Not even once.
Dr. Stephanie: Never crossed my mind. I’ve never wondered if your expressive vocabulary is high average, or exceptional. Never once crossed my mind.
Dr. Sharp: Okay.
Dr. Stephanie: And by offering us tests that say we can measure those things, we start measuring those things. We start being like those researchers who are measuring the cockpit and the pilot in 140 different [00:39:00] dimensions, none of which were useful to understanding the problem, and we can start saying that a high score is a strength, whether or not the person can do anything about it, whether or not it helps them, whether or not they even understand what you mean, whether or not it’s even really a thing, and we can get misled by that.
And so, as a little bit of a teaser to where I’m going, I want to give some examples of the types of things I’d rather be able to say about someone like you. For example, if you came to my office and I was thinking about your experiences, your expectations, your exceptions, and the experiments, those four E’s. I’m going to take a guess at some of these things. So we’re not going to hold me that these all have to be perfect, but I’m just going to give some [00:40:00] examples of things that I might say.
So I might think about all those stories that I know about you, of everything that you have started; the podcast, the testing Facebook page, some report writing software, crafted retreat in the summer, all of those things and say that one of your dynamic trade-offs, one of the real strengths that you have is that in the face of uncertainty, you favor taking action. You’ll do something even when the outcome is not guaranteed or not sure, even if you have to then retreat, reconfigure, figure it out, if it doesn’t work out right.
Dr. Sharp: Sure. Okay. I can get on board with this.
Dr. Stephanie: Okay. And then I’d like to talk about a core belief that you might have, and I know this from your coaching or from conversations that I’ve [00:41:00] heard you have on the podcast, which is that in most situations, I think you truly think there are small tweaks that will make things better, but that’s just a core fundamental belief that you carry around.
Dr. Sharp: Yeah, I think that’s probably true.
Dr. Stephanie: And then maybe talk about some of your coping patterns, like your focus that you’ve talked about on previous podcasts on exercise, nutrition, a healthy lifestyle that includes breaks. And then here’s a guess, a coping pattern that you might do under stress, which is maybe overfocusing on optimizing, overfocusing on improving. You’re nodding.
Dr. Sharp: Oh yeah.
Dr. Stephanie: Okay. And then maybe be able to talk about some of your core values, like making people feel heard and connecting people, getting them having conversations with each other. [00:42:00] And then also some of the resources that you have that are not even necessarily centered in your own person, but that you have access to.
So, from attending one of your crafted retreats, for example, I know that you have access to your spouse’s amazing group facilitation ability and her incredible aesthetic eye for colors, websites, and things like that, right?
Dr. Sharp: Sure.
Dr. Stephanie: So instead of getting a sense of your numbers, your score on a visual short-term memory task, and an expressive vocabulary task and trying to assign some sort of static label, I’m talking about trying to think about how we get more personalized insights about the person that feel like real strengths, that maybe they could then rely on or understand about themselves as they tackle the [00:43:00] challenges they’re facing or the challenges that they’re going to face in the time ahead.
Dr. Sharp: I like this question. What are your thoughts on the overlap between the two? Is there research, we’ll just say research as a shortcut, or some support for the idea that these cognitive abilities that we can measure, that they do actually flow into “strengths” or some of these more personalized qualities, attributes that you’ve outlined?
Let’s take a break to hear from a featured partner.
Y’all know that I love TherapyNotes, but I am not the only one. They have a 4.9 out of 5-star rating on trustpilot.com and Google, which makes them the number one rated Electronic Health Record system available for mental health folks today. They make billing, [00:44:00] scheduling, note-taking, and telehealth all incredibly easy. They also offer custom forms that you can send through the portal.
For all the prescribers out there, TherapyNotes is proudly offering ePrescribe as well. And maybe the most important thing for me is that they have live telephone support seven days a week, so you can actually talk to a real person in a timely manner.
If you’re trying to switch from another EHR, the transition is incredibly easy. They’ll import your demographic data free of charge so you can get going right away. So if you’re curious or you want to switch or you need a new EHR, try TherapyNotes for two months absolutely free. You can go to thetestingpsychologist.com/therapynotes and enter the code “testing”. Again, totally free. No strings attached. Check it out and see why everyone is switching to TherapyNotes.
Our friends at PAR have released the NEO Inventories Normative Update. The NEO [00:45:00] Inventories measure the five major dimensions of personality and the most important facets that define each. Now with an updated normative sample that’s more representative of the current U.S. population and fewer components for easier purchasing. Visit parinc.com/neo.
Let’s get back to the podcast.
Dr. Stephanie: Absolutely. This is a circle that a lot of people are trying to square, that a lot of people are trying to map the things that we naturally do in assessment, to trying to make meaning about how we understand the person.
One of the probably most notable examples would be the personalized medicine field, personalized psychiatry, personalized assessment, where they’re trying to get a sense of the individual’s personal correlations, personal patterns, personal causes. So if people are interested in [00:46:00] that, you could look up personalized assessments.
People are trying to look at ways to use the test data to inform about things that are meaningful. Certainly, people are trying to develop new tests or new assessments that look at some of these other factors. So I have resources if anyone wants them on HiTOP-friendly assessment measures, or ways to measure values scientifically, or ways to measure core beliefs scientifically, things like that that we could talk about if we have time at the end, or that I can give people resources to.
But people are trying to look at this because I think what you’re hitting on is that we do still need the numbers. [00:47:00] I’m not suggesting you just go into an assessment with some vibes and come out with no numbers, but that we put numbers a little bit more in their place and recognize that even though we often use testing and assessment as synonyms, we are not technicians, we are consultants who have a lot of information that we gather and analyze about a person through our understanding of our interview, our record review, our observations, other data that we gather, the videos that the parent sends you of their child at home or homework that you assign to monitor something in between testing session 1 or 2, that we have a lot of data other than just the testing that we can [00:48:00] use our principles of scientific assessment, scientific analyzing, that don’t necessarily have to be just numbers.
We are not technicians, and more importantly, we’re not just an expensive substitute for artificial intelligence. We don’t want to reduce testing to the point where AI could do a better job of it than we can. We don’t mind AI helping write our reports, but if AI could just get some numbers about a person and then spit out a diagnosis and a templated list of accommodations, that reduces our valuable role in helping people.
Dr. Sharp: You’re right. Just reflecting on that, but that’s another can of worms to open.
Dr. Stephanie: Isn’t it?
Dr. Sharp: But for now, we’ll leave it closed, and I’ll just say that I agree. [00:49:00] I think the context and these soft skills, so to speak, are very important. But I would love to talk about how we operationalize this. You’re starting to dip into that.
Dr. Stephanie: What I am thinking about here is returning to those four Es. Returning to assessing people’s experiences, their expectations, and their exceptions, and then trying to do some experiments.
And so with our pilot, if you’ll recall, we asked, what was the experience of being in the plane like for you? How did it compare with your expectations? Well, we can do the same thing when the question is about a 3rd grader who struggles with completing homework, or a college student who is experiencing academic difficulties for the first time, or a teen who is struggling in social [00:50:00] situations, or a 12-year-old who’s physically aggressive at home, or a 23-year-old who’s having psychotic experiences.
We’re getting information about what that experience is like for you? How does it compare with what you expected? What are the exceptions? What are the times when the problem’s not present or when things are going better? How does this compare to the past? What are some experiments that we could try doing right now to see if we can relieve some of this distress?
And so I thought maybe we could talk about those four E’s in a little bit more detail and I can flesh out a little bit more of what I’m talking about.
Dr. Sharp: Yeah, that sounds great.
Dr. Stephanie: Okay. So when I’m thinking about experiences, I’m often trying to get a sense from the individual of more of their transdiagnostic experiences. I’m trying to [00:51:00] ask less about a specific symptom, especially if I have a specific diagnosis in mind that’s leading me down a path that might not be useful, and trying to ask more about symptoms and experiences that people have that cross diagnostic boundaries.
So the way I do it in my mind is I think what’s going on with this person’s homeostatic layer? What’s going on with their social layer? What’s going on with their emotional layer? So by homeostatic …
Dr. Sharp: Can I ask you real quick, what do you mean by…
Dr. Stephanie: Absolutely.
Dr. Sharp: Oh, you’re answering it. Go ahead.
Dr. Stephanie: Did you just say homeostatic? Where are you going? Yes, thank you.
Dr. Sharp: Do I have that? Where is that on me?
Dr. Stephanie: Is that on my bingo card? I don’t remember. Exactly. So the homeostatic layer, by that, this is what, if you’re familiar with the Research Domain Criteria, RDoC, they have seven different [00:52:00] domains that they assess as part of that. One of their domains they call the arousal and regulatory layer.
And that’s basically what I’m talking about here. How sensitive is this person? What is their nervous system like? What upsets it? What signals danger? What makes them excited? How quickly do they return to baseline? Do they have control over that or is it just something that happens? Can they accept co-regulation from other people, like their parents? Just on a nervous system perspective, what’s going on with this individual?
And then that social layer, again, RDoC calls this system for social processes. I’m thinking about what the person’s attachment is like? How much do they trust other people? How interested are they in status? How interested are [00:53:00] they in belonging to a group? How much do they monitor what other people think about them? What’s going on in that layer for that person?
And then the emotional layer, which is what RDoC calls Negative Valence Systems and Positive Valence Systems, I’m thinking about what direction do this person’s characteristic emotions go in? How intense are they? How much emotional granularity does the person have? Can they name all these different emotions? Do they have the emotional vocabulary equivalent of aquamarine and turquoise, or do they only just know blue, or just a cold color? Can they understand their emotions on a really granular level? How well can they regulate those things?
So I’m trying to get at those more [00:54:00] transdiagnostic experiences, mostly, as you can see through my interview. But I do want to point out that RDoC and also HiTOP have good lists of some of these transdiagnostic experiences as well as whole pages on their website. If you go to hitop-system.org, they have this whole list of measures that you can use to try and get at some of these things if you want numbers about them.
And then, when I’m thinking about what RDoC calls the Cognitive Systems layer, or the sensorimotor systems layer, I’ve been framing it in my mind as developing processes or developing competencies. What skills does this person have to meet their goals?
[00:55:00] Specifically, I’m thinking about them in the Ross Greene model of lagging skills rather than a static truth about this individual. I’m trying to get a sense from stories that I hear about the child, observations of the child during testing, their test scores, work samples of can this person meet the expectations that other people have for her, or that she places on herself, or some combination thereof.
And these are going to be just your regular things that you assess during testing. You’re just thinking of them a little bit differently. So can this person develop an approach when faced with a novel problem? Does she feel confident in pulling up information that she already knows and applying it to this situation? Can she direct her attention to what she wants to focus on? Can she control her impulses and delay [00:56:00] gratification? Things like that? In the service of meeting her goals, can she do these things?
I’ve also really gotten into trying to think about the test scores less as static and more as trade-offs. So a lot of the things that we do, we talk about them as if they’re unidimensional, but they aren’t necessarily. For example, probably the most famous trade-off in neuropsychology is speed versus accuracy. You’re faced with, you have to draw as many strange wiggles in two minutes as you can. Do you do it as fast as you can, or do you do it as accurately as you can, or do you try to balance those?
When we talk about processing speed, we tend to talk about it as if it’s a unitary skill. He can process [00:57:00] quickly or he can process slowly. It’s often a trade-off. For example, gifted individuals are known to favor accuracy over speed.
So, how many gifted kids have you tested where their processing speed score is their lowest score because they try to be accurate during that? How many reports have you read that have called that a weakness or a disability when it’s a trade-off, a conscious way that the child is trying to solve a problem?
Other examples would be focusing on the big picture versus focusing on details, approaching sources of a problem or avoiding sources of a problem, getting along versus getting ahead using that spotlight attention where you focus your attention [00:58:00] all on one thing versus a more what Alison Gopnik calls lantern consciousness, where your attention is more broad and diffuse and you’re taking everything in.
Whether you persevere or whether you switch gears and try a new strategy, all of those would be more trade-offs. And here I’m going to stop and shamelessly plug that I’m going to be talking about these trade-offs as informed by functional and neuroanatomy at the AAPdN Virtual Conference on April 25th.
Dr. Sharp: Cool.
Dr. Stephanie: I’m going to use vignettes from the game show TaskMaster. Have you seen that game show?
Dr. Sharp: I haven’t.
Dr. Stephanie: Oh gosh, you have to watch it. It’s amazing. It’s all on YouTube. Anyway, I’m going to use vignettes from that to show in real time people making these trade-offs to try and solve problems that they’re faced with.
Dr. Sharp: Awesome. That sounds super fun. We’ll put the link in the show notes.
[00:59:00] Dr. Stephanie: So these are some examples of ways that I focus on what the experience is like for the person. I also try to get a sense of what their characteristic problem-solving strategies are for them. But I want to move now more to their expectations and how we get a sense of a person’s expectations.
Dr. Sharp: That sounds good. So, can I clarify just for a second before we dive into the expectations?
Dr. Stephanie: Absolutely.
Dr. Sharp: I get the sense in learning about someone’s experiences, a lot of this is happening through the interview, but then especially in the context of these trade-offs, you’re looking at how that might happen during testing as well, how they’re approaching problems, and speed versus accuracy, and so forth. Is that fair?
Dr. Stephanie: Exactly. That’s exactly right. I made a bit of an outline for our discussion today, as you might be able to tell [01:00:00] if you’re listening. I wasn’t sure where to put in the, okay, but where do you do this? What’s the practical, concrete things? Even though having known you for 10 years, I knew that you would ask, okay, but how do you concretely do this?
Dr. Sharp: I’m nothing if not concrete.
Dr. Stephanie: It’s one of your core beliefs that you believe that making things more concrete helps people have practical, actionable takeaways for leaving this. And so you’re right that I do a lot of these things through trying to hear stories.
I try to get actual examples, actual data about experiments that the person’s done in their own life, or actually see it happen, if I can, or walk through an example of that, or talk with them after testing and ask what that was like for them. Ask if we could try it a different [01:01:00] way. See what happens if I provide more structure. See what happens if they stop and think about their problem-solving approach first. See what happens if they talk themselves through the task. Things like that.
But a lot of it I’m getting through stories. I’m trying to hear what the person does when faced with ambiguity.
The reason that I’m trying to do that is because that’s often where our personality, the things that we might have some control over, really tends to bubble up more. Things that we might have some leverage over, that we might be able to change, adjust or receive help with, that’s the area where that impacts it.
You’re reminding me of a story that I heard. This is from the book, the Upside of Your Dark Side from Todd Kashdan. [01:02:00] He talks about the best way to test who’s going to do well in the elite special forces. The way they do it is pretty simple. They tell all their recruits to show up with all of their gear on and just go for a jog. It’s down a remote road. There’s nowhere else around. They just tell them to go for a jog.
The trick is they don’t know how long they’re jogging for. They have no idea how long this run is going to be. So do they sprint and try to beat everyone? Do they pace themselves because it might be 30 miles? Do they focus on themselves, retreat into themselves and try to pull on their own reserves of strength? Do they run with others and try to be encouraged by them? Do they try to show leadership skills and encourage the rest of the group?
That ambiguous situation is how [01:03:00] they see what the person’s real strengths are, what the experience of running is like for them, what they draw on, what they do. And I try and get stories like that, but about 8-year-olds without having to put them in full gear.
What did they do in this situation? What did they try? How did that work? Then what did you do? What else did you consider that didn’t work or that you wish you’d done? Things like that. Get those types of stories.
Dr. Sharp: 8-year-olds on an ambiguous jog. Noted.
Dr. Stephanie: That is what the new tagline for my assessment business is.
Dr. Sharp: Perfect.
Dr. Stephanie: That would be … I was going to say that would be terrible, but then I thought about some of the actual things that I make kids do, like the CPT.
Dr. Sharp: It’s better than some of the things we make kids do.
Dr. Stephanie: Maybe they’d [01:04:00] prefer it.
Dr. Sharp: Yeah. Just go run.
Dr. Stephanie: Exactly.
Dr. Sharp: Nice. Okay.
Dr. Stephanie: These are also ways that I’m trying to get at their experiences, their real strength. It’s also how I’m trying to get at the exceptions to the rule. I’m trying to get at when the problem isn’t present. Those are real-life experiments.
When you hear that a child is inattentive at home and not at school, you think, oh, one of the raiders must be wrong, but the research says that’s not actually true. We are contextual. Different parts of ourselves show up in different situations.
Bram and Peebles, in their book, Psychological Testing That Matters, which I will never stop recommending. So people who’ve heard me talk have heard me recommend this book 1,000 times. They [01:05:00] call that the conditions under which the person thrives, struggles or is having difficulties.
I’ve also heard them called if-then signatures. If you’re in a situation by yourself, then you are more inattentive, and it’s harder for you to stay on task, but when you’re with other people, you have an easier time paying attention. We’re not collections of static abilities. We’re dynamic, strange webs of if-then signatures.
I’m trying to get a sense of this person’s if-then signatures so that I can then start combining them with the trade-offs and being able to write reports that say things like in situations when he’s relying on external motivation [01:06:00] instead of internal motivation, he favors speed over accuracy. Or when she’s with her peers, her fears of being overwhelmed by not being in control intensify, and in those situations she favors dominance-based problem solving strategies, getting ahead over warmth-based problem solving strategies, getting along, things like that.
Dr. Sharp: Yes.
Dr. Stephanie: Does that make sense?
Dr. Sharp: Sure.
Dr. Stephanie: I’m also trying to get a sense with the expectations of the person’s values, what meaning they place on this. Of course, the contextual family, systemic, dynamic, and cultural factors. I’m giving those a little bit of short shrift because you’ve had so many guests on your podcast talk about all of those things, and I’m not going to say anything in [01:07:00] those areas that is necessarily going to be too new.
So I’ll highlight two things that I do that I think might be a little bit different, which is that I’m also trying to get a sense of the person’s underlying beliefs about the world. Their core beliefs about themselves, their core beliefs about other people, their core beliefs about the world and any of their central tensions or preoccupations.
So do they think the world is safe? Do they think other people are dangerous? Do they think they’re a source of help? Do they think they’re fundamentally broken? Do they think that revealing any kind of shortcoming would be dangerous and unsettling, destabilizing for them? Some of those types of things.
I borrowed those from the Psychodynamic Diagnostic Manual, which is, [01:08:00] Lingiardi and Nancy McWilliams. The first chapter is worth the price of the book alone. So definitely read that, but also from the UPenn research on Primal Beliefs. They have a scale that you can use to measure your primal beliefs if you’re wondering. It’s myprimals.com. Things like that to get a sense of that.
And then I’m observing the kid during testing and trying to put it all together by using scientific thinking principles and making sure that I’m looking for repetition. I’m not just basing this off of one story or one test score. When I talked about your dynamic trade-off of being willing to take action in the face of uncertainty, I have a lot of stories about that to support that that’s something that you regularly do and have done and could draw on in the future.
I’m looking for [01:09:00] convergence. I’m making sure that I’m not getting too far outside the data and drawing my own fanciful conclusions, but occasionally I’m emphasizing what Brahmin people call the singularity.
Sometimes you hear a story that’s so unique, so interesting, that you don’t need more than one. You just hear that one story of what they did and have some interesting ideas about what their core beliefs are, what they value, or things like that.
I want to be mindful here of the time and basically just emphasize this call to action of broadening our evaluations to really focus on those four E’s instead of the DNA, and seeing if we can’t together come up with ideas of ways to make testing meaningful.
I also just want to say that if anybody is still listening to this [01:10:00] and is interested in these ideas, please talk to me. Let me know that you’re out there, and if you want to collaborate on something, please let me know because this is basically the animating passion of my professional life right now.
Dr. Sharp: Talk about a call to action, if anyone wants to join the animating passion of Stephanie’s professional life, the invitation is out there.
Dr. Stephanie: That sounds like a great name for my autobiography.
Dr. Sharp: Yeah. Seriously.
Dr. Stephanie: Yes.
Dr. Sharp: Write that down.
Dr. Stephanie: Yeah.
Dr. Sharp: This is so good. There’s so much that we could talk about here and so many paths we could go down. I know this is just scratching the surface and it has been good. I appreciate how you come at it from this philosophical or idea perspective, but then make it concrete. I love making things concrete.
The measure for me of how helpful something is, is could I [01:11:00] use it tomorrow or this afternoon if I wanted to? And that’s 100% true with this material. I’m thinking, okay, how am I going to do my interview tomorrow differently? What might that look like? What can I tell my tech for the rest of the testing today? And that’s going to be super helpful. So I appreciate you being here.
I’m going to try to bookmark or reserve some of your time to talk about rethinking recommendations because I feel like that is important and deserves a lot of time.
Dr. Stephanie: I have so many thoughts on that in particular, and that’s the part that I was like, well, we’re running out of time, maybe I won’t necessarily talk about what to do instead of the accommodations. I am definitely thinking a lot about that as well because I think many of us feel like we get to the end of a really long assessment, and even if we have gotten those four E’s rather than just the DNA, and you feel like [01:12:00] you have mapped this child, that you understand him like a sunset, and then you still just have that list of 42 templated recommendations for ADHD, that’s all I’ve got. So I have a lot of thoughts on that and would be more than happy to talk with anybody about some of the things that I’m trying to do or trying to think about.
Dr. Sharp: That sounds great. That’s so funny. It’s like this sunset needs preferential seating. That’s important.
Dr. Stephanie: Somehow, it needs more time to unfold.
Dr. Sharp: This sunset needs time and a quarter based on the processing speed. Yes. It’s a pleasure, as always. Thank you for being here and sharing your ideas. I always love hearing what you’re thinking about.
Dr. Stephanie: Thanks so much for the time and for accidentally sparking this discussion in my brain.
Dr. Sharp: Sure. Anytime. [01:13:00] All right, Stephanie. Until next time.
All right, y’all. Thank you so much for tuning in to this episode. Always grateful to have you here. I hope that you take away some information that you can implement in your practice and your life. Any resources that we mentioned during the episode will be listed in the show notes so make sure to check those out.
If you like what you hear on the podcast, I would be so grateful if you left a review on iTunes or Spotify or wherever you listen to your podcasts.
And if you’re a practice owner or aspiring practice owner, I’d invite you to check out The Testing Psychologist mastermind groups. I have mastermind groups at every stage of practice development: beginner, intermediate, and advanced. We have homework, we have accountability, we have support, we have resources. These groups are amazing. We do a lot of work and a lot of connecting. If that sounds interesting to you, you can check out the details at thetestingpsychologist.com/consulting. [01:14:00] You can sign up for a pre-group phone call, and we will chat and figure out if a group could be a good fit for you. Thanks so much.
The information contained in this podcast and on The Testing Psychologist website is intended for informational and educational purposes only. Nothing in this podcast or on the website is intended to be a substitute for professional, psychological, psychiatric, or medical advice, diagnosis, or treatment.
Please note that no doctor-patient relationship is formed here, and similarly, no supervisory or consultative relationship is formed between the host or guests of this podcast and listeners of this [01:15:00] podcast. If you need the qualified advice of any mental health practitioner or medical provider, please seek one in your area. Similarly, if you need supervision on clinical matters, please find a supervisor with expertise that fits your needs.
