Podcast: Play in new window | Download | Embed
Join Dr. Susie Raiford and I as we talk through the ins and outs of remote assessment, or telepractice, with the WISC-V and KTEA. We spend the first part of the episode talking through the research on telepractice, covering topics like equivalency, norming, and equating as we evaluate different modes of test administration. The conversation then shifts to practical guidelines for administering the WISC-V and KTEA remotely. Here are a few things we cover:
- Is remote assessment a viable, valid alternative to in-person administration?
- What is needed on the examiner side?
- What is needed on the client side?
- You don’t have to mail blocks to the client!
- Can we still get a full scale IQ through remote assessment?
Cool Things Mentioned
- Presence Learning
- All About Q-interactive episode
- Essentials of WISC-V Integrated (affiliate link)
- WISC-5 Remote Assessment Guidelines from Pearson
- Research on remote admin and equivalent tasks
- Subtest-by-subtest guide for remote admin of the WISC-V
- KTEA Remote Assessment Guidelines
- Pearson’s telepractice page
- Testing Psychologist Community on Facebook
The Testing Psychologist podcast is available for CEU’s!
I’ve partnered with At Health to offer CE credits for podcast episodes! Visit this link to access current and past episodes available for CE credit. You can use code “TTP10” for a discount on ALL the course credits you purchase from At Health!
About Dr. Susie Raiford
Dr. Susie Raiford is a senior research director at Pearson, where she is involved in all aspects of test creation and revision conceptualization. Dr. Raiford has helped design the research plan for test creation and revision, write the manuals, develop items, and oversee data collection as well as creation of all final test components. For tests where there is no live author, she serves as the tests’ ghost authors.
Susie has been a primary developer of the Wechsler Intelligence scales since March of 2004. She was the primary research director on the WISC-V and the WISC-V Integrated, along with the WAIS-IV and the WPPSI-IV. Her present project is the WAIS-5.
Susie has published four books on the Wechsler scales and maintains a part time psychological assessment practice.
Prior to becoming a psychological test developer, she primarily engaged in clinical work for ten years, providing assessment, psychotherapy, and consultation to patients of all ages in inpatient, outpatient, school, and residential settings in both the private and public sectors.
About Dr. Jeremy Sharp
I’m a licensed psychologist and Clinical Director at the Colorado Center for Assessment & Counseling, a private practice that I founded in 2009 and grew to include 12 licensed clinicians, three clinicians in training, and a full administrative staff. I earned my undergraduate degree in Experimental Psychology from the University of South Carolina before getting my Master’s and PhD in Counseling Psychology from Colorado State University. These days, I specialize in psychological and neuropsychological evaluation with kids and adolescents.
As the host of the Testing Psychologist Podcast, I provide private practice consulting for psychologists and other mental health professionals who want to start or grow psychological testing services in their practices. I live in Fort Collins, Colorado with my wife (also a therapist) and two young kids.
Ready to grow your testing services? Click below to schedule a complimentary 30-minute pre-consulting call!
Comments 74
Great podcast with Dr. Raiford
Author
Thank you for the feedback!
Frankly, I think it’s ridiculous that Pearson is pushing remote assessment during a global pandemic. These instruments were not standardized for remote use, let alone remote use during a pandemic. Your decisions are reckless and unethical.
Laura, It is a very serious accusation to say someone is unethical and reckless. I suggest you consult the APA ethical standards for teleassessment during this unprecedented time and consider redacting your comment as nothing I have said in this podcast is unethical.
Second, I understand this information was a lot to take in, especially if you are unfamiliar with test development. I would suggest listening again to the beginning of the podcast when I discuss norming versus equivalence and the test development industry standard to establishing how to evaluate whether you can apply normative information in a new format. I clearly explained it. In case you don’t have time to listen again, I have summarized below.
New norms are not how it works. The test development industry standard when mode or format changes is first to study equivalence, not collect new norms, unless they aren’t equivalent. If they aren’t, typically you equate or create new norms. There was a good webinar presented by the INS that reviewed the evidence to date and the APA standards for telepractice during COVID also list some of them, as do the WISC guidelines on the product page.
At this point I think the bigger problem is for clinicians to gain competency in telepractice and think about ethical and legal issues, and whether it is appropriate for your patient, which is what the INS and the APA have emphasized. I highly recommend that INS webinar recording.
Dear Dr Raiford I am an Educational Psychologist. Part of my work load involves assessment of students in HE. It does not look as though my past method of assessment – WAIS-IV, WIAT Word Reading and Spelling etc will be able to continue – ie 1 – 1 assessment – at least for the present. Is it possible to assess remotely using the above instruments? Than k you for reading this.
Dear Dr. Smith,
Yes, you can use both the WAIS-IV and the WIAT-III through Q-global screensharing (Qg digital components are accessible and free at least through June 30 (here are the instructions for how to get your free trial: http://view.ocrm3.pearson.com/?qs=3f63228ab73a66da6ea0d3f62829bb059c7dca412acbb0e01d33a479bfcc87e6061cae6b125355e337b173cad57c85d3662e897c16a660fb74bec2a271b732a5988f27cb5b0132d63a01491c56c61b5f)
and using your paper record forms and response booklets.
Here are the instructions for how to use the WAIS-IV via telepractice:
https://www.pearsonassessments.com/professional-assessments/digital-solutions/telepractice/telepractice-and-the-wais-iv.html
And here are the instructions for how to use the WIAT-III via telepractice:
https://www.pearsonassessments.com/content/school/global/clinical/us/en/professional-assessments/digital-solutions/telepractice/telepractice-and-the-wiat-iii.html
I hope this helps! Let me know if you need anything further.
Laura, this post is remarkably naïve and short-sighted. You, as a school psychologist, may have the luxury of delaying assessments until school resumes. However, many of us use instruments like the WISC in evaluations that directly impact medical decision making.
The “ethical implications” of providing no assessment are real. Moreover, it is very naïve to think that the field will not move in this direction; the current pandemic simply accelerated the timeline. There are already several studies showing the equivalency of remote versus in-person assessments, all very promising. Remote assessment certainly raises unique privacy concerns, but nothing that is insurmountable.
Psychologists need to learn to anticipate the direction of the field and get involved as the experts in assessment and psychometrics or other fields (e.g., occupational therapy) WILL take over. I’d add that such assessments have the added benefit of providing services to heretofore unreachable populations (e.g., geographically remote). To flatly claim that this endeavor is inherently unethical is patently ridiculous… and I say that as the author of 4 book chapters related to ethics in neuropsychological testing.
I listened to the entire podcast and I am left with several questions. The first is under what circumstances should one be conducting cognitive assessments at this time. To be clear, I am not ruling out that there may be some cases in which it would be important to test right now, but I would like to hear your opinions about sample/hypothetical cases in which it would be crucial to complete testing in the current conditions of global pandemic. My second question is to what extent, in your opinion, should a clinician consider the environmental stressors and the child’s emotional state, both in general and at this moment. I was surprised that in a podcast of this length and detail on administration, there was no mention of considering the impact of stress on the child’s performance. As a school psychologist with a background in trauma and crisis intervention I find it unsettling to use data collected in the midst of extreme transition and uncertainty. In my experience children do not perform typically when experiencing major life transitions.
Although research is presented that suggests remote assessment produces what appear to be equivalent results, these studies were not conducted during a traumatic event (i.e. the current global pandemic). I am concerned that while care and attention was directed to accounting for things like the removal of block design, there was not even mention that “results should be interpreted with caution” not necessarily due to remote administration, but rather due to administration due to a global pandemic.
Dear Dr Rious,
I wish the podcast had been longer. As it was we didn’t even have time to talk about subtest specifics.
The purpose of this podcast was limited in scope and length. It is not intended to be a comprehensive resource for the many issues that must be thought through.
As stated in the WISC-V guidelines, consult local, state, and federal regulations and your professional organizations’ best practice recommendations in this process. There are also numerous forums where psychologists are discussing these questions and posting many references to address your questions. Here are some of the resources I’ve gathered and I hope this helps.
Yours
Susie Raiford, PhD
APA guidelines:
Guidance on psychological tele-assessment during the COVID-19 crisis
INS webinar:
https://event.webinarjam.com/replay/13/pyl2nayhvspsp09
IOTC guidelines:
https://static1.squarespace.com/static/50a3e393e4b07025e1a4f0d0/t/5e8260be9a64587cfd3a9832/1585602750557/Recommendations-Guidance+for+Teleneuropsychology-COVID-19-4.pdf

WISC telepractice guidance
https://www.pearsonassessments.com/professional-assessments/digital-solutions/telepractice/telepractice-and-the-wisc-v.html
How to access Pearson test digital stimuli on Q-global:
http://view.ocrm3.pearson.com/?qs=3f63228ab73a66da6ea0d3f62829bb059c7dca412acbb0e01d33a479bfcc87e6061cae6b125355e337b173cad57c85d3662e897c16a660fb74bec2a271b732a5988f27cb5b0132d63a01491c56c61b5f
Dr. Raiford,
Thank you for your reply and the links to resources. However neither the reply nor the links address my questions which are not primarily about tele-assessment, but rather the necessity of cognitive assessment generally in the middle of global pandemic. I will restate the questions in brief :
1. Under what circumstances, in your opinion, would it be essential to conduct a cognitive assessment in this moment
2. In general, do you consider assessments conducted in the midst of or following traumatic events/major life stressors to be valid?
Respectfully,
Jenny
Author
I’m sure that Susie and any others who have conducted research in this area will have thoughts on this, but I’ll jump in as well. Yes, we need to know the impact of environmental and personal trauma on assessment results and keep those in mind if/when we administer assessments to each client. That said, I think we have to keep in mind that individual circumstances differ. Not everyone is responding to this event in the same way, and many may not yet (or ever) consider it “traumatic.” So it’s difficult to answer these questions as if they apply to all.
Excellent point Jeremy. I think in some cases families have been waiting a long time for an evaluation. If we can offer some support and some services virtually at this time, it could be of great help to a family as they plan for a new school year in the fall. The need for virtual evaluation and comfort level for families during this time will vary of course.
Children are affected by trauma whether they have tele-assessment or in person assessment. Children can have traumas that are not related to the pandemic at all. I don’t think the answer is to not assess children, but to do a multi-modal assessment that provides more information than one test does. This can include interviews of student, care givers and teachers, reviewing work samples, trial teaching, observations in their daily context, and possibly social-emotional and behavioral scales. This kind of assessment gives enriched information that may confirm or bring into question test results when a child may not at their best. For example, some children get low test scores when anxious or upset, but observation and adult information about how they communicate, function and do on a variety of tasks and in a variety of situations can be helpful in interpreting or assigning validity to those scores. This is based on working in a school with troubled children for 17 years.
I wonder if we would use trauma/anxiety/depression questionnaire tools for parents and children to help evaluate potential social-emotional concerns as a part of/prior to cognitive assessment.
We would also likely consider the benefit/risk analysis for each client.
Once again, great sticking with the question! I have not seen the webinar, but I would likely have the same questions!
Thank you, Dr. Rious. These are thoughtful comment and concerns. I think we have consensus that the impacts of trauma and stress should always be considered when assessing.
Great Question!!
Dr. Raiford mentions that the scoring for the Non-motor IQ is available in Qglobal. Further documentation by Pearson states that the “Essentials of WISC–V Integrated Assessment book” appears in the Q-global Resource Library. I do not see this book in the resource library. In the WISC-V section I only see 2 documents: “About the WISC-V Integrated” and a sample report. Neither of which provide information about how to calculate the Non-motor IQ.
If anyone has successfully found a WISC-V manual which includes detailed information about how to calculate the Non-motor IQ, please advise. Thank you!
Dear Sammy,
These will be posted in Q global on Friday April 10. Also, if you’d like to read the book excerpt in advance I have posted it in the Testing Psychologist Facebook group. If you can’t find it please let me know.
Yours
Susie Raiford
I have serious concerns about remote assessments under normal circumstances, but under the current circumstances I find it unethical and even immoral. I see no way for results to be valid.
Agreed!!
I’m posting my reply above to answer this same question:
+++++++++++++++
This post is remarkably naïve and short-sighted. You, as a school psychologist, may have the luxury of delaying assessments until school resumes. However, many of us use instruments like the WISC in evaluations that directly impact medical decision making.
The “ethical implications” of providing no assessment are real. Moreover, it is very naïve to think that the field will not move in this direction; the current pandemic simply accelerated the timeline. There are already several studies showing the equivalency of remote versus in-person assessments, all very promising. Remote assessment certainly raises unique privacy concerns, but nothing that is insurmountable.
Psychologists need to learn to anticipate the direction of the field and get involved as the experts in assessment and psychometrics or other fields (e.g., occupational therapy) WILL take over. I’d add that such assessments have the added benefit of providing services to heretofore unreachable populations (e.g., geographically remote). To flatly claim that this endeavor is inherently unethical is patently ridiculous… and I say that as the author of 4 book chapters related to ethics in neuropsychological testing.
+++++++++++++++
Adding to this prior post, You’ve stated that the results would not be “valid”? What the heck does that even mean? Valid for what? The vast majority of the children I evaluate are about to undergo neurosurgery. I’m guess that’s about as stressful as school being cancelled during a pandemic. Are none of the test results I’ve ever gotten “valid”? You are thinking about IQ tests as if it is some sort of monolithic number that is set in stone when everyone is well aware that it is dependent on MANY factors, including emotional state. It is a mere “snapshot in time”… nothing more. So if you need it for treatment planning, waiting until after treatment is no longer needed is not an informed approach.
There are children who face anxiety just WALKING into a school building, for several different reasons. For these children, taking an assessment online, in a setting where they feel most comfortable, such as at home, reduces anxiety for them. There are students and families who decide that online school/home school/hybrid school is best for themselves, their children. We all know, as school psychologists/mental health professionals, there are myriad reasons for this. Allowing a child to be assessed in a setting where they feel comfortable, and have reduced anxiety, is NOT immoral and unethical. I am surprised that someone who has worked with children who have anxiety in school settings would suggest this. I can think of more than one student I have worked with who would receive a more valid and reliable score taking an assessment online than they would in person. There are also students who live in remote areas, or areas where school psychologists are hard to find. These children still deserve services, they deserve to be assessed if necessary. As someone above stated, the world is moving toward this direction in many areas: teletherapy with psychologists and therapists, telehealth with psychiatrists and physicians, online music therapy, online OT/PT, etc. are being offered and welcomed by society. We need to be able to be flexible in our thought process as well. It’s new for us, and perhaps it causes some frustration and anxiety for us. However, calling it immoral and unethical is off the mark.
Thank you for the episode, I found it informative. Dr. Raiford mentioned a teletesting google group, how can I find/subscribe to that group? Thank you!
Dear Sanam,
You can join telepractice testing psychologist group here:
https://groups.google.com/a/peakgifted2e.com/forum/?utm_medium=email&utm_source=footer#!forum/telehealth-testing-psychologists
Were the equivalency studies for WISCV virtual assessments done with parent as proctor in the child’s home?
It seems unrealistic to expect the vast majority of parents to facilitate testing from home, especially with the rather high technology demands. I could see using this model as a telepractitioner in rural areas if I had someone trained on site to help the child, but expecting parents to do this right now in the current situation seems unwise.
Hello Dr. F.,
Most studies that have established telepractice and face-to-face mode equivalence involve an onsite facilitator who is in a professional role. However, preliminary research conducted with parents serving as an in-home facilitator who managed audiovisual needs and response booklets found no significant differences across modes (International Neuropsychological Society, 2020). International Neuropsychological Society [INS]. (2020). Teleneuropsychology (TeleNP) in response to COVID-19. Retrieved April 8, 2020, from https://www.the-ins.org/webinars/
I hope this helps. Dr. Harder described her protocol and parent involvement in good detail.
Hello Dr. Hunt,
Most studies that have established telepractice and face-to-face mode equivalence involve an onsite facilitator who is in a professional role. However, preliminary research conducted with parents serving as an in-home facilitator who managed audiovisual needs and response booklets found no significant differences across modes (International Neuropsychological Society, 2020). International Neuropsychological Society [INS]. (2020). Teleneuropsychology (TeleNP) in response to COVID-19. Retrieved April 8, 2020, from https://www.the-ins.org/webinars/
I hope this helps. Dr. Harder described her protocol and parent involvement in good detail.
Thank you for the timely information regarding remote administration of tests. I was in the middle of testing a child, and we had to postpone the rest of the testing because of current situation of having to stay at home. Now I can do remote testing and finish up what we have started.
Hi Susie,
I’m not seeing the WISC-V stimulus books in the resource library. I do see them for the KTEA. Wondering when the WISC-V will be made available as we do have clients on the schedule who want to move forward with their evaluations! Thanks for all you are doing. We have been using telepsychology for years and find that for some families it really is the best option.
Hi Anna,
I heard from Susie previously that these will be released on April 10. Fingers crossed!
Those stimulus books are coming on Friday April 10
Some highlights from FASP. I couldn’t agree more. Lots of points the WISC presentation did not address, especially the human factor.
FASP is “strongly recommending that Florida School Psychologists halt the continuation of evaluations.”
“Conducting piecemeal evaluations is in violation of best practices for comprehensive evaluations.”
“There are no norms for how children perform in a pandemic. We strongly recommend that schools do not base or make decisions about a child’s needs from limited information or their performance in a time of crisis.”
We strongly recommend that School Psychologists not be penalized in any way for discontinuation of face to face education, intervention, or evaluations during this time.”
3 points for discontinuation:
(1) “Questionable Validity: What students are currently experiencing is something that nearly all living generations have ever experienced before. It is impossible to ascertain whether or not we are truly capturing the typical performance of our students when conducting our evaluations.”
(2) “For instruments that MAY be able to be utilized virtually, many school psychologists are not typically trained or competent in the virtual delivery of assessment tools. As such, continuing evaluations would be a violation of law #….”
(3) “Deviation from standardization often invalidates the results of these tests.”
Thank you so much!!
I have several open referrals for IQ testing and very eager to use the telepractice.
Hi Susie,
I will wait for the I WISC-V stimulus books in the resource library of Friday, April 10th so I can go on with the cases where it is possible. My question is if this resources can stay or will be available after the COVID-crisis.
Thank you for this presentation. I’ve listened, read all the attachments and links, and if I was still in private practice, I would be all over this . . we are talking about our livelihoods. Working in public schools for a few decades, I do not see this as applicable unless NYS Ed Dept puts forth guidelines and we enter a future where shelter and place is the new normal. Hopefully, I’ll be retired by then.
My biggest concern is test security. When I started in the field, I completed Gifted evaluations for an affluent District. The preferred tool was the SB L-M (yes I’m old). On more than one occasion, students had memorized test items. They were coached. How can we, as assessment professionals, protect the integrity of the test? What is to prevent the parent from recording every item presented? Or searching in real time for the answers? So much of the assessment process is how the student got there, not how many did they get correct. Honestly, if a remote-administered evaluation crossed my desk and their was a clear agenda on the part of the parents, I’d be highly suspicious. Maybe that is my nature.
My .02
Thank you again
the WISC-V remotely. From the documents published on Pearson + the testing psychologist podcast interview with Dr. Raiford we understand the nonmotor IQ is an acceptable substitute for the FSIQ and more conducive to remote testing because the subtests therein do not require testing materials. The subtests for each Index follow:
Nonmotor Visual Spatial Index (NMVSI) = Visual Puzzles
Block Design Multiple Choice
Nonmotor Nonverbal Index (NMNVI) = Visual Puzzles
Block Design Multiple Choice
Matrix Reasoning
Figure Weights
Picture Span
Nonverbal General Ability Index = Similarities
Vocabulary
Block Design Multiple Choice
Matrix Reasoning
Figure Weights
Nonmotor Full Scale = Similarities
Vocabulary
Block Design Multiple Choice
Matrix Reasoning
Figure Weights
Digit Span
Naming Speed Quantity
All of the above subtests are available on Qiactive except Block Design Multiple Choice. I would love to hear from someone who has completed the nonmotor batter and can clarify whether it makes sense to do the following:
1. Set up the battery with all available subtests in Qiactive
2. Score as you go using Qiactive while presenting the materials using the digital stimuli on Qglobal.
3. Complete Block Design Multiple Choice using the digital stimuli on Qglobal.
4. Add the raw scores from the Qiactive subtests and BDMC into the excel scoring tool on Qglobal (supposedly will be published on April 10)
The scoring tool yields the 4 index scores above. It is my understanding the the nonmotor indices have their own specific norms. So in using the nonvoter scoring tool you are forgoing some of the typical indices from the WISC-V including: Verbal comprehension (Similarities + Vocab), Fluid Reasoning (Matrix Reasoning + Figure Weights) and Working Memory (Digit Span + Picture Span).
For those conducting WISC-V testing remotely, do you plan to use both the non-motor scores and the traditional indices listed above (VC, FR, WM)?
I understand that the nonmotor FSIQ reportedly correlates more highly with the FSIQ and the test re-test for FSIQ (I was encouraged to hear this!) Specifically for educational purposes, do you think school districts will prefer to see some of the WISC-V traditional indices in addition to the nonmotor indices? Does anyone know why the Nonmotor battery which includes both subtests for calculating the Verbal Comprehension index, did not include this as one of their indices?
Thank you. Very informative and something to think about at this time and in the future.
Thank you for all of the information. I agree that telepractice is an area practitioners, including myself, need more training in.
I had several questions I thought of during the podcast, but I’d like to focus on a question I have about the trained facilitator. I’m thinking hypothetically as I am not sure administering assessments virtually is going to be possible for students I work with.
Is there a standardized training a parent would need to participate in, or would this mainly happen spontaneously as testing is occurring? I also have concerns with the parent being in the room for several reasons, but if we did need them to facilitate (& deemed them fit to do so), I’m thinking they would need to stay in the room with the child and provide assistance throughout if needed.
What is your guidance on training facilitators?
Thanks!
Is there a video available that provides visual demonstration of administration of subtest-by-subtest for WISCV?
Thank you
First of all, thank you for taking the time to address some of the concerns that many psychs have presented regarding the validity and feasibility of remote test administration. I appreciate you drawing everyone’s attention to the equivalency studies (and explaining how to interpret them), as well as the detail you provide on remote administration logistics. Unfortunately, in my estimation, the logistics are unrealistic for many (if not most) public school psychologists.
Although you’ve shown that remote assessment is feasible in some cases, my question is (still) whether it is ethical. I work in a district in which 99% of the population is economically disadvantaged. However, there are vast wealth differentials within our population, with families ranging from affluent to homeless. Despite concerted efforts, many of our families still do not have access to functional technology to support their children’s remote learning, and many more lack internet access.
In this podcast, you make a point that the remote testing environment should be made to resemble the school testing environment. I completely agree. Yet, accomplishing that is much easier done for some families than others. As an example, you suggest that family members should leave the room/house and go play in the backyard. Many of our children live in apartments or shared homes and do not have a safe (or any) backyard. They cannot go stand outside in the snow (yes, that is still happening) for an hour while I ask their sibling to interpret a bunch of picture patterns. We have children in need of assessment who are one of ten children living in a two-bedroom apartment with two chromebooks to share between them. Other children have only their parent’s iPhone to use for schooling. They certainly don’t have a surplus camera lying around at home (such as the “cheap throwaway tablet” you mention). Can such an environment be structured in a way that is conducive to remote assessment?
My concern is, because such discrepancies in access and opportunity exist, engaging in remote assessment seems to present even greater possibility of discrimination than testing within a school building (which we all know is already rife with fairness issues, even if empirically demonstrated to be unbiased). If we offer this practice to some families, we would have to offer it to all, despite the differences in family circumstance. However, due to the differences in available resources and supports at home, it stands to reason that some children would be getting a “better chance” to demonstrate their knowledge and skills than others.
At the end of the day, this podcast seemed to be geared primarily toward those working with a privileged population in private practice. This is simply not the reality in our nation’s public schools, in which most psychological assessment services are delivered. I am interested in hearing your, and others’, thoughts on how remote assessment could be implemented in other environments.
Dr. Raiford mentions that the Correlation is stronger between in-person vs. remote administration of the WISC than test- retest of the WISC. Has anyone been able to find the literature that supports this?
Author
If you look in the “Cool Things Mentioned” section of the post, there’s a link to the table with all the research studies and citations. I forget if it’s Table 1 or 2, but one of them will have that info 🙂
The table of equivalencies mentions a reference list, but I cannot find the reference list anywhere. Specifically, the table is “research on remote testing and equivalent tasks” but the reference list mentioned is nowhere to be found. Thanks for your help!
I cannot find the References either.
Scroll to the bottom of this website for a list of the equivalence studies. We’re updating it all the time, and I think this one is more recent than the WISC-V one, although that one is in process of being updated. https://www.pearsonassessments.com/professional-assessments/digital-solutions/telepractice/telepractice-and-the-wais-iv.html
Hi Susie and Jeremy,
Thank you for this very informative interview! I appreciate the time and energy dedicated to preparing and providing these resources, which are foundational to resuming assessments with some clients. That you have solicited resources from Wiley and produced this interview in the midst of the many other demands upon you at this time is outstanding.
Based on listener comments, there remains much ‘grist for the mill’ as clinicians, regulatory bodies, and other stakeholders consider the best path forward for remote assessment in these unprecedented times of uncertain duration. Should sufficient resources be available, I would welcome another interview that addresses some of the considerations raised by practitioners from varied work contexts. IMO, we cobble together the way forward one iteration at a time.
Again, thank you!
Hi Jeremy and Susie,
Could you please advise how to locate and open the excel file to calculate the non-motor fulls scale score? I see it referenced in the essentials of the WISC-V integrated document, but I am having trouble finding it. Thank you for the unprecedented access to these resources , your time, and assistance.
Alison, please check the Qglobal restricted folder; it’s been there a few weeks.
Yours, Susie
Weighing the pros and cons of remote testing is a Herculean task. While this is the wave of the future-and the future seems to be now-recent guidance from APA and NASP raises serious questions about remote testing. Primary instruments used in cognitive, achievement, and neuropsych testing are difficult if not impossible to administer in their entirety. We cannot assure the security of instruments if we use workarounds like mailing test response booklets to parents nor is it easy to control the testing environment, the skills of parent/facilitators, and whether the equivalency studies (which I have not yet read but would like to) have been peer reviewed. All this and more from a seasoned evaluator who wants to make this work. Tackling the technological problems is very challenging due to uncertain wifi connections, the difficulty screen sharing and watching clients perform tasks without multiple screens, and the clarity of what we are saying and what clients are saying, particularly on tests of listening comprehension or phonemics. The platforms available do not seem to yet have the capacity to assure all of the above can be done while therapy online is easier to facilitate-as long as the connection does not go down!
On top of all this are the legal/ethical issues. Let’s say you manage to do a remote administration and the results are disappointing to parents or schools do not accept the results because they were not administered according to standardizstion norms. This is a setup for a lawsuit.
I would like this to work. Any feedback would be appreciated.
Dear Dr. Raiford,
Thank-you so much for this informative podcast. I have learned a great deal. I decided against doing true remote assessment, but am trying to think through the possibility of doing ‘no touch’ assessment in a large room, since I live in an area with relatively low covid case rates and we have been told this type of contact may be feasible at some point in the future. This seems possible with digital materials – that a child and I could work at separate desks where we could see and hear each other well but keep a safe distance, although of course, I think that every psychologist has to weigh the risks and benefits of this approach for each case.
What I am mainly concerned about now is how academic scores may be affected by a world in which children have not been in conventional school for two months. Can the test publishers put together any helpful data on this? Does what we know about the ‘summer slide’ provide some basis for comparison? I intend to search for this literature myself, but I would love an expert opinion.
Hi Dr. Lilley,
I like your thinking.
Pearson is putting out a piece on your academic question, which has been written by the eminent achievement test developer and researcher Dr. Kristina Breaux. Watch our website for that. I will try to return and let you know when it’s available.
Yours
Susie Raiford
Thanks so much, I’ll watch for it!
If we only use the Block Design Multiple Choice subtest from the WISC-V Integrated with the rest of the WISC-V subtests is the Full Scale IQ derived from the WISC-V scoring or the WISC-V integrated scoring?
I have a similar question. If we replace Block Design with the multiple choice version, but we give Coding and Symbol Search (mailing the protocols to the client with recommended security procedures), is there a way to calculate the traditional Full Scale Score? Is it OK to do subtest substitution or prorate? In many cases I want to see the graphomotor output piece, so I would likely choose to use the paper response books pretty regularly.
Jamie and Dinah, you can just give Visual Puzzles in this case. the BDMC solution is designed for a fully telepractice experience. Options could be any of the following:
1. Use all the regular subtests in a combination of Qglobal and paper, and come in and out of the examinee’s room only for Block Design and the response booklet subtests. Do the rest by telepractice from another room.
2. Same as 2, but use all of the regular subtests except Block Design, substitute Visual Puzzles for FSIQ, and don’t get a Visual Spatial Index, but get everything else. This option could also be done remotely only, if you use the sealed envelope method described here: https://www.pearsonassessments.com/professional-assessments/digital-solutions/telepractice/telepractice-and-the-wais-iv.html under Telepractice Environment & Equipment dropdown menu in the Capturing response booklet performance section. (I’m sending you to the WAIS telepractice page because we haven’t updated the WISC-V telepractice page with this information yet, but it’s coming, because WAIS was done more recently and more clearly answered customer queries). If the examinee is testing from home, there are some ideas about how to use any smartphone or tablet in the Peripheral camera or device section of that same dropdown menu, including this, so you can see the response booklet progress. You don’t have to use the third device/camera, these are just suggestions.
I would also suggest you read the entire WAIS telepractice document because we deal more thoroughly with the idea of facilitators and how to manage that aspect. Again, WISC-V is getting updated with this information soon.
3. This option avoids the smartphone camera. It’s the same as 2, but you opt not to try to use response booklets, so you need some alternate composite scores. I created some new nonmotor composite scores that utilize BDMC from the WISC-V Integrated for a book, Essentials of WISC-V Integrated (2017). Note that these are not using substitution in any way; they are new norms I created using the Pearson norming method that meet all of our typical quality standards.
For BDMC, Q-global provides the manual section, access to the stimulus book that’s shareable through a teleconference platform, and a printable 1 page record form to go along with it.
One possible use for these nonmotor scores is obviously for accessibility for kids who have motor impairment. But, a 100% contaminant-free telepractice option is another application!
All the reliability, validity, and clinical utility information is available in that book excerpt so that you can evaluate the use of these nonmotor composite scores with your population, but I think they are quite good generally! You can download excerpts of that book for free on Qglobal in that WISC-V Integrated restricted folder. It has all the norms tables you might need too. If you choose this option, you might find the excel tool (also in the same folder) helpful. Note that it derives all nonmotor composite scores if the right subtests are available, and also applies some interpretive rules of thumb that I developed for the book (these are optional and you’re free to use them or not, but maybe you will find them helpful). There is no scoring within Qglobal for this because this was done for a book.
o Using this option, you can already get VCI, FRI, and WMI.
o I suggest considering using the Naming Speed Index since you can’t get the Processing Speed Index without response booklets, so that you have some information about information processing speed. Many other published tests use these types of measures as their only measures of processing speed. You can also consider using the Symbol Translation Index if you are missing that aspect of associative memory usually partially measured with Coding.
o You can get the Nonmotor Full Scale Score instead of Full Scale IQ: SI, VC, BDMC, MR, FW, DS, and NSQ. I provide the rationale for why I selected Naming Speed Quantity to stand in for Coding, see the book excerpt. Subtest composition is in Rapid Reference 4.3 in the book
o If you need them, there are also Nonmotor NVI and Nonmotor GAI that merely use BDMC instead. Look at Rapid Reference 4.3 in the book.
o You can also get a Nonmotor VSI: BDMC+VP.
Please let me know if you have any other questions or if I can assist further.
Is there an FSIQ type score that uses the PSI workbooks instead of Naming Speed Quantity? What is the correlation between CD/SS and NSQ?
Thank you, Ashley
Very much in the curious and researching phase, but are there any specific mirroring apps you recommend? I would like more guidance on how to use a mirroring app to present items via Q-interactive. Trying to really figure out these logistics. And weighing trying to lend a document camera to the family vs. using the parent’s iphone. I understand if you used a phone as a document camera, you would just “invite” that phone to the “meeting.” But if you used a document camera instead, are you able to see both the document camera and the student’s face at the same time? Or would you have to pick? When I looked at “using two cameras” via Zoom, the zoom website talks about how to switch between cameras, but not do both at the same time.
Author
Hi Delia – a lot of folks are using Reflector to mirror (https://www.airsquirrels.com/reflector). You can use two cameras via Zoom, but it’s much easier if you have two monitors on the clinician side and set your Zoom settings to “Use dual monitors.” I’m hoping to put together a video walkthrough or at least a few photos soon to illustrate the setup.
Using Q-Global or Q-Interactive, is there a possibility of conducting the FULLY REMOTE assessments and still calculate our FSIQ – given that the student being remotely at home, we are unable to have them complete the cancellation and coding subtests (i.e., processing speed subscore)? Does that meant that if we cannot deliver the paper versions to the home of the client, that we cannot conduct the assessment remotely in FULL – hence, unable to obtain FSIQ.
I’d like to hear more about what changes or accommodations need to be made to the Q-Interactive in order to be able to conduct the proper and complete cognitive assessment (WISC-V, WAIS-IV) FULLY REMOTELY.
Thank you.
Has anyone provided guidance about the KTEA-3 grade (or age) level norms when children have missed months of school (some parents doing better than others with online learning). After summer, many will have been off for 6 months. I’d assume kids will have lost more learning than during a typical summer.
Thanks,
-Ashley
Hi. If I want to substitute block design with the multiple choice option, how do I swap out the score in the g-global scoring? Is that possible? Or, must I figure out how to use the excel spreadsheet? Thanks so very much!
During the podcast on Remote Testing you referenced a scoring rubric when using the WISC V-Integrated- BDMC substitution for BD (WISC-V) You indicated that this was available. Does Pearson have this?
\Where is this located or where is it available.
I purchased the WISC-V Integrated Assessment-Wiley, but don’t see it in the book.
Thank you
Bruce Hartman
I am wondering if others are having issues accessing the Block Design Multiple Choice material. I have not been able to find it on Q-Global. Has it been taken down or am I looking in the wrong place? Any help would be greatly appreciated.
Author
I believe they stopped offering it for free on August 15. Now you can purchase the digital stimulus materials through Pearson’s website.
Hi Jeremy, Thanks for getting back to me so quickly. You are absolutely right. They have included Block Design Muliplechoice in a new WISC V Integrated product, which apparently supplements the WISC V by offering tests less reliant on expressive language skills. It is a shame having to buy a whole new expensive product for the one item I need to administer the WISC V remotely. I really appreciate your informative podcast, thanks for what you do.
Author
Agreed! You’re not the only one wishing Pearson would make the digital materials free if you’ve already paid for the print version. Thanks for listening and the kind words 🙂
Dr. Raiford,
I just wanted to say thank you for all of your insight and expertise. I train telepsychologists across the country. Whenever I have a question and am on the fence about a process I always check here first, so thank you for offering your guidance and expertise to the rest of us. It’s greatly appreciated.
I am late to the party haha. Thank you so much for this insight into how we use our “tools.” But I was hoping to restate a previous question to see if there is an update with Q-Interactive (11/2020).
“Using Q-Global or Q-Interactive, is there a possibility of conducting the FULLY REMOTE assessments and still calculate our FSIQ – given that the student being remotely at home, we are unable to have them complete the cancellation and coding subtests (i.e., processing speed subscore)? ”
Thank you again! As a psychologist, I truly appreciate the time and effort it takes to mold these tools and services in order to better serve our children.
Author
Hey Nick – thanks for the kind words and for listening! I’ll let Dr. Raiford comment here since she’s the expert, but yes, my understanding is that you can calculate a non-motor FSIQ. Here’s what she said on another comment:
Hope that helps! Susie, feel free to add more if you’re still reading these comments 🙂
Love your podcast, I work as a psychometrician in a private practice and (specifically for this episode) would benefit form the transcript of the information shared. It would be infinitely valuable to help me share with the neuropsychologists I work with,
Author
Hey Sarah – thanks for listening! I’ve had a few people ask lately about transcriptions, so I’m looking into options to make that happen. I’ll make sure to announce it when that’s all set up.
Pingback: 172. The Best Episodes of 2020 - The Testing Psychologist