Validation of a Neurofitness Device to Assess Cognitive Measures: A Pilot Study

INTRODUCTION

Neurocognitive technology-based tools are recognized for their ability to detect concussions, assess cognitive function, and support cognitive performance training. These tools have taken shape in electronic, portable formats like smartphones, tablets, and laptops, offering brief, repeatable assessments that are valuable in clinical and sports settings (1,2). Few companies were early adopters and have developed device applications that offer these timely capabilities as new companies continue to enter the market.

ImPACT Applications Inc. is one such company, with their development of the Immediate Post-Concussion Assessment (ImPACT) Test, a Food and Drug Administration (FDA) Class II medical device that administers a neurocognitive battery including visual and verbal memory, reaction time, and processing speed (3). This computerized test has displayed strong convergent validity against other cognitive measures (82%–89%), and moderate-good sensitivity (81.9%–91.4%), specificity (69.1%–89.4%), and test-retest reliability among athletes (≥90% confidence interval [CI]) (4–6).

While the ImPACT computer-based test is the most widely recognized, the Sway Medical mobile application (Sway), developed by Sway Medical Inc., is a recent FDA-cleared computerized cognitive assessment aid that stands out for its accessibility on most smartphones and tablets and compatibility with iOS and Android operating systems (1,7,8). Like ImPACT, Sway broadly assesses cognitive function, including visual processing speed, neuromotor response, inhibition control, memory, and executive function, along with balance (7). This mobile test has shown initial findings of convergent and discriminant validity when compared with ImPACT’s Quick Test, with 12 of 24 correlations reaching statistical significance (r = 0.22 to −0.46), and demonstrates adequate test-retest reliability in community-dwelling adults (interclass correlation coefficient [ICC] = 0.68–0.88) (1,2).

The Reflexion Edge, developed by Reflexion Inc., is an emerging device that claims to be a cognitive training system with the ability to track and monitor cognitive functions (9). The Reflexion Edge differs from Sway and ImPACT in its usage of a large touchscreen device, which the developers justify is essential for targeting an athlete’s entire field of view (9–11). Reflexion Edge has made broad claims regarding its ability to train and monitor various cognitive functions, including reaction time, peripheral vision, decision-making, and hand-eye coordination (9).

Beyond consumer reviews and endorsements from the developers, limited studies have been conducted on the Reflexion Edge to assess its validity. The purpose of the current study was to examine the concurrent validity of the Reflexion Edge against the ImPACT computer-based test and the Sway Medical mobile application. We hypothesized that some scales measured in each Reflexion Edge task would show significant correlations with those of ImPACT and Sway, by cognitive domain.

METHODS

Participants and Procedures

Undergraduate college students were recruited through fliers and word of mouth. A Wichita State University (WSU) Institutional Review Board–approved informed consent form was obtained from each participant. Exclusion criteria included known neurocognitive conditions, seizures or traumatic brain injury within the year, visual or hearing impairments, and drug or alcohol use. Data collection was conducted in a quiet, controlled setting within the WSU Human Performance Laboratory. In a within-subjects design, participants, in a single testing session, completed assessments using each of the 3 devices (ImPACT, Sway, and Reflexion Edge) in a randomized order. Each device included a built-in familiarization trial followed by a recorded trial, with the specific task for each device described in Table 1 (3,7,9). Participants completed each task at their own pace and were offered a 2-minute rest between devices and tasks.

TABLE 1.Description of Reflexion Edge, ImPACT, and SWAY tasks.^a

View Fullscreen

RESULTS

A total of 22 participants (age = 23.54 ± 9.81, 54% male) was screened and included. The findings are presented in Table 2. Starting with reaction time/processing speed tasks, Reflexion Edge Big G correlated moderately with ImPACT Symbol Match (r_s = 0.58, P < 0.01) in 1 of 14 correlations, and moderate-strong with Sway Inspection (r_s = −0.62 to 0.60, P < 0.05) in 4 of 14 correlations. Reflexion Edge Complex Reaction correlated moderate-strong with Sway Reaction Time (r_s = −0.62 to 0.64, P < 0.01) in 2 of 4 correlations, and moderately with Sway Inspection (r_s = 0.45, P < 0.05) in 1 of 4 correlations. Reflexion Edge Go/No-Go showed no significant correlations. In attention/memory tasks, Reflexion Edge n Plus One showed no correlations, while Reflexion Edge Memory Saccade, moderately correlated with ImPACT Word Memory (r_s = 0.49–0.52, P < 0.05) in 3 of 9 correlations, and ImPACT Three Letter Memory (r_s = −0.43, P < 0.05) in 1 of 12 correlations. Lastly, for executive function tasks, Reflexion Edge Minefield strongly correlated with Sway Reverse Number Counting (r_s = −0.78 to 0.77, P < 0.001) in 4 of 6 correlations. Reflexion Edge Go/No-Go showed no significant correlations.

TABLE 2.Correlations between Reflexion Edge tasks and ImPACT/Sway tasks.

View Fullscreen

DISCUSSION

In this pilot study, we sought to validate the Reflexion Edge against the ImPACT computer-based test and the Sway Medical mobile application. The hypothesis was partly supported across the reaction time/processing speed, attention/memory, and executive function domains, as some Reflexion Edge tasks did show significant correlations with their measured scales, while some tasks did not. Though the findings suggest that select Reflexion Edge tasks could assess specific cognitive function, a larger sample is needed to fully evaluate the device’s validity.

Reflexion Edge’s reaction time/processing speed tasks showed some meaningful correlations. However, the requirement for hand-eye coordination and peripheral vision in these tasks may have confounded the measures, as ImPACT and Sway tasks do not target these cognitions (12–15). Big G showed weak correlations, likely due to these added cognitive demands, potentially obscuring the true measurements. Complex Reaction showed more consistent correlations, possibly due to its simple and intuitive design for assessing basic reaction time. In contrast, the absence of any correlation from Go/No-Go, despite its demand for strong inhibitory control, a component present within ImPACT and Sway’s reaction time/processing speed tasks, remains unclear and warrants further investigation.

Among the attention and memory tasks evaluated, limited correlations were found for those implemented by Reflexion Edge, with associations observed only for the Memory Saccade task. The design of this task closely resembles established virtual saccadic memory paradigms, supporting its comparability (16,17). In contrast, the n Plus One task showed no correlations, perhaps because it involved cumulative memory retention, requiring participants to retain and recall all items as they were added, which differed from the one-time recall tasks from the other devices.

Lastly, within the executive function domain, strong correlations were observed only for Reflexion Edge’s Minefield task. Its design effectively integrated key elements of executive function: working memory (tracking the location and detonation of targets), cognitive flexibility (switching priorities between proximity and detonation), and inhibitory control (resisting the proximal but far-from-detonating target) (18). In contrast, Go/No-Go also did not show significant correlations within the executive function domain, warranting further examination of its design.

In this study, we are among the first to assess the concurrent validity of the Reflexion Edge against 2 other cognitive tools. However, given that the sample of college students was small and lacked generalizability, further research is needed with larger and more diverse samples.

CONCLUSION

In conclusion, early findings suggest that select Reflexion Edge tasks demonstrate comparability to specific ImPACT and Sway tasks in engaging particular cognitive domains. However, given the unique differences among these cognitive tools, further research is necessary to fully validate the Reflexion Edge as a reliable and effective instrument.