To succeed as a biomedical researcher, the ability to flourish in interdisciplinary teams of scientists is becoming ever more important. Institutions supported by the Clinical and Translational Science Awards (CTSAs) from the National Institutes of Health have a specific mandate to educate the next generation of clinical and translational researchers. While they strive to advance integrated and interdisciplinary approaches to education and career development in clinical and translational science, general approaches and evaluation strategies may differ, as there is no single, universally accepted or standardized approach. It is important, therefore, to learn about the different approaches used to determine what is effective. We implemented a Web-based survey distributed to education leaders at the 60 funded CTSA institutions; 95% responded to the survey, which included questions on the importance of preparation for interdisciplinary team science careers, methods used to provide such training, and perceived effectiveness of these training programs. The vast majority (86%) of education leaders reported that such training is important, and about half (52%) of the institutions offer such training. Methods of training most often take the form of courses and seminars, both credit bearing and noncredit. These efforts are, by and large, perceived as effective by the training program leaders, although long-term follow-up of trainees would be required to fully evaluate ultimate effectiveness. Results from the survey suggest that CTSA education directors believe that specific training in interdisciplinary team science for young investigators is very important, but few methodologies are universally practiced in CTSA institutions to provide training or to assess performance. Four specific recommendations are suggested to provide measurable strategic goals for education in team science in the context of clinical and translational research.
- team science
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
The Clinical and Translational Science Awards (CTSA) program supported by the National Institutes of Health (NIH) represents a consortium of approximately 60 biomedical research institutions across the United States. The goals of the consortium are “to accelerate the translation of laboratory discoveries into treatments for patients, to engage communities in clinical research efforts, and to train a new generation of clinical and translational researchers.”1Research education is clearly a major objective of the CTSA program. The Education Core CompetencyWork Group, a joint effort of the NIH and the CTSA Education and Career Development (EdCD) Key Function Committee (KFC), undertook an effort to define core competencies for master’s degree programs in clinical and translational research, which include 14 thematic areas defining the knowledge, attributes, and skills that are essential to success in clinical and translational science.2Two of the 14 thematic areas specifically address competency in interdisciplinary team research, as summarized in Table 1.
Whereas individual CTSAs strive to advance integrated and interdisciplinary approaches to education and career development in clinical and translational science, they may do so in very different ways, as there is no single approach practiced uniformly in all the centers. Perhaps not surprisingly, therefore, strategies, resources, evaluation processes, and effectiveness may vary substantially between institutions. It is important to learn about the different approaches and, ultimately, determine what works and what does not. That process starts with examining methods that are currently in place.
With this goal in mind, members of the EdCD KFC of the national CTSA consortium created a new working group on “Team Science Training,” with the objectives of assessing, describing, and critiquing approaches to preparing scholars for careers in interdisciplinary team science (defined below). To begin the process of examining different approaches to preparing scholars for interdisciplinary science careers, the committee developed a survey instrument to distribute to the education leaders at 60 CTSA institutions nationwide. The survey asked about each institution’s approaches for “teaching” or fostering team science skills and strategies and the perceived utility and effectiveness of these efforts. The purpose of this article was to present the findings from that survey questionnaire, which incorporated responses to multiple-choice questions as well as qualitative analyses of open-text responses. We present these findings with a view to providing a reference aid for future program design and evaluation efforts in training for interdisciplinary science.
Note that for the purposes of this investigation, we use the taxonomy of interdisciplinary science provided by Rosenfield3: “Researchers work jointly but still from [their] disciplinary-specific basis to address [a] common problem.” Similarly, we adopt the definition of team science proposed by Stokols et al.4: group initiatives “designed to promote collaborative—and often cross-disciplinary—approaches to analyzing research questions about particular phenomena.” Because interdisciplinary science most often involves team efforts, we restrict attention in this article to “interdisciplinary team science,” that is, team projects that involve contributions and ongoing collaboration by scientists representing at least 2 distinct disciplines as they address together a common research question. Thus, our findings are applicable to research projects involving interdisciplinary teams.
MATERIALS AND METHODS
To learn about beliefs, perceptions, and approaches to “team science training” being undertaken by CTSA institutions, we created a Web-based questionnaire. Clinical and Translational Science Awards education leaders across the nation (n = 60) were contacted through e-mail and asked to participate in the study from August 2012 to September 2012. A direct link to the survey was provided in an e-mail generated by the REDCap survey Web application,5with 3 e-mail reminders and 1 “last chance” e-mail sent to maximize overall response rate. A cover letter about the study was sent with the survey request and was accompanied by the list of competencies in translational teamwork and leadership (Table 1).
The questionnaire asked about each institution’s approaches for “teaching” team science skills and strategies and the perceived utility and effectiveness of these efforts. The purpose of this article was to present the findings from that survey questionnaire, which incorporated responses to multiple-choice questions as well as qualitative analyses of open-text responses (Table 2).
The responses to multiple-choice questions are summarized through percentages and bar charts. Comparisons of qualitative characteristics across groups are evaluated by use of the χ2 test. All analyses were conducted in STATA 12. (StataCorp, College Station, TX).
Open-ended text generated from the survey questionnaire items were reviewed to extract important indicator quotes. Analyst triangulation was used throughout the data analysis process to enhance confirmability of the findings.6A list of hierarchical codes was developed by consensus of the coders.
Two investigators independently reviewed the indicator quotations to identify constructs. The second investigator coded the indicator quotations with the first investigator’s coding removed. Saturation was achieved when no new codes emerged from the data, confirming the adequacy of the sample size and providing an end point to the coding portion of the data analyses. A total of 45 constructs were agreed upon to classify the data, derived from 44 indicator quotations. After the initial identification of constructs, the team reexamined the data and identified additional categories of constructs, reorganized and labeled existing constructs, and identified relationships between constructs. To assist in data reporting, constructs were categorized by types of factors as described in the Results section. Whenever divergent interpretations of constructs occurred, indicator quotations were re-reviewed and discussed until consensus was achieved.7
This analysis technique has been used in other qualitative studies without quantifying qualitative results.8Qualitative research designs are not meant to provide quantitative estimates. Qualitative research probes beyond the level of survey data and allows a deeper understanding to explore dimensions that quantitative studies cannot uncover.9
Survey Response Rate
Fifty-seven of 60 individuals responded to the survey, for an overall response rate of 95%. Thus, we believe that we have captured the views of nearly all the CTSA institutions and can be confident that these results are representative of the target audience.
The marginal response percentages for each multiple-choice question are given in Table 3. Overall, 86% of respondents felt that interdisciplinary team science training is important (very or somewhat) for young investigators. Of the 8 (14%) who felt that this training is not very or not at all important, 2 endorsed the notion that interdisciplinary skills are not important for success as a clinical/translational scientist. Three stated that although these skills are important, they cannot be taught. Just over half of institutions (29 institutions, or 52% of the total respondents) stated that they offer training in interdisciplinary team science skills as part of their CTSA educational portfolio, and 72% of these require such training of their scholars. Most of the institutions who offer interdisciplinary training (62%) do so in collaboration with another school or unit within the university (e.g., with a school of business, education, law, or communication).
Of the 29 institutions that offer interdisciplinary team science training, the majority offer a formal, credit-bearing course based in the health sciences (55%) or a noncredit seminar or workshop (59%). Most of these training options can be described as case-based courses in which students review case studies in team science and develop strategies for pursuing interdisciplinary science despite the obstacles to it. About 28% require an experiential activity outside the classroom, such as a research rotation.
Most education leaders (89%) perceived the training they offer as at least somewhat effective. There was little variation in perceptions of effectiveness across types of training opportunities provided (question 6), with percentages reporting somewhat or very effective ranging from 88% to 100% (Fig. 1A).
Perceptions of effectiveness also did not vary substantially by type of course (content of classroom-based or seminar-style courses), with percentages of somewhat or very effective falling between 86% and 100% (see Fig. 1B).
In the survey, open-text opportunities were offered in questions 2, 6, 7, 8, and 11. Themes and subthemes of these responses can be found in Table 4. It is important to highlight and clarify how these responses were elicited within the context of the survey. In question 2, most education leaders agreed (86%) that opportunities in interdisciplinary team science were “very important” or “somewhat important,” although some education leaders disagreed (14%), selecting interdisciplinary team science as “not very important” or “not at all important,” in which the survey was programmed to branch to 3 additional choices to their initial response.
The first choice “I don’t believe these skills are important to success as a clinical/translational scientist” and “I don’t believe these skills can be taught effectively” were the most common, with a third choice “other” in which an open-text response was an option. As seen in Table 4, 3 statements under “other” illustrate some of the tensions and ambiguities between the terminology “team science” and interdisciplinary research. The first construct is the belief of an established successful paradigm: “I have (been) part of interdisciplinary teams for 35 years both in research and as a clinician.”
The second construct is that of the conceptual uncertainty of the term “team science” as having many attributes of skills and competencies similar to interdisciplinary research and therefore rendering it difficult to distinguish definitively between these 2 concepts. “I have an issue with the term team science. How (is) team science different than interdisciplinary research, are the competencies different, are the skills different?”
When asked in question 6, “…what types of training opportunities are provided (in team science),” respondents could select more than 1 option. These options included “a formal credit bearing course offered by (a) health sciences school or (b) another school (e.g., business, education, engineering)” or (c) a noncredit seminar or workshop or (d) an experiential requirement outside the classroom (e.g., a rotation or practicum placement in an interdisciplinary setting) or (e) “other, please describe.”
Respondents of “other” indicated an integration of team science training opportunities into existing noncredit and credit-bearing educational course offerings, as well as integration of “team science” in existing curriculum such as bioinformatics, computational biology, clinical informatics, journal club, and annual symposia. There was also an indication of a “when needed” response to implementing “team science” in which mentoring teams consisting of 2 or more mentors from differing disciplines would require team science training; however, no further information was provided as to what that training entailed.
In question 7, we asked respondents to “…describe the content of your course or workshop on interdisciplinary team science.” If their description was not a choice in our survey, they could write a description in “other.” We received a variety of ways in which interdisciplinary team science knowledge and skills were integrated into existing noncredit bearing courses and/or embedded into weekly KL2 seminars with an emphasis on conflict and team management through experiential learning. Others adopted distance learning opportunities and took advantage of either “Team Science Course” offerings such as “online resource from NUCATS [Northwestern University Clinical and Translational Science Team Science Course]” or through outcome-based learning strategies such as team collaboration (competition) for pilot funding or through distance-based experiential learning. Didactic course offerings in team science leadership were also given as course content.
Evaluation of Team Science
Question 8 asked respondents to share “…strategies/methods for evaluating the effectiveness of the interdisciplinary training opportunities at your institution.” Responses varied greatly and were coded under evaluation as when (time point), who (provides information), what (subject area content), and how (evaluation instruments) evaluation processes of team science. A total of 109 comments were coded from 27 survey responses, as summarized in Table 5. Sample survey responses are shown in Table 4.
When: Respondents indicated postcourse/training (74%) as the most common time point for evaluation, with others stating a combination of precourse and postcourse (15%) and during (11%) course training as data collection time points.
Who: The majority of responses indicated the scholars or trainees (78%) provided evaluation information, with others indicating both mentor and scholar supervisor (11%) or an external entity (11%).
What: The 2 most common content areas identified as the subject of evaluation included competency-based team performance measures (34%), such as collaborative projects, manuscripts and coauthorships, social network analysis (SNA), and generic course evaluations or posttraining evaluations (24%).
How: The most common method of evaluation was a formal evaluation (96%), but details of the method collecting information were usually not specified.
Shared Community Experience
In question 11, we asked “…are there any other thoughts, suggestions, or comments you’d like to share with us at this time?” and we received a wide spectrum of suggestions and comments leading to categories of importance and queries as to what other CTSAs were doing. Many of the responses indicated a need for sharing curriculum content, course syllabi, and evaluation instruments. Some respondents stated they were in the “developmental stage” of their team science curriculum and expressed great interest in what other CTSAs are doing. Perhaps the most urgent stated: “The new RFA puts great emphasis on this [team science education], but it is clear that there are few, if any accepted models or standards for this kind of education. One has no clue as to whether one’s attempt to address this will be adequate or not.”
In summary, these qualitative data reveal the tensions, ambiguities, and diversity of perceptions, attitudes, and applications of the term “team science” in clinical translational sciences education and career development. These data suggest that there is a need for guidance in curriculum development, instructional strategies, and pedagogical methods to support interdisciplinary team science training and education.
It is widely perceived that the research landscape is changing for biomedical research.4,10–15The biomedical literature is increasingly filled with team-based science, as reflected by larger numbers of authors (larger teams) on published articles.10–12Cutting-edge research often takes place at the interstices between established disciplines; thus, the skills and competencies to succeed as members and leaders of interdisciplinary teams have been cited as a critical need among young investigators.13–15Meyers et al.16argue that there is a need for a “qualitatively different investigator” than in years past and that “curricula designed to promote teamwork and interdisciplinary training will promote innovation” when it is delivered in combination with strong disciplinary skills. The unsettled question, however, is how to define interdisciplinary team skills and how to deliver them. Several authors have attempted to tackle this question; see, for example, the articles by Gebbie and colleagues17and Borrego and Newswander.18If training programs are to succeed in fostering interdisciplinary skills, then an analysis of the work of these authors (and many others) suggests that we focus on strong disciplinary skills (so as to be perceived as a valuable member of interdisciplinary teams), regular interaction with colleagues from multiple disciplines, exposure to the theory and practice of building and sustaining a high-functioning team, and opportunities to practice interdisciplinary team building, management, and communication skills. A number of different strategies could be adopted that would provide training in these areas. We decided, therefore, to take a systematic approach (i.e., a survey) to identify and describe the range of strategies currently used and the perceived utility of those strategies in practice.
In this exploratory cross-sectional study, we describe beliefs, activities, and perceptions of CTSA institution education directors about interdisciplinary team science and related training opportunities offered to young investigators as part of their training portfolio in clinical and translational science research. The results suggest that although the majority believe that offering training in team science is very important, only about half of the institutions offer specific training opportunities, and just over a third has made team science a formal program requirement. Among those with formal training programs, 89% believe that their efforts are at least somewhat effective; only about 30% believe that their educational efforts have been very effective, which may be due to the dearth of methods for evaluating interdisciplinary “readiness.”
The CTSA survey results suggest 2 broad themes. The first is that specific training in interdisciplinary team science for young investigators is very important, especially given the changing nature of how science is being conducted in the 21st century,12and the second is that there is no one methodology universally practiced in CTSA institutions designed to meet this identified need. This outcome (i.e., lots of experimentation and little uniformity across CTSA institutions) is neither surprising nor unexpected: when a newly emerging problem or opportunity arises, there is always a proliferation of strategies trying to fill the niche. A lack of uniformity across the CTSAs, when it comes to teaching interdisciplinary team science, is a natural evolutionary stage. It is the intention of this article to move CTSAs beyond the present stage of evolutionary development, when it comes to teaching young investigators interdisciplinary team science, through the natural process of winnowing.
Another important observation, however, is that not all CTSAs have had the award the same length of time. As a consequence, some institutions are, no doubt, still building their educational portfolios. Moreover, whereas teams of scientists working together might be dated, at a minimum, to the Manhattan Project (1942–1945), the formal study of “interdisciplinary team science” is more recent in the biomedical research realm, and there is an understandable lack of clarity about many things, including a curriculum designed to train young investigators for this type of work. That may explain in part why 89% of the respondents to the survey considered the training they offer at their sites as at least somewhat effective, yet no 2 institutions conducted their training programs in exactly the same manner.
The challenge of interdisciplinary team science “training,” reflected in the CTSA survey results, may simply reflect the challenges that appear repeatedly in the literature about interdisciplinary team science.19–21Like interdisciplinary team science itself, any robust training program for young investigators will require that various points of view be represented. These differing points of view, however, will at times result in conflict.20The challenge will be to manage these “conflicts” productively within the training program, which will provide young investigators with the skills needed to do so once they are working in interdisciplinary teams.
To answer the primary question implied in the CTSA survey, that is, “What methodology works best for training young investigators in interdisciplinary team science?” long-term studies will need to be conducted that define, with precision, what is meant by “effectiveness” as an interdisciplinary investigator, and then young investigators will need to be followed over time to gauge their effectiveness based on the type of training they received. We should note that a limitation of this study is the lack of a definition of “effective” in survey question 9, leaving the definition of effective training subject to interpretation by the program directors completing the survey. This points to an additional concern raised by this study that not only do CTSA institutions use different training approaches for team science, but also there is no consistent or widely accepted method for assessing interdisciplinary or team skills and evaluating trainee progress.
We view the results of these analyses as a “call to arms” for CTSA institutions nationwide. We cannot leave these questions unanswered if we are to achieve our goal of training a workforce that is well prepared for interdisciplinary team science. Therefore, we strongly recommend that the following 4 steps be taken by the members of the CTSA consortium:
Recommendation 1. The EdCD key function committee for the CTSA should conduct a review of already described competencies and develop a common, agreed-upon set of competencies specific to interdisciplinary team science for all scholars and trainees. For example, Gebbie et al.17used a Delphi survey process with a panel of interdisciplinary center directors to identify competencies that were identified in 3 overarching domains: conducting research (e.g., “Use theories and methods of multiple disciplines in developing integrated theoretical and research frameworks”), communicating (e.g., “Advocate interdisciplinary research in developing initiatives within a substantive area of study”), and interacting with others (e.g., “Engage colleagues from other disciplines to gain their perspectives on research problems”).17Similarly, Borrego and Newswander18specified 4 domains of competency: grounding in multiple traditional disciplines, integration skills and broad perspective of the interdisciplinary domain, teamwork, and interdisciplinary communication. Most recently, Holt22used a Delphi panel approach to identify 24 competencies for interdisciplinary research collaboration, which were clustered into 5 domains: intrapersonal; disciplinary awareness and exchange; integration; teamwork, management, and leadership; and fruition. These 3 competency sets are overlapping and similar and provide a solid foundation for establishing a uniform recommended set of CTSA competencies for team science education and training.
Recommendation 2. Core curricular components from various institutions should be broadly shared on the CTSA Web site. These materials may include course syllabi, case studies, instructions for student projects, links to Web-based training, and other materials that educators have deemed valuable in this domain. The desire for a shared community experience was evident in the survey results. As more curriculum components are shared and become available for comparison, some consensus may be reached as to best practices, which may be of particular benefit not only to new CTSA institutions, but also to non-CTSA institutions that become interested in expanding the scope of their biomedical research training programs.
Recommendation 3. New methods of instruction may be needed to support the development of team science competencies. Rather than relying on teaching methods that promote passive learning, such as traditional didactic lectures and even some Web-based online learning, the social nature of the competencies involved in interdisciplinary team science suggests that more effective teaching strategies will support active learning and incorporate small group teaching such as problem-based learning and team-based learning (TBL).23–25Such strategies have been used extensively in health sciences education, but less so in research training. Team-based learning was recently suggested to be particularly useful for nurturing the ability to engage in interdisciplinary team science,26which is supported by the recent report that TBL supports the reasoning strategies and social mechanisms that underlie ethical decision making required for the responsible conduct of research.27In the TBL method, students learn collaboratively, working through individual and group tasks and holding one another accountable for completing assignments and preparing for in-class activities. In addition to developing new methods of instruction to foster interdisciplinary team science skills, CTSA educators may lead faculty development efforts to support this training on a broad scale, across the consortium.
Recommendation 4. The CTSA evaluation key function committee should undertake a review of the literature and select a set of assessment metrics that all CTSAs can use to gauge success of their trainees in interdisciplinary team engagement and productivity. These may include “process” measures that are assessed before, during, or immediately after training, as well as “outcome” measures that reflect career achievements over the longer term. A working group of the EdCD KFC recently identified a set of metrics that may be used to evaluate career success for clinical and translational scientists and that includes metrics to assess collaboration and team science.28For example, networking may be assessed using the 6-item Cross-Disciplinary Collaborative Activities Scale, which measures participation in cross-disciplinary collaboration, and the 10-item Research Orientation Scale, which measures researcher preference for unidisciplinary, multidisciplinary, or interdisciplinary research. Both of these measures were developed by the Transdisciplinary Research on Energetics and Cancer initiative of the National Cancer Institute.29The 23-item Research Collaboration Scale, developed by the Transdisciplinary Tobacco Use Research Center Program, measures trust and respect for team members, satisfaction with collaboration, and perception of the impact of collaboration.30Additional assessment tools are available from the National Cancer Institute’s “Team Science Toolkit,”31from the Science of Team Science Website (sponsored by the Northwestern University Clinical and Translational Sciences Institute),32and from another NIH publication: Collaboration and Team Science: A Field Guide.33Another option may be to try to assess, over time, the degree of interdisciplinarity of an individual’s set of publications. Porter and colleagues34have proposed using measurements based on the range of subject categories in the Thomson Institute for Scientific Information Web of Knowledge Website for citations; that is, they suggest counting the number of subject categories cited by all the articles published by an individual author, in addition to the number of subject categories referenced by the publications that cite that author’s work. In combination, these factors would represent the “reach” of the author’s work across disciplines. Although intriguing, these measures are cumbersome to compute and could require years of follow-up to be meaningful. Nevertheless, it would be useful to the CTSA consortium and to the field to identify several key metrics to calculate over time to enable valid, reproducible, commonly used strategies for evaluation of interdisciplinary team skills and productivity.
As the CTSA Consortium enters the next phase of its evolution through a strategic planning process as recommended by the recent Institute of Medicine report about the opportunities that lie ahead for advancing clinical and translational research,35we hope the results of this survey and our recommendations are useful in developing measurable strategic goals for team science in the context of clinical and translational research. The groundwork has been laid for collaborations across CTSA institutions to identify and disseminate best practices for education and training in interdisciplinary team science and for individual institutions to implement them as best fits the needs and resources specific to their environments.