Examining resource availability and usefulness in a National Health Service Research Interprofessional Postdoctoral Fellowship Program

Postdoctoral fellowships are costly: institutions incur substantial monetary costs, and fellows suffer the opportunity cost of delaying entry into their professional careers. Nevertheless, fellowship training is a beneficial academic investment; the right resources can attract high-quality candidates and maximize return on investment for all parties. This study examined the availability and perceived utility of training resources in a national, multisite interprofessional health services research fellowship program and examined differences in resource perception between alumni and directors as well as M.D. and Ph.D. alumni. One-hundred thirty-one alumni and 15 directors from a multisite interprofessional postdoctoral fellowship completed surveys regarding fellowship resources. Results from the fellowship sample as a whole revealed that mentoring and seminars were the most commonly available resources in fellowships and alumni from the same site often disagreed about resource availability. When we compared alumni and directors’ responses from the same site, we found they often disagreed about resource availability, with directors often being more likely to respond that the resource is available than the alumni. Finally, M.D. alumni reported availability of more resources and found resources to be more useful overall than Ph.D. alumni. Mentoring and seminars are important and commonly provided resources for trainees in fellowship programs; however, M.D.s and Ph.D.s vary in perceived usefulness of other resources, suggesting that one resource does not fit all. Given the gap, postdoctoral fellows may benefit from direct communication of available resources. Moreover, as Ph.D. fellows reported less resource availability and usefulness, attention should be given to meeting their unmet needs. Taken together, this will optimize their fellowship experience, thus better preparing them for their career and, ultimately, their impact on health care.


Introduction
Each year graduates from research and clinical programs delay entrance into careers as professionals to gain additional expertise in their chosen field and further invest in their future via postdoctoral fellowships. In 2019, the National Institutes of Health (NIH) spent nearly $31.4 billion in competitive grant awards for medical research (NIH, 2019). For aspiring research scientists, such fellowships have become increasingly necessary for

Open Access
Innovation and Education *Correspondence: Jennifer.Bryan1@va.gov 2 Houston VA HSR&D Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey VA Medical Center, (MEDVAMC 152), 2002 Holcombe Blvd., Houston, TX 77030, USA Full list of author information is available at the end of the article advancement (Robertson, Klingensmith, & Coopersmith, 2007;Su, 2011). Compared to their peers, postdoctoral fellows are more likely to follow a research career path and have more published research manuscripts (Jacob & Lefgren, 2011), even after controlling for candidate characteristics such as prefellowship productivity and number of years since receiving their doctorate. Fellowship alumni are also more likely to obtain employment in their chosen field and to obtain leadership positions than fellowship finalists (Wall et al., 2017).
Despite their clear benefits, fellowships have costs for both trainees and the training institution. Fellowship stipends are considerably lower than salaries for comparable positions in academia and industry for new doctoral graduates, translating to an opportunity cost to the fellow of up to 3 years of professional-level salaries. For instance, the NIH Ruth L. Kirschstein Research Service Award first-year postdoctoral fellowship stipends were capped at $ 42,840 for fiscal year 2015 (NIH, 2014). By contrast, median salary for all Ph.D. psychologists was $85,000 in 2015 (American Psychological Association [APA], 2017); the median starting salary for internal medicine physicians was $238,227 in 2015 (Faber, Shivarn & Ebell, 2015).
Fellowships are costly to the training institution, as they provide many resources not covered by the stipend. Resources offered to fellows may range from office space to mentoring time (often a leading expert in the field) to administrative support (Rybarczyk, Lerea, Whittington, & Dykstra, 2011;Tucker et al., 2017). Here, resources are defined as programmatic strategies that support the professional development and everyday working life of fellows. Specific resources include relationships (e.g., mentoring, and networking), tangible and personnel resources (e.g., software and statisticians), and tailoring/ structure of the fellowship (e.g., seminars and auditing courses). This is important because postdoctoral fellows neither have classmates to whom they may turn for support and networking, nor do they experience the institutional integration that may enhance their quality of life (Bruckman & Sevbestyen, 2017).
Considering the high cost and expected outcomes of postdoctoral health fellowships, it is important for institutions to ensure a high-quality fellowship experience, including availability of resources that will help prepare fellows to embark successfully on the next stage of their careers. An evaluation of the National Cancer Institute's Cancer Prevention Fellowship Program highlighted the importance of mentoring and structured training for career success (Faupel-Badger, Raue, Nelson & Tsakraklides, 2015). Moreover, the Next Generation of Biomedical and Behavioral Sciences Researchers: Breaking Through report has recommended that postdoctoral fellowships provide fellows with a number of resources, including learning opportunities both in professional development and research, mentoring, and research experience (National Academies of Sciences, Engineering, and Medicine, 2018). Despite evidence that structural elements and high-quality mentoring are important for postdoctoral fellows' outcomes, we know little about specific resources offered by fellowship programs, the awareness of these resources from leadership and fellows' perspectives, and their impact on the training of fellows.
Ph.D. programs and clinical training programs are traditionally siloed and produce scientist-practitioners with limited perspectives regarding interdisciplinary collaboration. Interdisciplinary collaboration and investigation have become priorities, as they enhance the scientific discovery process in ways not possible with traditional training approaches (NIH, 2008). The Agency for Healthcare Research and Quality defines health services research (HSR) as a "…multidisciplinary field of scientific investigation that studies how social factors, financing systems, organizational structures and processes, health technologies, and personal behaviors affect access to health care, the quality and cost of health care, and ultimately, our health and well-being" (Agency for Healthcare Research and Quality, 2014), and HSR fellowships include interdisciplinary collaboration as a critical training element. Interprofessional learning environments facilitate the acquisition of interdisciplinary competencies (i.e., interprofessional teamwork, team-based interdisciplinary care; Taylor Fatima, Lakshman, & Roberts, 2017;Canadian Health Services Research Foundation., 2012) such that systematic interprofessional education may result in individuals recognizing the importance of interdisciplinary collaboration (Blue, Mitcham, Koutalos, Howell, & Leaphart, 2015). Although interprofessional training is beneficial in several respects, it also presents unique challenges.
Postdoctoral fellowships in HSR train interdisciplinary fellows. These disciplines range from medical doctors to clinical psychologists to public health experts to social scientists. Moreover, the amount of research skills developed in doctoral training programs depends on the purpose of the training program. Medical doctors are trained to provide medical care; clinical Ph.D.s are mostly trained to provide patient care and participate in research and, lastly, nonclinical Ph.D.s are substantially trained in research (APA, 2016;Public Health, 2020). Thus, fellows have varying degrees of previous research experience when entering a health service research fellowship, despite having the same goal upon graduating, to be a health services researcher.
Overall, there is a need for interdisciplinary investigation and practice by researchers and clinicians to improve the science of health services. Further, it is incumbent upon training programs to be more responsive to the needs of research and clinical practice trainees during the training process. The purpose of this study was to examine types and perceived usefulness of resources from the alumni and directors' perspective and cross-site differences in resources availability in a national two-year advanced fellowship program in HSR. We examined differences. Additionally, differences in resource availability and usefulness by professional background are also evaluated, giving insight to the unique needs of M.D. and Ph.D. alumni. The results of this study will inform programmatic areas that can be improved to optimize fellowship experience.

Methods
This retrospective observational study comprised a telephone survey of directors at 15 HSR postdoctoral fellowship sites, followed by a web-based survey of their (N = 222) alumni.

Setting
Sponsored by the U.S. Department of Veterans Affairs (VA), the Advanced Fellowship Program in Health Service Research is a national postdoctoral fellowship program with established program sites in 15 U.S. cities (Ann Arbor, Michigan; Bedford, Massachusetts; Boston, Massachusetts; Durham, North Carolina; Hines, Illinois; Houston, Texas; Indianapolis, Indiana; Iowa City, Iowa: Los Angeles, California; Minneapolis, Minnesota; North Florida/South Georgia/Tampa, Florida; North Little Rock, Arkansas; Palo Alto, California; Pittsburgh/ Philadelphia, Pennsylvania; Seattle, Washington). All fellowship sites are located at a VA medical center with at least one academic affiliate. Alumni are interprofessional, drawn from clinical (e.g., medical, nurse practitioner, pharmacy, social work) and research (e.g., various specialties of psychology, public health, health economics, medical anthropology) backgrounds. Upon graduation, alumni are expected to lead, develop, conduct, and evaluate innovative HSR in a variety of clinical settings, including the Veterans Health Administration and other healthcare institutions that seek to improve care for Veterans and the nation (Veterans Health Administration, 2019).

Participants
We invited fellowship directors from the program sites (N = 15) that had graduated alumni at time of evaluation and all alumni that separated from these fellowship sites from 2000 to 2015 (N = 222) to participate in our evaluation.
All current directors of HSR fellowships with fellowship alumni participated (N = 15). Of the 246 potential participants, 222 met eligibility requirements (HSR fellowship alum completing their fellowship between 2000 and 2015). Of those eligible, 131 participated (59% response rate). Reasons for nonresponse included lack of or incorrect contact information (n = 8), declined to participate (n = 9), or did not respond to phone or email (n = 64).

Fellowship director telephone survey
Telephone surveys were used to maximize response rate. We scheduled individual appointments to administer the telephone survey to each site's current director(s). Directors received questions before the appointment, to allow opportunity to research the answers if needed. Directors were asked what resources (e.g., mentoring, seminars, courses, etc.) were available to alumni at their site from a checklist. In addition, the Office of Academic Affairs (OAA) provided a list of alumni who had separated from the HSR fellowship between 2000 and 2015, as well as their most recent contact information. Fellowship directors were asked to corroborate the OAA list and correct or update it as needed. Directors' responses were compiled and informed the questions asked in the alumni survey.

Alumni web-based survey
After we had updated their contact information, fellowship alumni were invited to participate via email, which contained a link to the web-based survey. For fellowship alumni whose contact information was not available from OAA or their fellowship program, we searched other sources, including the VA internal Global Address Book, university websites, and professional social media outlets (e.g., LinkedIn, Research Gate, Twitter). Further, current publication searches (e.g., in PubMed or Web of Science) and contacts with other alumni also aided in identifying alumni's current contact information. All alumni with available contact information who appeared to be eligible were invited to participate. The project manager contacted each fellowship alum via email up to three times or until the survey was complete; participants with available phone numbers were contacted up to five times via telephone if participants were nonresponsive to emails.
Fellowship alumni were first asked to confirm that they were an alumnus from the HSR fellowship and their year of graduation. Next, they were asked demographic questions including, race, gender, ethnicity, and degree. Fellowship alumni were asked to rate the extent to which they found specific resources (seminars, journal clubs, courses, audit courses, earning a degree, mentoring, and research practicum) useful during their fellowship. Response options ranged from 1 (not useful at all) to 5 (very useful); alternatively, participants could indicate that the resource was not offered during their fellowship. Resource availability was determined to not be available if they selected it was not offered during their fellowship. However, if they gave a rating on the usefulness item, we coded them as having the resource available. Finally, alumni answered three open-ended questions; "What resources/training could have made your fellowship experience more useful?", "What were the most valuable resources/training/experiences in your fellowship?" and "Any other comments?".

Data analysis
Descriptive statistics were calculated and used to examine the demographic distribution of our sample. We calculated the percent of alumni at each site who endorsed availability of each resource. Alumni data on resource availability were compared with director data on resource availability, and discrepancies were noted. In addition, Fisher's exact tests were used to examine post hoc differences of perceived availability of resources by degree type. Logistic regression was used to determine differences between clinical and nonclinical Ph.D.s, with M.D.s as the comparison group. We analyzed perceived usefulness of resources by averaging the usefulness scores for each resource and conducted t tests to compare usefulness by degree type.

Analysis of open-ended questions fellows survey
The responses from the three open-ended questions in the survey were imported into ATLAS.ti (v. 6.2), a software program that allows for storing and managing qualitative data. Two members of the research team independently reviewed every qualitative response to identify initial categories of themes within the given responses. The reviewers verified each other's categorizations and identified discrepancies, which were then resolved by consensus: reviewers discussed until an agreement was reached as to the most appropriate category for that particular code. Categorizations were then broadened or reduced as deemed appropriate. After this initial review, the reviewers repeated this process over multiple iterations, aggregating or splitting given categories as necessary to better represent the data. Table 1 summarizes which resources were available at each fellowship program site, as reported by alumni survey respondents. Cells with a Y (n = 44) indicate instances when at least 80% of respondents from a site reported that the given resource was available. Cells with an N (n = 9) indicate instances when at least 80% of respondents within a site reported the resource as unavailable to them. Finally, cells with an M (n = 62) indicate instances when there was less than 80% agreement regarding availability of the resource within a site. Fellowship alumni reported that all sites offered mentoring and some type of seminar as part of training. The perception of availability of the other five resources was far more mixed.

Cross-site differences in availability of program resources
Next, we compared the resource availability responses of directors to those of alumni. Table 2 overlays these responses and presents findings from the comparison of the availability of resources from the perspectives of directors and alumni. In 63% of comparisons (n = 76), alumni and directors disagreed about availability of a given resource at a site. The only resource alumni and directors consistently agreed upon was mentoring. Few instances (n = 3) appeared when alumni were aware of a resource that the director reported was unavailable. More instances were reported of directors reporting resource availability and fellows alumni reporting the resource was not available (N = 7). The most common cell type was directors reporting a resource as available and disagreement among alumni (n = 44).

Exploratory analyses: explaining reported discrepancies in resource availability
Two possible explanations could account for discrepancies between directors and alumni in reports of fellowship resource availability: year of separation (resources currently offered might not have been available in earlier years of the program at a site and vice versa) and degree type (different resources might be available to Ph.D.s vs. M.D.s). We examined these possibilities post hoc with a Fisher's exact two-sided test, using the full sample of fellowship alumni. Year of fellowship separation did not predict availability of any resource. However, for the five resources (earning degrees, journal club, auditing courses, taking courses, and research practicum) for which alumni disagreed regarding availability at their site, M.D. alumni were more likely to report the resource as being available than Ph.D. alumni (p > .025, FET) (see Table 3). Additional post hoc logistic regression analyses were conducted to examine whether there were group differences between clinical Ph.D.s and nonclinical Ph.D.s with the comparison group of M.D.s. Results revealed no significant differences between clinical Ph.D.s and M.D.s for journal club, taking courses, and auditing courses (p > .05); while group differences emerged (p < .05) for nonclinical Ph.D.s and M.D.s for the five previously noted resources (earning degrees, journal club, auditing courses, taking courses, and research practicum).

Usefulness of fellowship resources
M.D.s reported almost all resources (including seminars, journal clubs, mentoring, ability to take courses, and opportunity to earn a degree) as more useful than their Ph.D. counterparts (see Table 4.). In contrast, M.D.s and Ph.D.s reported no significant differences in perceived usefulness of auditing courses or engaging in research practica. Notably, all resources were rated as more useful by M.D.s than by Ph.D.s.
As seen in Table 4, both M.D.s and Ph.D.s rated mentoring and seminars among the top three most useful resources. However, there are discrepancies in relative usefulness among many of the remaining resources. To further explore these findings, we conducted a post hoc series of t tests to investigate whether there were differences in usefulness of resources. Clinical Ph.D.s resembled their nonclinical counterparts on many resources, reporting seminars, journal clubs, courses, earning degrees (p < .05), and auditing courses (trending, p < .10) less useful than M.D.s. However, for the usefulness of mentoring and research practica, there were no differences for M.D.s and clinical Ph.D.s; while trending differences emerged for clinical Ph.D.s and nonclinical Ph.D.s, with clinical Ph.D.s reporting the resources more useful (p < .10).

Qualitative findings from alumni
Alumni responses to our open-ended questions regarding utility and value of available and desired resources also revealed important insights. Many of the same resources cited as most valuable by those who had access to the resource were also reported as lacking but would have been useful during fellowship. Figure 1 summarizes the frequency with which resources were reported as available and valuable versus missing yet highly useful had they been available. These resources can be categorized into relationships, tangible and personnel resources, and tailoring/structure of the fellowship. We expand on each category below.

Relationships
Mentoring and networking were most often reported as most valuable and most useful. Many alumni stated that they received support and one-on-one mentoring. They spoke of the value of having formal and informal mentors as well as the relationships with other individuals who were beneficial to their experience as well as their career. Examples are as follows: A nonclinical PH.D. alum (2010-2015) stated that, "Mentoring relationships formed were invaluable and directly led to long-term Career Development Award (CDA) funding" An M.D. Ph.D. (2005)(2006)(2007)(2008)(2009)) alum commented, "Assistant fellowship director was dedicated, caring and made all the difference. I enjoyed my peers -we provided a lot of support to each other. " A nonclinical Ph.D. alum (2000)(2001)(2002)(2003)(2004) also said, "The opportunity to collaborate with my VA HSR&D colleagues in my center and in other centers was hugely valuable, as was the mentoring I received, not only from my formal mentors but from informal mentors outside my center. " In contrast, alumni who reported needing more or different mentoring than what they experienced often noted needing mentoring aligned with their primary interest and goals. Other mentoring challenges reported by the alumni included mentor responsiveness and protected time for mentoring, as well as inexperience as a mentor.   D. alum (2010-2015) said, "My mentor did not know how to be a mentor. "

Tangible and personnel resources
The tangible and personnel resources category contained responses on funds for books, software and hardware, programming support, conference registrations, journals, travel, statistical support and administrative support. We again see an almost equal number of fellows stating it was most valuable and would have been useful to have. One clinical Ph.D. alum (2010)(2011)(2012)(2013)(2014)(2015) commented that "Computers had access to multiple journals, library assisted in accessing any other journals needed. " A nonclinical Ph.D. alum (2005)(2006)(2007)(2008)(2009) said, "I appreciated that there was money that could be used for IT (information technology) and travel when I went through the program. " An M.D. alum (2010)(2011)(2012)(2013)(2014)(2015) commented, "The people and access to VA data and research support. Mentors, research assistants. "

Flexibility and structure
Fellowship alumni valued the balance between flexibility to tailor the fellowship experience to meet their needs and structure. The ability to audit classes as well as earn a Master's degree in public health was also highly valued.  alum (2010-2015), commented as follows:

Not having a clinical degree or background creates a disadvantage working in HSR. My mentors suggested attending weekly rounds in geriatrics at one of the local hospitals (non-VA), and that helped a lot…I think more opportunities for nonclinicians to engage and learn about the clinical setting would be useful
Structure appeared only in the question asking what would have been more useful. Alumni felt their fellowship needed more structure in curriculum, expectations, goals, and training. A nonclinical Ph.D. alum (2010)(2011)(2012)(2013)(2014)(2015) said, "Any structure, meetings, journal club, more information about how to get research accomplished without funding, more direct mentorship, an academic affiliation for classes/seminars, clear expectations with clear information about how to meet those expectations, mentors who are less overburdened" "I was not provided any structured training or education as part of I had the freedom to take advantage of freely-available training/education opportunities but may have benefitted from more structured information, " said a Ph.D. alum (2010)(2011)(2012)(2013)(2014)(2015).

Discussion
This evaluation compared the availability and perceived utility of resources reported by fellowship alumni and directors across 15 sites nationally for an advanced, interprofessional postdoctoral fellowship program in HSR sponsored by the VA. We found that the only resources universally reported by alumni and directors as being available at all sites were seminars and mentoring. These two resources were also perceived as among the most useful by fellows. Additionally, M.D.s reported having more resources than Ph.D.s during fellowship and perceived those resources to be more useful than their Ph.D. counterparts found them. Comparing perceptions of resource availability, it was most common that directors reported resources as available, while alumni reported availability as mixed. Post hoc analyses revealed that the variability in alumni reports tended to be the result of disagreement between M.D.s and Ph.D.s, with M.D. s more likely to report access to a resource. Additionally, qualitative analysis revealed that the resources alumni reported as most valuable were often times the same resources alumni reported would have been beneficial. Moreover, alumni reported that structure would greatly enhance the fellowship experience. This study is timely, with new recommendations from the National Academies of Sciences, Engineering, and Medicine recommending that postdoctoral fellowships should offer resources including mentoring, research training, and professional development to both M.D. and Ph.D. fellows (National Academies of Sciences, Engineering, and Medicine, 2018). As funding agencies start to require these resources for postdoctoral training, the responsibility to provide these resources will fall on the institution, department, and principal investigator. This study contributes to the literature by evaluating availability and usefulness of research and highlights the importance of disseminating and communicating resources available to postdoctoral fellows in a training program. Cultivating a centralized way to disseminate resources to fellows could optimize university, department, and principal investigator resources and maximize fruitfulness of the fellowship.

Resource availability as reported by alumni
Numerous resource inconsistencies were reported within sites, over and above what could be expected due to programs' adding or removing resources over time.
Rather, it appears that some fellowship alumni from the same site may have had different perceived or actual resources available than their peers. This trend emerged at all 15 sites, indicating that dissemination efforts about resources need to be universally addressed; such that fellows are better aware of all resources available. Although we are unsure exactly why this occurred, it is possible that discrepancies were due to mentors' awareness of resources and their subsequent guidance to their fellows. For example, Mentor A might have suggested her fellows enroll in a particular audit course at their institution, whereas Mentor B might not have been as well connected to the institution and, therefore, unaware that said audit courses existed. The impact of the mentor on fellowship experience is supported by the qualitative results. If this is so, fellowship programs should make concerted efforts to standardize information flow to ensure all mentors are aware of all resources available to their fellows.

Discrepancies in fellows' versus directors' perceptions of available resources
We also observed differences in alumni' versus directors' responses on available resources. The results show that often, when alumni disagreed about whether the resource was available, the director would say that it was. It is plausible that directors either thought a given resource was available when in fact it was not, or that alumni were not aware that a given resource was available. Additionally, with organizational tenure and leadership positions, directors may be more aware of opportunities. However, it may be difficult for directors to remember that not everyone, especially those new to the organization, is aware of these opportunities. Low resource availability has been expressed by postdoctoral fellows (van der Weijden, Teelken, de Boer, & Drost, 2016). Universities often exist in silos (Grigsby, 2015). In an interdisciplinary setting, such as HSR, it is possible that a mentor with a background in psychology knows of resources offered by the psychology department, while a mentor with big data expertise is more familiar with resources offered by the engineering department. While this may be suitable to transfer content knowledge, universities, departments, and mentors may be providing redundant resources that fellows could universally benefit from, such as a seminar series on writing a K award or a professional development talk on networking at a conference. We recommend that directors be cognizant of this issue and ensure that the existence of all available resources is communicated to all fellows and mentors. Given the cost of many resources, lack of awareness means resources go unused and dollars are wasted for the institution. Significant differences in resource availability were reported by M.D. and Ph.D. fellows. There are several reasons why discrepancies may have occurred. First, it is possible that alumni who were very interested in a particular resource sought out that experience (e.g., courses), while others who were less interested did not pursue such resources and, therefore, might not have been aware of their availability. Another possibility is that availability of resources was not adequately disseminated to all fellows. It might have been perceived that M.D.s could benefit more from certain resources than Ph.D.s and that information was, therefore, communicated more to the former group. This is supported by the difference in resource usefulness.

Resource utility: M.D.s versus Ph.D.s
Overall, M.D.s found resources more useful than Ph.D.s. In addition, although both valued seminars and mentoring highly, there were differences in relative usefulness of many other resources. M.D.s appeared to value more formal training opportunities (i.e., they rated courses and opportunity to earn a degree as among the most useful), whereas Ph.D.s appeared to value less formal opportunities (i.e., they rated courses and opportunity to earn a degree low in utility but rated audit courses and research practicums as among the most useful). One potential explanation is that the more formal resources, such as earning a research-focused masters, closely resemble Ph.D. graduate school training, while they are vastly different than the training received in medical school. In other words, many courses offered (e.g., research methods, biostatistics) were more than likely requirements for fellows to obtain a Ph.D., while, if they were offered to M.D. alumni during medical school, they were likely electives and introductory level. These differences in graduate and medical schools are seen, as the desired outcomes of the training programs are different. Graduate schools train Ph.D.s for research (Jepsen, Varhegyi, & Edwards, 2012), while undergraduate medical education prepares students to practice medicine. Perhaps a tailored approach (more formal coursework for M.D.s, more informal research training for Ph.D.s) could help with this optimization and return on investment. Having background-specific training opportunities was reflected in the qualitative results.
The differences in perceptions between Ph.D. and M.D. fellow are consistent with vocational interest and personality theory. Holland's RIASEC theory of personality and vocational interests, the dominant vocational interest theory in the literature, posits that both people and occupations can be categorized into six personality types (Realistic, Investigative, Artistic, Social, Enterprising, and Conventional) (Holland, 1997). According to O*Net (the Department of Labor's online database of occupational characteristics and data (2020)), the dominant personality type for all fields that tend to apply to a health services research fellowship is (unsurprisingly) Investigative.
Where M.D.s and Ph.D.s differ is in the dominance of the remaining personality types. For M.D.s (e.g., internists, hospitalists, preventive medicine physicians, and family practitioners), Social and Realistic consistently round out the top three dominant personality traits. In contrast, the only dominant trait Ph.D.s (e.g., psychologists, epidemiologists, sociologists, and economists) tend to share (both with other Ph.D.s and with M.D.s) is the investigative. The rank order of the remaining vocational characteristics varies from field to field (e.g., for clinical psychologists, the top types are Investigative, Social, Artistic, in that order; by contrast, for an economist, the top types are Investigative, Enterprising, Conventional, and Investigative; for a sociologist, they are Social and Realistic). Given the difference between Ph.D.s and M.D.s in their dominant vocational characteristics, as well as the skills learned in their respective doctoral programs, it stands to reason that the resources they would value or find useful in their fellowships would differ as well.
These findings taken together illuminate the need for increased communication between fellows, mentors and directors. To provide the highest return on investment for all parties, processes for bidirectional communication should be implemented. It is possible that fellows aware of current resources but not using them have other unmet needs. Perhaps Ph.D.s might benefit from exposure to a healthcare setting or other experiences not provided in their doctoral training program. Furthermore, mentoring and seminars were the resources with the most consistent responses from alumni and directors, which may be an avenue programs can use to check in with fellows. Perhaps a seminar on "How to get the most out of your fellowship" led by the director and mentors could be used to disseminate such information. Tailoring mentorship to the needs of diverse fellows is recommended by the National Academies of Sciences, Engineering, and Medicine (National Academies of Sciences, Engineering, and Medicine, 2019). Additionally, a needs assessment could be conducted. If fellows express unmet needs, a novel approach may be warranted. Perhaps a rotation of experiential learning might be valuable. This could be tailored to fellows' needs, such as project management, mentored team leadings and internal grant review committees with structured time for feedback on these skills from mentors, which would be invaluable in forging their career path after the program's end.

Limitations
The strengths of our findings should be considered in light of our study's limitations. Not all alumni responded to requests to complete our survey. It is possible that the sample was biased by responses only from individuals who had a particularly good or bad experience with the fellowship, which could make them more likely to participate in a survey up to 15 years after completing their fellowship. Responses from long-standing alumni could have also been colored by the traditional biases and limitations of long-term memory. However, a representative sample did respond to the survey from all sites, both M.D.s and Ph.D.s, from all years since fellowship separation and with sufficient variability in responses to make the likelihood of halo bias or socially desirable responding low.

Future directions and conclusions
Fellowship training is valuable and costly for all parties involved. To maximize the return on investment of fellowship training, directors must be attuned to the needs of their fellows, active in disseminating currently available resources, and cultivating a culture in which a bidirectional conversation of needs and expectation is welcome. Our study found that seminars and mentoring are the most universally available resource, as well as the most useful, regardless of doctoral training background. From a fellowship perspective, fellowship directors need to at least ensure that the quality of these two programmatic elements is of high caliber. Fellowships need to give the necessary didactic and mentoring that are needed. Future research should look at what elements are needed to support this, such as training on how to mentor and protected time for mentors and course directors. The needs of the Ph.D.s differ from those of M.D.s. This might mean that it is important to pay attention to tailoring the fellowship experience. The needs of the two different degrees are different, so programs much be tailorable to meet the needs of their fellows.