The Advisory Panel Needs to be
Prepared to Track Important Outcome
Measures in Order to Determine if its Initiatives
are Successful in Retaining Health Care Professionals
in Rural Areas

The Advisory Panel has Developed
a Well Designed Website


The Rural Health Advisory Panel was established to be the decision making body for the West Virginia Rural Health Education Partnerships (WVRHEP). The West Virginia Rural Health Education Partnerships grew from the integration of two programs, the Kellogg Community Partnerships Initiative and the West Virginia Rural Health Initiative (RHI). The partnership between the state of West Virginia and the W.K. Kellogg Foundation is cultivating an environment supportive of long range, creative strategies to address the problem of critically limited primary health care in rural and medically under served areas of the state.

In the fall of 1992 the first students begin rural placements at the Kellogg sites and in the spring of 1993 at the West Virginia Rural Health Initiative (RHI) sites. Working with the Vice Chancellor of the University System, the Office of Community and Rural Health Services laid the foundation for the collaboration and ignited the partnership spirit between these two state agencies.

By April 1994, the RHI and Kellogg Community Partnerships had developed strong ties and the community and program leaders began the process of integrating both programs into one statewide program. In the fall of 1994, the University System of WV Vice Chancellor mandated that all system supported health science students, except Dentistry, must complete a minimum of three months of clinical rotations in rural areas of the state.

On March 9, 1995, the West Virginia State Legislature passed S. B. 161 amending the Rural Health Initiative Act and providing for the official and legal integration of the Rural Health Initiative and the Kellogg Community Partnerships program. These two programs are now a statewide program consisting of 13 training consortia or networks of community based health, social, and education agencies, covering 47 of West Virginia's most under served counties. This enabling legislation called for the appointment of an integrated state advisory panel, which reports to the Vice Chancellor for Health Sciences of the University System in the development and implementation of the restructured program. The Vice Chancellor served as the project director of the Kellogg Community Partnerships and the Rural Health Initiative and now heads the integrated program. The 1995 legislation prescribed the membership and duties of the Rural Health Advisory Panel appointed by the governor, that reports to the Vice Chancellor for Health Sciences. Based upon the experiences in the Rural Health Initiative and the Kellogg programs, this panel has articulated the vision, values, mission, and goals of the restructured and integrated program. The Advisory Panel approves all policies for the organization. The functions and duties of the state panel are to establish and maintain the vision and mission of the program. The Panel oversees development and implementation of policy in governance and administration including personnel policies, operations and management, and finance. This body also reviews, through its various committees, the following standard reports at appropriate times: committee reports, network level and centralized budgets, matters of state level policy, site coordinator reports, annual review of affiliation agreements, and annual reports.

Issue Area 1: The Advisory Panel Needs to be Prepared to Track Important Outcome Measures in Order to Determine if its Initiatives are Successful in Retaining Health Care Professionals in Rural Areas.

The Rural Health Initiative Act of 1991 declares that refocusing health sciences education will aid in the recruitment of health care professionals and their retention in the state, and improve the availability of health care services in the state, especially in rural areas. The Act also created the Rural Health Advisory Board to oversee the implementation of the rural health initiatives. The Legislature established 15 goals under the Rural Health Initiative Act. Some of these goals include:

The Legislative Auditor's review indicates that the Advisory Panel has accomplished many of the Legislature's goals. These goals serve as the foundation or infrastructure of a refocused health sciences educational process towards improving the state's retention rate and improving the availability of health care serves in under served areas. It is expected that with much of the refocus in place, the desired outcomes will be achieved. However, the initiatives take several years before any changes can be measured. Also, it is not clear to what extent the initiatives will be successful. The Advisory Panel will soon approach a critical point in which the restructured health sciences educational process will begin to show results. However, the Advisory Panel is not in good position to measure outcomes that will indicate if the program has been successful and to what extent. Currently the Advisory Panel receives its tracking data from different agencies. These agencies are not linked and the data reported from these agencies by the advisory Panel leaves gaps in trying to follow progression of Health Care Professional Students. The time has come for the Advisory Panel to establish a consolidated tracking system. Through this in-depth tracking the Advisory Panel can help in identifying the effectiveness of its programs in retaining health care professionals, especially primary care physicians in the rural parts of the state. This is important because if the program results in little or no success, problems and solutions will have to be identified.

Residency Program and Clinical Rotations

In 1992, the Advisory Panel program consisted of 12 training sites. This has been expanded to a statewide program consisting of 13 training consortia or networks covering 47 of West Virginia's most under served counties (see Table 1). At this time the 13 regional consortia consist of 255 training sites. These sites include the lead agency sites and the satellite sites and programs which train students. Also, within the state the Advisory Panel has placed Learning Resource Centers (LCRs) with computer stations and educational materials at 18 locations, ten of which are connected to statewide educational programs through MDTV (interactive telemedicine).

Table 1
The Advisory Panel's Thirteen Training Consortia

Name of Consortium Counties Served
Cabin Creek Health Center Kanawha 
Cabwaylingo Health Education Consortium Cabell, Wayne, and Lincoln
Country Roads Consortium Summers, Monroe part of Greenbrier
Eastern WV Rural Health Education Consortium Tucker, Pendleton, Grant, Mineral, Hardy, Hampshire, Morgan, Berkley, Jefferson
Little Kanawha Area Consortium Calhoun, Gilmer, Wirt, Pleasants, Ritchie and Tyler
Mountain Health Partners Consortium Clay, Braxton, Lewis, Upshur, Randolph, Harrison, Barbour, Marion, Taylor and Preston
Rivers and Bridges Consortium Raleigh and Fayette
Rural Mountain Consortium Pocahontas and Greenbrier
Rural Ohio Valley Education Resources Consortium Marshall and Ohio
Southern Counties Consortium Boone, Logan, Mingo, Wyoming and McDowell
Webster-Nicholas Education Consortium Webster and Nicholas
Western Counties Consortium Mason and Putnam
Winding Roads Health Consortium Jackson and Roane

The growth of the Advisory Panel's training sites has increased the demand for medical field faculty. The Advisory Panel has interpreted the term medical students to mean all health professions students. In 1992 there were 60 health care providers of all disciplines providing training to students. In 2000 the total of all rural practitioners training students was 493. This included 259 physicians in the fields of Family Medicine, Internal Medicine, Obstetrics and Gynecology, Pediatrics, Medicine-Pediatrics and Emergency Medicine. In addition to these medical field faculty, there are currently 28 Dentists, 35 Physicians Assistants, 62 Pharmacists, 61 Nurses and Nurse Practitioners, 32 Physical Therapists, 6 Nurse Midwives, 4 Medical Technologists, 4 Social Workers and 2 Occupational Therapists who also train students. This is an important accomplishment of the goal to increase the number of health professionals in rural areas of the state.

Through the building of its infrastructure, the Advisory Panel is showing increases in training sites, field faculty and community contact in rural West Virginia (see Table 2).

Table 2
WV Rural Health Education Partnership Growth

1997 1998 1999
Student rotations 1,573 1,713 1,989 1,596
Student rotations WVRHEP sites 1,166 1,123 1,558 1,232
Weeks of training at rural sites in the state 6,295 7,347 8,429 7,304
Training sites 163 187 211 255
Field faculty 344 422 455 493
Community services provided to West Virginians 23,611 59,039 100,564 156,628

Policies and Procedures for Educational Curriculum

The Advisory Panel began the change in the educational process by establishing policy and procedures to reflect rural health requirements. Some of these policies and procedures included:

These changes help direct the medical schools to incorporate rural health in the training process. Also, the Advisory Panel made significant changes in the health profession programs. This ensured that the curriculum during these rotations includes discipline-specific clinical training, interdisciplinary case management, community service and/or community-based research. These policies and procedures developed by the Advisory Panel have impacted the development and improvement of curriculum in the medical schools of West Virginia.

Creating an Educational Pipeline

In building the rural health program, it became evident that efforts were needed to develop support programs to bring more students into rural health. The Advisory Panel directed the development of programs to build a full educational pipeline for rural medicine. This pipeline will funnel students into the state's medical programs. Through this, the Advisory Panel hopes to increase the number of rural medicine students in the population of medical students in the state.

In building the pipeline, the Advisory Panel has guided the development of the Health Sciences Scholarship Program. This program started in January 1996. The scholarship requires a two-year service obligation to an under served rural area upon completion of the student's residency training. Although a very strong program, the Advisory Panel will see slow results from this pipeline due to the length of training (see Appendix B). To date, there have been 96 scholarships awarded with 60 to Physicians, 19 to Nurse Practitioners and 17 to Physician Assistants.

Another program developed to fill the educational pipeline is the Health Sciences and Technology Academy (HSTA) program. This program reaches out to the students of the secondary schools of the state. Students in this program progress through four years of summer training with their local school teachers and campus-based faculty. They develop networking skills, communication skills and the ability to pull together resources and a feeling of ownership in providing solutions to community problems. From the programs first year in 1994 to the present, the HSTA program has shown continuous growth in students and participating counties of the state (see Table 3). Ninety-five (95) percent of those students who successfully completed the HSTA program as of 1999 are in college and 90% are attending West Virginia colleges and universities (see Table 4). In 1997, the Legislature gave state supported schools the authority to grant full tuition and fee waivers to successful HSTA students.

Table 3
Health Sciences Technology Academy (HSTA) Program Growth

1994 1999
Students (9th-12th grade) 44 500 (+)
Teachers 9 53
Counties involved 2 22

Table 4
1999 HSTA Program Graduates

61 seniors completed HSTA program this year
College-going rate percent (%) 95.1%
Dropout rate percent (%) 1.6%
Average Grade Point Average above 3.0
Average ACT score for HSTA Graduates 22
Average ACT score for West Virginia 20.1
Average ACT score for the United States 21

These programs will continue to feed new students into the Rural Health Program and help the Advisory Panel achieve its goal of increasing the number of rural medicine students in the population of medical students in the state.

Tracking of Health Care Professionals
The state identifies under served areas through the use of the Health Professional Shortage Area (HPSA) designation process. This is a federal process for identifying areas with a shortage of primary care professionals. In West Virginia, 41 of the 55 counties are designated as partial or whole-county HPSAs.

The West Virginia Rural Health Advisory Panel has stated in its mission statement:
The mission of the West Virginia Rural Health Education Partnerships is to achieve greater retention of West Virginia trained health science graduates in under served rural West Virginia communities by creating partnerships of community, higher education, health care providers, and governmental bodies.

The Advisory Panel is required to issue a report to the Legislature on the recruitment and retention of medical personnel. The West Virginia Code §18B-16-6 states in part:

... The report shall address the success of the state's primary care physician and other health care related provider recruitment and retention efforts.

This report is issued to the Legislative Oversight Commission on Education Accountability and the Legislative Oversight Commission on Health and Human Resources Accountability. The first independent report was produced in 2000. Prior to this, the report was part of the annual report of West Virginia Rural Health Educational Partnership.

In review of the reports from 1997 to 2000, it appears that from 1997 to 1999 there are no data on the retention of medical personnel (those completing training and starting practice in West Virginia). The reports contained informational charts on the following:

(1) WVRHEP Community Service Contacts;

(2) WVRHEP Training Consortia Infrastructure;

(3) Student Rotations by County;

(4) Student Rotations by School / Discipline; and

(5) WVRHEP Student Rotations and Student Weeks.

In the 2000 report of the Recruitment and Retention Committee, the committee displayed its retention figures in a chart containing seven disciplines. The seven disciplines on the chart of West Virginia Health Professions provided accumulative information from 1991 to 1999. This data was provided by sources outside the Advisory Panel and put together by the Panel for its report. The data on practice sites of graduates was provided by the three medical schools in West Virginia then verified by the WVRHEP site coordinators. This data is most likely a low estimate since only those practitioners who could be verified were counted. The data on financial incentive programs were provided by the Bureau for Public Health and the University Systems of West Virginia (now the West Virginia Higher Education Policy Commission). First, the Advisory Panel is using accumulative information in its reporting. This makes it impossible to view the year-to-year trends. The committee is not tracking the number of West Virginia residency graduates entering rural practice in West Virginia to see if the State is retaining an increased number each year or if we are we losing these trained individuals to other locations. The committee is not tracking the total number of physicians in rural practice to see if the program has impacted the rural areas by increasing the number of physicians each year, if the state is maintaining the same number year-in and year-out or if the number of rural physicians is decreasing each year. The committee did not establish a baseline of numbers from the time prior to the creation of the rural health program on the numbers of West Virginia medical school graduates, West Virginia residency graduates, West Virginia residency graduates entering practice in rural West Virginia, and Physicians in practice in rural West Virginia. This information does not appear to exist for any of the graduation year group.

In a handout chart to the Legislature on retention of medical school graduates in West Virginia, the panel's chart contained five questions with information to be provided in two columns 1987 to 1992 and 1989 to 1994, on each medical school and a total of all three (see table 5). This data was collected from the Higher Education Report Card produced by the medical schools in the state. Report Cards were gathered for each year that the Advisory Panel would use in their Chart. This Report Card contains information pertaining to the medical schools and what they required to report out. In review of this handout, the following information to answer these questions could not be found:

(2) Of those practicing in West Virginia, how many came from West Virginia residency programs?
The information that was provided was not clear about who it covered and why there was an overlap in years. It was explained that the chart shows retention by year for the most recent six-year cohort of medical school graduates who have completed residency training. The first column, graduates from 1987 to 1992, shows retention for reporting year 1997. This was the first year the Advisory panel began this reporting format. This was established as the baseline year. The next column shows retention in reporting year 1999. The Advisory Panel along with a committee from the medical schools and the Central Administration Research Staff of the Department of Higher Education chose this process of reporting as it believed it would take a "moving snapshot" every year of the most recent six-year cohort of medical school graduates. They chose a six-year cohort as a better indicator of trends because numbers can fluctuate widely when based on individual graduating classes. It was stated that there is an overlap of years because physicians can relocate their practices at any time, and the advisory Panel is tracking retention every year.

Table 5
Retention of Medical School Graduates in West Virginia

Total WV Medical School Graduates 1987-1992 1989-1994 % change
No. of Graduates 960 963 .3 %
No. of Graduates with completed Residency Training 880 924 5.0 %
No. Practicing in WV, All Specialties 317 360 13.6 %
No. Practicing in WV, Primary Care 168 206 22.5 %
No. Practicing in WV, in Non-Urban Areas of WV 89 90 1.1 %

The Advisory Panel was then asked to provide a Retention of Medical School Graduates Report broken down for each year from 1987 to 2000 (see Table 6). A question was added to the table on the number entering West Virginia Residency Programs.

Table 6
Retention of Medical School Graduates in West Virginia (1987-2000)

Total WV Medical School Graduates 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000
No. of Graduates 163 166 148 163 165 152 158 177 185 180 191 202 187 NA
No. Entering WV Residency Programs


























No. of Graduates Who Have Completed Residency Training Programs *




























No. Practicing in WV, all Specialties




























No. Practicing in WV, Primary Care




























No. Practicing in Non-Urban Areas of WV




























* Number who have completed residency training, both in and out of state.

** The 1995 data are incomplete because some graduates have not finished residency training.

The agency was asked to explain the numbers reported in Table 5 and Table 6. In reviewing these two tables, the following questions were asked:

(1) How many were in rural practice before the program started?

(2) How many of the number of graduates with completed residency training were trained in West Virginia?

(3) How many came from training outside of West Virginia programs that are now practicing in West Virginia?

In response to these questions, the agency provided a statement from the Program Coordinator of the West Virginia Higher Education Policy Commission which stated that:

WVRHEP does not have a tracking system that tracks individual students through their medical education, rural rotations, residency training, and practice sites. Consequently, it was a complicated process to produce the data requested. The data came from different sources and reporting years in the Report Card. The schools report on number of graduates and number entering WV residency programs, while the alumni association reports on the location and specialties of physicians practicing in West Virginia. These data sources are not linked to the WVRHEP student tracking system. The WVRHEP tracking system, TRACKER, currently tracks students and rural rotations according to the executive director of WVRHEP.

In a statement from the Associate Vice President for Rural Health / Executive director of WVRHEP, it was stated that:

This report was very difficult to generate because we do not have a good method to track individuals once in practice by their practice address and then verify if this practice address is located in an underserved area. To verify these locations we contacted our WVRHEP site coordinators and/or by calling the individual provider. Adding a tracking and reporting feature to our TRACKER system that would somehow link to licensure data for practice addresses would be wonderful and so very helpful. This would enable us to give much more meaningful reports to both our consortia and policy makers.

In a statement from the Vice chancellor for Health Sciences, it was stated that:

This year, WVRHEP staff are working with a consultant to put in place a longitudinal tracking system that will enable us to gauge the impact of rural training and financial incentives over time. We are also developing a survey instrument to identify the factors that influence student career choices and location decisions. The purpose of the survey is to see if curriculum changes are needed and to improve recruitment and retention of graduates. Although we will not see the full impact of rural training on the location of medical school graduates for several years, we have seen an increase in the number of graduates entering residency training in West Virginia and choosing primary care fields. Both of these hold promise for retention of our graduates.

Contact with the West Virginia Board of Medicine revealed that it produced information on the number of physicians in West Virginia and where they practice through license renewal. This information could be obtained from the West Virginia University Office of Health Services Research in Morgantown. The information is broken down by county, specialty, and specialty in county.

The Legislative Auditor also contacted the Office of Research and Statistics of the State Budget and Control Board of South Carolina and found that they have what is called the Licensed Health Professions Database. This system uses information from licensing boards to track health professionals and analyze the impact of state's education programs on the retention of such individuals throughout the state. Therefore, establishing a successful tracking system is an accomplishable task.

In review of the educational curriculum and training process, the Advisory Panel has addressed and made significant improvements in the issues put forth by the West Virginia Legislature. These improvements include surpassing legislative requirements for the number of rural training sites; requirements for students to perform rural rotations; and the creation of educational pipelines to increase the number of rural of rural medicine students. Now that the Panel has successfully put in place the infrastructure of its program, it is time for it to establish a tracking system that can help in identifying the effectiveness of its programs in retaining health care professionals, especially primary care physicians in the rural parts of the state.

Recommendation 1:
The Advisory panel should establish a tracking system that can identify how many West Virginia residency graduates are being retained in practice in rural West Virginia.

Recommendation 2:
The Advisory panel should establish a tracking system that can identify how many physicians are in practice in rural under served areas of West Virginia each year to determine if the program is impacting those areas by increasing the number of physicians or if the number is staying the same or decreasing.

Recommendation 3:
Until it has improved its own system, the Advisory panel shouldmakeuse of the various health care profession licensing boards in obtaining information on healthcare professionals in practice.

Recommendation 4:
The Advisory panel should establish a baseline of numbers in the categories tracked. This will show what impact the creation of the Rural Health Act has had on improving the number of medical personnel in rural areas.

Issue Area 2: The Advisory Panel has Developed a Well Designed Website.
The Rural Health Advisory Panel has developed an in-depth website. This site provide information about the Panel, the actions taken by the Panel and the results of those actions. This site has enabled the Panel to provide accessibility to its members, to other state agencies and to the general public. Providing accessibility to information on the agency through the means of the internet has increased the amount and speed in which this information is communicated.

The website can be reached at the address of This site contains most of the information published by the Rural Health Advisory Panel. The website is user friendly and easy to navigate. Information within the website includes:

In review of the Rural Health Advisory Panel's website, the Advisory Panel has vastly improved the accessibility of information. This site has enabled the Panel to provide accessibility to its members, to other state agencies and to the general public.