Prescribing Psychologists: DOD Demonstration Participants Perform Well but Have Little Effect on Readiness or Costs (Letter Report, 06/01/99, GAO/HEHS-99-98).
Pursuant to a legislative requirement, GAO reviewed the Military Health System's (MHS) Psychopharmacology Demonstration Project (PDP), focusing on: (1) how PDP graduates have been integrated into MHS; (2) the quality of care they provide to military personnel and beneficiaries; (3) their effect on medical readiness; and (4) comparing the costs of the program graduates to those of other military psychologists and psychiatrists.
GAO noted that: (1) the 10 PDP graduates seem to be well integrated at their assigned military treatment facilities; (2) the graduates generally serve in positions of authority, such as clinic or department chiefs; (3) they also treat a variety of mental health patients; prescribe from comprehensive lists of drugs, or formularies, and carry patient caseloads comparable to those of psychiatrists and psychologists at their same hospitals and clinics; (4) also, although several graduates experienced early difficulties being accepted by physicians and others at their assigned locations, the clinical supervisors, providers, and officials GAO spoke with at the graduates' current and prior locations--as well as a panel of mental health clinicians who evaluated each of the graduates--were complimentary about the quality of patient care provided by the graduates; (5) however, granting drug prescribing authority to 10 military psychologists cannot substantially affect the medical readiness of an organization staffed by more than 800 psychiatrists and psychologists; (6) according to military psychiatrists and psychologists GAO talked to, it is unlikely that the graduates' prescribing abilities and knowledge of psychotropic drugs would be needed during wartime because these types of drugs are not generally the treatment of choice in combat; (7) rather, in treating combat stress, the preferred course of treatment is adequate rest, counseling, and a quick return to the front lines; (8) nonetheless, clinic and hospital officials told GAO that the graduates--by reducing the time patients must wait for treatment and by increasing the number of personnel and dependents who can be treated for illnesses requiring psychotropic medications--have enhanced the peacetime readiness of the locations where they are serving; (9) GAO projects that the Department of Defense (DOD) will spend somewhat more on these 10 prescribing psychologists than it would have spent to provide similar services without the prescribing psychologists; and (10) primarily because of their high training costs, GAO estimates that over the course of the PDP graduates' careers, DOD will spend an average of about 7 percent more (or about $9,700 annually) per PDP graduate than it would spend on a mix of psychiatrists and psychologists who would treat patients in the absence of the PDP graduates.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: HEHS-99-98 TITLE: Prescribing Psychologists: DOD Demonstration Participants Perform Well but Have Little Effect on Readiness or Costs DATE: 06/01/99 SUBJECT: Health care personnel Performance measures Military personnel Human resources utilization Medical education Human resources training Health services administration Drugs Cost effectiveness analysis Mental health care services IDENTIFIER: DOD Psychopharmacology Demonstration Project DOD Medical Readiness Strategic Plan DOD Military Health Services System
****************************************************************** ** This file contains an ASCII representation of the text of a ** ** GAO report. Delineations within the text indicating chapter ** ** titles, headings, and bullets are preserved. Major ** ** divisions and subdivisions of the text, such as Chapters, ** ** Sections, and Appendixes, are identified by double and ** ** single lines. The numbers on the right end of these lines ** ** indicate the position of each of the subsections in the ** ** document outline. These numbers do NOT correspond with the ** ** page numbers of the printed product. ** ** ** ** No attempt has been made to display graphic images, although ** ** figure captions are reproduced. Tables are included, but ** ** may not resemble those in the printed version. ** ** ** ** Please see the PDF (Portable Document Format) file, when ** ** available, for a complete electronic file of the printed ** ** document's contents. ** ** ** ** A printed copy of this report may be obtained from the GAO ** ** Document Distribution Center. For further details, please ** ** send an e-mail message to: ** ** ** ** <email@example.com> ** ** ** ** with the message 'info' in the body. ** ******************************************************************
Cover ================================================================ COVER
Report to the Chairman and Ranking Minority Member, Committee on Armed Services, U.S. Senate
PRESCRIBING PSYCHOLOGISTS - DOD DEMONSTRATION PARTICIPANTS PERFORM WELL BUT HAVE LITTLE EFFECT ON READINESS OR COSTS
DOD Prescribing Psychologists
Abbreviations =============================================================== ABBREV
ACNP - American College of Neuropsychopharmacology DOD - Department of Defense MHS - Military Health System MRSP - Medical Readiness Strategic Plan PDP - Psychopharmacology Demonstration Project USUHS - Uniformed Services University of the Health Sciences VRI - Vector Research, Inc.
Letter =============================================================== LETTER
June 1, 1999
The Honorable John Warner Chairman The Honorable Carl Levin Ranking Minority Member Committee on Armed Services United States Senate
The Military Health System (MHS) provides for the mental health care needs of the approximately 8 million active-duty members, retirees, and their dependents. To meet these needs, MHS employed 431 psychiatrists and 430 clinical psychologists in fiscal year 1999. Some functions of psychiatrists and clinical psychologists overlap. As physicians, however, psychiatrists are trained in and licensed to practice medicine and are permitted to prescribe medication for the treatment of both mental and physical conditions. Because no medical training is required to practice clinical psychology, clinical psychologists--whether in the military or the civilian sector--historically have not been permitted to prescribe drugs. In 1991, however, MHS instituted the Psychopharmacology Demonstration Project (PDP), which was designed to train and use military psychologists to prescribe psychotropic medications.\1 By June 1997, when the project was terminated, 10 psychologists had completed the training and were subsequently assigned to various Air Force, Army, and Navy military medical facilities across the country.\2
At the time of our review, 9 of the 10 program graduates were still treating patients and prescribing medications at military hospitals and clinics.
The Senate report accompanying the fiscal year 1999 National Defense Authorization Act directed us to study the results of this program, including the use and performance of the PDP graduates. Based on the Senate report and subsequent discussions with your offices, our evaluation (1) describes how PDP graduates have been integrated into MHS, (2) provides information on the quality of care they provide to military personnel and beneficiaries, (3) discusses their effect on medical readiness, and (4) compares the costs of the program graduates to those of other military psychologists and psychiatrists. To address these issues, we talked with all 10 PDP graduates and other providers and officials at the facilities where the graduates were practicing or had practiced. Although one graduate left the military during the course of our review, our evaluation includes information about this graduate's service as a prescribing psychologist before leaving the military to reflect the full range of information available on the performance of the graduates. We also reviewed the PDP graduates' credentials files,\3 performance reviews, and relevant reports.
Our work was performed from June 1998 through May 1999 in accordance with generally accepted government auditing standards. Further information on our scope and methodology is included as appendix I.
-------------------- \1 These drugs affect psychic function, behavior, or experience.
\2 In April 1997, we issued a report on PDP, Defense Health Care: Need for More Prescribing Psychologists Is Not Adequately Justified (GAO/HEHS-97-83, Apr. 1, 1997).
\3 The credentials files contain information on education, licenses, performance evaluations, and other information, as well as a record of any quality problems that resulted in adverse outcomes.
RESULTS IN BRIEF ------------------------------------------------------------ Letter :1
The 10 PDP graduates seem to be well integrated at their assigned military treatment facilities. For example, the graduates generally serve in positions of authority, such as clinic or department chiefs. They also treat a variety of mental health patients; prescribe from comprehensive lists of drugs, or formularies;\4 and carry patient caseloads comparable to those of psychiatrists and psychologists at the same hospitals and clinics. Also, although several graduates experienced early difficulties being accepted by physicians and others at their assigned locations, the clinical supervisors, providers, and officials we spoke with at the graduates' current and prior locations--as well as a panel of mental health clinicians who evaluated each of the graduates--were complimentary about the quality of patient care provided by the graduates.
However, granting drug prescribing authority to 10 military psychologists cannot substantially affect the medical readiness of an organization staffed by more than 800 psychiatrists and psychologists. Moreover, according to military psychiatrists and psychologists we talked to, it is unlikely that the graduates' prescribing abilities and knowledge of psychotropic drugs would be needed during wartime because these types of drugs are not generally the treatment of choice in combat. Rather, in treating combat stress, the preferred course of treatment according to service readiness officials and field commanders is adequate rest, counseling, and a quick return to the front lines. Nonetheless, clinic and hospital officials told us that the graduates--by reducing the time patients must wait for treatment and by increasing the number of personnel and dependents who can be treated for illnesses requiring psychotropic medications--have enhanced the peacetime readiness of the locations where they are serving.
We project that the Department of Defense (DOD) will spend somewhat more on these 10 prescribing psychologists than it would have spent to provide similar services without the prescribing psychologists. Primarily because of DOD's higher training costs, we estimate that over the course of the PDP graduates' careers, DOD will spend an average of about 7 percent more (or about $9,700 annually) per PDP graduate than it would spend on a mix of psychiatrists and psychologists who would treat patients in the absence of the PDP graduates.
-------------------- \4 As used here, "formulary" refers to the set of prescription drugs that a provider is permitted to prescribe to patients when treating illnesses.
BACKGROUND ------------------------------------------------------------ Letter :2
The principal mission of MHS is medical readiness. As defined by DOD, medical readiness encompasses both wartime and peacetime components. The wartime mission is primary, according to DOD's Medical Readiness Strategic Plan (MRSP), requiring MHS to provide top quality health services, whenever needed, in support of military operations.\5 In peacetime, according to MRSP, the military medical departments are to maintain and sustain the well-being of the fighting forces in preparation for war. Finally, MRSP states that the military may provide care to dependents or retirees in peacetime, when not employed in preparation and training for the wartime role. The Army, Navy, and Air Force all use military and civilian health care providers to meet their readiness needs.
PDP was established by DOD in response to a conference report dated September 28, 1988, which accompanied the fiscal year 1989 DOD Appropriations Act (P.L. 100-463). The report directed DOD to establish a demonstration pilot training program in which military psychologists may be trained and authorized to issue appropriate psychotropic medications under certain circumstances."
This training program began in August 1991 with four participants. Training for the initial class consisted of 2 years of classroom training at the Uniformed Services University of the Health Sciences plus 1 additional year of clinical training. For subsequent classes, however, the training was modified to consist of 1 year of classroom training and 1 year of clinical training. PDP participants obtained their clinical experience on inpatient wards and at outpatient clinics at Walter Reed Army Medical Center in Washington, D.C., or at Malcolm Grow Medical Center located at Andrews Air Force Base in Maryland. During the clinical part of the training, participants were trained to take medical histories and incorporate them into treatment plans and to prescribe medication for patients with certain types of mental disorders.
Two prescribing psychologists graduated from the initial training class in 1994. The three subsequent graduating classes included 1 prescribing psychologist in 1995, 4 in 1996, and 3 in 1997--for a total of 10 graduates.\6 These 10 graduates--three women and seven men\7 --represented each of the three services: 4 from the Navy and 3 each from the Air Force and Army. In 1995, as part of the program, guidelines were issued on the graduates' roles, including a suggested drug formulary that they would use, a scope of practice limited to patients between the ages of 18 and 65, and the level of supervision or proctoring of graduates for 1 year after graduation.
Several evaluations of the program have been completed since its inception. The American College of Neuropsychopharmacology (ACNP),\8 under contract to DOD, conducted six annual assessments of PDP and issued a final report on the program in 1998. In conducting these assessments, an ACNP evaluation panel interviewed PDP participants and graduates, program officials, classroom instructors, clinical supervisors, and others. Vector Research, Inc. (VRI), also under contract to DOD, conducted an evaluation of the program to determine its cost-effectiveness and feasibility. VRI's report was issued in May 1996 and concluded that PDP was cost-effective.\9 In our April 1997 report, we expressed concern about VRI's analysis because in our view it was based, in part, on unrealistic assumptions.
Additionally, as required by the National Defense Authorization Act for fiscal year 1996 (P.L. 104-106), GAO conducted a study of PDP, which included (1) an assessment of the need for prescribing psychologists in MHS, (2) information on the implementation of PDP, and (3) information on PDP's costs and benefits. In our resulting 1997 report, we concluded that training psychologists to prescribe medication was not adequately justified because MHS had not demonstrated a need for prescribing psychologists, the cost of the program was substantial, and the benefits were uncertain.
In response to the same act, PDP was terminated in June 1997. However, those psychologists who had graduated from or were currently enrolled in the program were permitted by the legislation to continue prescribing psychotropic medication.
-------------------- \5 See DOD, Medical Readiness Strategic Plan (MRSP) 1998-2004 (Washington, D.C.: Aug. 1998), p. 22.
\6 Three participants left the program during the training.
\7 To safeguard the graduates' privacy, we use only masculine pronouns in this report.
\8 ACNP is a professional association of about 600 scientists from disciplines such as behavioral pharmacology, neurology, pharmacology, psychiatry, and psychology.
\9 VRI, Cost-Effectiveness and Feasibility of the DOD Psychopharmacology Demonstration Project: Final Report (Arlington, Va.: May 17, 1996).
PDP GRADUATES ARE WELL INTEGRATED INTO MHS ------------------------------------------------------------ Letter :3
PDP graduates are well integrated into MHS. They hold positions of responsibility, such as clinic or department head, and treat a broad spectrum of patients, including active-duty personnel, retirees, and dependents. They can prescribe medication from comprehensive drug formularies and have patient caseloads that are comparable to those of psychiatrists and other psychologists who practice at their clinics and hospitals. Although the graduates were initially supervised closely, all but two have been granted independent status, meaning that they are subject only to the same level of review as psychiatrists at their locations. However, although the graduates are currently well integrated, several experienced early difficulties being accepted at their locations.
PDP GRADUATES HOLD POSITIONS OF RESPONSIBILITY, AND MOST TREAT A MIX OF PATIENTS ---------------------------------------------------------- Letter :3.1
The nine program graduates remaining in the military at the time of our visits are serving as the chief of a clinic or department, suggesting the high professional esteem in which they are held. For example, one serves as the chief of an Army division mental health clinic, one as the commander of an Air Force mental health clinic, and another as the chief of a Navy hospital's mental health department. Serving as clinic or department chief includes performing administrative duties, such as supervising other mental health providers and managing the day-to-day operations of the clinic. The one graduate who left the military did not serve as clinic or department chief during his year of post-PDP service.
Although PDP guidance limits graduates to seeing patients between the ages of 18 and 65, most graduates see a mix of patients, including active-duty personnel, retirees, and dependents. Two graduates serve in clinics that treat only active-duty personnel, and one serves in a clinic that treats primarily active-duty personnel but also treats dependents when mental health providers are available. The remaining seven treat a mix of active-duty personnel, dependents, and retirees.
PDP GRADUATES PRESCRIBE FROM COMPREHENSIVE DRUG FORMULARIES ---------------------------------------------------------- Letter :3.2
To guide medical facilities when granting prescribing privileges to the program graduates, a suggested drug formulary listing psychotropic drugs by name was created as part of PDP.\10 Six of the 10 graduates are assigned to facilities that granted the graduates drug formularies that are at least as comprehensive as the drug formulary recommended for them. The remaining four graduates have formularies that lack some drugs listed on the suggested formulary but contain additional drugs not listed on the suggested formulary. Although these four graduates' formularies do not include all drugs on the recommended formulary, none noted that this lack of some drugs reduced their effectiveness in providing patient care.
Some graduates' authority to prescribe is broader than others'. While four of the graduates have formularies consisting of lists of specific drugs they can prescribe, five have formularies listing classes of drugs from which they can prescribe. Formularies listing drugs by class, rather than by name, allow the flexibility to prescribe a new medication if it falls into a class of drugs already authorized. Otherwise, the graduates have to petition to have the new drug added to their authorized drug formulary. One graduate's formulary is even more flexible, granting the graduate broad authority to prescribe psychotropic drugs and their adjuncts.\11
-------------------- \10 Although all graduates received training in the use of psychotropic drugs to treat mental disorders in patients, they may not prescribe medications until granted prescribing privileges by the medical facility where they are assigned. Each facility is responsible for establishing the list of drugs, or formulary, from which providers at the facility can prescribe.
\11 Adjuncts are drugs that are commonly used in the treatment of the side effects of psychotropic medications.
PDP GRADUATES' AVERAGE MONTHLY CASELOADS ARE COMPARABLE TO COLLEAGUES' ---------------------------------------------------------- Letter :3.3
Eight of the 10 graduates' caseloads are comparable to those of psychiatrists and other psychologists at the same location. (The remaining two graduates practice at locations without psychiatrists or other psychologists, so their caseloads could not be compared to other mental health providers'.) For example, one graduate sees an average of 47 cases per month--higher than both the average for other psychologists at the same location (40 cases per month) and the average for psychiatrists at the same location (30 cases per month). Another graduate--the chief of the clinic in which he works--sees between 60 and 70 cases per month. Although this is lower than the average of 100 cases per month seen by the psychiatrist in the same clinic, the graduate told us that 30 to 50 percent of his time is spent on administrative duties associated with his position as chief.
Variation in the graduates' average monthly caseloads--which range from 40 cases for one graduate to 185 cases for another--results in part from the graduates' locations and responsibilities. For example, the graduate with the lowest monthly caseload is stationed overseas and treats only active-duty personnel and their dependents who have been screened for suitability for overseas assignment. In addition, this graduate is the chief of the mental health department and of the hospital credentials committee and serves on the medical staff executive committee. Conversely, the graduate with the highest monthly caseload was the only graduate not serving as a clinic or department chief, allowing this graduate more time to treat patients.
MOST GRADUATES HAVE BEEN GRANTED INDEPENDENT STATUS ---------------------------------------------------------- Letter :3.4
Initially, all graduates received close supervision by psychiatrists, in accordance with guidance issued as part of PDP. For example, each graduate's supervisor reviewed the graduate's charts for patients receiving medication. Other elements of supervision varied but included observing patient sessions or meeting separately with patients; holding formal weekly meetings to discuss cases; and requiring written approval for either starting, stopping, or changing the dosage of medications. The level of supervision was subsequently reduced for all graduates, seven of whom were granted independent status--meaning that they are subject only to the same level of chart review as other providers at their location. Another graduate has been granted independent status for treating outpatients--the bulk of the graduate's caseload--but is supervised when treating inpatients. Granting these graduates full or partial independent status indicates hospital officials' belief that the graduates need no more supervision than do other prescribing providers.
The remaining two graduates have not been granted independent status. Officials stationed at one graduate's location told us that they had anticipated granting him independent status; however, before officials reevaluated his status, the graduate was transferred to a new location.\12 The second graduate serves at a facility that has a policy requiring continued supervision of all physician extenders (such as prescribing psychologists, physician assistants, and nurse practitioners) who prescribe medication, regardless of length of service or level of performance.
-------------------- \12 According to the graduate, hospital officials at the graduate's new location have not yet determined whether he will be granted independent status.
SOME GRADUATES EXPERIENCED INITIAL PROBLEMS WITH ACCEPTANCE ---------------------------------------------------------- Letter :3.5
While ultimately well integrated at their locations, some graduates experienced some initial difficulty in this regard. For example, a graduate from one of the first PDP classes waited 10 months at his initial location to receive prescribing privileges and waited another 3 months before treating a patient requiring medication. Another graduate told us he learned that certain drugs on his formulary had been eliminated only after being informed by a patient that the hospital pharmacy had rejected a prescription written by the graduate. However, both graduates have been reassigned to different locations, and both have been accepted at their new locations.
Some of the graduates encountered initial skepticism from supervising psychiatrists, primary care physicians, nurses, and hospital officials who were uncomfortable with the idea of allowing psychologists to prescribe drugs. For example, one graduate told us that a physician at his location was so opposed to giving him prescribing privileges that the doctor resigned from the credentials committee after these privileges were granted. One psychiatrist at another location told us that upon learning that he was assigned to supervise a PDP graduate, he contacted the American Medical Association to inquire about the ethical propriety of a psychiatrist serving as a proctor for a prescribing psychologist. However, nearly all of the physicians and others we spoke to told us that the graduates' performance subsequently convinced them that the graduates were well trained and knowledgeable. Several physicians also told us that they came to rely on the graduates for information about psychotropic medications.
GRADUATES ARE REPORTED TO PROVIDE GOOD QUALITY OF CARE ------------------------------------------------------------ Letter :4
Overwhelmingly, the officials with whom we spoke, including each of the graduates' clinical supervisors, and an outside panel of psychiatrists and psychologists who evaluated each of the graduates rated the graduates' quality of care as good to excellent. Further, we found no evidence of quality problems in the graduates' credential files.
The graduates' clinical supervisors have the most extensive knowledge about the graduates' clinical performance because they have been responsible for reviewing the graduates' charts, discussing cases with the graduates, and observing the graduates' interactions with patients. Without exception, these supervisors--all psychiatrists--stated that the graduates' quality of care was good. One supervisor, for example, noted that each of the graduate's patients had improved as a result of the graduate's treatment; another supervisor referred to the quality of care provided by the graduate as phenomenal. The supervisors noted that the graduates are aware of their limitations and know when to ask for advice or consultation or when to refer a patient to a psychiatrist. Further, the supervisors noted that no adverse patient outcomes have been associated with the treatment provided by the graduates.
External evaluators also provided information on the graduates' quality of care. In 1998, an ACNP panel composed of board-certified psychiatrists and licensed clinical psychologists performed a final evaluation of the graduates--interviewing the graduates, their supervisors, and other officials, and reviewing a portion of each graduate's patient charts. In its resulting report, ACNP described each graduate's location and role, discussed the results of interviews with the graduates' clinical supervisors and others, and discussed the results of patient chart reviews. In its report, ACNP stated that the graduates had performed well in all the locations where they were assigned, that they had performed safely and effectively as prescribing psychologists, and that no adverse outcomes had been associated with their performance.\13
-------------------- \13 During our review, we received allegations regarding certain graduates' performance from two individuals involved in overseeing or evaluating the graduates. In all cases, we reviewed available evidence and held discussions with relevant officials. In all but one case, we found that there was not sufficient evidence to support the allegations. In the one case, the hospital's chief of medical staff considered the issue insignificant.
GRADUATES' EFFECT ON READINESS IS MINIMAL ------------------------------------------------------------ Letter :5
Although the graduates have been well integrated and have been reported to provide good care, their effect on DOD's medical readiness could not be more than minimal. DOD has approximately 400 psychiatrists and 400 psychologists; granting prescribing privileges to 10 psychologists is unlikely to affect combat readiness. Further, because psychotropic drugs are not used extensively during combat, the graduates, if deployed in combat, would likely have little effect on readiness beyond their role as clinical psychologists. However, evidence we gathered suggests that the graduates have modestly enhanced the peacetime readiness of military personnel at their current locations.
GRADUATES ARE UNLIKELY TO NEED PRESCRIBING ABILITY IN WARTIME ---------------------------------------------------------- Letter :5.1
Many officials--including service readiness officials and field commanders--told us that the graduates would likely have little effect on readiness in combat because psychotropic drugs are not generally the treatment of choice in combat and thus prescribing authority would not be in great demand. Because none of the PDP graduates have been deployed to a combat zone, however, no data exist on the actual use of the graduates in wartime situations.
According to many officials with whom we spoke, the preferred course of treatment for combat stress is adequate rest, counseling, and a quick return to the front lines. Soldiers who require medication are generally evacuated to hospitals located away from combat areas. Psychologists' counseling skills can be valuable front-line tools to handle stress, although this can be accomplished without the special training given to prescribing psychologists. A service-level medical readiness official told us that the most effective techniques to minimize combat stress are proactive--that is, counseling troops upon their arrival in the combat zone to reduce their anxiety level before combat. According to officials, the social workers, psychologists, and psychiatrists who provide this type of proactive counseling have a far greater effect on the well-being of the troops in battle than those who treat personnel after combat stress has set in. This proactive approach does not require prescribing authority.
GRADUATES CONTRIBUTE TO READINESS AT THEIR LOCATIONS ---------------------------------------------------------- Letter :5.2
Although the PDP graduates' prescribing skills may not be needed in combat situations, the graduates reportedly improve medical readiness at their peacetime locations. According to officials, the graduates improve readiness by reducing the time that patients must wait for treatment or by increasing the number of patients who can be treated.
Before the graduates were stationed at their current locations, some patients requiring mental health care received both psychotherapy from a psychologist and drug therapy from a psychiatrist because psychologists had not been permitted to prescribe drugs. Patients who needed to see two providers for treatment could, according to officials, wait up to 3 weeks to get an appointment with a psychiatrist. Prescribing psychologists, however, can treat some patients needing drugs who otherwise would require an appointment with a psychiatrist. Since these patients see only one provider--their prescribing psychologist--the time and effort needed to receive treatment is reduced.
Other benefits may accrue as well. For example, one official told us that when only a portion of the units in his division--which is staffed with a psychologist and a psychiatrist--get an order to deploy, the division has to consider which providers should remain at the division's permanent location so that the division as a whole has adequate medical support. In the past, if the division decided to deploy its psychiatrist, the permanent location would be without a prescribing mental health provider. Having a prescribing psychologist enables the division to deploy one prescribing provider while keeping another at the division's permanent location.
The graduates may also contribute to medical readiness through the care of dependents. According to several officials with whom we spoke, personnel who are worried about whether their family members are receiving adequate care may be affected in their ability to carry out their duties. One official told us that the PDP graduate in his unit--who primarily treats dependents--contributes to readiness in this manner. Because the facility did not have enough psychiatrists to care for dependents before the graduate was assigned to this location, those who needed to see a psychiatrist were referred to civilian psychiatrists in a nearby city. According to this official, many dependents did not seek care from these psychiatrists because they could not afford the copayment. The PDP graduate gives the facility the additional capability to provide care to dependents without charging them. The official believes that, consequently, more dependents seek and receive the care they need and fewer active-duty personnel worry about their family members' treatment.
PDP GRADUATES ARE MORE COSTLY THAN TRADITIONAL PSYCHOLOGIST AND PSYCHIATRIST MIX ------------------------------------------------------------ Letter :6
We project that DOD will spend somewhat more on its 10 prescribing psychologists than it would have spent on providing mental health services using the traditional mix of psychologists and psychiatrists. When all DOD expenditures for various mental health care providers--including salaries and acquisition, training, and retirement costs--are averaged over the length of time the providers are expected to serve, the average yearly cost of a PDP graduate is about 7 percent higher than that of the combination of psychologists and psychiatrists who would have provided treatment similar to that provided by the graduates.\14
Adapting a methodology developed by VRI,\15 we analyzed and compared DOD's costs for providing salaries, training, retirement pay, and other career-related benefits to military clinical psychologists, prescribing psychologists, and psychiatrists. We found that mental health providers' overall yearly costs to DOD are not identical. Of the three types of providers we analyzed, the costs for military psychiatrists are the highest--in part because psychiatrists receive more yearly pay than military clinical psychologists or prescribing psychologists. The PDP graduates' costs are the next highest and are considerably more than clinical psychologists--primarily because the costs involved in training the graduates and evaluating them (including evaluations by ACNP and VRI) far exceed the training costs for clinical psychologists.
Considering all career-related costs, we project that, on average, the PDP graduates will each cost DOD about $9,700 per year--or about 7 percent-- more than the cost of the combination of psychologists and psychiatrists that would be used to treat patients in their absence. Appendix II describes our analysis in more detail.
-------------------- \14 Other physicians--such as family practice and internal medicine doctors--also prescribe psychotropic medications. However, psychiatrists are the only physicians included in our analysis.
\15 VRI previously evaluated PDP, under contract to DOD. We updated VRI's model with more current information.
AGENCY COMMENTS ------------------------------------------------------------ Letter :7
In comments received April 26, 1999, responding to a draft of this report, the Executive Director of DOD TRICARE Management Activity stated that DOD agreed with the report and had no further comments.
Copies of this report are being sent to Representative Floyd Spence, Chairman, and Representative Ike Skelton, Ranking Minority Member, House Committee on Armed Services; and to the Honorable William Cohen, Secretary of Defense. Copies will also be made available to others upon request. If you have any questions about this report, please call me at (202) 512-7101 or Ronald J. Guthrie, Assistant Director, at (303) 572-7332. Other major contributors to this report are Steve Gaty, Sigrid McGinty, and Arthur D. Trapp, Senior Evaluators; and Timothy J. Carr, Economist.
Stephen P. Backhus Director, Veterans' Affairs and Military Health Care Issues
OBJECTIVES, SCOPE, AND METHODOLOGY OF OUR REVIEW =========================================================== Appendix I
The objectives of our review were to
-- describe how the 10 Psychopharmacology Demonstration Project (PDP) graduates have been integrated into the Military Health System (MHS);
-- obtain information on the quality of care they provide to military personnel, dependents, and retirees;
-- determine their effect on medical readiness; and
-- assess the cost-effectiveness of the PDP graduates.
To address the first two objectives, we visited the current or former duty locations of nine of the graduates and contacted the remaining graduate, who is stationed overseas, by telephone. At the locations we visited, we also interviewed the graduates' clinical supervisors, the hospital commander or designee, and various other clinicians and personnel to obtain information about the graduates' performance and level of integration.
Lacking a uniform definition of integration, we used several measures of how the graduates were used in order to assess their integration. We obtained information on each graduate's current position and role, scope of practice, drug formulary, average monthly caseload, and level of supervision received. We also reviewed the graduates' credentials files and performance reviews. We contacted all the members of an American College of Neuropsychopharmacology (ACNP) panel that performed a 1998 review of the graduates to obtain their views about the quality of care provided by the program graduates. We analyzed ACNP's May 1998 report and the report's supporting documentation, as well as prior ACNP evaluations of PDP.
To collect information on the PDP graduates' impact on medical readiness, we spoke with officials from each of the services and from the Office of the Assistant Secretary of Defense (Health Affairs), as well as officials at the graduates' locations. In addition, we reviewed DOD's Medical Readiness Strategic Plan to determine the role of MHS in supporting DOD's medical readiness.
To assess the cost-effectiveness of the graduates, we used a model developed by Vector Research, Inc. (VRI), under contract to DOD. Using updated data and assumptions, we calculated the life-cycle costs of the graduates, as well as those of other DOD psychologists, psychiatrists, and other physicians, and compared the annual life-cycle costs of these providers to determine the cost of the graduates relative to that of other providers. Appendix II provides a more detailed description of the model and the assumptions we used in calculating life-cycle costs.
ANALYSIS OF PDP GRADUATES' COSTS RELATIVE TO THOSE OF OTHER DOD PROVIDERS ========================================================== Appendix II
This appendix presents the methodology, data sources, and principal assumptions we used to calculate the career costs of military psychiatrists, psychologists, and prescribing psychologists. It also discusses how we compared the costs of prescribing psychologists to those of these other mental health care providers. Our analysis builds on a 1996 VRI study, in which VRI compared the cost of various types of military health care providers to the cost of a prescribing psychologist and assessed the relative cost-effectiveness of training the psychologists to prescribe medication and having them deliver this service in MHS.\16
For the purposes of this report, we have updated and extended the VRI analysis, most notably by
-- revising the figures used by VRI to represent the costs involved in training the prescribing psychologists and
-- estimating the career length of the graduates who currently remain in the military, based on their career length to date, and calculating their career costs.
Except where noted, the data we used--such as military pay rates and health care costs--were provided by VRI. However, we did not verify the accuracy of these data.
-------------------- \16 Other tasks in the study included identifying impediments to integrating prescribing psychologists into MHS and evaluating the potential roles and functions of prescribing psychologists in DOD.
COST ANALYSIS ------------------------------------------------------ Appendix II:0.1
DOD uses several types of providers to deliver mental health care, including psychologists, psychiatrists, family practice doctors, and internal medicine doctors. However, their career-related costs--including salaries, training, and retirement pay--are not identical and are generally lower for psychologists than for these physicians. For example, psychologists are not eligible for all special payments above salaries that physicians may receive.
We calculated the average career costs of the graduates and other providers and compared them to one another, using costs based on the anticipated career length and overall cost to DOD of the PDP graduates and other providers. Most PDP graduates spent a part of their military careers as clinical psychologists (before they entered PDP) and part of their military careers as prescribing psychologists (after they entered PDP). For comparison purposes, we assumed that the mental health services provided by PDP graduates as prescribing psychologists are comparable to those provided by psychiatrists\17 --that is, they are trained to perform a function (prescribing psychotropic medication) that psychiatrists would have to perform in their absence.\18
Because a PDP graduate's career, on average, is a combination of the functions performed by psychologists and psychiatrists, we compared the portion of a PDP graduate's career spent as a psychologist (that is, before the graduate became a prescribing psychologist) to the yearly cost of a military psychologist, and we compared the portion of a PDP graduate's career spent as a prescribing psychologist to the yearly cost of a military psychiatrist. For example, one PDP graduate served about 10 years as a military psychologist before entering PDP and, since then, has served about 4 years as a prescribing psychologist--for a total of 14 years. Thus, the graduate spent 71.4 percent (10 years) of his practicing career in the military as a clinical psychologist and 28.6 percent (4 years) as a prescribing psychologist. The yearly cost of the graduate could then be compared to 71.4 percent of the yearly cost of a psychologist plus 28.6 percent of the yearly cost of a psychiatrist.
Another PDP graduate served 3 years as a military psychologist before entering PDP and has served 3 years as a prescribing psychologist, for a total of 6 years. Thus, 50 percent of his practicing career in the military was spent as a clinical psychologist and 50 percent was spent as a prescribing psychologist. As a result, the yearly cost of this graduate could be compared to 50 percent of the yearly cost of a psychologist plus 50 percent of the yearly cost of a psychiatrist.
The 10 PDP graduates differed in the length of time they had served as military psychologists before entering PDP, ranging from not having served in the military to having served 10 years,\19 with a mean average of about 4.5 years as military psychologists. Similarly, the participants can be expected to differ in the length of time each remains in the military as a prescribing psychologist. We calculated the average length of their projected careers as prescribing psychologists, based on the length of their military service to date and the rates at which DOD psychologists have historically left the military. Using these data, we project that each program participant will serve an average of 6 years as a prescribing psychologist after entering PDP (including service to date as prescribing psychologists). Thus, we expect the participants to serve an average combined career total of 10.5 years in the military as clinical psychologists and subsequently as prescribing psychologists: an average of 4.5 years (or 43 percent of their careers) as clinical psychologists, plus an average of 6.0 years (or 57 percent of their careers) as prescribing psychologists. The average yearly cost of the graduates can thus be compared to 43 percent of the yearly cost of a psychologist plus 57 percent of the yearly cost of a psychiatrist.
Our estimates of the overall cost of the various types of providers included
-- acquisition costs that DOD incurs when recruiting someone into the military;
-- training costs to provide DOD-sponsored training to military health care providers;
-- force costs, which cover basic pay and allowances (such as allowances for housing), special pay, miscellaneous expenses, and health care benefits over the course of an active-duty career; and
-- retirement costs, which include retirement pay and retiree health care benefits over the expected life of the retiree.
-------------------- \17 Some--including ACNP and the American Psychological Association--have pointed out that the graduates are not intended to replace psychiatrists. ACNP wrote, PDP was not designed to replace psychiatrists . . . and it did not do so. Instead, the program products' were extended psychologists with [the] value-added component prescriptive authority provides.
\18 Other physicians--such as family practice and internal medicine doctors--also prescribe psychotropic medications. However, their annual life-cycle costs are higher than those of psychiatrists, primarily because they serve shorter careers than psychiatrists and, thus, their overall costs are larger on an annual basis. Because psychiatrists' costs were the lowest of the physicians' costs analyzed, we used their costs in order to provide the most conservative comparison.
\19 Two PDP graduates entered PDP immediately upon joining the military.
DATA AND ASSUMPTIONS ------------------------------------------------------ Appendix II:0.2
Although our analysis resembles VRI's--and in most cases relies on VRI's data and assumptions--in several instances we used data or assumptions that differed from VRI's. These differences reflect our emphasis on incorporating data that reflect, to date, the actual costs and experience of the program as it was implemented by DOD, rather than VRI's projections of how the program might be implemented. We discussed these changes with a VRI official, who stated that while he disagreed with our estimate of the cost of classroom training, the assumptions we used in our calculations were reasonable given the history of the program. The remainder of this appendix discusses the major assumptions we made in performing our analysis and explains where and how our data or assumptions differed from VRI's.
DIFFERENT SCENARIOS ---------------------------------------------------- Appendix II:0.2.1
In calculating the cost-effectiveness of PDP, VRI used two case scenarios: start-up and optimal. Costs in the start-up scenario included the nonrecurring, fixed costs associated with PDP development and initial implementation, such as the cost of the external evaluation by ACNP, as well as other costs that VRI believed would diminish or disappear in the long run.
The optimal scenario represented PDP in a long-term, steady state during which no nonrecurring costs associated with program start-up would accrue. In this scenario, VRI set the cost of supplies and training to levels that indicate long-term efficiency.
In contrast to VRI, we did not project different scenarios because the program has been terminated. Instead, we used data that reflect, to date, the actual costs and experience of the program as it was implemented by DOD.
PRE-PDP SERVICE ---------------------------------------------------- Appendix II:0.2.2
VRI assumed that PDP participants would have at least 6 years of experience as military clinical psychologists when they entered PDP. However, we found that although the 10 PDP graduates served an average of almost 7 years in the military before entering PDP, on average only about 4.5 of those years were spent as a clinical psychologist. We did not include nonpsychologist years in our cost comparison.
VRI assumed that the yearly continuation rates--that is, the probability that a given provider will stay within a given service occupation during a given year--for program participants before entering PDP were identical to those for military psychologists, including some psychologists who leave the military each year after the first 2 years of service. In contrast, based on the experience of the program, we used yearly continuation rates that reflect the fact that no participants left the military before entering PDP.\20
-------------------- \20 The continuation rate used affects the length of service calculated by the model. Because annual costs depend in part on this expected length of service, different continuation rates will result in different annual costs.
PDP CHARACTERISTICS ---------------------------------------------------- Appendix II:0.2.3
VRI used two different estimates of class size, depending on the scenario. In the start-up case, VRI assumed that, on average, 3.25 psychologists would enter each PDP class, from which 2.25 prescribing psychologists would graduate. These numbers were based on the program experience at the time of VRI's report: 13 psychologists had entered the program and, according to a VRI official, it appeared that 9 would graduate. VRI set the retention rate during the program to reflect the assumption that 9 of 13 participants would graduate.
In the optimal case scenario, VRI assumed that, on average, 8.7 psychologists would enter PDP each year, while 6 prescribing psychologists would graduate. The continuation rate during the program was identical to that used in the start-up case.
However, of the 13 participants, 10--not 9--graduated from the four PDP classes. Consequently, we used an average of 3.25 (that is, 13/4) psychologists entering PDP each year and 2.5 (that is, 10/4) graduating. We set the continuation rate during the program accordingly. Further, in order to reflect the fact that 13 psychologists entered PDP--effectively leaving the services' clinical psychologist force for cost-comparison purposes--we used a continuation rate for clinical psychologists that differed slightly from the historical DOD rate to account for these psychologists.
Our estimates of the cost of training the graduates also differed from those used by VRI. For its cost model, VRI estimated the overhead costs associated with the program to be $2,890,343. However, based on ACNP's annual reports (some of which were not yet published when VRI conducted its study) and our interviews with the former PDP training director, we estimated the overhead costs to be about 14 percent lower at $2,474,578.
While our estimate of overhead costs is lower than VRI's estimate, our estimate of 1 year of classroom training at the Uniformed Services University of the Health Sciences (USUHS) is markedly higher than that used by VRI. VRI estimated the classroom training costs (which do not include the PDP overhead costs it estimated) for participants to be $39,969, based on its 1995 study of the costs of graduate medical education and on a survey of the costs of graduate medical education in the Washington, D.C., area. However, based on our previous analysis of USUHS costs,\21
we estimated the classroom training costs to be $110,028--or about 175 percent higher than VRI's estimate.
-------------------- \21 Military Physicians: DOD's Medical School and Scholarship Program (GAO/HEHS-95-244, Sept. 29, 1995).
POST-PDP SERVICE ---------------------------------------------------- Appendix II:0.2.4
To project how long the PDP graduates could be expected to serve as prescribing psychologists, VRI assumed no graduates would leave the military for the 2 years immediately following the program. VRI also assumed that the rate at which the graduates leave the military thereafter would be identical to the rate at which other clinical psychologists leave.
In contrast, our projections of the graduates' post-PDP careers were based on their actual length of service to date. Because all graduates completed at least 1 year of post-PDP service, we set the continuation rate for the first year after the program to 1. However, the yearly rate for the second year was set to 0.9, because only 9 of the 10 graduates completed a second year of post-PDP service. To estimate how much longer the graduates who are still in the military could be expected to remain in the military, we used information gathered during our interviews with the graduates (such as the graduates' future plans for military service) as well as historical continuation rates for DOD clinical psychologists. Based on these calculations, we estimate that the participants will serve an average of about 6 years as prescribing psychologists, including the productive portion of their training.\22 (We conducted a sensitivity analysis, described at the end of this appendix, to determine the effect this estimate had on our final cost estimates.)
VRI also assumed that the PDP graduates posed no more of a malpractice risk to DOD than any other mental health providers delivering the same treatment to the same types of patients. Further, VRI assumed that PDP graduates did not receive the special pay paid to psychiatrists and other physicians in the military, assuming instead that the salary for PDP graduates was identical to that for military clinical psychologists. We also used these assumptions.
-------------------- \22 In accordance with VRI's estimate, we assumed that PDP participants were not productive (that is, saw no patients) during the classroom portion of their training and were 50 percent productive (that is, were half as productive as fully trained clinicians) during the clinical portion of their training.
SUPERVISORY TIME ---------------------------------------------------- Appendix II:0.2.5
VRI estimated that the PDP graduates would require 5 percent of a supervisor's time for the remainder of their careers. However, based on our fieldwork, we reduced that estimate to zero. Although two graduates have still not been granted independent status, supervision of the graduates in general has been reduced significantly. For example, one graduate required about 1 hour per week (or less than 3 percent) of supervisory time during the first 18 months after the program; during the subsequent 18 months, this graduate has required about 0.5 hours per month (or less than 0.3 percent) of supervisory time. Eight of the graduates currently require less than 1 hour per week of supervisory time. However, not all supervisors were able to quantify the amount of time they spent supervising the graduates. Even when supervisors could quantify this time, it was often less than 1 percent, and as a result we used an estimate of zero to provide a conservative estimate of the cost of the graduates. Had we used a percentage larger than zero, our estimate of the PDP graduates' costs would have been higher. (We conducted a sensitivity analysis, described at the end of this appendix, to determine the effect this assumption had on our final cost estimates.)
RETIREMENT COSTS ---------------------------------------------------- Appendix II:0.2.6
Based on DOD figures, VRI calculated pension rates based on an average service time for military retirees of 22.5 years. However, our estimates of the graduates' expected length of service yield an average service time for retirees in this group of 23.8 years. In other words, the graduates who serve at least 20 years in the military--and are thus eligible to earn a pension--will likely have served an average of 23.8 years. We calculated retirement costs accordingly.
Further, since only some of the graduates' years of service before entering PDP were spent as military clinical psychologists and because some of the retirement costs for the graduates are associated with service as neither clinical psychologist nor prescribing psychologist, we believe it is not appropriate to include this portion of retirement costs in our cost comparison. As a result, retirement cost estimates for the graduates were reduced.
UPDATED COSTS ---------------------------------------------------- Appendix II:0.2.7
The data used in VRI's earlier calculations were in 1996 dollars. For our analysis, we updated the figures to 1999 dollars using the most recent estimates of the DOD medical consumer price index.\23
-------------------- \23 Neither we nor VRI discounted the costs included in these calculations. Discounting determines the present value of an amount of money that will be spent in the future. For example, a dollar paid by the government today is more costly than a dollar paid at some future date because it increases the burden of making interest payments on the national debt. See Office of Management and Budget, Guidelines and Discount Rates for Benefit-Cost Analysis of Federal Programs, Circular A-94 (Washington, D.C.: Office of Management and Budget, Revised Oct. 29, 1992).
RESULTS OF ANALYSIS ---------------------------------------------------- Appendix II:0.2.8
Table II.1 shows the results of VRI's calculations and our calculations.
VRI's Cost Estimates and GAO's Cost Estimates
Yearly life-cycle cost per full-time equivalent (1999 dollars) ---------------------- Provider group VRI total GAO total ---------------------------------- ---------- ---------- Psychiatrist $188,472 $188,472 Psychologist 96,819 92,703 Psychologist and psychiatrist 136,895 147,532 combination Prescribing psychologists (start- 133,942 \a up case scenario; graduating class size set to 2.25) Prescribing psychologists (optimal 120,463 \a case scenario; graduating class size set to 6) PDP graduates (based on program \a 157,226 experience) ---------------------------------------------------------- \a Not applicable.
VRI's estimates for the annual cost of the prescribing psychologists in both the start-up case ($133,942) and the optimal case ($120,463) were less than that of the combined psychologist and psychiatrist cost ($136,895). VRI concluded that the program was cost-effective. On the other hand, our estimate of the annual cost of prescribing psychologists ($157,226) was higher than that of the combined psychologist and psychiatrist cost ($147,532), by about $9,700.
Our estimate of the cost of the graduates is higher than VRI's because of the different data and assumptions we used, our estimate of the cost of the psychologists is lower than VRI's because we adjusted the psychologist continuation rate slightly, and our estimate of the combination of psychologist and psychiatrist costs is higher than VRI's because our estimates of the length of time the graduates served as military clinical psychologists and will serve as prescribing psychologists differ somewhat from VRI's estimates. Because the combination of psychologist and psychiatrist costs depends on the proportion of time the graduates spend as clinical psychologists and prescribing psychologists, differences in these proportions will result in different estimates for the combination of psychologist and psychiatrist.
SENSITIVITY ANALYSIS ---------------------------------------------------- Appendix II:0.2.9
To assess the influence that our assumptions of length of service and supervisory time had on the results of our calculations, we performed a sensitivity analysis on each of these assumptions. To perform each analysis, we varied our assumptions about length of service or supervisory time while holding all other values constant.
First, we performed a sensitivity analysis on our projections of the length of time the graduates can be expected to remain in the military. Using DOD's historical continuation rate for psychologists, we projected that the participants will serve for about 6 years as prescribing psychologists, including service to date. This resulted in our estimate that the annual cost of the graduates is about $9,700 more than the combined psychologist and psychiatrist costs used for comparison. If the participants were to serve for 7 years as prescribing psychologists, the estimated cost differential between the PDP graduates and the combined psychologist and psychiatrist costs is reduced to about $6,300. Projecting an average length of service of 8 years as prescribing psychologists reduces that differential to about $3,800; 9 years, to about $2,100; and 10 years, to about $800. Thus, given this program's experience, the graduates would not be less expensive than the combined psychologist and psychiatrist unless they served as prescribing psychologists for an average of more than 10 years.
In addition, because we could not precisely quantify the amount of supervisory time required by the graduates, we assumed in making our calculations that the supervisory time was zero. To determine the effect that this assumption had on our final cost estimates, we performed a sensitivity analysis using other estimates of supervisory time. First, we used VRI's estimate that the graduates would require 5 percent of a supervisor's time throughout their career. This assumption raised the estimated differential between the cost of the graduates and the combined psychologist and psychiatrist cost from $9,700 to about $11,800. Assuming 3 percent of a supervisor's time raised the estimated cost differential to about $11,000 per year; assuming 1 percent of a supervisor's time raised the estimated cost differential to about $10,100 per year.
*** End of document. ***