At the Indiana University School of Medicine (IUSM) in Indianapolis, physicians are more likely to be associated with their personal digital assistants (PDAs) than with lab coats and black bags crammed with reference books. For the past two years the medical school, acting on the recommendations of IUSM's Educational Technology unit, has made handheld devices as integral to med students' clinical education as stethoscopes.
Recently, the Liaison Committee on Medical Education (LCME) established a mandate that all medical schools must collect outcome data to demonstrate the extent to which its educational objectives are being met. Collecting this data from all students presents a significant data collection challenge. As we grappled with satisfying the mandate, we realized the potential benefits of adopting a school-wide PDA-based data collection system.
By investing in a standardized platform, we had an opportunity to rescue the university from the increasing financial and operational burdens of a paper-based system of data collection, storage entry and reporting. A streamlined data capture process is beneficial to the school because the information collected is of critical importance to the school's continued accreditation.
IUSM, the second largest medical school in the country with nine campuses throughout the state and an enrollment of 1,200 students, recognized the opportunity and began a comprehensive program to educate its students and faculty on the benefits of using mobile technology. Through the department of Educational Technology, the school took action to make sure that new initiatives were being incorporated into the medical curriculum. In 2002, our Educational Technology unit began introducing faculty and students to PDAs through the use of periodic exhibitions and lunch-and-learn sessions. Later, in 2004, based on the LCME mandate, our PDA program segued into the centerpiece for our Clinical Encounters Real-Time Tracking System (CERTTS).
In addition to providing our students with mobility and a faster, more reliable way to record clinical encounters throughout their required third- and fourth-year clinical rotations, we also wanted to grant them rapid access to vital reference materials. By supplying future clinicians with treatment information at the point of care, we facilitate learning opportunities. Our desired project outcomes — for students and the educational system — included faster access to clinical information, fewer data collection errors, reduced likelihood of lost data and a centralized collection and reporting process. In addition, the IUSM hopes to foster ideas for solving problems that future healthcare professionals will face as the need for accurate clinical and research information continues to grow.
For security — a top concern for all medical school IT departments — the system encrypts the transfer of data. The student log data focuses on clinical experiences or encounters and does not contain identifiable patient data. However, as a precaution, our system has been reviewed in light of HIPAA concerns and approved by our internal Compliance Office.
System development
In the latter half of 2003, we initiated a campaign on campus centered on the potential for IT solutions to satisfy LCME mandates while simultaneously investigating various hardware and software products. Based on our research, in 2004 the administration of IUSM established a requirement that all upcoming third-year medical students purchase a PDA. To maximize operability between the school and medical center and to minimize training, we decided to standardize students on the Palm operating system using Pendragon Forms Enterprise from Pendragon Software Corporation of Libertyville, Ill. as the framework for collecting data. The most important criterion used to make this choice was the need to collect signatures from doctors, nurses and residents to verify the students' clinical experience. An internal review was done in early 2004 to determine what type of system could satisfy this need.
At the time, Pendragon Forms was the only vendor we found that offered a practical way to electronically collect a signature that could be rendered as an image for viewing on the web. In December of 2006, IUSM again reviewed available technology. At this point, two other systems were determined as feasible — digital pens and OMR (Optical Mark Recognition) scanning. After obtaining vendor quotes, digital pens were over the budget allowed for the project. OMR scanning was viable, but required a return to paper. Logistically, the dissemination of paper forms was too difficult to manage for students as they rotate through clinics, hospitals and private offices statewide during their third year of medical school.
In early 2004, Educational Technology assumed the challenge of converting IUSM's paper forms to a digital system. When the system launched in June only a "raw" view of captured data was available. Clerkship directors could look at an individual student and see a list of clinical encounters in table form. After being exposed to these Web-based reports, directors were able to formulate ideas about what criteria would be used to evaluate student performance. In order to make custom reports available on the system, we met with each of nine clerkship directors and considered the specific needs of each.
Throughout the first year of operation, the reports were customized in consideration of these needs. By the end of 2004, clerkship directors had a custom report available that allowed them to quickly assess the clinical experience of each student rotating through their clerkship. The same report was available to the individual student. These reports are accessed through the IUSM's curriculum management tool, ANGEL, from ANGEL Learning of Indianapolis, Ind. All of the data is housed on a MS SQL database server, with reports coded in PHP and HTML.
Our next step was to select the device. Our choice to go with Pendragon Forms limited our device options since the company only supported Palm OS hardware in 2004. Today, this software supports Palm OS and Windows Mobile devices. For the first year of operation, we chose the Palm Tungsten C., a web site which facilitated the ordering, hardware return and warranty process that was proposed by CDW Healthcare of Vernon Hills, Ill., and was made accessible to all students through a link in the ANGEL system.
We kicked off the program in June 2004 with good results. Students can now record their patient encounters on a single, easily accessible device and transmit the data directly to a secure Microsoft Windows 2003 server. We maintain stations in the medical library and other buildings that allow students to upload data and access CERTTS reports at any time. Students also have the option of uploading their logs via a Wi-Fi connection. ANGEL, IUSM's course management system licensed through Angel Learning of Indianapolis, is used to centralize access to our CERTTS software downloads, documentation and reports. The middleware that links Pendragon Forms Enterprise data to the ANGEL system was designed and written by the department of Educational Technology.
To help students extend the value of their PDAs, our team regularly conducts training programs on how to use the handheld's on-board software and clinical reference software, in addition to the CERTTS software. Educational Technology has also negotiated a deeply discounted per-seat license for Epocrates Essentials clinical reference suite from Epocrates of San Mateo, Calif., and is currently licensed by IUSM for all third- and fourth-year medical students. When negotiating a discount for this type of product, it is important to mention the marketing opportunities available by exposing medical students to the product through two years of medical school. Although we stress to our students that there are competing products and we demonstrate other products in orientation sessions, they tend to use what is familiar and easily available.
System costs
The startup cost in the first year totaled $182,000. This cost included $300 per handheld, paid by 280 third-year students, and $98,000 paid by the school for server hardware ($5000), software ($20,000 one-time licensing fee) and Educational Technology staff salary for development, training and support ($73,000). The IUSM did not consider purchasing the hardware in advance and handing out to students. Instead, many students received their devices as gifts from family members, or used student loan funds to purchase the hardware. Recurring costs include a new round of handhelds each year purchased by the students and $52,000 paid by the school for server upgrades, continuing development, and a dedicated training and support position.
The return on investment is described best by the achievement of full compliance with the recent LCME mandate. The system design phase forced clerkship directors to evaluate the educational goals of their curriculum and establish baselines for measuring student's progress towards these goals. Now that the system is operational, the immediate availability of reports has allowed IUSM directors to modify clinical exposure during a rotation to ensure these goals are met. Custom reports defined by clerkship directors are coded by Educational Technology and made available to the directors at any time through the ANGEL system, and accessed via a Web browser.
What we learned
In the first year that this system was implemented, there was significant student resistance. Although the system was rolled out to support third and fourth year students, the IUSM administration made an exception that fourth year students would be exempt from recording clinical experience on the PDA. The exception took some of the load off of support personnel, and eased the transition to an electronic system. In retrospect, Educational Technology could have used one additional high-level developer, and one student support technician.
Now that the system has been in place for four years, these positions are in place. The amount of support time needed to make such a system work was originally underestimated. The heavy technical support load negatively impacted the time that was available for other Educational Technology initiatives until positions were established to handle PDA support needs.
Changing with the times
To make the most of the technology's ongoing improvements, we moved to the Palm TX in June, 2006 to take advantage of its built-in wireless and flash memory capabilities. Looking ahead to the Fall of 2007, we will be supporting the Palm OS-based Treo line of smartphones.
By mandating that medical students purchase a Palm OS device at the end of their second year, Indiana University School of Medicine is making use of mobile information technology in ways that help us broaden the curriculum, comply with regulations on clinical encounters reporting, reduce errors and, most important of all, enhance the education of future clinicians.
Michael Bangert is the operations manager for educational technology and Amy Hatfield is digital initiatives librarian at the Indiana University School of Medicine, Ind.
