HRSA SPNS Grantees Meeting

San Diego, CA

January 7-9, 2002

Monday – December 7, 2002

Participants: Kari Maier, Veronica Salcido-Harding, Michelle Valverde, John Wiebe, Tony Chavez, Alisa Olshefsky, Maria Luisa Zuniga, Rosana Scolari, Steve Trujillo, Ken Burton, Barbara Estrada, Tony Estrada, Sandi Duggan, Yolanda Cantu, Bob Smith, Michael Tarter, Kurt Organista, Tim Brittingham, Herman, Curiel, David Barney, Betty Duran, Richard Moore, Richard Seaton, Pam Reid Duffy, Robert Castrillo, Melody Bacha, Heather Baldwin

Meeting Goals and Objectives:

Introductions by participants. Welcome by Mari & Rosana to San Diego. Review of Agenda by Herman. David asked that projects share lessons learned during their 15-minute presentation. He announced that George Huba will be in this evening and will attend meeting on Tuesday.

HRSA Update: Richard Moore

Richard thanked projects for efforts expended in last 18-months with implementing the demonstration projects. Acknowledged Evaluation Center for implementing multi-site evaluation. Noted that September 11^th Critical Incident has impacted services along the border, as well demonstration projects. Stated it is important to be aware that projects have been able to respond to the impact of the critical incident. HRSA has been working with victims and survivors in providing health care and homeland security, working on strategic planning for biological warfare threats, and with the Secretary of Health focusing on border health issues by visiting with providers in the border states. Within HRSA, Titles III & IV have continued to provide additional funding to support border projects. One example was the funding for a Bi-National Border Health meeting to be hosted by the Arizona Border HIV/AIDS Care Project. Today, SPNS is announcing the initiative for American Indians and Alaskan Natives which seeks to incorporate HIV services with emphasis on substance abuse and mental health. The initiative will also fund a technical assistance center to work with five grantees under this initiative. He introduced Richard Seaton who is working with HRSA and grantees to develop products (reports) on issues of border health. Richard Seaton is a marketing expert and writer with many years experience in HIV/AIDS. Mr. Seaton will be available to meet with each project during the next two days. He will work with projects on disseminating what they have learned to the larger community and funding sources. He will looking a highlighting specific issues for each project. Richard Moore also introduced Melody Bacha of the HRSA San Diego office.

Dissemination Committee: Tony and Barbara Estrada

Tony referred to pink sheet previously distributed at Grantees meeting with HRSA research questions. Tony and Barbara Estrada presented a PowerPoint presentation. Tony began with stating that this an opportunity for grantees to brainstorm about potential dissemination articles. Michael Tarter suggested that grantees select a topic or issue they want to pursue as soon as possible, then concentrate their efforts on developing the data to support their publication. Tony advised that it is also important to determine what is an adequate sample size for the topic selected. Mari said benefit is that no one else has worked in this arena and thus we can really take the initiative to properly disseminate the finding on the projects efforts. Tony said Outreach is an area of focus in which we can dissemination information on what it takes to recruit participants, work with the individual to agree to testing, and when diagnosed, what it takes to get the individual into HIV treatment. He advised it is important to consider what type of journals would be receptive to publishing the group’s articles. Michael suggested that papers presented in conference proceedings can also become published papers as well. He further suggested that continuous variables such as CD-4, viral loads, and medical outcomes can support reports that have small sample size.

Optimizing local service delivery systems, Question #3 - Mari said that the data collected to date indicate that clients are very pleased with the health care they are receiving. This poses some difficulties in analyzing the impact on health care. It was pointed out that cultural factors also impact how clients respond to rating services. Bob Smith stressed that we work with a very disadvantaged population, therefore they are hesitant to critic services for fear of loosing the services.

Providing Quality of Care – This is impacted with improvement in HIV treatment and increased length of life expectancy. Michael said that progression by gender may be an area that group may want to study. Women are typically diagnosed in later stages of the disease. Social Desirability Scale could be quite useful for use in assessing cultural competency by providers.

Other Research Questions/Issues: Tony informed the group that dissemination activities can focus on local issues as well as multi-site. Promotoras is a local evaluation issue that projects may also want to disseminate. Pam suggested that identifying resources in border communities, as well as who controls those resources, is very important to address in publications. Also impact of issues like border patrol located near clinics and its impact on access by clients. Melody suggested that dissemination on service models developed for border populations is also important. Another issue is correlation between quality of life and satisfaction with health care. Kurt suggested a literature review of border health issues and HIV. He suggested we could examine how project services compare with what is currently being provided and if the project outcomes are similar to other services being delivered, or if there is difference in response by clients to different services providers.

Timeline and Interest Groups – Requested from Richard Moore that Helen Schietinger’s HIV Initiative Report be provided to all Border grantees when completed. Richard anticipates her report will be completed by end of February 2002. Projects were encouraged to look at dissemination issues on their local model developed to address local issues. John Wiebe suggested that a descriptive paper could be the focus of the group to support a later seminal paper. Kurt suggested a review of the existing literature would also be beneficial (Kurt has already begun this activity). John emphasized that a paper with client demographics with charts of on client demographics, barriers to care, risk factor data, and other specific issues could be included. Also critical issues such as border crossing time delays as affected by 9/11 incident. Baseline descriptive paper would be first initiative for dissemination. John said addressing unmet needs could be compared to existing national data. Tony advised that first descriptive paper would require approval from dissemination committee. Betty suggested that one representative from each project and one from evaluation center be appointed on a committee to begin work on a descriptive paper. Persons appointed were Yolanda Cantu – VAC; Michelle Valverde – NM, Maria Luisa Zuniga – San Ysidro; John Wiebe – Centro de Salud Familiar La Fe, David Barney – Centro de Evaluación; and Tony Estrada – Arizona Broder HIV/AIDS Care Project. Betty will coordinate a conference call for designated committee when instructed by Tony.

Richard Moore asked if grantees had planned to present at the All Ryan White Titles Grantees meeting in Washington, DC in August. Betty advised that Barbara was informed that all border grantees desired to attend the meeting, then follow-up with our regular SPNS grantee meeting while in DC. Richard advised HRSA was interested in having the group present at this meeting and will follow-up when he returns. Division of Training and Technical Assistance are lead for putting the conference together. Tenative dates are August 19-22, 2002 in Washington, DC.

US/Mexico Bi-National Border Health HIV/AIDS Conference: Richard Moore and Steve Trujillo

Richard advised that HRSA has issued a grant to El Rio Health Center to conduct a Bi-National Border Health conference during the week of September 9, 2002. Steve advised that providers, clinicians, community representative, and state officials from both sides of US/Mexico border will be invited to the conference to discuss health disparities, community development, collaboration, and addressing local needs of border populations. This is an opportunity for border grantees to collaborate with Arizona to develop a comprehensive meeting along with El Rio Health Center. Ken said this would be a follow-up to a border conference hosted by El Rio two years ago. Tentative plans are to develop this into an on-going bi-national activity that will sustain the energies coming out of this conference. Steve advised they are looking for conference sites in Tucson. Anticipate at least 300-400 participants will attend the conference. Melody stated that information shared by Mexican providers and government representatives is very valuable and she hopes they will be provided ample time to share their information at this conference. El Rio would like a representative from each project to serve on the steering committee, as well as a state health representative. El Rio is also seeking funding from state agencies to support the conference expenses. They are presently working with Mexican officials to assist them in contacting potential participants from Mexico. A representative from Mexico will also serve on steering committee. Ken advised that some funds will be available to defray expenses for speakers who have limited resources, as well as scholarships for other participants. El Rio is seeking additional resources to support many of the associated expenses for the meeting. Steve asked grantees to share their mailing lists with El Rio in order to help them identify other potential participants. Translation and interpretation equipment are also essential for this meeting. Primary medical care, auxiliary services and research tracts (2-tracks) will be topics addressed in presentations during the conference. Pam suggested that the National AETC faculty should be included as presenters at this conference. Stated that many of these faculty are leaders in their fields and by inviting them, could reduce costs for honorariums. Steve advised that it is also a desire to have publications come out of this conference, such as conference proceedings. Mari cautioned the group by stating that any information presented at this meeting and published in a conference proceeding, potentially cannot be published in a refereed journal. Therefore, she suggested we think seriously about what is disseminated. Michael suggested involving some professional societies to support the conference and related expenses, this would also add credibility to conference proceedings. Steve is also interested in including AETC regional offices as part of the planning process. UNIDOS will meet in November and we may find ourselves competing for the same key speakers who may have already been invited to attend the UNIDOS conference (border and health issues). UNIDOS hold bi-annual meetings. Representatives appointed from the Border projects are Bob Smith, Veronica Salcido-Harding, Tony Chavez, Heather Baldwin, Herman Curiel, Wayne Salseda, Robert Castrillo and Steve Trujillo.

Training Strategies that work – clinic shadowing: Dr. Pam Reid Duffy and Robert Castrillo

Pam - Advised there is no existing model that explains the complexity of services of HIV along the border. Arizona is working at developing a co-management model for primary care in collaboration with multiple service providers. They are not using a expertise approach, but working as a partner learning about the needs and issues that must be addressed in providing HIV primary care in a border community. Their role is to learn from the border providers as to what the AETC must teach to meet their medical training issues. As trainers they are responsible for determining who are providers in the community and how to meet their knowledge needs. Often providers are not professional staff by lay people from the community who are the stabilizing staff in community services. Training is designed based on local need and not based on how AETC perceives everyone should provide the service. A principal goal is to engage local providers to participate in training and provide them tools and supplies (example was the blow fish which typically is given to doctors, but AZ AETC provides to promotores and other lay workers). Important to increase comfort level of border providers.

Robert – AETC model for clinic shadowing is to get a low volume provider to work in a high volume setting, allowing the provider to observe and practice HIV medical care. Important for doctors who do not have extensive experience in treating HIV patients. Has been used with HIV case managers and promotoras to learn how to speak with their clients about issues such as impact of travel to access medical care, how responding to treatment regimens, and other issues which affect their ability to respond to care. When shadowing, it is important to inform providers that person observing clinical care is there to learn and not to translate for the provider or perform other duties. Have found it very beneficial to observe how the trainees provide their services in the local setting and the type of conditions they must work in. Communities have raised concern about always participating in needs assessments but never receiving feedback or support to address issues identified in the assessment process.

Rosana shared with the group a problem that occurred when the promotoras were provided data modules that needed to be completed with outreach contacts. When San Ysidro staff conducted a follow-up visit with the promotoras to inquire why some data points were not being completed in the module, they learned that the promotoras were not comfortable in asking specific questions which they perceived as intrusive. The resolution was to train the promotoras on how to ask the questions. This resulted in a substantial increase in the number of completed data modules by promotoras. Mari said it was also important to provide promotoras an option to note on the modules that “Client is not comfortable in responding” which also allowed more modules to be properly completed. Robert said another critical issue to acknowledge is that promotoras have been conducting health promotions in their communities for many years and must be listened to when they make recommendations or express concerns on specific issues that affect their ability to conduct their outreach.

A few pointers shared with the group were: 1) Respect who your are training regardless of level of education. 2) Flexibility – think outside the box. 3) Focus beyond just the professional worker. 4) Prevention and Intervention must both be addressed. 5) Looking at training as part of needs assessment, community dynamics. 6) Use people of community as co-trainers. 7) Training across professions – Example: medication use training - include case managers, pharmacist, physician, etc. 8) Keep lots of options open for new ideas.

Mari said there are many barriers promotoras must deal with in conducting outreach. An example was when they go into the agricultural fields, need to ask Patron permission to provide education to workers which may require explaining to Patron exactly what they will say and do.

Rosana shared with the group how as service providers, we are often not aware that support staff are not always comfortable in providing services to HIV positive clients.

Bi-national Migrant Health Initiative: Dr. Kurt Organista (Handout)

California Endowment has set aside 50 million dollars to support this initiative and funds are available to support any applications that would support or qualify under this initiative. California Endowment is looking at working with Rockefeller Foundation that would enable them to provide funding to agencies on both sides of the border that would enhance migrant workers health status. Presently, the California Endowment is prohibited from providing funding outside the US.

Tuesday – January 8, 2002

Participants: Rosana Scolari, Maria Luisa Zuniga, Alisa Olshefsky, Tony Chave, John Wiebe, Michelle Valverde, Veronica Salcido-Harding, Kari Maier, Steve Trujillo, Ken Burton, Barbara Estrada, Tony Estrada, Richard Moore, George Hube, Richard Seaton, Robert Castrillo, Armida Ayala, Healther Baldwin, Reymundo de los Cabos, Terri Whittaker, Bob Smith, Yolanda Cantu, Sani Duggan, Michael Tarter, Kurt Organista, Tim Brittingham, Herman Curiel, David Barney, Betty Duran

Medical Chart Review Planning: George Huba (power point presentation & book on Broward County Study)

Outline of power point presentation

1. What charts are reviewed - Projects need to set criteria as to when client should have been in medical care in order to be included in medical chart reviews.
2. Time Interval to be used
3. Chart reviewers
4. What is coded – includes a time series on each client
5. Coder Reliability and Validity
6. Types of Data – Encounter level data (time series) – Patient level data (relative time)
7. Time Linking Period
8. Definitions of Success – Drop in Viral Load – Increase in CD4 Counts – Delay in or absence of Opportunistic Infections
9. Comparison Groups
10. Internal Comparisons
11. Caveats and Pitfalls – New HIV drugs – Sensitivity of lab tests – Differing viral loads
12. Data Massaging is a critical step – manipulation to get into correct form for analysis
13. Other Considerations – need a good history of treat form

Broward County Study:

• Five primary medical providers
• Major questions to be addressed need to be determined
• Design considerations – type of charts and types of data to be collected
• Initial Assumptions and Boundaries – Therapies and evaluation vs. research
• What this Evaluation study can do
• Results on Services provided – Time trends in services provided
• Results on Patient Outcomes:

*Overall patient change in medical status (required that client been seen at least during 2-quarters in a 4-quarter period)

*Effects of Combination Therapy on clinically significant change

* Estimated cost for chart review and analysis was approximately $125,000 of which about half went to medical chart extraction

Design of Study:

· Eligibility Criteria

· Participate Providers

· Data Points at Intake

· Data Points each Quarter

· Patient Characteristics

· Caveat on Co morbidity Data

1. Homelessness

2. Substance Abuse

3. Psychiatric Diagnoses

4. Hepatitis

Two tasks that grantees may want to assess:

Do you see improvement at least one time in client chart?

Comparison of “optimal” level of CD4 and viral loads – changes over time

Handouts:

1. HIV/AIDS Medical Services and Outcomes in the Broward County Ryan White Title I Eligible Metropolitan Area
2. Standard dosing chart for anti-HIV drugs
3. FW: HIV drugs (follow-up information to Broward Study)

David advised grantees that cost for medical chart reviews will be paid by the University of Oklahoma. Projects need to obtained client consent forms prior to scheduling chart reviews. Steve advised that some medical charts are located at clinic sites that may be 3-4 hours from the main clinic, therefore, the consultant will need to travel to the rural clinic to review charts or arrangements will need to be made to have the charts transferred to the main clinic for one or two days. Medical records personnel are critical, therefore they should be included in coordinating all chart reviews. Two projects anticipate that local CHCs may be resistive to allowing access for chart reviews. Rosana suggested that a letter signed by HRSA’s Bureau of Primary Health, and the local project be mailed to each CHC advising them that medical chart reviews are an evaluation requirement that they must support, could be beneficial. Richard Moore said he would speak with Joe Baldi about such a letter. Mari said their IRB included approval of medical chart reviews. David will send OU IRB letter of consent to all projects. Kari suggested that letter from Bureau of Primary Health Care list specific dates for chart reviews. Mari said that the projects provide a client URN number on the signature line of the consent form versus a real signature in order to maintain client confidentiality. David said OU must have an original consent form signed by the client. George said that he did not require client consent for the chart reviews in Broward County due to the fact that the review is intended to enhance medical care, while client confidentiality is still maintained. George suggested that hiring a licensed medical consultant and with local state law permitting, then the projects could potential not require client consent. Typical chart review of 10-quarters takes approximately 2 hours, with most taking only one hour. George recommended that some line of communication be established with medical records staff to inform them of the process and obtain their participation/support of the client chart reviews. Mari suggested that the consent be obtained and kept in the medical chart, with a block out signature copy with sent to OU. Betty advised that the blocked out signature copy would require that a staff person sign a statement on the form stating the original signature is on file in the client’s medical chart. Bob recommended that projects include in the contract with the CHCs a statement requiring them to allow medical chart reviews in Year 03 and Year 05. John Wiebe stated their consent form allows the local project to have access to client medical charts. He asked it would be possible for OU to contract with local projects to conduct the chart reviews? The local project would then need to have the data collector trained by OU. Richard said if the group agreed to have the local projects conduct the chart reviews, then the supplement funds allocated for this effort would be awarded to the demonstration sites versus being channeled through OU. Betty will aide projects with projecting anticipated costs for conducting the chart reviews locally. The budget projection will then be made available to Barbara Aranda-Naranjo and other HRSA staff for budget planning. John made the motion that OU contract with the local projects to conduct the medical chart reviews. He added that OU identify the nurse consultant who the projects would hire under contract to conduct the chart reviews locally and that the nurse consultant be trained by OU and data collected would go directly to OU. Motion was approved by all grantees. Costs per project will be based on a per chart cost. Local consent will only be required. Data forms will be mailed to OU for data entry. Nurse consultant travel and training costs must be included in budgeting local expenses for contract. Barbara asked if it would be best to have local projects key in data, then send data files to OU. George suggested centralized data entry be set-up in order to have better control over quality control. Final decision: local projects will copy data forms and submit copy to OU for data entry. Motion passed. Mari said it would be helpful to provide their partner sites with an idea as to what data will be collected and with a possible timeline. George suggested that each project identify a physician or nurse who would participate in development of data modules for chart reviews who is also a gatekeeper to the medical systems which have the client files.

Variables for chart review: (form is 4-pages) includes service dates by quarter; list all pharmaceuticals; all CD4 counts with dates taken, etc; all viral loads and methods; all opportunistic infections with dates; and other major events such as drug abuse, etc. The Broward Report includes a list of all suggested variables in Appendices G, page 175. David has copies of actual instruments that will be sent to all grantees.

Project agreed to attempt to locate a local nurse consultant who project will hire to conduct all medical chart reviews locally. All projects will send OU paper data copies. Nurse consultant will be trained by OU. George advise that The Measurement Group often trains the nurses via telephone. Each local site will designate a local medical staff person to assist with overseeing the chart reviews locally. This person will also educate the nurse consultant on local standards and policies.

Supplemental Handouts by The Measurement Group:

· AETC modules

· Health Care Provider Characteristics and Perceived Confidence from HIV/AIDS Education. AIDS Patient Care and STDs, Vol 14(11), 11/11/00

· Systems Change Resulting from HIV/AIDS Education and Training. Evaluation & The Health Professions, Vol 22 (4), 12/99.

· Do Characteristics of HIV/AIDS Education and Training Affect Perceived Training Quality? Lessons from the Evaluation of Seven Projects. AIDS Education and Prevention, 12(5), 2000.

· Trainee Characteristics and Perceptions of HIV/AIDS Training Quality. Evaluation & The Health Professions, Vol 23(2), 6/2000.

· Effects of HIV/AIDS Education and Training on Patient Care and Provider Practices: A Cross-Cutting Evaluation. AIDS Education and Prevention, 12(2), 2000.

San Ysidro Focus Group Update: Maria Luisa Zuniga (power point – posted on web page)

Acculturation Scale: Tony & Barbara Estrada (power point – posted on web page)

Comments: Syntax file posted on web page is to be revised. Barbara demonstrated measurement using acculturation scale using Arizona client data from Module B. Variable 223 on Module B is not part of acculturation scale.

Bob asked if projects should not also consider assessing cultural competency of provider to determine if they are responding to acculturation of clients. Some competency is assessed in the local AETC modules used by Arizona in regards to medical providers. Tony informed that methodological issues need to be considered in guiding what the local projects want to assess in regards to acculturation. The group will need to seriously consider how the data is analyzed and the findings reported.

David reminded the group to remember not to run analysis on original data as SPSS reverses the codes. Always use a copy of your data set. Barbara demonstrated an analysis run for acculturation only on Hispanic clients. Two sub-scales are included in the acculturation scale, language and ethnicity. Barbara will send David analysis sample in which Tony & Barbara have used 3 sub-scales in past studies to assess acculturation. Mari and Barbara will develop a summary sheet and out examples on how to interpret analysis to be post on web page.

PROJECT REPORTS: All reports presented in power point – all posted on web page. Order of presentation: Arizona - New Mexico - El Paso – California - Valley AIDS Council

Multi-site data report: David Barney – power point presentation – posted on web page

Data Quality – Tim advised data submission has been good, has received some duplicate cases which he checks to ensure that they are duplicates, he eliminates one of the cases and informs project. Tim advised that forms created in TELEform cannot be modified when a site wants changes. Changes must be developed in a new format in order to avoid data entry errors. Quality Assurance is difficult to maintain at OU but requires the support of local projects to ensure that case information is correct by running a local frequency report.

David reviewed data by modules as post on the web page. Asked grantees to periodically check the web and returned data reports to confirm that the client numbers are correct. Showed grantees where the SPSS Syntax file is located on web, as well as information from the Measurement Group. Demonstrated where all grantee presentations are posted.

Reviewed Upcoming Events – International AIDS Conference abstracts due by January 21, 2002. Advised that HRSA is unable to approve participation/travel for an international conferences. US/Mexico Border Conference abstract is also due on January 15, 2002. UNIDOS is scheduled for November 2002. October 2002 National AIDS Update Conference in South Padre. All Ryan White Titles meeting in Washington, DC on August 19, 2002.

Review of frequencies for HIV positive cases. 126 cases post-SPNS and 293 pre-SPNS. Variable 126 – we have received revisions to previous data submission, but the numbers for presenting issues appears to be low. John asked if David could run a frequency by site for review at Wednesday’s meeting. Alisa asked if Module D responses affect the reporting of presenting issues. These figures could be affected by age of clients due to maturation. Need to consider a client at risk for re-lapse or recidivism for issues such as mental health, alcohol abuse, drug abuse, and methadone.

C Module need to be developed and would could issues such as discharge date, status of discharge, and length of time in program. The module could also include any other information grantees feel is important that may have been overseen in the other data modules. Other information relevant to discharge could also be considered.

Advised that new modules are in conference file for grantee review: Hispanic Stress, Health Services Accessibility Scale.

Wednesday – January 9, 2002

Participants: Tony Estrada, Barbara Estrada, Steve Trujillo, Ken Burton, Richard Moore, Richard Seaton, Robert Castrillo, Melody Bacha, Armida Ayala, Michael Reyes, Healther Baldwin, Reymundo de los Cabos, Bob Smith, Yolanda Cantu, Sandi Duggan, John Wiebe, Tony Chavez, Alisa Olshefsky, Maria Luisa Zuniga, Rosana Scolari, Kari Maier, Veronica Salcido-Harding, Michelle Valverde, Michael Tarter, Tim Brittingham, Herman Curiel, Betty Duran, David Barney

Herman welcomed everyone to the final day and reviewed the agenda for the day.

HRSA Semi-annual Report: Richard opened floor for questions regarding semi-annual report. Bob asked how he could request an extension. Richard advised her contact his project officer and request the extension citing reasons.

Variable v126: David Barney (showed cross-tab of variable).

• All projects equally assessed transportation issues.
• Most of our clients are having on the average, three treatment issues.
• Michael Tarter suggested that we “multiple comparisons” to show differences in v126.
• Highest alcohol problems reported by AZ, lowest reported by NM.
• Betty stated that studies report alcohol at approx. 70% for the general HIV/AIDS population, and approx. 40 – 60% for substance abuse. Tony Estrada stated that those studies do not reflect inferred problems.
• Betty stated that under reporting of var. 126 will be ongoing and we will need to continue looking at these variables and the factors that affect them.
• Mental health issues generally parallel national numbers.
• NM lowest drug abuse, AZ highest.
• CA highest in homelessness, VAC lowest. (Bob stated this figure is affected by residency requirements.)

Barbara Estrada suggested that we look at v126 and each project characterize why those numbers are where they are.

C Module Discussion:

Adan joined in on a call at 9:45am, with Lydia and Robyn. Betty reviewed where we were in the discussion.

• Bob agreed that we need the module to cover missing variables. (ie, pregnant women, death, etc)
• David Barney stated that is will include case status, and may include client profiles. Suggested that each project submit variables that they see missing in e-mail form, and we will begin developing the module. Decided to discuss the module on all projects conference calls. Betty advised that the evaluation center will reminder the projects to send proposed variables to be included in this module via e-mail.
• Michael Tarter suggested that transportation issues that affect client keeping appointments be included.
• Tony Estrada suggested that we capture border crossing times/delays.
• Issue raised that guidelines may vary by each program (ie, length of time before closing files)
• May need to develop two modules: one for exiting program, another for missing variables.
• Alisa Olshefsky stated concern that chart reviews will not allow for a full exploration of tracking the participation of the client.
• Module to be discussed in further detail and reviewed during grantees meeting in Washington, DC in August 2002..

AETC Evaluation Discussion:

Betty reminded grantees that salmon colored documents in meeting packet is information on AETC collected from all demonstration projects. David updated the grantees by reviewing history of AETC in regards to multi-site evaluation. Initially, the Evaluation Center and demonstration sites were advised that AETC was not a multi-site evaluation piece. Local site were to collaborate with the regional AETC office to assess local training efforts. Currently, HRSA would like to examine the overall impact on community capacity in providing HIV medical care. Discussion issues were as follows:

· Betty stated that AETC’s training and evaluation tools are not standardized across all regional programs. Types of training also vary, as well as who is eligible for training

· Identification of instruments currently used, as well as instruments used by The Measurement Group are included in packet for review and consideration

· David stated that there are methodological problems with this evaluation as it appears it would be program evaluation

· Heather Baldwin asked why the national evaluation center hasn’t been asked to do this evaluation. Lydia said that was a decision of Laura Chiver. Richard Moore stated that this is not an evaluation of the AETC, but of the training component of these models. Evaluation of the collaboration of the AETC’s and the programs.

· Heather Baldwin stated that before funding of Columbia, there was not standardized evaluation. Now Border modules number 7 and 8 are primarily duplicates of that standardized evaluation. Every AETC is supposed to use the standardized form. Sandi Duggan stated that they are using “PIFs.” Yolanda Cantu stated that physicians are using the PIFs and no other forms were identified for VAC’s use. Yolanda and staff developed their own local evaluation modules, using the question: Does increased capacity and knowledge of providers increase capacity of system? Stated things weren’t clear. Lydia stated that border projects were not evaluating the AETC.

· Adan Cajina stated that projects are not doing program evaluation, but wanted to measure impact of training and we should stay away from AETC. Need of clarification of goals and objectives and review of the instruments. Ledia Martinez stated that HRSA is not dictating what the projects should do. None of the current instruments evaluate community capacity.

· Robert stated that AETC’s border providers show resistance to new forms(esp. evaluating training) and how will that effect them? Richard Moore asked if current forms covered our concerns. Heather said evaluating impact has been dealt by national AETC evaluators and we are grappling with the same issues.

· John Wiebe asked if were interested in evaluating training component of our local model? All agreed to “yes.” Suggested to have each project develop that evaluation of their own model. David stated that HRSA wants this standardized. Richard Moore reaffirmed.

· Sandi Duggan reviewed development up until this point. AETC is locally imposing their evaluation of training two years later. Nationally she see’s something else.

· Bob Smith suggested adding a paragraph on semi-annual report address AETC evaluation at the local level. Ledia Martinez wants measurement of direct impact at the system level, and Bob’s suggestion would not capture this. Bureau of Primary Care would like to have this assessed at systems level, then is successful, be able to replicate it nationally.

· Sandi suggested that we compare evaluation at each project, then develop a common instrument.

· Robert read letter from Pam Duffy, opposing standardized forms and in favor of building partnerships. Complete discovery process first, then proceed from there.

· Heather gave a brief over-view of AETC. She stated every site is different. Different faculty funded at each site, and very different training across the nation. Each have different capacity. AETC goes into community to solicit training and community also comes to AETC.

· David stated we are looking at impact on system and current modules do no measure that effect.

· Richard Moore restated that the “paragraph on semi-annual report” would not be adequate. He asked if we could develop an instrument that measures impact at each site?

· Michelle Valverde stated that after review of Huba’s materials, she anticipates it would take a great amount of time, and would be task intensive. The interview module is a 22 page instrument. David stated that he anticipates the results of Huba’s study would be the same as what we would get. Yolanda said that they follow-up with providers by asking routinely qualitative questions about outcomes of the training, and use that information to identify gaps in training. Heather asked how you measure differences in impact from the start and what the clinicians responsibilities are. Each are very different. All will be affected. There are no instruments to capture systems change currently.

· Steve Trujillo suggested that HRSA clarify what they want to achieve. John Wiebe said this could be done qualitatively, not quantitatively late in the project. He stated each project is evaluating impact of AETC training locally, although evaluation is impact on providers and not system change. Richard Moore stated that each site has a training component and how do we advocate this to become part of the Ryan White training model on a greater scale? Betty reviewed discussion - grantees do not feel we have the capacity to assess system change as a result of AETC training. Providers trained vary from region to region, type of training varies as well, medical professionals have high rates of turn-over, no standardized system for assessing training of individuals, no data on persons/agencies previously trained, and no on-going assessment of gaps in training. Yolanda also expressed concern about the exploration of training needs at local levels and the difficulties in capturing this information. Maria Zuniga said clarity is needed from HRSA in regards to what they mean by capturing system change. What are the expectations, the anticipated measures, what is being carried out by Columbia. David suggested that we could invite the National AETC Evaluation center to participate in our grantees meeting in Washington, DC and present the AETC evaluation plan.

· Barbara Estrada questioned why this has to be done at a national level. Why can’t it be accomplished locally? Richard stated that if it can’t be done as cross-site, then it may have to be accomplished locally.

· Group discussion bough out that the five projects do not have enough of an effect size to accomplish meta-analysis.

· Maria Zuniga identified human and financial needs required for accomplishing this goal. Lydia said some of the forms identified many of the variables that they want to accomplish. Stated that some of the evaluation issues desired by HRSA are already being done. Richard stated that this may have to be done locally, because training methods and systems are so different. Betty stated that the AETC’s are working to meet individual needs and the projects are working to accomplish other needs as well. You can get individual changes captured on current forms, but you can’t capture system change with existing forms. The project again suggested this information be captured at the end of the grant. Concern is that it will be time consuming, requiring more effort, and costing more financially as well. Ledia said that we agree about impact at individual level, but we still need to work on system level. Grantees stated that clear guidance is required on the process to achieve this task.

· Steve Trujillo said $10,000 was awarded to local AETC’s to collaborate with SPNS. Maybe this money can be used for evaluation in collaboration with the projects. Ledia stated that the differences may be so different w/each AETC that it wouldn’t be appropriate for HRSA to dictate the spending. The allotment is $10,000 each per year.

· David proposed that one solution the achieving assessment on system impact would be for the Evaluation Center to conduct a focus group with grantees during the DC meeting. Two hours would be set aside for discussion about the perceived impact of AETC training on community capacity during the term of the existing demonstration projects. The focus group would be transcribed, data coded, and analyzed. Bob Smith suggested this could be followed up with a second focus group of the same individuals at the end of Year 05 of the grant. Robert Castrillo asked OU to invite those AETC’s working with the project to participate in this focus group, including Peter from the National AETC Evaluation Center. Adan stated that this should also include the discussion of the guidelines that Lydia has suggested. A general consensus was obtained from grantees.

Pregnant Women as part of multi-site data: Yolanda Cantu (power point posted on web page)

David presented frequencies and cross tabs on women in the study.

Comments on power point presentation:

• Melody said that it is important to distinguish if the woman is receiving medical care in Mexico related to the pregnancy.
• Mari stated that there has not been a birth transmission reported for past seven years because of treatment interventions.
• AETC (Mary) is providing training to hospitals/clinics in the San Diego area on handling HIV+ pregnant women and births.
• Arizona doesn’t have very many women presenting at clinic and those are referred out to a high risk clinic.
• NMX has not had a pregnant woman to date. Should such a case be found, the woman would likely be referred to a specialty clinic in Albuquerque.
• Tony Chavez has several cases and they are referred out for speciality care, but are included in all other services.
• Rosana Scolari stated that HIV+ women are expressing interest in becoming pregnant. Many projects are experiencing this and needing to deal with the issue.
• VAC co-manages HIV+ pregnant women and their children. VAC has seen 10 pregnancies in the past year, all HIV-births.
• Mari expressed concern about including this population in IRB application, as they are a vulnerable population.
• It was suggested we could develop a local module to track this information. Yolanda will take the lead in developing a proposed module to collect data. Chart reviews could provide insight into much of the information that is to be collected. Betty expressed concern about referring outside to specialist and how that would be tracked in chart reviews.
• David moved that Yolanda draft a local module for pregnant women that could be completed voluntarily by the projects. Motion carried.

Update on Qualitative Study: David Barney

David stated that funds have been provided by HRSA to conduct a qualitative study. Betty stated that the Evaluation Center will hire a professional consultant to:

• to assist in the development of formative questions,
• to be conducted interviews in Spanish or English or both,
• to be completed in Feb/Mar.
• two groups: men, women
• Betty will contact projects for scheduling dates/locations.
• Evaluation Center will provide incentives to participants
• will not video tape
• 7-9 participants per group
• approximately 90 minute interviews
• all sites will be included
• travel reimbursement will be a factor for people participating (to be discussed by evaluation center and projects)
• Betty or Herman will be moderating the groups
• Should be Post-SPNS participants
• Confidentiality of subjects must be considered (great concern among some projects)

Projects expressed interest in key informants’ interviews v. focus group work. Selection bias is likely a problem in focus group. Due to the personal nature of the questions and sensitivity of the subject matter, projects expressed concern that bringing together people in a group could result in limited information. Felt it would be more productive to provide a larger incentive and conduct long interviews with individuals. Privacy of the interview could enhance the quality of the information being obtained.

The projects decided to replace focus group work with key informants.

• Selection of participants will be project responsibility
• Maximum of 6 participants per site
• Incentives will increase
• David will draft a profile of participant for projects to follow
• Interviewer will be bi-lingual

Future meeting planning:

• In Washington, DC
• Inviting AETC regional programs
• Change module
• In conjunction with All Titles meeting
• 2 day agenda August 19^th and 20th
• before All Titles meeting

Arrangements for the meeting will not be finalized until information on the Ryan White All Titles meeting is obtained and discussed with HRSA staff.

Training Need Assessment by Pacific AETC: Michael Reyes (power point presentation)

• Michael gave an overview of the Pacific AETC and the proposed training needs assessment
• Looking at training needs along the US/Mexico border and exploring the idea of a center to meet that need
• Decision to be made in July if the center is to be funded
• Pacific AETC received the grant and subcontracted to UCLA to conduct needs assessment

Armida’s PowerPoint – UCLA Border AETC

Discussion Items:

• Is there a need for a border AETC?
• Local processes?
• What data would you collect? Who would you target as respondents?

• Location of center is important
• Variation among populations at projects is great
• Training for more than health care providers.
• Front line people, greetings, etc
• Medical records people, proper handling of case files
• Making sure case manager’s have the skills they need to also meet the needs of the population (a possibility to include these folks as key informants)
• Restrictions on trainings and training materials for Mexican providers`
• Medical terminology translation
• Possibility of requiring state mandates that include HIV training hours for providers
• There are personnel issues that can’t be confronted with AETC trainings.
• Mexican American v. Mexican
• Discrimination
• Need realistic outcomes

Much of this should start in medical school and is engrained in people, taking years to alter.

Meeting adjourned at: 1:55p.m. on Wednesday, January 9^th.