Mr. Brad P. Buggar
U.S. Department of Energy
Idaho Operations Office
Idaho Falls, ID 83401-1562
Phone: 208-526-0833
Fax: 208-526-8789
E-mail: buggarbp@inel.gov
Number of Human Subjects projects reported: 7
| INEEL-90-95-001 | "INEEL/TSA Explosives Detection Program (Formerly Federal Aviation Administration (FAA) Explosive Detection System and Independent Validation and Verification Project)" |
| INEEL-03-001 | ""A Research-centered and Performance-based Approach to Advanced Emergency Response Planning Support System Design"" |
| INEEL-03-002 | "Revolutionizing Safety Training Using Safety Simulations and Model-Centered Instruction" |
| INEEL-03-003 | "Identification of Predictive Societal Issues Associated with Acceptance of Bioremediation Strategies" |
| INEEL-03-004 | "Making Sustainable Decisions Using the KONVERGENCE Framework" |
| INEEL-03-02-001 | "Human Factors Evaluation of Robotic Control Architectures for Navigation" |
| INEEL-03-02-003 | "System and Equipment Reliability and Performance" |
Other projects of interest associated with this site:
| NIOSH-92-002 | "Cohort Mortality Study of Idaho National Engineering and Environmental Laboratory (INEEL)" |
| PACE-97-DE-FC03-97SF21512 | "Medical Surveillance for Former INEEL Workers" |
| NIOSH-97-009 | "Multi-Site Case-Control Study of Lung Cancer and External Ionizing Radiation" |
"INEEL/TSA Explosives Detection Program (Formerly Federal Aviation Administration (FAA) Explosive Detection System and Independent Validation and Verification Project)"
Principal Investigator: Ms. Carla Miller, Idaho National Engineering and Environmental Laboratory
Project started in: 1990
This project ended in fiscal year 2003.
Funding for Human Subjects Research:
This project does not involve the use of multiple protocols/subprojects.
Institutional Review Board (IRB) Review:
Type of Review: Full Board
Approving Institution: Idaho National Engineering and Environmental Laboratory
Most recent approval: 05/22/03
IRB approval number: INEL-IRB-001
Additional IRB approvals from other institutions:
Type of Review: Full Board
Approving Institution: INEEL
Most recent approval: 05/22/03
IRB approval number: n/a
Number of human subjects who participated in this project/protocol/subproject in the last reporting period: 6
Reporting period for number of human subjects:
Fiscal Year 2003
Type(s) of Human Subjects Involvement:
Objective and Methodology:
This is an ongoing project. The PI has received funding in 2003 to perform contamination studies using a human-improvised explosive device (IED) builder. As a person handles or manufactures an explosive device, small particles are deposited on their hands and clothing. The ability to detect small amounts (trace quantities) of explosives on hands and clothes will give the Transportation Security Administration (TSA) a tool in preventing aircraft-related terrorist activities. The objective of this study is to continue to investigate how the particles are distributed on hands, clothing, and luggage and how long the particles can be detected. The procedure for this study involves toughing or handling various types of explosive materials. The human subject will perform a few simple tasks that are designed to simulate the construction and carrying of an improvised explosive device and/or transfer particles from a fingerprint to other materials such as cardboard, plastics, etc. After the tasks, samples will be collected from hands, face, and clothes using filter paper or a small vacuum that collects particles onto filter paper or by walking through a prototype portal explosives detection system. The IED will be placed in a suitcase and the suitcase will be monitored. The PI will track explosive contamination resulting from the building of an IED with subsequent placement of the device on a human subject or in a suitcase. The results have provided information to the TSA on how and where to sample in airports.
Risk:
The risks from these activities are minimal. The potential concern of skin sensitization by contact with explosives did not materialize. Safety analysis (EF-30-9-042, Rev. C) shows the risk of death from explosion or fire from these tests to be extremely small, about one in 4 million/year. Material Data Safety Sheets will be present onsite for each type of explosive tested. The benefits are an improvement in the detection of explosives at airports resulting from an understanding of explosive particle transport on people, an evaluation of state of the art explosive detectors, and the determination of levels and lifetimes of explosives contamination on explosive handlers.
Privacy:
Fingerprints are required only because of the unique property of fingerprints to transfer particles to other materials. In no way will the identity of the participants be linked to any fingerprinting or sampling. Informed consent was reviewed and signed by the PI on 05/05/2003.
""A Research-centered and Performance-based Approach to Advanced Emergency Response Planning Support System Design""
Principal Investigator: Dr. Jerry L. Harbour, Idaho National Engineering and Environmental Laboratory
Project started in: 2003
Funding for Human Subjects Research:
This project does not involve the use of multiple protocols/subprojects.
Institutional Review Board (IRB) Review:
Type of Review: Expedited
Approving Institution: Idaho National Engineering and Environmental Laboratory
Most recent approval: 05/22/03
Number of human subjects who participated in this project/protocol/subproject in the last reporting period: 60
Reporting period for number of human subjects:
Fiscal Year 2003
Type(s) of Human Subjects Involvement:
Purpose:
The purpose of this research is to examine how peoples’ ability to detect changes in slightly altered visual satellite images is affected by how the images are presented. The events of September 11, 2001 tragically reinforced the need for effective, responsive, and robust emergency management systems. A critical task of such systems is making the right decision at the right time. Significant concerns have been expressed, however, that emergency management systems are becoming primarily technology-driven, with little if any associated understanding of how such technologies actually enhance (or even if they do) emergency management effectiveness and associated decision making. A key goal of this research is to better define needed design characteristics and components of an advanced emergency response support system that more effectively supports human cognition of geospatial data and associated time-critical, high-consequence decision-making. Using a research-centered and performance-based approach, this research will explore how personnel can more effectively use geospatial information (satellite- and airborne-imagery and maps) to enhance emergency understanding and response action selection for both fast- and slow-onset disasters. An important task in such situations is detecting changes in system state.
Experimental Plan:
We propose the following experiment to test our hypotheses that people will have different abilities to detect differences between two potentially different satellite images when those images are in one of two different serial presentation formats versus when those images are presented in a parallel format. We will specifically test if image registration and resultant “motion” generation significantly improves the ability to detect subtle changes in imagery sets.
Based on the cognitive psychology research on change detection and change blindness, an experiment can be designed to test for people's ability to use geospatial information to detect both gross and subtle changes. Using a paradigm already validated in the cognitive psychology literature, testing for peoples’ ability to detect changes in satellite images is simply a matter of systematically presenting two images where one has had objects or shapes changed, added, or deleted and then recording their response. While there are other changes that can be made to the images (e.g., changes in color, distance, perspective, etc.), we are primarily interested only in peoples’ ability to detect shape change and object additions/deletions.
All participants will be asked to verbally identify (i.e., a cognitive response measure) as well as physically locate with a computer mouse, what possible shape changes and additions/deletions that they see in the two different images presented to them. Participants will be asked to wear a microphone connected to a headset so that their verbal responses can be recorded. Participants will also be told that the computer screen will be recorded so that their mouse movements and pointing can be recorded. No images of any participant, however, will be collected or recorded in any manner.
To vary the presentation format from serial to parallel, we will use previously standardized techniques as summarized by Rensink (2002). In the experiment, one-third of the participants will be randomly assigned to the parallel presentation condition, and have the image set presented on two different computer monitors in a side-by-side fashion. Another third of the participants will be randomly assigned to a serial “flip-flop” condition, and have the image set presented one at a time on the same computer monitor. In this condition, the participant will be allowed to toggle back and forth between the two images as many times as they wish as they try to identify possible differences. The last third of the participants will be randomly assigned to a serial “flicker” condition. The presentation of images in this condition is similar to the “flip-flop” condition except that there will be a light gray colored interference screen that pops up between the two potentially different images for 150 ms. The reason for the gray colored screen is to provide interference to the visual cue of “motion” from the changing images. Participants in the “flicker” condition will also be allowed to toggle between the images as many times as they wish. All participants will be asked to perform the change detection task as quickly but as accurately as possible. In order to have sufficient statistical power, we anticipate needing 60 to 80 participants.
In the proposed experiment, an e-mail will be sent out to INEEL in-town employees advertising our experiment in which they may voluntarily participate. Participants who have volunteered will go through the standard informed consent process.
Risks and Benefits:
There is minimal risk associated with participation in this experiment. Minimal risk means that the potential for harm or discomfort is not greater than that encountered in daily living or during the performance of routine physical or psychological exams or tests.
Protection of human subjects primarily comes from having all participants read and sign an informed consent form. In that form, participants are told 1) what they will be doing, 2) what the risks and benefits are in doing the task, 3) that they are free to ask questions, 4) that their responses will remain anonymous and confidential, 5) that they are free to leave the experiment at any time without penalty, and 6) that there are people they can contact if there are unresolved problems. Participants will receive a signed copy of the informed consent form to keep for their records. Participants will also be asked to read a copy of the INEEL “Experimental Subjects Bill of Rights.”
All voluntary protection program (VPP) and integrated safety management system (ISMS) guidelines will be followed to the best of our ability. We will remind the participant that they are free to stop and leave the experiment if they feel like they are at risk. There will be no deception manipulation in the experiment, besides not revealing the hypotheses we are trying to test until after the participant is finished with the experiment. A full debriefing session will occur at the end of the experiment to explain what we are trying to test for and to answer any questions the participants might have.
There is no direct benefit to the participants in this study, aside from the interesting nature of the tasks to be performed. However, a better understanding of how perceptions of change as a function of how differences presented may help improve efficiency and decision making in both government and civilian arenas.
Cited Reference:
Rensink, R. A., (2002). Change Detection, Annual Review of Psychology, v. 53, pp. 245 – 277.
"Revolutionizing Safety Training Using Safety Simulations and Model-Centered Instruction"
Principal Investigator: Dr. Robert E. Richards, Idaho National Engineering and Environmental Laboratory
Project started in: 2003
Funding for Human Subjects Research:
This project does not involve the use of multiple protocols/subprojects.
Institutional Review Board (IRB) Review:
Type of Review: Full Board
Approving Institution: Idaho National Engineering and Environmental Laboratory
Most recent approval: 05/22/03
Number of human subjects who participated in this project/protocol/subproject in the last reporting period: 45
Reporting period for number of human subjects:
Fiscal Year 2003
Type(s) of Human Subjects Involvement:
Purpose:
Over the years, workers in the construction industry have been injured or killed due to safety-related accidents. To help eliminate workplace injuries, Bechtel National, Inc. (BNI) has adopted state-of-the-art programs and practices and is constantly on the lookout for ways to improve in this area. Last summer, a new training program was approved for initial development and testing. Using an award-winning simulation approach, the training is designed as a refresher for construction supervisors, field engineers, and safety professionals. It is targeted to help these leaders identify and eliminate or mitigate hazards and error-likely situations on the job. A fall protection segment was selected for initial concept and feasibility testing. Brevity and deliverability to remote international locations over slow Internet connections are important design constraints.
After last year’s economic downturn, Bechtel placed the project on hold. In its efforts to find possible collaborators to join in the project, BNI discovered the potential synergy of merging this corporate effort with similar ongoing efforts at the Idaho National Engineering and Environmental Laboratory (INEEL). Bechtel BWXT Idaho, Inc. (BBWI), as prime contractor at the INEEL, has for many years been conducting training and human performance improvement research, and has been acquiring lessons learned through their application of this research. In short, BBWI has developed human performance and instructional methods and tools that have immediate applicability to the Bechtel Corporate safety simulation effort. The discovery of this overlap has led to BBWI proposing and BNI signing a Corporate-funded Research and Development (CFRD) joint project to revive and expand the scope of the initial safety simulation project. This expanded project now includes the research and development of high-impact, web-compatible training and incorporation of additional human performance improvement principles and advanced instructional methods.
The combined team (BNI and BBWI) has six research and development objectives spanning two phases. These objectives are to:
1. (Phase I) Determine the best ways to sequence and use the situational scenario form of simulation created by Drexler and Marsh in combination with instructional tutorials.
2. (Phase I) Determine what methods of presentation make for the most interesting/motivating and effective tutorials.
3. (Phase I) Determine the best ways to incorporate a human performance improvement framework into the safety training modules already outlined.
4. (Phase I) Determine the feasibility of delivering the completed content to a significant percentage (i.e., 80 percent) of the target audience world-wide (56K modem or greater). This objective will be executed by BNI alone.
5. (Phase I) Given the results of Phase I, determine what tools would lead to greater efficiency through development of the most effective formats.
6. (Phase II) Create and test the application of those tools.
Experimental Plan:
The INEEL will work closely with BNI to explore alternative methods to incorporate a human performance improvement framework into the current safety simulation design. Dr. Andrew S. Gibbons, Professor of Instructional Technology at the Utah State University, and father of Model-Centered Instruction (MCI), a “next generation” instructional design approach, will also be a team member. We will use the principles of MCI to create a design guide for tutorials. The team will complete the Fall Protection Segment. This segment will include a set of four or more scenarios and two forms of appropriate tutorials. One form of the tutorial will incorporate the principles captured in the MCI design guide and the other will be a more typical, though aesthetically pleasing, linear text presentation.
The human subjects will be volunteers recruited from construction managers and supervisors, operations managers and supervisors, field engineers, safety professionals, and work planners employed at the INEEL. Participants can be male or female and will most likely vary from 22 to 65 years in age. When the materials are ready, the INEEL investigators will invite subjects into the Human/Intelligent Systems Laboratory to go through the training. The investigators will use standard instructional interactions and methods to pre-assess learner skills, knowledge, and attitudes relative to the content being taught. The investigators will pre-brief the participants as to the purpose of the instructional materials and their future potential use Bechtel-wide. They will be informed that we are asking them to take the fall protection web-based training module and that we will be recording their interactions and measuring their learning. Recording will be done using computer logging and videotaping, and the evaluators will interview the participants to obtain feedback. The learners will again be interviewed after a 4 to 6 week delay. Pre- and post-tests will also be used.
The CFRD research project includes the optional task of more formally studying the effectiveness of various alternative methods and content. The nature of that study is currently undefined but would be defined by completion of the research described above. If a study is determined to be useful, the Principal Investigator will submit additional Human Subjects materials to the IRB prior to undertaking any further study.
Risks and Benefits:
There is minimal risk associated with participation in this experiment. Minimal risk means that the potential for harm or discomfort is not greater than that encountered in daily living or during the performance of routine physical or psychological exams or tests.
Protection of human subjects primarily comes from having all participants read and sign an informed consent form. In that form, participants are told 1) what they will be doing, 2) what the risks and benefits are in doing the task, 3) that they are free to ask questions, 4) that their responses will remain anonymous and confidential, 5) that they are free to leave the experiment at any time without penalty, and 6) that there are people they can contact if there are unresolved problems. Participants will received a signed copy of the informed consent form to keep for their personal records. Participants will also be asked to read a copy of the INEEL “Experimental Subjects Bill of Rights.”
All voluntary protection program (VPP) and integrated safety management system (ISMS) guidelines will be followed to the best of our ability. We will remind the participant that they are free to stop and leave the experiment if they feel that they are at risk. There will be no deception in the experiment. A full debriefing session will occur at the end of the evaluation to explain what we were trying to accomplish, get additional feedback, and answer any questions the participants might have.
There is no direct benefit to the participants of this study, aside from their learning or refreshing their knowledge in the area of Fall Protection. They will not receive any credit (via TRAIN) for this training. Their participation and feedback will enable Bechtel to improve this training content and method for Corporate-wide use. Many of the principles learned will also have a direct bearing on instructional methods that could be used by INEEL in future courses.
"Identification of Predictive Societal Issues Associated with Acceptance of Bioremediation Strategies"
Principal Investigator: Ms. Julie Marble, Idaho National Engineering and Environmental Laboratory
Project started in: 2003
Funding for Human Subjects Research:
This project does not involve the use of multiple protocols/subprojects.
Identifier or number: 003
Institutional Review Board (IRB) Review:
Type of Review: Expedited
Approving Institution: Idaho National Engineering and Environmental Laboratory
Most recent approval: 05/22/03
Number of human subjects who participated in this project/protocol/subproject in the last reporting period: 40
Reporting period for number of human subjects:
Fiscal Year 2003
Type(s) of Human Subjects Involvement:
Purpose:
The purpose of this research is to investigate how information presented about bioremediation affects the perception of risks associated with bioremediation. Specifically, we are interested in how stakeholder selection of remediation options and their acceptance of bioremediation technologies are affected by how they process bioremediation information presented to them.
Experimental Plan:
This INEEL Laboratory Directed Research and Development project is an experiment that uses a hypothetical test case to assess factors that affect perception of risk and scientific ethics. Use of a hypothetical test case has proved extremely valuable in previous research to elicit stakeholder concerns regarding remediation and stewardship. Preliminary research had indicated a test case could be based on the bioremediation course taken at Dover Air Force Base (AFB), DE. Use of this event as a test case provides a forum to elicit public perceptions regarding the remediation technology decision-making process. It also provides an instance where the public’s perception of the ethics of the decision to go forward with the pilot study in the face of evidence that the selected technology may have been insufficient and that expected results were not achieved can be explored.
In order to elicit factors influencing the psychological distance to the decision-makers, several variants of the Dover AFB bioremediation effort will be created. The variants will differ in the stated location of the Air Force base. Participants will make judgments regarding the bioremediation strategy for areas proximal or distal to them. For example, participants could be asked to make the judgments regarding the implementation of a strategy that is stated to occur at a ‘military base’ either in Lincoln, NE (intermediate distance to both Lake Forest College, Idaho State University, and INEEL), outside of Chicago, IL (proximal to the area of Lake Forest College), or outside of Salt Lake City (proximal to both INEEL and Idaho State University), with the location of the ‘military base’ randomized among participants at each location. In this manipulation, the distance from the location of the decision-maker to the stated location of the implementation strategy would be examined for its relation to responses. Similar manipulations will be used to identify factors that affect the decision-maker’s perception of the risk of the strategy. In previous work, we used an interrupted decision-making task, in which participants are periodically asked the state of their current decision before they have acquired all the available information. Such an interrupted decision-task may allow us to investigate the participants’ perceptions of scientific ethics of the researchers, their trust in the researchers, and other factors which may affect perception of risk, based on whether the participants have yet been made aware of the outcome of the strategy. If, however, initial research indicates that other task types are more effective at garnering information, the most effective methodology will be used.
Specifically, participants will be given the details of the remediation procedure in three sections and then asked a series of questions between each section. The questions will be Likert scales to make administration and recording easier. It is expected that the entire administration procedure will take approximately 45 to 60 minutes and will occur in groups.
Experimental design outline:
1. Environmental Concerns scale, pre-test
2. New Environmental Paradigm (NEP), pre-test
3. First section of chronology (i.e., the hypothetical AFB test cases)
4. First questions (note: They will be asked the same 20 questions at each break.)
5. Second section of chronology
6. Second set of questions
7. Third section of chronology
8. Third set of questions
9. Post-test demographics questions
10. Debriefing (Tell them that this actually occurred at Dover AFB in DE.)
Thus, the experimental design is a 3 x 3 matrix, as follows: Location of participants: Chicago (Lake Forest College), Pocatello (Idaho State University), and Idaho Falls (INEEL); and location of Air Force Base in story: Chicago, Lincoln, NE, Idaho Falls.
In order to have sufficient statistical power, we anticipate needing 30 to 50 participants. In the proposed experiment, an e-mail will be sent out to INEEL in-town employees advertising our experiment in which they may voluntarily participate. Participants who have volunteered will go through the standard informed consent process. It is anticipated that various aspects of the experiment will take the remainder of FY2003 to complete (including journal write-up), will require the use of a conference room, and require two or three human factors scientists.
Risks and Benefits:
There is minimal risk associated with participation in this experiment. Minimal risk means that the potential for harm or discomfort is not greater than that encountered in daily living or during the performance of routine physical or psychological exams or tests.
Protection of human subjects is accomplished by having all participants read and sign an informed consent form. In that form, participants are told 1) what they will be doing, 2) what the risks and benefits are in doing the task, 3) that they are free to ask questions, 4) that their responses will remain anonymous and confidential, 5) that they are free to leave the experiment at any time without penalty, and 6) that there are people they can contact if there are unresolved problems. At the end of the experiment, participants will be fully informed that the remediation case study presented was based on remediation efforts in Dover, DE and that the purpose of manipulating where the remediation had supposedly occurred was to learn how this affects their decision making. Participants will receive a signed copy of the informed consent form to keep for their records. Participants will also be asked to read a copy of the INEEL “Experimental Subjects Bill of Rights.”
All voluntary protection program (VPP) and integrated safety management system (ISMS) guidelines will be followed to the best of our ability. We will remind the participant that they are free to stop and leave the experiment if they feel like they are at risk. There will be no deception manipulation in the experiment except changing the location of the bioremediation study among participants and not revealing the hypotheses we are trying to test until after the participant is finished with the experiment. A full debriefing session will occur at the end of the experiment to explain what we were trying to test for, that the events described actually occurred in Dover, DE, and to answer any questions the participants might have.
There is no direct benefit to the participants in this study, aside from the interesting nature of the tasks performed. However, a better understanding of how perceptions of change as a function of how bioremediation information is presented may help improve efficiency and decision making in both government and civilian arenas.
"Making Sustainable Decisions Using the KONVERGENCE Framework"
Principal Investigator: Dr. Steven Piet, Idaho National Engineering and Environmental Laboratory
Project started in: 2003
Funding for Human Subjects Research:
This project does not involve the use of multiple protocols/subprojects.
Identifier or number: 004
Institutional Review Board (IRB) Review:
Type of Review: Expedited
Approving Institution: Idaho National Engineering and Environmental Laboratory
Most recent approval: 05/22/03
Number of human subjects who participated in this project/protocol/subproject in the last reporting period: 60
Reporting period for number of human subjects:
Fiscal Year 2003
Type(s) of Human Subjects Involvement:
Purpose:
The purpose of this research is to investigate how the group decision-making process that is used affects: 1) the group’s ability to make decisions and 2) individual participants’ attitudes and perceptions of the decision, the decision-making process, and other participants. Specifically, we are interested in whether our decision-making process, called KONVERGENCE:
· Improves the group’s satisfaction with the decision-making process.
· Improves the likelihood that a group, consisting of individuals with relatively diverse environmental attitudes, will come to consensus on a decision for an environmental issue relative to other, less complete decision-making processes.
Background:
Some cleanup decisions, such as cleanup of contaminated sites or disposal of spent nuclear fuel, have proven difficult to make. Such decisions face high resistance from stakeholders – possibly because they do not trust the decision makers, view the consequences of being wrong as too high, etc. We have developed a Model for Sustainable Decisions, called KONVERGENCE, which emphasizes how to resolve disagreements and provides ways to make progress on stalled or intractable decisions. KONVERGENCE does this by visually showing how our Knowledge, Values, and Resources for a given alternative must converge for a decision to be successful. In other words, all participants in the decision-making process must agree that the proposed solution/alternative is something we know how to do, is consistent with everyone’s values, and is within our means. Our model addresses decisions involving complex and/or unusual relationships (between decisions, among stakeholders, etc.), conflicting views, high “stakes”, and ramifications that extend over long time periods. Ultimately, the goal of KONVERGENCE is to make environmental cleanup decisions easier to make, implement, keep, and sustain. By sustainability, we mean decisions that work better over the entire time period—from when a decision is made, through implementation, to its end point.
To assess whether the KONVERGENCE Model can really improve decision-making, however, we need to test it. Volunteer human participants need to work through a challenging, complex, high “stakes” environmental issue using the KONVERGENCE Model to determine if it is more effective at generating decisions that are perceived by the participants as being easier to make, implement, keep, and sustain (i.e., satisfying) than less complete decision-making processes.
Two features of the KONVERGENCE Model that we would like to test in particular are Adaptability and Values. While we believe that all of the features that are included in the KONVERGENCE Model are essential (i.e., not superfluous) to the decision-making process, the ability to sustain a decision through implementation to its end point is often challenging because many environmental issues have an extremely long life (i.e., some environmental hazards will remain dangerous for generations). Consequently, there is a need to emphasize that decisions remain Adaptable over time. We hypothesize that emphasizing Adaptability as an important concept to consider when making decisions, versus not mentioning the concept at all, will decrease the resistance to make, implement, and keep decisions.
Along with the idea that Adaptability is needed for successful decisions, another feature of KONVERGENCE that is particularly important to emphasize is Values. While Knowledge and Resources are also critical to successful decision-making, understanding and using the Values of the relevant stakeholders/participants as a basis for a decision, rather than relying upon Regulations, is a crucial difference between the KONVERGENCE Model and other, less complete decision-making processes. We hypothesize that using Values, as opposed to using Regulations, as a basis for a proposed solution/alternative will increase the likelihood that an alternative will be chosen and remain acceptable over time.
Experimental design outline:
Specifically, in the experiment involving volunteer human participants, we will perform the following:
· Administer the New Environmental Paradigm (NEP) company-wide as a pre-test screen through the daily electronic company memo (iNotes) sent to everyone’s e-mail address. The iNote will invite employees to take five to 10 minutes to fill-out the NEP on-line through their Lotus Notes program. See supporting document, “iNote message.doc.”
· Select individuals who scored in the upper and lower quartile of the NEP and assign them to one of four experimental groups (Lotus Notes has the ability to record the demographic information of the individuals who fill out the NEP survey). See supporting document, “Follow-up participant advertisment.doc.” All four groups will work through the same environmental issue. The first group (Group A) will work through the environmental issue using the KONVERGENCE Model in its complete form: specifically, Adaptability and Values will be features that are included. Group B will use an incomplete version of the KONVERGENCE Model – Regulations will be used as a surrogate for Values. Group C will use a version of the KONVERGENCE Model that does not mention Adaptability as being an important concept to consider in environmental decision-making. Finally, Group D will use a version of the KONVERGENCE Model that uses Regulations as a surrogate for Values and does not mention Adaptability. Thus, the experimental design is a 2 x 2 matrix that orthogonally compares Values to Regulations and Adaptability to No Adaptability.
· Once the groups meet for their first experimental session, the experimenters will facilitate the four different group decision-making processes. The processes include the following general elements:
a. Introduce why the group is together – what is expected of them, and what to expect.
b. Introduce the KONVERGENCE Model, problem statement, and environmental issue.*
c. Answer questions, get feedback, collect additional information.*
d. Generate alternatives or decision options.
e. Refine alternatives with the goal of achieving KONVERGENCE.
f. Explore ways to keep KONVERGENCE.
*Note: The information given on the KONVERGENCE Model (Adaptability vs. No Adaptability) and type of feedback solicited (Values vs. Regulations) will systematically vary for the different groups in these steps. For details on what these decision-making processes steps entail, see the supporting document, “Details of proposed research.doc.”
· In addition to the tests above, we hope to conduct another test of the KONVERGENCE Model on a different environmental issue, assuming we have the funding and time. Specifically, we would test the KONVERGENCE Model on the environmental issue of Nuclear Power Plant siting. The procedures for this test, using Group E, will basically be the same as the procedures for Group A. As before, we would select individuals who scored in the upper and lower quartile of the NEP survey and then have them go through the following KONVERGENCE process. See the supporting document, “Details of proposed research.doc,” for more information.
All participants will be debriefed on the nature of the experiment. In order to have sufficient statistical power, we anticipate needing 32 to 60 participants. Participants who have volunteered will go through the standard informed consent process. It is expected that the pre-test screening will take employees five to 10 minutes to complete and the follow-on experiment is expected to occur over two 3-hour sessions. It is anticipated that various aspects of the proposed experiment will take the remainder of FY2003 to complete (including journal write-up), and will require the use of a conference room, and three to five experimenters.
Risks and Benefits:
There is minimal risk associated with participation in this experiment. Minimal risk means that the potential for harm or discomfort is not greater than that encountered in daily living or during the performance of routine physical or psychological exams or tests.
Protection of human participants in the pre-test survey phase will come from having all employees who choose to access the survey read a disclaimer page before they are permitted to take the survey. This disclaimer page will have all of the basic elements of a standard informed consent form (e.g., confidentiality, risks and benefits, participant’s rights, etc.), and will require the employee (i.e., potential participant) to click on an “I accept” button at the bottom before allowing them to take the survey. If they do not click on the "I accept" button, the system will not permit them to fill out the NEP. This disclaimer page will operate similarly to the “license agreement” pages for software packages that show up on screen before the user is allowed to install the new software onto their computer. See supporting document, “On-line NEP Informed Consent.doc.” The data will be collected on a protected network server that employees will access through their Lotus Notes program. Only the researchers will have access to the employees’ responses.
Protection of human subjects is accomplished by having all participants read and sign an informed consent form. In that form, participants are told 1) what they will be doing, 2) what the risks and benefits are in doing the task, 3) that they are free to ask questions, 4) that their responses will remain anonymous and confidential, 5) that they are free to leave the experiment at any time without penalty, and 6) that there are people they can contact if there are unresolved problems. There is no deception involved in this experiment, except not revealing what the hypotheses are until after the experiment is completed. At the end of the experiment, participants will be fully informed about purpose of the experiment. This debriefing session also serves as an opportunity to answer any questions the participants might have. Participants will receive a signed copy of the informed consent form to keep for their records. Participants will also be asked to read a copy of the INEEL “Experimental Subjects Bill of Rights” before they decide to participate in the experiment.
The experimenters will also be told that they must respect the rights of the participant and attempt to answer every question the participant may have. In addition, all voluntary protection program (VPP) and integrated safety management system (ISMS) guidelines will be followed to the best of our ability.
There is no direct benefit to the participants in this study, aside from the interesting nature of the tasks to be performed. However, a better understanding of how group decision-making is affected by the decision-making process the group is taught may help improve the acceptance of environmental decisions.
"Human Factors Evaluation of Robotic Control Architectures for Navigation"
Principal Investigator: Ms. Julie Marble, Idaho National Engineering and Environmental Laboratory
Project started in: 2003
This project ended in fiscal year 2003.
Funding for Human Subjects Research:
This project does not involve the use of multiple protocols/subprojects.
Institutional Review Board (IRB) Review:
Type of Review: Full Board
Approving Institution: Idaho National Engineering and Environmental Laboratory
Most recent approval: 05/22/03
IRB approval number: n/a
Number of human subjects who participated in this project/protocol/subproject in the last reporting period: 12
Reporting period for number of human subjects:
Fiscal Year 2003
Type(s) of Human Subjects Involvement:
The purpose of this research is to assess the usability of different control modes or architectures and the efficiency of adapting the degree of autonomy for the control of the robot. Adjustable autonomy or adaptive control is a novel approach in human-robot interaction. It is not clear what method of implementing this type of control architecture will lead to the greatest usefulness of robots in search or navigation type tasks.
Human subjects are involved in this research because it is their perceptions that the PI wants to examine in order to increase the acceptance and usefulness of this technology. It is not possible to simulate the interaction of humans with robots in the mode of adaptive control, because this type of control architecture has not previously been implemented. In addition, the purpose of this research is to study the interaction of humans and robots with this type of control architecture. Informed consent will be obtained.
The human subjects will be employees of the INEEL, solicited by email from one of the company buildings. Participants can be male or female and will vary in age from 22 to 65. The PI is interested in employees who do not have significant experience in the control of robots (i.e., less than one to two times per week), either in real life or through video games, because the PI wants the participants to be accurate reflections of users who may only infrequently need to use robots.
Participants will be asked to navigate an ATVR Jr. robot through a softwall “maze.” Participants will navigate visually by video feed to the computer that controls the robot. They will navigate the “maze” eight times. They will be asked a series of questions after each navigation regarding their subjective assessment of the ease of use of the interface and the task. Their total participation may take as long as two hours.
There are minimal risks associated with participation in this experiment. Participants may need to make multiple rapid keystrokes during the robot navigation task, which could aggravate an injury, such as carpal tunnel syndrome or other cumulative trauma disorder (CTD). To minimize the potential for risk, the participant will be screened for a history of CTD or injury to their wrists or shoulders. The participants will be encouraged to stand or walk and stretch between trials. Because the participants will be separated from the robots, there is little potential for injury from contact with the robots. The PI feels the benefits outweigh the risks primarily because the risks are minimal and the benefit of the INEEL implementing adjustable autonomy in robots will reduce the need to expose human workers in risk to tasks such as waste clean up.
All participants will be advised they are free to stop and leave the experiment if they feel like they are at risk. There will be no deception manipulation in the experiment. A full debriefing session will occur at the end of the experiment to explain what the PI is trying to test and to answer any questions the participants might have.
There is no direct benefit to the participant in this study aside from the interesting nature of the tasks to be performed and the opportunity to learn about potential uses of semiautonomous robots. However, increased ease of use and usability of robots will allow implementation of robots in tasks that are dangerous or inaccessible to humans. This is especially critical for organizations that require humans to enter high-risk or dangerous situations, such as those associated with the Department of Energy or the Department of Defense.
All of the data collected during the study will be controlled by the PI to ensure privacy. The data will be kept in a locked cabinet, accessible only by the investigators. The data will be kept for three years. All of the data will be identified only by a subject number; the participants name will not be associated with any data. The results of the study, including laboratory or any other data, may be published for scientific purposes, but the participants' name will not be published nor any identifiable references to the participants.
"System and Equipment Reliability and Performance"
Principal Investigator: Mr. David Gertman, Idaho National Engineering and Environmental Laboratory
Project started in: 2003
Funding for Human Subjects Research:
This project does not involve the use of multiple protocols/subprojects.
Identifier or number: 02-003
Institutional Review Board (IRB) Review:
Type of Review: Expedited
Approving Institution: Idaho National Engineering and Environmental Laboratory
Most recent approval: 08/07/02
Explanation of IRB approval:
Protocol was initially approved in August 2002, but funding did not come through until after the beginning of FY2003.
Number of human subjects who participated in this project/protocol/subproject in the last reporting period: 40
Reporting period for number of human subjects:
Fiscal Year 2003
Type(s) of Human Subjects Involvement:
Purpose:
During the next two years, in support of quality assurance (QA), hazards analysis and operability (HAZOP) studies, the international standards organization (ISO) standards accreditation process, and vulnerability studies, a number of potential systems on site or in Idaho Falls will be evaluated for DOE HQ and others in their effort to develop or improve systems and equipment performance. As part of this evaluation, the availability and efficiency of selected systems will be modeled and evaluated, and various types of data recording equipment will be used to measure equipment performance and functioning. During the course of the evaluation of equipment, personnel performance in conjunction with the functioning of these systems may be videotaped or audio taped. Because the human factor is involved in the functioning or restoration of systems, an IRB review is required.
Experimental Plan:
The proposed research will compare three conditions for assessing systems reliability and performance:
1. Laboratory studies (conducted in a controlled computer-based laboratory)
2. Staged studies where equipment configurations can be changed to allow focused system studies to be conducted
3. Field studies where the configurations and conditions studied are those found their natural setting (work environment(s))
Laboratory Studies: General
The participants for the proposed research will be recruited from the INEEL employee population. It will be limited to persons with self-reported normal hearing, normal vision, and an engineering or operations background or job. For this experiment, normal vision means vision uncorrected or corrected using glasses or contacts to approximately 20/20 and ability to read a computer screen without discomfort or changing any screen settings. Normal hearing means no hearing impairments (e.g., deafness), and the ability to hear and understand sound and speech from a computer without any additional aids. There will be approximately 20 to 40 people participating in the study. If the information from the laboratory study is deemed valuable, then a follow-on study with approximately the same conditions but with a slightly different mock-up of systems is anticipated for the FY2004 time frame.
The experimental task will be to role play the part of an operator, shift supervisor, maintenance personnel or plant manager and to read/view and evaluate information and come to a decision to recommend to the decision maker that is in accordance with operating principles and or procedures available during the course of the experiment. Prior to the presentation, each participant will be asked to complete a subject matter pre-test (which will test how much they know about the experimental task subject matter (i.e., control room response) before the presentation.). The experimenters reserve the right to waive a basic knowledge test if the subjects are currently certified in the subject area.
Prior to any participation, the subjects will sign an informed consent form. At the end of the laboratory study, the participants will receive a de-briefing from the experimenter regarding the experimental manipulation of equipment performance similar to the engineering debrief that they would receive in the field. For example, they would be told that a pump failed to start, a valve failed to close fully, or cooling systems were unavailable due to testing. Subjects will participate with full knowledge that the decisions and actions that they take during the experiment will be video taped and audio taped. The names of or types of agencies interested in the generalized scientific findings from this type of research may or may not be divulged to subjects. At the conclusion of the experimental session(s) subjects will also fill out surveys indicating their situation awareness and the degree of workload and complexity experienced during the sessions. Just as in the case of the staged and field studies, subjects will be assigned a subject number, their information will be controlled and key locked, and this information destroyed three years after the study completion.
Those viewing raw or processed data will only have access to subjects’ numbers when reviewing raw data or processed data sheets. A data management process has been designed to establish configuration control. In addition, subjects’ supervisors, personnel department, or anyone else that could impact their career or financial status will be precluded from viewing their data.
Participants may debrief the experimenter(s) on why they made particular recommendations or decisions. This information will not contain the subject’s identity. Finally, general questions about the experiment and the participant’s attitude toward it may be administered. Very limited demographic information (e.g., age, education, or years of experience) about the subject will be collected. Total participation time is estimated to be 2-1/2 hours.
Laboratory Studies: Data Collection
During the task, participants will sit in front of a computer monitor or walk between displays to check status and take readings much as they would in a real world environment. All computer input and output will be recorded. They will be video taped and audio taped when working as a team to solve problems. Experimenters will also video record eye movement, posture, and evaluate the type of information accessed by subjects to support their decision-making. For example, use of procedures, process mimics, and subsequent dialog among subjects will be recorded. All subjects’ answers to questionnaires, scales, test, and final questions will also be recorded. The subjects’ identities will be protected using a numbering scheme that is used to track data, and information shall be locked in a safe. Staged studies (i.e., focused system studies conducted in close to real work conditions) and field studies (studies conducted in the real work environment) are expected to rely less upon questionnaire and rating as sources of data than the laboratory studies.
Staged Studies (focused system studies):
Staged studies encompass situations in the field where equipment configurations can be changed to mimic a series of real world conditions. In these situations, subjects agree to participate, however, the precise start and stop times are not known ahead of time in order to make their responses to conditions as true to life as possible.
In order to increase the realism of proposed studies of equipment performance and reliability, a series of staged studies conducted in the field, i.e., locations on site, where equipment can be configured in various ways, is proposed. For example, valve line-ups (valve number and status (open, closed)) could be configured where a combination of plant automatics and required human action is required to response to abnormal conditions. The advantage of this type of research is that it allows for the study of existing systems configured in a slightly different way to more precisely anticipate the reliability and performance of these systems in actual situations.
Focused system studies typically have the following characteristics:
· involve fewer subjects than laboratory or field studies.
· are less disruptive of overall system activities.
· allow for defining and testing system configurations that are rarely observed.
· data collection periods may not be apparent to subjects.
The potential for any harm caused by subjects being unaware that they are being observed is reduced to an acceptable level by two actions contained in the present approach:
1.) Prior to any participation, the subjects will sign an informed consent form. The informed consent form explains why the studies are being conducted, acknowledges that personnel such as themselves may be video taped or audio taped, may be surveyed or interviewed afterwards and describes how their identity, career, and financial status is to be protected.
2.) At the end of the staged studies, the participants will receive a debriefing from the experimenter regarding the equipment performance; similar to engineering procedures used when any untoward event occurs on site.
Field Studies (studies conducted in the normal work environment):
Field studies occur in the natural work environment where equipment configurations and conditions are as close to their real-world setting as possible. The highest degree of realism for determining the reliability and performance of equipment and systems is obtained by conducting field studies. They typically have the following characteristics:
· are more expensive to run.
· involve more subjects.
· provide less control over the variables being measured
· provide a basic understanding how people think and respond to technology in real-world settings.
· data collection periods may not be apparent to subjects.
Since the exact start time for data collection is not shared with subjects ahead of time, informed consent and subject debriefing will be carried out as described above. As part of this process, subjects working in areas where staged studies might be conducted will sign an informed consent form. They will be told that they may be video taped and audio taped during the course of studies conducted; may be surveyed or interviewed afterwards; and are informed how their identity, career, and financial future is to be protected.
At the end of studies, they will also be debriefed in accordance with engineering procedures on site that will explain any equipment malfunctions that occurred through manipulation and will be assured that it was not due to their performance. All data from the three types of studies conducted in the proposed research (laboratory, staged, field) will have the subject's identity protected by assignment of a numeric identification code, and this information will be locked in a secure area. Line management for any subjects involved in the proposed research, as well as human resources personnel, will not have access to any performance data. Subjects will only be performing duties that they perform as part of their training and everyday activities. Hence, no risk above the normal everyday risk that they assume in their jobs will be posed to them during this research.
Risks and Benefits:
There is minimal risk associated with participation in this experiment. Minimal risk means that the potential for harm or discomfort is not greater than that encountered in their routine work activities. During the laboratory research phase, subjects will be seated in front of a computer monitor or at a table (to read the paper-based presentation). The subjects will check instrument status, take readings, and check equipment status by phone or face-to-face much as they would in their everyday work life.
There is no direct benefit to the subjects participating in this study. However, the system evaluations will support a variety of important laboratory, DOE HQ, and national initiatives. The benefits of this research will be a better understanding of equipment performance and reliability including availability in regards to human ability to recover systems, or restore systems from faults including approaches to diagnosis and decision making. General insights will be drawn regarding the basis for decision making from the use of a relatively complex task with specific scientific information, in conjunction with experimental control of equipment status and information presentation types. It will also be possible to draw inferences regarding what type of information is accessed by subjects to help them successfully execute their tasks. For example, equipment functioning including the way in which information associated with equipment status is alarmed, the interpretability of this information against background noise, and other potential distractions are of considerable interest to the Department of Energy and all who are concerned with reliability and performance of strategic systems that pose no undue risk to subjects.