Developing a framework to improve information literacy in an entry to practice paramedicine program - Barr
Download as PPTX, PDF1 like268 views
The document discusses embedding information literacy education in an entry-level paramedic program. It begins by acknowledging contributions from various groups and individuals. It then discusses the changing student intake and teaching practices that emphasize the need to develop students' information literacy skills. An external study with paramedic academics found that they define information literacy as including finding/collecting information, critiquing/evaluating, and integrating - seen as both lower and higher-order skills. An internal student survey at the university found that students' use of diverse resources and understanding of concepts like synthesis increased over the course of the program, though gaps remained. The document proposes several "big easy" solutions to better develop students' information literacy skills in a
Developing a framework to improve information literacy in an entry to practice paramedicine program - Barr
- 1. Embedding information literacy education in an entry to practice program Dr Nigel Barr
- 2. Acknowledgements • CSALT – USC Learning and Teaching Grant Scheme • Reference group • Student life and learning – Margot Reeh • Librarian – Roger Carter • C-SALT – Theresa Ashford • NAPA – in kind support 2
- 3. Who am I & where is USC? 3 Smart Steps Case based simulation Evidence based practice
- 4. Why the fuss over IL? 4 Changing student intake • the suite of competencies and attributes of students entering university has changed over the last 2 decades Research • universities in Australia are not successfully equipping their graduates with information literacy skills to enable life- long learning. Changing teaching practice • How do we teach our students to move beyond reproducing current thinking to critical thinking?
- 5. Ambulance and Paramedic Industry Skills Forecast Industry Reference Committee; 2018 • Certificate II to diploma • VET sector • Bachelor PhD • Tertiary sector • National registration for paramedics in Australia begins 2018 • Degree now baseline qualification for paramedics 5
- 6. The IRC ranked 12 generic skills 1 • Information literacy 2 • Communication 3 • Leadership… 6
- 7. The project Literature review External Data Collection Network of Australian Paramedic Academics Internal Data Collection USC Bachelor of Paramedic Science 7
- 8. NAPA study What did our external partners SAY? 8
- 9. We asked NAPA participants (n=29) to define what Information Literacy meant to them... Theme % of comments Skill level Find / collect 84.6 Lower Critique / evaluate 53.8 Higher Integrate 38.5 Higher Understand 38.5 Lower Identify knowledge required 34.6 Higher Communicate 19.2 Higher Manage data 3.8 Lower 9
- 10. The benefits of developing Information Literacy skills in Paramedics Students who develop information literacy skills Generally agree (%) can engage in independent learning through constructing new meaning, understanding and knowledge. 94.7 can search for and use information for decision making and problem solve. 94.7 can demonstrate social responsibility through a commitment to lifelong learning and community participation. 73.7 derive satisfaction from using information wisely. 68.4 10
- 11. “Standard” . “Element” . “Learning Objective” Standard 1. Recognises the need for information and determines the nature and extent of the information needed Importance Point in program 1.1.1 Explores general information sources to increase familiarity with the topic High Early 1.1.4 Identifies key concepts and terms in order to formulate the focus questions High nc 1.2.1 Understands how information is organised and disseminated High Early 11 Points of consensus
- 12. Points of consensus 1. Recognises the need for information and determines the nature and extent of the information needed 1.1 Defines and articulates the information needed Activities/context: Explore information sources, identify key concepts and terms 1.2 Understands the purpose, scope and appropriateness of a variety of information sources Activities/context: Topic context is discipline bound; differentiates between a variety of information sources; identifies primary and secondary information sources 12
- 13. Student survey USC Internal data 13
- 14. Making sense of student understandings of Information Literacy: Sources of information students use for assignments • Peer Reviewed Journal 97.3 • Textbooks 82.0 • Government reports 64.9 • Online reference 60.4 • Collaborative Encyclopaedia 12.6 • News Media 2.7 • No statistically significant changes across the years 14
- 15. • Use of diverse resources to develop assignments increased with year in program: Entry year 24.3% final year 65.0% • HOWEVER - no associations were found for year in program regarding: • checking whether a concept is still current while developing an answer for an assignment. • use a systematic process to help judge the relevance of the information. • use of bibliographic software to manage collected information. 15 Making sense of student understandings of Information Literacy:
- 16. • Associations were found for year in program • Use of multiple sources to reach a conclusion • Early year 10.8% Final year 60% • Use the USC style guide to ensure accuracy in citations • Early year 9.4% Final year 20% • No associations were found for year in program - • Working out how to explain a new idea effectively • Synthesis - setting out the main ideas from each source and then combine them to generate new knowledge for an assignment • Use and understanding of statistical/research data 16 Making sense of student understandings of Information Literacy:
- 17. The ‘big easy’ a solution or strategy that can be implemented using your current resources and network. 17
- 18. Our Big Easy Our solution was based on developing tools to help recognise and map how our program would support the development of a student’s ability to recognise, find, critique and synthesise 18
- 19. Solution 1 Critical alignment within a program, informed by the data collected externally and internally Vertical integration decision tool Horizontal integration map 19 IL Standards Standard 1 Year 1 outcome Year 1 Course 1 Element outcomes Activities Assessment Year 1 Course 2 Element outcomes Activities Assessment IL Standard USC standard Program standard defined by CAUL defined by university defined by program Year 1 Standards Assessment Target courses Year 2 Standards Assessment Target courses Year 3 Standards Assessment Target courses
- 20. 20 Standards and elements Year standard Scaffolds 1. Recognises the need for information and determines the nature and extent of the information needed Year 1 - With direction the student interprets the nature and extent of information the task requires. Teacher led deconstruction of task requirements Year 2 - To work with minimal direction to interpret the nature and extent of information the task requires. Student led deconstruction of task requirements Year 3 - To work independently to interpret the nature and extent of information the task requires. Independent deconstruction of task requirements. 2. Finds needed information effectively and efficiently Year 1 - With direction find information required to answer a task using an appropriate search strategy Teacher led session to develop the skills necessary to use multiple strategies to find information Year 2 - To work with minimal direction to find the information the task requires. Student led session to develop effective strategies to find information required. Year 3 Independently finds needed information using a structured approach Independently uses appropriate search tools to efficiently find required information. 3. Critically evaluates information and the information seeking process Year 1 - With direction evaluates the relevance of information. Teacher led session on assessing the usefulness and relevance of the information. Year 2 - Identifies themes within the information and constructs a logical argument. Student led session on defining and applying criteria for evaluating the information. Year 3 - Critically explores themes and critiques the evidence discovered Student led session on critical evaluation of information.
- 21. Solution 2 A structure informed by the literature review and USC resources 21 Working Party (me) Program reference group Course coordinator Program coordinator Academic developers
- 22. Solution 3 Develop a scientific communication rubric for the program that shows increasing complexity of the pass mark across consecutive years. Criteria & Weight Year 3 pass (Developing) Year 2 pass (Developing) Year 1 pass (Developing) Year 1 fail (inadequate) Formation of an argument Content / Scientific Merit The assessment piece should be well-researched, drawing on a suitable range of quality sources of information to answer the question posed Sound theoretical base with all material wisely selected Sound theoretical base with no irrelevant material included. Theoretical base mostly relevant. Inadequate theoretical base with mostly irrelevant or low-quality material included. Analysis The assessment piece should show evidence of exploration of important themes identified in the scientific literature Themes developed in a balanced way; developing logical relationships and internal consistency; Logical relationships are indicated; consistent internally. Themes are described without being connected; internal inconsistencies are present. lack of breadth in identified themes, absence of logical relationships or many internal inconsistencies Synthesis The assessment piece should show evidence of critiquing the scientific evidence and weaving together of ideas to support a logical argument. All arguments are fully and clearly supported by evidence based on research which is presented showing comparisons and synthesis. Most arguments are generally supported by appropriate research-based evidence with some synthesis of the evidence cited. Some arguments are supported by evidence however there is infrequent critical analysis of evidence from sources used. Little, if any of the evidence cited supports the arguments or evidence from sources is treated uncritically. 22
- 23. Solution 3 Develop a scientific communication rubric for the program that shows increasing complexity of the pass mark across consecutive years. Criteria & Weight Year 3 pass (Developing) Year 2 pass (Developing) Year 1 pass (Developing) Year 1 fail (inadequate) Presentation of the argument Structure The assessment should be structured to meet the specifications contained in the task information. Elegant and imaginative presentation Structure appropriate to assignment task. Some inconsistencies in style; adequate statement of intention appropriate layout. Appropriate introduction with clear statement of intention; well-structured overall with appropriate conclusion; Structure and style Badly designed; Grammatical Conventions Your work should be well- presented in a style that meets the prescribed criteria or contemporary conventions All presentation criteria met with no spelling, syntax or grammatical errors. All presentation criteria met but with a more than five spelling, syntax or grammatical errors. Appropriate style that was consistent with the task Inappropriate style that was inconsistent with the task. Referencing and citations Your work is original and the sources of ideas are acknowledged where appropriate and citations reflect the prescribed style guide Consistent and complete Vancouver style referencing techniques with a few errors Consistent and complete Vancouver style referencing techniques with moderate number of citation errors There are inconsistencies in the Vancouver style referencing technique. In adequate referencing style and technique. 23
- 24. 24
Editor's Notes
- #4: I have a nursing and paramedicine background I am a lecturer in the Bachelor of Paramedic Science program at USC, Qld The program I work in is a science based paramedicine program that teaches evidence based practice mainly through case based learning My interest in ILE stems is because IL is the backbone of evidence based paramedicine and clinical practice.
- #5: Changing student intake the demographic attributes of university students in Australia has shifted significantly over the last two decades changing suite of competencies of students who enter the classroom. Research universities in Australia and North America are not successfully equipping their graduates with the requisite skills in information literacy to enable life-long learning. Our research showed a lack of consensus among educators on critique and synthesis Teaching & Learning How do we teach our students to move beyond reproducing current thinking to critical thinking?
- #6: Education of paramedics - In 1996 the first university course for entry to practice paramedicine began. Now, in 2018 a national registration scheme is being introduced and the base line qualification for a paramedic is a bachelor degree.
- #7: How does the industry rate ILE? SkillsIQ. 2018 Industry Skills Forecast for Public Consultation. Ambulance and Paramedic Industry Skills Forecast Industry Reference Committee; 2018 In this changing environment the Industry Reference Committee met to discuss amongst other things 12 generic skills taught in courses. Information literacy was voted as the top skills IL is defined by the skills council as the ability to: identify a need for information, and to locate, evaluate, and effectively use and cite the information. develop a working knowledge of new systems. work without direct leadership and independently. 2 – Communication 3 – Leadership 4 – Technology and application 5 – Design mindset/thinking critically/ Problem solving 6 – Language, literacy and numeracy 7 – Data analysis 8 – Customer service 9 – STEM 10 – Entrepreneurial 11 – Financial 12 – Environmental / sustainability
- #8: Our project consisted of three stages Literature review- A systematic review of the literature was performed to identify information literary skills are described as critical for the health disciplines, and this search also sought to identify validated survey tools in the area of testing IL skills. Delphi study – (I think this may be too much for your time and audience) A modified Delphi process13 with three iterative rounds were conducted online We used the ANZIL model as the frame to collect data The ANZIL model has 6 standards and each has a subset of elements. These are on the notes on the tables We provided learning outcomes from major universities to seek consensus on important attributes for a paramedic program to focus on and where in the program that the attribute should be achieved by. We surveyed Paramedic academics to achieve consensus. Participants were all members of the Paramedics Australasia Network of Australian Paramedic Academics (NAPA). This group was chosen as the reliability of data from Delphi studies are improved when subjects are drawn from the specialized area of knowledge related to the target issue.14 USC student survey Custom designed online survey as the two validated surveys developed by CAUL were deemed by the reference group to not be useful in the context of this project Some of the questions seemed ambiguous, open to social identity issues, or irrelevant due to changes in technology. The survey was piloted with a small convenience sample of four academic staff and fourteen students from Bachelor of Nursing and Bachelor of Sport and Exercise Science programs.
- #9: We asked our external partners – paramedic academics from around the country NAPA – national association of paramedic academics We asked How they defined IL How they taught IL To rate 72 learning outcomes based on the ANZIL standards
- #10: The Australian and New Zealand Institute for Information Literacy definition a set of abilities enabling individuals to; recognise when information is needed and have the capacity to locate, evaluate and use effectively the needed information We asked NAPA participants in round one of the Delphi to define IL as a skill. This question elicited responses that emphasised lower order skills of finding or collecting information rather than critique and synthesis. This aligned with the ILOs that reached consensus as learning objectives pertaining to Standard Three: critically evaluates information and the information seeking process. The ILOs for critical thought were the predominant learning outcomes that did not achieve consensus. The findings show that the lower order IL skills appear to be more valued than higher order skills. This is unusual because IL is considered the basis for evidence-based-practice2 and is a necessary skill for paramedic-led healthcare and continuous professional development. The lack of consensus in this area is consistent with previously cited studies showing that IL is often underdeveloped in university graduates.
- #11: Benefits of attaining information literacy skills CAUL make 4 statements about the benefits of attaining IL skills. All the NAPA participants indicated that students undertaking an entry-to-practice paramedic program need to develop information literacy skills. NAPA participants generally agree with the benefits of ILE as described in the ANZIL framework. (shown here) and perceived it as important in paramedic undergraduate entry-to-practice programs and of value to students. However, the comments describing how ILE is taught indicated that ILE is not systematically embedded in many paramedic programs. Four overarching principles: 1Engage in independent learning through constructing new meaning, understanding and knowledge; 2Derive satisfaction and personal fulfilment from using information wisely; 3Individually and collectively search for and use information for decision making and problem solving; and 4Demonstrate social responsibility through a commitment to lifelong learning and community participation.
- #12: There are 6 ANZIL IL standards. Each has a number of elements. In the NAPA Delphi we presented learning outcomes for each element to the participants. We asked the participants the level of importance of each learning outcome where in the program these should be achieved. We showed participants how others responded and gave an opportunity to change their answer. Consensus was if 80% of participants agreed. This slide shows an example of ILOs that reached consensus on importance – high, moderate or low. Note that not all learning outcomes that reached consensus on level of importance reached on consensus for place in program. Participants indicated that many outcomes should be taught at the commencement of a program of study. Front loading the foundations of IL skills makes sense as skills of finding, describing, storing and managing information, support the higher level skills such as information analysis and evaluation. The other surprise was that critical thinking outcomes of did not reach consensus. It is my position after this project that this may tie into why universities are poor at developing IL skills in graduates and why this attribute was only mentioned by about half of the NAPA participants in their definitions. The first two numbers in the numbering of the ILO shows which standard and element it is assigned to. The third number is added to provide the ILO with a unique identifier.
- #13: From these findings we developed the important themes or activities and context for teaching in an entry to practice paramedicine program. What is shown here is an example of two elements for standard 1. This has allowed us to make better choice around curriculum development.
- #15: Peer Reviewed Journal 97.3 Textbooks 82.0 Government reports 64.9 Online reference 60.4 Collaborative Encyclopaedia 12.6 News Media 2.7
- #16: Statistical values for association Fishers exact test=9.455, P=0.027 Q1: I use diverse sources of information to develop my assignment. Choose one response only. Q2: I check whether a concept is still current while developing an answer for an assignment. Q3: I use a systematic process to help me judge the relevance of the information Q11: Do you use bibliographic software such as Endnote to manage collected information?
- #17: Regarding the use of multiple sources to reach a conclusion Early year 10.8% C/- final year 60%; FET=23.467, P=0.002 Use the USC style guide to ensure accuracy in citations Early year 9.4% C/- final year 20%; FET=13.330, P=0.022 We also found in the data that Students perceive that basic statistical information such as P value, Confidence Intervals, Odds Ratio, Limitations of study design, Level of evidence, Number needed to treat, is important to consider when choosing an article to use for an assignment But only about a quarter of students reported using this data often or always when it was provided, Even less were able to correctly identify the correct definition for these concepts from a range of options. . Concept Important informationΩ (%) n=74 Actively considerΨ (%) n=101 Able to define concept (%) N=76 P value 66.7 28.7 19.5 Confidence Intervals 41.4 23.8 17.1 Odds Ratio 62.7* 18.6* 14.3 Limitations of study design 84.1* 50.6* Level of evidence 84.5 72.2 Mean 81.6 Median 76.3 NNT 14.5
- #18: The big easy – is a strategy that can be implemented using your current resources and network. These are the start of change These actions start momentum for change.
- #19: We shouldn’t assume that understanding scientific communication and information literacy occurs automatically with a program of study A lot of service teaching occurs for our program, so it is not just individual programs with responsibility to develop IL skills of students So the question became how wouldour program scaffold the teaching of students to move beyond reproducing current thinking to critical thinking.
- #20: We are moving away from course-centric to a program centric view. We are looking at the CAUL standards and defining within our program what students should achieve at each year level. We are then mapping this to courses. I will work through this slide to demonstrate what I mean. Vertical integration For each CAUL IL standard we must define it at program level and then year level within the program and decide what assessment tasks would best detect evidence of achieving that standard. The next decision is to decide which courses are best suited to target the teaching. Horizontal We take the information from the decision tool and apply it to the selected courses. The student data has shown us where the problem areas are The Delphi data provided information on what Paramedic Educators think are the important aspects to be concentrating on. We will also be concentrating on critical thinking, although no consensus was reached. We are used this template to map ILE across the program and courses. Wang L. Developing peoples information capabilities: Fostering Information Literacy in educational workplace and community contexts. Library and Information Science. 2013;8:31-49.
- #21: As a team we produced a concensus statement on the ANZIL standards for each year level. PAR101, PAR102, PAR103 PAR211, PAR202, PAR203, PAR222, PAR212 PAR301, PAR311, PAR302
- #22: To guide our staff the following flow of information and assistance will be implemented. Roles within the framework Program reference group: expertise external to the program that can advise on guidelines and pedagogy to enable information literacy education with a program. external Program coordinator: the program lead for the design, alignment and quality assurance of curricula activities within a program of study Course coordinators: responsible for the design and delivery of individual courses within a program of study C-SALT: USC body responsible for assisting program and course coordinators to deliver quality programs of study.