Statistics for Librarians, Session 4: Statistics best practices
Download as PPTX, PDF2 likes992 views
The document outlines best practices in statistics, emphasizing the importance of understanding known and unknown variables, using valid measures, having a data entry plan, and knowing the data well. It details statistical concepts including types of variables, levels of measurement, methods of data analysis, and approaches to hypothesis testing while advocating against relying solely on p-values. Furthermore, it highlights the significance of effect sizes, confidence intervals, and the necessity of using appropriate statistical methods tailored to research questions.
Statistics for Librarians, Session 4: Statistics best practices
- 2. Know what you know and what you don’t know Have a comparison group Use validated measures Have a Data Entry Plan Get to know your data If it doesn’t fit, change it Place your bets before you collect the data Use the best methods of analysis for your question & your data Go beyond the p-value BEST PRACTICES
- 3. What is Statistics? •Study of Data •Collecting •Organizing •Summarizing •Analyzing •Presenting •Storing & Sharing Why is it Important? •Make sense of the data •Explain what happens and (possibly) why •Make sound decisions •To know how close we are to the truth.
- 4. Results Bias? Sampling Error? Invalid Measures? Random Error? Other Factors? PURPOSE OF STATISTICS
- 5. BEST PRACTICE: KNOW WHAT YOU ALREADY KNOW, WHAT YOU WANT TO KNOW AND WHAT YOU DON’T KNOW
- 6. How do users differ when (searching, finding, selecting) (articles, books, Web sites)? What are the effects of ___________On ____________? Whichis better at improving _________? How are people (finding, selecting, using) _______? What are factors associated with ___________? STARTING WITH YOUR RESEARCH QUESTION
- 7. KINDS OF VARIABLES Independent Subjects Factors Effects of… Dependent Objects Outcomes Effects on…
- 8. Nominal •Counts by category •No meaning between the categories (Blue is not better than Red) Ordinal •Ranks •Scales •Space between ranks is subjective Interval •Integers •No baseline •Space between values is equal and objective, but discrete Ratio •Interval data with a baseline •Space between is continuous LEVELS OF MEASUREMENT (NOIR)
- 9. •Counts by Categories •Ranks •Scales Qualitative •Measurements •Composite scores •Simple Counts Quantitative ANOTHER WAY
- 10. LIKERT-TYPE SCALE? Arbitrary Few Levels Individual Questions Ordinal? Symmetrical Many Levels Composite Score Interval?
- 11. BEST PRACTICE: HAVE A COMPARISON GROUP
- 12. WAYS OF COMPARING… Time Periods Other Libraries National Surveys Patron Types Material Types
- 13. •Qualitative •Comparison Expected ranks or ratios •Quantitative •Correlations Two variables •Quantitative or Qualitative •Paired or Not Paired Samples or Groups KINDS OF COMPARISON
- 14. BEST PRACTICE: USE A VALID MEASURE
- 15. Are you actually measuring what you are trying to measure? VALIDITY OF MEASURES
- 16. USE A TOOL WITH ESTABLISHED VALIDITY Approaches and Study Skills Inventory for Students (ASSIST) User Engagement Scale (UES)
- 17. ESTABLISH VALIDITY OF MEASURES •ConsistencyReliability •Common sense Content or Face Validity •Based on theory Construct Validity •Comparison with other valid measures Criterion Validity
- 18. BEST PRACTICE: HAVE A DATA PLAN
- 19. GOAL OF DATA COLLECTION IN STATISTICS Reliability Bias
- 20. BIAS Systematic (not random) deviation from the true value (Statistics.com) Selection Bias Measurement • Observer Bias • Non-response Bias Analysis Bias
- 21. DATA INPUT Have a data entry plan Train the inputters Use data validation tricks Double-entry
- 22. BEST PRACTICE: GET TO KNOW YOUR DATA
- 24. • Average • For Quantative data • Excel function: =Average(range) Mean • Middle • For Quantitative or Rank data • Excel function: =Median(range) Median • Most common • Primarily for Qualitative data • Excel function: =Mode(range) Mode MEASURES OF CENTRAL TENDENCY
- 26. DISTRIBUTION OR SPREAD OF QUALITATIVE DATA Tables •Counts •Percentages/Ratios •Averages of Counts Excel •Pivot Tables
- 27. PIVOT TABLES IN EXCEL Select Data •Highlight table •Insert->Pivot Table Select Variables •Categories (Row Labels) •Values Change Settings •Percentage of Grand Total •Average
- 28. DEMONSTRATION OF PIVOT TABLES FOR SPREAD OF QUALITATIVE DATA
- 29. GRAPH & CHART RULES OF THUMB Trends Connection across the X- axis Categorical Comparisons Grouped Stacked Relative Stacked Categorical Few Categories Differences are Wide
- 31. John W. Tukey Exploratory Data Analysis Examining your data visually. Stem & Leaf Hinges Box plots Scatter plots, etc. EXPLORATORY DATA ANALYSIS
- 32. STEM-AND-LEAF Stem Leaf 0 01112222222222222233333344445556 666677788899 1 0000000011122223333356778899 2 00122234444799 3 0245 First digit(s) Last digit Years at UNT 0 5 13 1 6 13 1 6 13 1 6 13 2 6 15 2 6 16 2 7 17 2 7 17 2 7 18 2 8 18 2 8 19 3 11 29 4 11 29 4 12 30 4 12 32 4 12 34 5 12 35 5 13
- 33. FROM STEM-AND-LEAF TO HISTOGRAMS
- 34. Stem Leaf Count 0 1122223334445555666666677777899 31 1 000011122222222333346677889 27 2 0122234468 10 3 1112355888 11 4 12 2 Range Count 0-9 31 10-19 27 20-29 10 30-39 11 40-49 2 0 10 20 30 40 0-9 10-19 20-29 30-39 40-49 Histogram of Years at UNT
- 35. HISTOGRAMS IN EXCEL •Options •Add-ins •Manage Add-ins Analysis Toolpak •Equal Size Ranges •Ceiling (“more”) Set ranges •Data •Data Analysis •Histogram Create Histogram •Insert Bar Chart •Highlight histogram •Select bars & Format Selection •Gap Width=0% Create Graph For Histogram 9 19 29 39 49
- 36. DEMONSTRATION OF HISTOGRAM IN EXCEL
- 37. SPREAD OF QUANTITATIVE DATA How variable is the data? Range Quantiles Standard Deviation
- 39. Box plots Median Upper & lower quartiles Outliers PRESENTATION OF SPREAD
- 40. Measure of dispersion of data Square root of the average variation from the mean STANDARD DEVIATION
- 41. Greater variation, less certainty Lower variation, more certainty WHAT DOES THE SD TELL YOU?
- 45. DEMONSTRATION OF DISTRIBUTIONS Distribution of the Population The “Truth” N is the # of samples n is the number of items in each sample Watch the cumulative mean & medians slowly merge to the population
- 46. Transform ation of data BEST PRACTICE: IF IT DOESN’T FIT, CHANGE IT
- 47. WHY TRANSFORM? 0 5 10 15 20 25 30 35 40 45 50 0-9 10-19 20-29 30-39 Years at UNT 0 2 4 6 8 10 12 14 16 Log10(Years at UNT)
- 48. Y=a+bx Log(Y)=Log(a+bx) 1/Y = 1/(a+bx) HOW TRANSFORMATION WORKS
- 49. Evaluate the distribution of raw data Select a transformation method Transform the data Normally Distributed? Statistically Test Transformed Data HOW TO BECOME NORMAL Express the result in the terms of the transformation
- 50. BEST PRACTICE: PLACE YOUR BETS BEFORE YOU START
- 51. INFERENTIAL STATISTICS Tests of hypotheses •Associations •Expectations Accounts for uncertainty •Random error •Confidence interval
- 53. EXAMPLE HYPOTHESIS >=75%* <75%* *…of journal articles cited by UNT PACS faculty in journal articles published between 2008-2011. UNT Libraries provides access to…
- 56. BEST PRACTICE: CHOOSE THE BEST METHOD FOR YOUR QUESTION AND DATA
- 57. Assumptions Limitations Appropriate data type What the test tests KNOW THE TESTS
- 58. Variable Type What is being compared Independence of units Underlying variance in the population Distribution Sample size Number of comparison groups FACTORS ASSOCIATED WITH CHOICE OF STATISTICAL METHOD
- 59. USE A FLOW CHART
- 60. BEST PRACTICE: GOING BEYOND THE P- VALUE
- 61. AND THE P-VALUE SAYS… Much about the distributions More about the H0 than H1 Little about size of differences
- 62. MORE USEFUL STATISTICS Effect Sizes •Tell the real story Confidence Intervals •State your certainty
- 63. Correlations •Cohen’s guidelines for Pearson’s r Differences from the mean •Standardized •weighted against the standard deviation •Cohen’s d 𝑑 = 𝑥1 − 𝑥2 𝑠 EFFECT SIZES OF QUANTITATIVE DATA Effect Size r> Small .10 Medium .30 Large .50
- 64. Based on Contingency table • Odds of event A divided by odds of event B • Case-control studies Odds ratio • Uses probabilities rather than odds • Experiments, RCTsRelative risk EFFECT SIZES OF QUALITATIVE DATA Test A/B Yes No Total Yes 10 15 25 No 50 25 75 Totals 60 40 100
- 65. Point estimates Intervals Based on Expressed as: •Single value •Mean •Degree of uncertainty •Range of certainty around the point estimate •Point estimate (e.g. mean) •Confidence level (usually .95) •Standard deviation •The mean score of the students who had the IL training was 83.5 with a 95% CI of 78.3 and 89.4. CONFIDENCE INTERVALS
- 67. Know what you know and what you don’t know Have a comparison group Use validated measures Have a Data Entry Plan Get to know your data If it doesn’t fit, change it Place your bets before you collect the data Use the best methods of analysis for your question & your data Go beyond the p-value BEST PRACTICES
- 68. RESOURCES Rice Virtual Lab in Statistics Excel Tutorials for Statistical Analysis Khan Academy - videos Basic Research Methods for Librarians Descriptive Statistical Techniques for Librarians