Kwon Ph.D. Dissertation 2016
Download as PPTX, PDF2 likes227 views
The document discusses various projects focused on visualizing citation patterns and impact metrics within academic research, notably through platforms like Google Scholar and Scholar Plot. It highlights several academic conferences and publications related to these topics, outlining methods and tools designed for citation analysis, funding considerations, and visualization of scholarly impact. The effort emphasizes bridging data representation with user experience in scholarly communication.
Kwon Ph.D. Dissertation 2016
- 6. … 30, 40, 50 ... pages
- 8. + Free; inclusive - Publications only; little visualization + Publications; funding - Requires extensive set-up Missing about 2,000 citations, 16 h-index
- 9. Article Author Year Conclusion “Visualization of the citation impact environments of scientific journals” Journal of the American Society for Information Science andTechnology L Leydesdorff 2007 Effort focused on visualizing citation patterns using a journal data set “Augmenting the exploration of digital libraries with web-based visualizations” IEEE Fourth International Conference on Digital Information Management (ICDIM 2009) P Bergstrom D Atkinson 2009 Exploring patterns in the literature using a static data set at CiteSeer “SciVal experts: A collaborative tool” Medical Reference Services Quarterly EVardell T Feddern-Bekcan M Moore 2011 Summary of researchers’ profiles using Scopus “Scholarometer: A system for crowdsourcing scholarly impact metrics” Proceedings of the 2014 ACM Conference onWeb Science (WebSci 2014) J Kaur M JafariAsbagh F Radicchi F Menczer 2014 Citation analysis using Google Scholar, but no Impact Factor and no funding information
- 11. 1 3 2
- 12. A. Google Scholar Profile B. CurriculumVitae C. Scholar Plot
- 14. Histogram of IF IF Frequency 0 10 20 30 40 50 0100030005000 IFs Journals #1 <= 2 5554 #2 2 - 4 1948 #3 4 - 16 808 #4 16 >= 62 * IF - Impact Factor IF Journals
- 17. A
- 18. B
- 19. CBA
- 29. HTTP request: (jQuery, Ajax) HTTP response: (JSON) HTTP request: id = author name HTTP response: author details 1 Web Server (PHP 5.5) Client (Web browsers) 2 3 Database Server (MySQL 5.5) 6 Table Scholar authors, Impact Factor, NSF, NIH, etc. Table Table 7 Render in SVG (HTML5+CSS3, nvd3) 4 5 SQL Query Google Scholar Authors Impact Factor NSF, NIH, NASA Dynamic data – On Demand Yearly Update
- 30. I Pavlidis First Initial + Lastname
- 31. Daniel M. Smith Daniel Michael Smith M % Daniel Daniel M % Daniel Michael Daniel Michael Google Profile Funding
- 32. • 1.1 :The scheme should reveal causal relationships among merit criteria • Funding + pre-production credit + post-production credit • 1.2:The scheme should be scale invariant • Individual or Department or College (composite personhood)
- 33. • Scholar Plot is good for individuals • Not scalable to groups No!!!
- 34. • 1.1 :The scheme should reveal causal relationships among merit criteria • Funding + pre-production credit + post-production credit • 1.2:The scheme should be scale invariant • Individual or Department or College (composite personhood) • Scholar Plot is good for individuals • Not scalable to groups • Scholar Plot draws from Google Scholar,Thompson Reuters, and OpenGov • It is a public product working flawlessly! (ScholarPlot.com) • Scaling interface was still pending Work-in-progress Done Done Done Work-in-progress
- 38. Earth and Atmospheric Sciences PhysicsBiology and Biochemistry
- 40. • Local – same department • Global – same discipline
- 41. • Local - same department • Global - same discipline
- 42. • Local – same department • Global – same discipline
- 49. • 1.1 :The scheme should reveal causal relationships among merit criteria • Funding + pre-production credit + post-production credit • 1.2:The scheme should be scale invariant • Individual or Department or College (composite personhood) • Scholar Plot is good for individuals • Not scalable to groups • Scholar Plot draws from Google Scholar,Thompson Reuters, and OpenGov • It is a public product working flawlessly! (ScholarPlot.com) • Scaling interface is still pending • Validates the design choice of the three criteria for the visualization Done Done Done
- 50. Fall 2011 (1st year) Spring 2012 Fall 2012 (2nd year) Spring 2013 Fall 2013 (3rd year) Spring 2014 Fall 2014 (4th year) Spring 2015 Fall 2015 (5th year) Spring 2016 S Taamneh, M Dcosta, K Kwon and I Pavlidis "SubjectBook: Web- based Visualization Of Multimodal Affective Datasets", ACM Human Factors in Computing Systems, CHI 2016, San Jose, CA D Majeti, K Kwon, P Tsiamyrtzis and I Pavlidis "Dissecting Scholarly Patterns in Biology and Computer Science", The Science of Team Science, SciTS 2015, Bethesda, MD K Kwon, D Shastri and I Pavlidis "Information Visualization in Affective User Studies", The IEEE Visual Analytics Science and Technology, IEEE Information Visualization, and IEEE Scientific Visualization, VIS 2014, Paris, France K Kwon, D Shastri and I Pavlidis "Interfacing Information in Affective User Studies", The 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Ubicomp 2014, Seattle, WA T Feng, Z Liu, K Kwon, W Shi, B Carbunar, Y Jiang and N Nguyen, "Enhancing Mobile Security with Continuous Authentication Based on Touchscreen Gestures", The twelfth annual IEEE Conference on Technologies for Homeland Security, HST 2012, Waltham, MA J Lee, Z Liu, X Tian, D Woo, W Shi, D Boumber, Y Yan, and K Kwon, "Acceleration of Bulk Memory Operations in a Heterogeneous Multicore Architecture", 21st International Conference on Parallel Architectures and Compilation Techniques, PACT 2012, Minneapolis Conference Presentations K Kwon, "Design Principles: Information Visualization in User Studies", Proceedings of the 2015 US-Korea Conference on Science, Technology and Entrepreneurship, UKC 2015 Atlanta K Kwon, "Interfacing Information with Mixed Methods", Proceedings of the 2014 US-Korea Conferenceon Science, Technology and Entrepreneurship, UKC 2014 San Francisco, CA Activities / Membership 2012 PhD Student Association Officer 2014 Computer Science PhD Showcase 2014 Graduate Research and Scholarship Projects (GRaSP) 2015 Graduate Research and Scholarship Projects (GRaSP) 2016 Volunteering Judges M.S.Switched Lab Released Released
- 52. Scholar Plot
Editor's Notes
- #2: Thank you for coming for my presentation today. My name is Kyeongan Kwon. Today, I am going to present my PhD dissertation about scalable data visualization for academic careers.
- #3: This is overview about today’s presentation. In introduction, I am going to talk about the research problem and related works. In Design philosophy and methodology, I am going to cover design philosophy and how the philosophy apply to the product.
- #4: Before starting to present my research, I would like to talk about a little bit what data visualization is exactly. There are many benefits of it, however, I point out two main things. Qualitative analysis is the scientific study of data that can be observed, but not measured.
- #5: Because of the way the human brain processes information, using charts or graphs to visualize large amounts of complex data is easier than poring over spreadsheets or reports.
- #6: A Good Research Problem There are difficult and time consuming tasks when Appraising academic careers. For examples,
- #7: CV is still very popular to evaluate. - it is where your publishing and research history as well as your memberships and fellowships. Why it is inconsistent. For example, the way people list their publications is inconsistent. Journal (IF or not) , Conference (Acceptant rate or not )
- #8: Three are three goals of research.
- #9: Some related works is out three. Software products.
- #10: Also there are some literature related work.
- #11: There is no definitive scheme for evaluating academics. Because everybody has own ideas. Big journal is good, many cited publication is also good. This isn’t clear, how it is related to each other.
- #12: Let me tell you how I structured this visual abstract based on three merit criteria. - Impact of the intellectual products, what happened after you published. - Prestige of the venues where intellectual products appear, How famous’ places where you published. - Funding that enables intellectual production Why impact on the vertical scale is not a Impact Factor, but citation because of the fixed scale of Impact Factor (1-60), citations could reach 100,000
- #13: I have three figures of the same scholar in order to see the difference. Google Scholar Profile. Only one bar chart ; publication list Curriculum Vitae, a brief account of a person's education, qualifications, and previous experience. However, Scholar Plot brings more but simply. It includes all the publications with different colors and symbols which lead people could distinguish type of publication quickly. It also includes funding information.
- #14: This is one result of a scholar who have about 300 publications includes all Journal/ Conference and Patents. It is easy to identify what type of publication powers the individual’s scholarship.
- #15: How Differentiate different prestige Journal has little different prestige because it has only a widely accepted ranking system that is the Journal Impact Factor. Conference has acceptance rates and some general guide line to tell the rank of conferences like A+ / A. However, some has no these information. Patents have no this type of information. We chose disks to be the symbols of journal publications, as disk scaling can be done very effectively by simply varying its radius (A ∼ r2). Size How do I present this value from lower to higher IF. I did some data analytics for this. Frequency of Journal published by Thomson Reuters Four categories based on histogram analysis. I use this to commensurate the size of disk.
- #16: Two different scales views to create a standardized scale for the y-axis for comparison, I introduced a log10 scale for the default plot and an option to toggle to the decimal scale view. If you have a senior records, and put it as a decimal scale, it suppress visualization. However, junior records isn’t important because you hardly can see the difference. That’s why Logarithm is default. It makes the difference visibility.
- #17: Scholar Plot also depicts the NSF/NIH/NASA funding of an individual as a multiline in log2 scale Line Chart is proper for me to use visualizing funding data because it is commonly used like stock markets.
- #18: There are some patterns of scholarly profiles. I bring three example. not necessarily look at the main plots, you can simply see the vertical projection and tell the type of patterns.
- #21: Now I had a prototype including publication records and funding records. I wanted to make it improved. So, I gave two evaluations both focus group and user study.
- #22: A total of n = 15 participants from various natural, mathematical, and social sciences participated in the user study. Usability, accuracy and intuitive understanding of Scholar Plot scored the highest (μ = 4.2). The participants were planning to use Scholar Plot frequently (μ = 4.1). Overall, the survey confirmed that the base level of Scholar Plot is a user- friendly tool that academic users find of interest and value.
- #23: through a focus group I added 4 panels below scholar plot results. Team science information. - the number and intensity of collaborations for the depicted scholar. Impact information - Summarize highly cited papers. Prestige information - the specific journals where the scholar publishes most often and their impact factors.
- #24: We have another feature that interact an each plot. When you interact with a plot, it shows the tooltip, which includes paper title, year of publication, the number of citations, journal or conference name and journal impact factor. this is optional
- #29: Data sources Impact which is citations from Google Scholar Prestige which is Impact Factor from Thomson Reuters
- #30: User visit to Scholar Plot Type the scholar name to send HTTP request in Ajax(asynchronous JavaScript and XML) and jQuery Web server fetches scholar data from Google Scholar Profile with user input parameter Web server connects database server in order to fetch the data (Author, Impact Factor, Funding information) Web server returns data as a Json format to user with HTTP response Web browsers renter the data in SVG using HTML5+CSS
- #31: The biggest problem is middle name
- #32: Matching Title will be placed 1. remove 2. matching
- #35: I presented Scholar Plot (SP), a compact and scalable (individual-department-college) visualization interface for academic merit. We released SP at http://scholarplot.com. The basic idea behind SP is to facilitate instant deeper insights regarding different strengths of academic records, supporting the work of evaluation committees and the curious academic in search of an advisor or department. One of SP’s strengths is that it draws data from open sources that are inclusive.
- #37: 1. Funding information - NSF/NIH/NASA 2. Citation 3. Impact Factor - size and colors
- #38: You can simply understand the concept and information of department
- #41: Local - Local scale Y-axis is automatically scaled to the highest citation received by faculty in the department. Quartiles are calculated using the faculty within the same department. Citation / Impact Factor / Funding Global – Quartiles are calculated using the faculty within the same discipline. We used CIP (The National Center for Education Statistics designed the Classification of Instructional Program) codes to map the departments to disciplines. In Global, scale below the cloud is fixed with 20,000 which is 90 percentile of all faculty. scale above the cloud is fixed with 300,000 which is approximately the highest citation of faculty
- #42: Guess how do you think local, Truly everyone in department are good.
- #44: To validate design choices – prominent group (chaired) 1. Ground truth (standard, base) 2. Linear Model – proof 3. Variables - Mirror to visualization I ran data analysis for validation of academic garden design. CS – n= 248 of 515 , n=61 of 130 Bio – n=152 of 381 , n=32 of 97
- #45: This linear model evaluates the correlation chaired professors with the three variables displayed in Academic Garden. Total Citations, Mean IF, Total Funding (NSF, NIH, NASA) Is_chair (1) – if the faculty is in the top quartile for either of the 3 criteria – funding, mean IF, total citations Is_chair (0) – if the faculty is not in the top quartile for all the three criteria Quartiles are calculated with respect to the department to which the faculty belongs to. ex., if a faculty is in the top quartile for citations but not for funding and impact factor, he still gets 1 according to this measure/metric
- #46: All three criteria as separately factors In computer science, citations are very important. And you can clearly see the chaired faculty. Significantly related to citation. Because they don’t publish so much journals, impact factor are not significant. Also, our funding sources are not included everything which computer science faculty received funding from.
- #47: local_top_q_any_of_three 1 – if the faculty is in the top quartile for either of the 3 criteria – funding, mean IF, total citations 0 – if the faculty is not in the top quartile for all the three criteria Quartiles are calculated with respect to the department to which the faculty belongs to. ex., if a faculty is in the top quartile for citations but not for funding and impact factor, he still gets 1 according to this measure/metric
- #48: All three criteria as separately factors In computer science, citations are very important. And you can clearly see the chaired faculty. Significantly related to citation. Because they don’t publish so much journals, impact factor are not significant. Also, our funding sources are not included everything which computer science faculty received funding from.
- #49: Linear Model: Academic Garden This linear model evaluates the correlation chaired professors have with the three variables displayed in Academic Garden. Why ? Validate design choices – prominent group (chaired) 1. Ground truth (standard, base) 2. Linear Model – proof 3. Quartile - Mirror to visualization This data analysis validates the design choice of the three criteria for our visualization!
- #50: We presented Scholar Plot (SP), a compact and scalable (individual-department-college) visualization interface for academic merit. We released SP at http://scholarplot.com. The basic idea behind SP is to facilitate instant deeper insights regarding different strengths of academic records, supporting the work of evaluation committees and the curious academic in search of an advisor or department. One of SP’s strengths is that it draws data from open sources that are inclusive.
- #52: I would like to thank Dr. Pavlidis being my research advisor. And I would like thank each of you for serving on my committee. Done.