Mpc.01-Introduction_EN.pdf
- 1. Introduction SCC5933 – Research Methodology in Computer Science Prof. Moacir Ponti www.icmc.usp.br/~moacir Instituto de Ciências Matemáticas e de Computação – USP 2018/1
- 2. Introduction What is research? I “Research is the process of gathering information about some subject, analyze them using the scientific method with the intention of increase the stock of knowledge” (Wikipedia)
- 3. Introduction Method I “The scientific method is a set of basic rules for a scientist to develop a controlled experiment in order to test and observe events, so that to reach conclusions and report those conclusions, that, in case of validity, are then applied to science” (Wikipédia)
- 4. Choose the research theme Theme I The choice can be made looking for: I Relevant: scientific, social, technologic, I Adequate: to those employed at university, institute and research lab I Check for time and feasibility to develop the research I Scope: it is not necessary to solve it all. It is better to limit then to be too broad.
- 5. Objective I The objective can be defined with a literature review I Should be an action that addresses some gap or existing problem I Must be coupled with a well defined hyphotesis
- 6. Objective I Warning: objectives as proposal are usually weak I If the objective of research is “to propose something”, then the mere proposal is sufficient? I Enunciate the problem in a precise way I Explain why the problem is important given the literature review I Make sure the premises are sound
- 7. Some examples I “...this project proposes the use of optimization methods for vehicle route problems...” I “...the main objective is to develop neural network algorithms for sentiment analysis in text...” I It is hard to grasp the actual problem to be addressed I It is not clear what exactly is the research question and its importance in those objectives
- 8. Hyphotesis I Good objectives are driven by good research hypothesis Hypothesis I Claim that will be tested to be true or false I The research project must investigate the claim in order to confirm or falsify this claim; I Defining a sound hyphotesis is what differentiates research from a technical work.
- 9. Literature Review I The research must keep reading throughout the research projects I It is ok to start with books and surveys I After you master the main techniques, then search for relevant work on good repositories I Read papers in a critical way: I LARAMEE, R. S. How to Read a Visualization Research Paper: Extracting the Essentials. IEEE Computer Graphics and Applications, Vol. 31, No. 3, 2011, pages 78-82. Disponível em http://www.cs.swan.ac.uk/~csbob/research/how2read/ laramee09how2read.pdf. I FOWLER, M. How to Read Signal Processing Journal & Conference Papers. http://www.ws.binghamton.edu/fowler/HowReadPapers.htm. Repositories I Scholar (http://scholar.google.com) I Scopus (http://www.scopus.com) I Web of Science (http://www.webofknowledge.com) I ...
- 10. Evaluation How to evaluate your research? I Define, as soon as possible, how to measure your results in order to understand how close you are to the main objetive I Try hard, but if necessary, drop/change the initial idea. I Since usually ∼ 90% of outcomes are actually failures, we have to make sure we are evaluating correctly the results, since the beginning I Understand all research has limitations and weak points I Example: little innovation, incremental contribution, results marginally different from state-of-the-art, application is restricted, it is not scalable, etc. I But: a negative result is also good if the method was correct!
- 11. To be exposed to research ideas I Exposition makes it easier to (re)define objectives and find ideas I Discuss your work with other colleagues and researchers I Frequently read papers (at least 1 monthly) I It is the responsibility of the student to bring ideas and possibilities to discuss with the supervisor! I Follow important researchers on social media (twitter, research gate, etc.)
- 12. Agenda Introduction Steps of the scientific process Research levels/tipes in Computer Science
- 13. 1: “Product or implementation” I Can be innovative or not I If lacking a hyphotesis, then it is not I When innovative, it is usually exploratory I If it is a system or reproduction, can be reported in a “Technical Report” I Acceptable for undergraduate final project (TCC), but hardly for Master or Doctorate degrees
- 14. Exploratory I It is acceptable to not compare with previous work I Biological computer that solves problems such as the travelling salesman Vic Norris et al. Computing with bacterial constituents, cells and populations: from bioputing to bactoputing. Theory Biosci. 130(3): 211-228, 2011.
- 15. System or implementation I Can be justified when there is a clear application, not yet explored I Health, I Education, I Agronomy, I ... I But, if so, then it is important to compare with previous work at least qualitatively!
- 16. 2: “Something different” I proposes a “different approach” to some problem, or a “different” implementation or application I require literature review and qualitative comparison
- 17. I Can be a different approach, not necessarily better I It is usual in problems that are well studied, but not sufficiently I Can result in case studies Deep Learning I For a while (∼ 4 years), just approaching something with deep learning was enough I Now those need more rigor Thanks to www.xkcd.com
- 18. I It is valid to propose “something different” when there is scarce data or time escassos. I A well defined case study can be a good way to start I But, make sure I premises are convincing I there is some hyphotesis
- 19. 3: “Something supposedly better” I A problem that is well studied, there is available data and papers reporting results on those well known datasets I Need to follow protocol, evaluation measures, that are previosly defined by the literature
- 20. I When there are already many solutions: you must justify why your approach is valid or better in some sense I Often results in an incremental contribution. I need a better discussion on the drawbacks and advantages Travelling salesman problem I Formulated in 1930, NP-hard. Brute-force solution isO(n!) Thanks to: www.xkcd.com
- 21. Example: image denoising Noisy image State-of-the-art Our method
- 22. 4: “Something better” I New results are better considering standard tests I Datasets that are known and widely used in the literature I Comparison is direct since everyone follows the same protocol I Advances the state of the art e.g. image classification I Benchmark: Caltech-101, ImageNet
- 24. I If your method is better, then you add a new ’line’ among the most relevant ones
- 25. 5: “Proof” I Need a good theoretical background I Involves the use of theorems, lemmas, in order to address some problem under some premises
- 26. I Following some theoretical framework, write a proof based on induction, deduction, contradiction, etc. I Modern computer science was born with such types of research, in the decades of 1930-1940 I First themes were: computability, algorithms, complexity, information theory, optimization, artificial intelligence, etc. Alan Turing. On computable numbers, with an application to the Entscheidungsproblem. Proc. London Math. Society, vol. 42, 230–265, 1937
- 27. A remarkable example P vs. NP I A problem for which there is an algorithm that finds a solution in polynomial time: class P I A problem for which there is an algorithm that verifies a solution in polynomial time: class NP I Prove if P = NP is one of the most relevant open problems in computer science.
- 28. Another example Compilers I Check machine code for 64 bits and multicore processors I Source code correctness
- 29. Computer Science research I Theoretical, I Empirical, I Exploratory.
- 30. The role of the supervisor I Offer criticism I Help interpreting and discussing the results, facilitating new ways to solve some problem I Recommend studies and papers I Read and give input to text, thesis, dissertation, papers, reports